Some eNom customers have experienced almost two days of downtime after a planned data center migration went titsup, leading to DNS failures hitting what users suspect must have been thousands of domains.
Social media has been filled with posts from customers complaining that their DNS was offline, meaning their web sites and email have been down. Some have complained of losing money to the downtime.
Affected domains include some registered directly with eNom, as well as some registered via resellers including Google Workspace.
The issue appears to have been caused by a scheduled data center migration, which was due to begin 1400 UTC on Saturday and last for 12 hours.
The Tucows-owned registrar said that during that time both reseller hub enom.com and retail site enomcentral.com would be unavailable. While this meant users would be unable to manage their domains, DNS was expected to resolve normally.
But before long, customers started reporting resolution problems, leading eNom to post:
We are receiving some reports of domains using our nameservers which are failing to resolve. Owing to the migration we are unable to research and fully address the issue until the migration is complete. This is not an expected outcome from the migration, and we are working to address it as a priority.
The maintenance window was then extended several times, by three to six hours each time, as eNom engineers struggled to fix problems caused by the migration. eNom posted several times on its status page:
The unexpected extension to the maintenance window was due to data migration delays. We also discovered resolution problems that impact a few hundred domains
eNom continued to post updates until it finally declared the crisis over at 0800 UTC this morning, meaning the total period of downtime was closer to 42 hours than the originally planned 12.
A great many posts on social media expressed frustration and anger with the outage, with some saying they were losing money and reputation and others promising to take their business elsewhere.
Help, it’s Monday morning in the UK, I have a team who need to work. I have customers who want to place orders. I have a new partnership in progress which will be ruined at this rate. Please please let us know what is happening. #enom #enomdown @enomsupport
— Karen Burns (@cooper_karen) January 17, 2022
@enom @enomsupport What on the world is going on? Every 3 hours we get another boilerplate message saying “another 3 hours” This is completely unacceptable. How are you going to fix this and how are you going to make people whose work sites have been down all weekend whole?
— Andrew Pershing (@Sci_Officer) January 16, 2022
I almost have begrudging respect for how badly @enom has fucked the pooch with their migration. Blowing up your service so badly that your customers can’t even take their business elsewhere is some galaxy brain shit.
— Keith (@FilthySpecs) January 16, 2022
Same. I’m in a group with 6 other bloggers who also have the same DNS issue. Our sites have been down since yesterday. We’re losing ad income & have readers who can’t access the information they need.
— Elaina Newton (@TheRisingSpoon) January 16, 2022
Some said that they continued to experience problems after eNom had declared the maintenance over.
@enom your service status website states incorrect status for Google services? My email is still down!!! Also, getting error when logging in to enom domain management console!
— Tomislav Pavosevic (@tompavosevic) January 17, 2022
eNom primarily sells through its large reseller channel, so some customers were left having to explain the downtime in turn to their own clients. Google Workspace is one such reseller that acknowledged the problems on its Twitter feed.
Some customers questioned whether the problem really was just limited to just a few hundred domains, and eNom seemed to acknowledge that the actual number may have been higher.
Looks like one of the most severe internet outages I’ve seen in years is underway and has not hit MSM yet! @enom, “largest wholesaler of Internet domains” has screwed up a migration and taken possibly up to 2M domains offline.
Certainly my measly 100+ are dead.
See @enomsupport—
Andy Stringer (@AndyStringer) January 17, 2022
At this point, we have been able to track a few hundred. If you have domains that are affected we encourage you to reach out to help@enom.com with the domain(s) in question.
— enomsupport (@enomsupport) January 16, 2022
I’m in contact with Tucows, eNom’s owner, and will provide an update when any additional information becomes available.
The post Nightmare downtime weekend for some eNom and Google customers first appeared on Domain Incite.
Original article: Nightmare downtime weekend for some eNom and Google customers
©2022 Domain Observer. All Rights Reserved.