You’ll never realize that a backup and and disaster recovery plan will help you sleep better at night if you run a website, even if you never have to recover your website.
Recently, two hosting platforms and their users suffered missteps.
a2 Hosting, a shared hosting provider I’ve been using since 2013, has had their Windows servers shut down for over a week as the company suffers from a ransomware attack. Additionally, the available backups the company has for customers appear to be over 2 months old.
DigitalOcean mistook a user’s script as a crypto-mining operation, and shut down a startup’s servers.
I’m fortunate to not be affected as my a2 Hosting account is Linux-based, and my DigitalOcean VPS is a low-profile risk. However, this is devastating for these company and their users. I’m sure there are terms of service policies that cover these hosting companies for situations like these to a certain aspect.
There’s a much to learn from these situations, and this is a good time to reflect on having plans for your website in situations like these.
Restorable backup plan
There’s no excuse not to have a backup plan and infrastructure for your computer, websites, and any important data. Here are some of the backups I have set in my digital life:
- For my main computer, I have a full weekly backup with daily incremental backups, that are then synced to my FreeNAS box, which are also synced to a Backblaze B2 Bucket.
- For my FreeNAS server, I have a backup of the config file that is backed up to Dropbox, Backblaze, and mirrored on a second USB drive.
- For my websites, my entire cPanel host instance is backed up each week, and then downloaded to my FreeNAS server. The individual websites have backups with BackupBuddy which have weekly and daily schedules relative to their respective performance, which are then synced with Dropbox. Some sites also backup to Amazon S3.
As you can see, take my data very seriously. Some data has 3 or 4 destinations. I’m ready to launch a new computer image, FreeNAS box, or return my entire cPanel instance or individual website back from the dead.
In fact, I recently had a botched release of improvements to this very website that went poorly. I was able to bring the site back up in less than 30 minutes because I have the infrastructure and documentation in place to recover.
Disaster Recovery Plan and Exercises
Just having a back up isn’t enough. Knowing how to recover those backups is an important aspect too.
In the case of a2 Hosting, had someone had recent backups of their website, they could have found a new service, restored their backups, and changed their DNS to the new service. After DNS propagation, a website could return to full operations within 24 hours at the latest.
At my day job, I’ve participated in Disaster Recovery Exercises where I help validate whether or not applications I use can perform critical tasks after the recovery begins. It’s a boring exercise, but I now see how important it really is.
My recommendation is to have a test environment that is nearly identical to your site’s live environment, do something to make it no longer work, and then restore the site from a backup. You might even want to try and see if you can also find a new vendor and restore your site to that vendor. Having that knowledge will help you sleep better at night.
Planning for the Future
Despite this situation, I’m still sticking with a2 Hosting and DigitalOcean for the immediate future. a2 Hosting has been a great partner, and I have only a few support tickets with them since I started. I know if I was on the other side of the table, I’d be furious. Companies make mistakes, and no company is infallible.
The moral of this story is: a company as large as these two companies are, they should have had their customer’s data backed up to a separate location (although customers should be responsible for their own data) and had a plan in place to return their service to functionality at a quicker pace.
You don’t have to be a2 Hosting or DigitalOcean, or their users. You now have the knowledge to be better.