A Recovery Disaster

My first task was to assess what I could salvage. The server room was soaked through, however, some kit was off the ground, but unfortunately, quite a few tower servers were on the floor and completely ruined.

After a couple of hours of unplugging what was dry and moving it, I was ready to try to get things back up and running to some degree. I managed to salvage a few things: 3 servers, a switch, an external tape drive and around 10 laptops that were on the desks downstairs.

Then I started connecting it all up with the view of creating a small network which would allow them to carry on as usual while the ground floor of the building was out of action. When connecting it all up, it soon became apparent that the servers on the floor were the domain controllers and the main application servers which housed the company’s active directory, DHCP and DNS, databases etc.

There was a problem, we had no hardware to recover the tape backups to, at that point in time they were as good as useless in getting the company back up and running.

Explaining this to the managing director was difficult, he couldn’t understand it. They were getting successful backups every night and they paid for a lot of tapes. So naturally, he wanted it working ASAP. I called the office to see if we had any spare server hardware that we could recover the servers to. Unfortunately, they had been loaned to another company who were in the process of migrating to a new system. There was only one thing for it, I had to speak to the account manager and get new servers ordered ASAP.

5 days later, with new server hardware and software I started recovering the servers. If you have ever tried to recover a domain controller from a tape backup you will know it’s no easy feat. The fact that business had been down for 5 days was bad enough, but paired with restoring the database server meant the pressure was on.

Me being me, I got the servers back up and running, but in total it took 8 days before they were back to a functional level. During this time the entire manufacturing process had stopped, they weren’t able to receive emails and staff had to do overtime to help with the clean-up process.

Anyhow. The point of this blog is to let you know that things don’t have to be like this anymore. As times have changed, new technologies have surfaced and there are quicker ways for companies to get back up and running in the event of a failure.

Whilst there are still a huge number of customers that run tape backup or backup to a NAS drive, there are also a huge number of customers that understand the risks of having no DR in place. So, ask yourself, could your business cope with 8 days of downtime?

New solutions on the market such as Datto mean that customers can take full image backups of their servers to a local appliance every 15 minutes. These images are replicated to the Cloud, giving you a FULL DR solution. If this technology was available years ago when I was a field engineer, I would have been speaking to every man and his dog about it.

For more information on backup and DR solutions visit our Cloud DR managed service page.

If you have a backup plan, it needs to be regularly tested to ensure it works effectively in the event of a disaster. Book your Free DR Review with us today and ensure business continuity in any situation.