Computer Technician of the Year-2011 Goes to......dum dum dum duuuuum:
Ed Steele of RT Software Systems, Inc., in Berkley, Michigan.
Like so many of my legal industry colleagues, our offices are located in a modern, gleaming, office building on dry land in a Detroit, MI suburb. As such, nothing in the world could have prepared me for the surprising call I received at 4:30 AM, last Thursday morning. An 8 inch water main burst, and within an hour, the building’s 75,000 square foot basement was transformed into a murky Olympic Sized Swimming Pool. The water reached the 1st floor (where my office is located) and water was pouring out the front door and into the parking lot. Water began seeping into my “right off the lobby” office suite.
Within minutes, all of our precious electronics, data and some of our paper records were sitting in silt and water. The office wasn’t open; there was no advance warning and no chance to power down and/or unplug equipment. There was neither opportunity to pull PC’s off the floor and onto a desk nor any chance to save the bottom drawers worth of records in each file cabinet.
Disaster had struck, as swiftly and unexpectedly as any we had experienced before. What do we do? Where do we start? Not only had our day to day operations been effected; our major concern, going forward, was: What was the retrieval status of our many years of protected and confidential client data? How were we going to recover this information? In fact, how were we, as a business, going to recover from this catastrophic event?
The most important consideration is, of course, knowing who to call first. Most people would call their insurance company. Certainly, I did that. But, who did I call first? Well..my “IT Guy”, of course! I called Ed Steele, CEO of RT Software Systems, at 4:45 AM and, as is his norm in our long working relationship, he answered the phone! At 4:45 in the morning! I frantically told him what had happened.
Without regard to the time (he was probably sleeping when I called), Ed answered the call to action and scrambled the jets. He drove to our office, met with office personnel and, somehow, convinced the Fire Department to escort him into our office suite. Once he was on location, he started lifting equipment out of the water. He retrieved my server and literally carried it on his back out the front door into the light. (OK, maybe he wheeled it out; the guy is terrific, but he isn’t Superman.) After physically protecting the machine and the precious date that was, hopefully, left inside, he returned to the quagmire and began pulling all of the remaining equipment to dry land.
Unfortunately, this is not where the story ends; Ed had our equipment out of danger within an hour of my original telephone call. However, there was no way to know if the power surges from water shorting out power in the basement circuit panels had fried them, or whether the rising water in our suite had reached the electronics and fried them alive. We could not know if anything would boot and work. Further, maybe worse, we had no office; the building was shut down, water all over the lobby and down each hallway, and without power. We had no place to set up shop, re-establish connectivity, and get back on line so customers could call and/or email, or work could start flowing again.
Again, Ed did not hesitate; RT Software Systems operates a 3000 Square Foot ultra-modern Data Center in Berkley, Michigan, about ten minutes away from our office building. The Data Center is State of the Art – raised flooring, all power gets fed from the top down into the building. It has five air conditioning systems, natural gas generator, and 1000 amp electrical service. There are four Internet connections from various carriers, 24/7/365 Security, Monitoring and Access Control. These are just a few of the features he told me about, not to mention that went I went to see him, the place looked like NASA’s Mission Control in Houston.
And, do you know what he did? He literally placed me “Into a Cloud”, as he put it, and he did this on the same day the event occurred. It turns out that the equipment was a total loss. The Server wouldn’t power on. The seven workstation PC’s were toast. The Firewall, Cable Modem and Switch that made up the backbone of our network were malfunctioning and damaged beyond repair. We had a major problem on our hands. We had remote off-site backup, but the data we were working on the last few days was vital to our operations. Ed Steele fired up his data recovery software and went to work to get our data running on replacement equipment.
Anybody who has gone through the pain of equipment failure knows that the process takes many, many hours of waiting due to the slow copying process from old equipment to new equipment. We were playing a waiting game. Needless to say, I was a nervous wreck. While I gnawed at my fingernails like an expectant father, Ed reassured me that no matter how it ended he would have our business up and running “as soon as possible”, as “early as late tonight”. That’s what he said. Yeah, I know, I didn’t believe him either! All I could do was wait, so I waited. And waited. And waited.
Finally, at around 2 AM the following morning, I received a text from Ed. “You are up and running”, were the glorious words that appeared on my IPhone screen. Access to our server data, and workstation data, had been restored! Ed had managed to provision a dedicated Internet IP Address for our email. He had managed to re-point all of our Intra net and Remote Access connections to his Data Center. He had managed to restore mail flow and made it possible for our five office personnel to connect remotely into the Cloud and have the exact same “XP Software Desktop” that each had before the flood! We were back in business, remotely (remember, we still had no office) before the sun came up on Friday morning. Phones were transferred to cell phones and home phones; we took turns fielding calls.
This was, in my opinion, no small miracle. Not only had Ed immediately answered the call to try to engineer a failed rescue attempt of the assets, but within 24 hours (I don’t think he or his crew got any sleep), he had managed to accomplish what mere mortals might in a week. He even brought back our Windows based Small Business Server from the dead by copying the data to new gear and restoring it “in the clouds”. He recovered and restored the software and local drives on five workstations and copied the data to temporary laptops, then to new desktop workstations. software copies to new equipment. Within days, all workstations were restored and available to all staff-members. Unbelievable!
Don’t get me wrong – things have been far from perfect; printer connections were slow to be established; files that had not been saved to disk were lost. Days of work are yet to be recovered, if they are recoverable, at all. Paper records were damaged and office supplies were lost. We are in temporary space (our landlord did a nice job in rescuing the remains of our office and moving it to dry space over the weekend, providing Internet access to our “cloud” and moving our telephone service to the new suite).
So, we are functional, not optimal, but fully functional. Our web site, email services, remote access are up and running; we were operational for a time on cell phones and laptops; within a week, thanks to Ed and building management, telephone service was restored, Ed secured and installed new desktops, and we are fully functional in new, albeit cramped (you can’t have everything) space.
What did I learn from this experience? A lot! Here are a few important things:
1. Have a competent IT Guy you can trust (He may not be Superman, like Ed Steele, but make sure he is competent and trustworthy. Ed’s contact information is listed below).
2. Subscribe to an off-site backup service – If you have a fire instead of flood, you have a shot at restoring on-line service.
3. Prepare for the worst. We had no expectation or warning – this disaster hit like a sudden, terrible, storm. Our livelihood was at risk; to my discredit, I had spent almost zero time thinking about a flood preparedness plan. Trust me – we will have a documented, laminated, disaster plan, moving forward.
4. Think long and hard about Cloud computing. When you go to the cloud, you increase your risk in one small area, and decrease it in 99 other big areas. The small risk increase? If you lost Internet service at your office (or anywhere in between you and your cloud provider, for that matter), you would have no access to your data until Internet service was restored or you retreated to a location that had Internet service. Your huge risk decreases? Raised floors. Overhead power. Multiple power sources. Multiple Internet connections. Multiple cooling solutions. Ed tells me that he has customers in the data center who have not rebooted for 600 or 700 days at a time. For a flat monthly rate, I have greater peace of mind, server maintenance, security and high availability. I can sleep again, even if Ed can’t.
I know what you are thinking; a Cloud provider such as RT Software Systems, or any other such provider, may “go down” or suffer a similar disaster to mine. It certainly could happen. But that risk is significantly decreased by having your tech center in the electronic version of Fort Knox.
If you want to reach Ed Steele and speak with him concerning his Data Center, or any of the services he provides such as Remote Backup, Application Hosting, Server Maintenance (he has provided many years of support to me and my office, while I had an on-site server–he can help with whatever type of service you have), or other Tech Project Support, please do. He can be reached at 248-229-3880 or via email at firstname.lastname@example.org. He is a typical IT Guy – buried in somebody’s server room or carrying a wet server up two flights of steps. If you get his message center, he will surprise you by calling you back or texting you. He provides “above and beyond” service and that’s what all of us want from our techno-nerds. That why Ed Steele is my choice for “Tech of the Year, 2011”.
As I post this from my space in the clouds, I recommend that you take a hard look at where your backups are, what kind of plan you have for failure of equipment, and whether it’s just simpler and less risky to retain an outfit like RT Software Systems to “put you in the clouds”.
It’s 2011. The cloud is here. The monthly service fee prevents you from spending thousands, up front, on expensive, local hardware. And pipes will keep bursting, no matter what year it is. I learned this the hard way.
You can check out pictures of RT Software’s Data Center at http://www.rtsoftwarehosting.com “
Mark Bello has thirty-three years experience as a trial lawyer and twelve years as an underwriter and situational analyst in the lawsuit funding industry. He is the owner and founder of Lawsuit Financial Corporation which helps provide legal finance cash flow solutions and consulting when necessities of life litigation funding is needed by plaintiffs involved in pending, personal injury litigation. Bello is a Justice Pac member of the American Association for Justice, Sustaining and Justice Pac member of the Michigan Association for Justice, Business Associate of the Florida, Tennessee, and Colorado Associations for Justice, a member of the American Bar Association, the State Bar of Michigan and the Injury Board.