How our Operations teams get ready for events like Winter Storm Jonas
Winter Storm Jonas, 2016’s first severe weather event in the United States only raged for three days. In that short time however, it dropped record-breaking accumulations of snow on the US east coast, from New York to south of Washington, DC. With two feet of snow on the streets, many brick and mortar businesses stayed closed for the duration. Data centres on the other hand, which many have argued have already replaced the shopping mall and the college campus as essential hubs, don't have that luxury.
We tend to think of data centres in terms of the extreme connectedness they enable, from social networks to online banking. But when a hurricane or severe snow storm hits, a data centre becomes a physically isolated entity. Each local data centre team takes on the task of protecting infrastructure and maintaining uptime under challenging conditions. Advance planning, proven processes, and impeccable execution once a storm is on the way are crucial for data centre emergency preparednessand maintaining service at each location. These factors allowed our facilities to preserve their uptime standards throughout the punishing 2014-2015 winter storm season.
Because Winter Storm Jonas hit so much of the East Coast of the United States at the same time, we checked in with Tom O'Donohoe, Digital Realty's Director of Property Operations, and Joseph Grassi, Director of Technical Operations for the Eastern Seaboard. Tom and Joe are responsible for the day-to-day regional operations including emergency response in a territory that stretches from Toronto, Canada to Miami, Florida. The large scope of the region and its differences in weather patterns make data centre emergency preparedness challenges the "most complex in the country," according to Tom.
How do you monitor weather events for the region?
Like most other aspects of data centre preparedness, weather watching is a sum of many parts, starting with the National Oceanic and Atmospheric Administration (NOAA)'s National Weather Service, which is tasked with providing the latest and best information about global weather patterns. According to NOAA, industries that are sensitive to weather and climate changes account for approximately one-third of the United States’ gross domestic product (GDP). The Digital Realty Global Command Centre, which provides 24/7/365 communication between all the properties in the portfolio and Digital's customers, subscribes to international weather updates. Also important for effective regional readiness are the "wellies on the ground" in local data centre sites. These individuals have not only been aware of the weather changes through updates, but can actually see what’s happening and fully grasp the impact of weather on the local environment and municipal services. Joe points out that just two inches of snow and ice in Atlanta can be more crippling than 30 inches in NY or Boston, where they have "tonnes and tonnes of salt and the right equipment" for responding to a storm.
What’s the most important piece of the plan?
Tom and Joe agree that the best emergency planning in the world is only as good as the people who carry it out. According to Tom, "The personal safety and welfare of our employees, vendors and customers is an integral part of emergency planning and management. It's people who get you through the tough spots during these emergencies, execute the checklists and ultimately drive the plan during storm events such as Winter Storm Jonas. Their personal safety is paramount."
With a proven plan in place, what still worries you?
Mother Nature is a wild card and any severe weather event presents factors beyond reach of the best planning. How long will a storm go on? Will the local government get the streets clean so data centre personnel can be refreshed? If the local utilities are down and the site's relying on backup generators, how long is the power going to be out? "If you suspect the outage will last," Tom says, "you can have an extra fuel truck stationed in the parking lot to prevent the tanks from running dry. The real worry is about those conditions and events that can't be foreseen. Having tested procedures and protocols in place is the best way to keep from losing sleep."
How do you communicate with customers during a weather event?
Internal communication is constant as an event escalates. Communication with customers is continuous, before and during the event, via detailed notifications and simple notes. For example:
15 inches of snow. Facility running smoothly.
With each customer, communication occurs according to a cadence established in advance that matches their preferences-hands on or hands off. How much and how often a customer wants to be updated tends to be a function of how central data is to their daily operations. Financial services companies are especially keen to know what's happening. Facilities like the Global Command Centre with its constant monitoring and updates make it possible for data centre operators to manage operations down to the suite level and provide continuous reports for both their customers and internal stakeholders.
What if network connections are impacted?
According to Joe, maintaining data centre connectivity is best assured by advance planning and good facility design. "Digital Realty can provide the electricity and the resiliency to keep the Meet Me rooms running in an emergency, but ultimately, external carriers providing service in our facilities need to have their own plans in place to move from a failed system to a working one. We do everything we can to support them and work with them, but what happens outside the data centre is their responsibility."
The Eastern Seaboard experiences both severe snow storms and hurricanes. Which would you rather deal with?
In recent years, Digital Realty's East Coast data centres have successfully weathered a variety of weather events, from Winter Storm Jonas to Hurricane Sandy, with no related interruptions to service. Both Tom and Joe agree they’d rather face a bad winter storm than a Category 4 or 5 hurricane any day of the year.
You can read more about how data centres prepare for severe weather in these articles:
Thank you Tom and Joe for your insight into this topic!