The Planet Blog

Do you have any plans for Tuesday, June 16, 2009, at 11:00am?

Well … cancel them.

You’re invited to join us as we celebrate the opening of our new 106,000-square-foot colocation data center facility in Plano, Texas!

How often to do you get the chance to witness the ceremonial shearing of a beautiful ribbon by the blades of comically large scissors? Whatever your answer is to that question, I would suggest that’s not often enough, so you should take advantage of this opportunity.

The grand opening event for this data center — our eighth — will feature remarks from The Planet Chairman & CEO Douglas J. Erwin and Plano City Manager Thomas H. Muehlenbeck. A light lunch and refreshments will be served, and you, as our guests, will have the opportunity to tour the ultra-secure, “green” facility.

On the technical front, the new data center uses the latest modular cooling technology from Turbine Air Systems (TAS), with high-efficiency, water-cooled chillers that eliminate the possibility of an interior water leak. The facility meets SAS 70 Type II standards and is protected by biometric security access.

If you’re keeping track at home, the addition of this raised-floor real estate brings The Planet’s footprint up to 224,000 square feet … more than five acres of enterprise-class server space.

For more event information and to register to attend, visit http://www.theplanet.com/New-Data-Center-Grand-Opening/.

If you wait too long to register, your “light lunch” might end up being a package of Ritz crackers and a jar of peanut butter, so sign up today!

-Kevin

You’ve probably been on pins and needles since you read our last blog post … anxiously anticipating the inside information of how we built a better-than-ideal-efficiency data center.

If you’re familiar with data center design, you’ve probably heard the terms “cold aisle” and “hot aisle.” A cold aisle is an aisle between racks of servers that sends cool air up through the floor (in the case of raised flooring data centers) and into the fronts of the servers. The air cools the server components and is exhausted through the back of the server as warm air, creating a hot aisle behind the server. Data centers are typically set up with racks of servers arranged front-to-front and back-to-back so that for every 2 rows of servers, only one cold aisle is needed. (If you’d like to see this setup in action, check out our popular Data Centric post.)

D6 Phase 3 uses those principles in a different way: hot and cold air are completely isolated.

Let’s take a look at how the phase is built to see what that looks like and why it is much more efficient.

In a raised-floor data center, your air conditioner blows air down under the floor into an air-tight compartment, and you insert vented floor tiles in the areas you want to cool. The large black unit in the image below is one of this phase’s air conditioners.

The metal posts you see on the ground are the braces used to create the grids upon which flooring tiles are installed.

The photo below is the bottom of a flooring tile. You’ll notice that there are no screws or bolts on the tile … it is simply laid on top of a grid of braces to make an air-tight seal. Each of the tiles is partially made out of concrete and is relatively heavy, so when a tile is installed, it’s not going anywhere unless you really want it to.

Below, you can see what the flooring grid looks like without floor tiles installed.

As we continue building the flooring, the data center appears to be taking shape … pretty standard process up to this point. But now, take a look at the air conditioning units below. Notice anything strange (aside from the fact that the covers aren’t installed)?

The return air plenum extends all the way to the ceiling of the room … now we’re getting somewhere.

I only retained a few things from my elementary school physics lessons, but one of them was that warm air rises and cold air sinks. This natural phenomenon is used in data center cooling: the air conditioners send the cold air down under the floor to cool the servers, when the servers send out warm air, the warm air rises to the top of the room, and the air conditioners pull the warm air from the top of the room to process and send back down as cold air. As Jeff mentioned in his interview, the higher the air conditioner pulls the air from, the warmer that air will be, and you don’t want to cool already cold air, so you should try and pull the hottest air in the data center.

Phase 3 goes a step further: it creates an airtight space above the ceiling tiles where all the warm air is exhausted and pulled in by the air conditioners.

So once we’ve got the floor tiles and the ceiling tiles installed, our fresh data center phase (without any server racks) looks like this:

This begs a big question: if the cold air is being sent through the floor to the servers and the warm air is being pulled from the ceiling, how do the servers pull the cold air and push the warm air without the heat being disseminated into the other areas of the data center?

Enter our new custom-made rack-mount cabinets.

The cabinet above is pulled out in one of our other data centers to demonstrate the design of the Phase 3 cabinets. The servers will pull cold air from the floor in front of the server but instead of exhausting the warm air out into an open aisle, the warm air will rise through a vent sealed to the ceiling (the sides, front and back of the cabinet are closed when the covers are installed).

So a completely installed server row looks like this:

One noticeable difference between this data center and the other data centers we’ve shown you in Houston and Dallas are the servers themselves: all rack-mount, no towers.

The rack-mount servers allow for a better power density throughout the data center, so we’ve got to make sure we can provide the power to all of our new servers in the event of a utility power outage.

We’ve got N+1 power redundancy, so for every phase, we have a dedicated backup generator, and for every data center we’ve got an extra backup generator in case any of the other generators fail. New phase = new generator. It’s pretty interesting to see the generator without its skin, right?

Now that power is accounted for, we can install our PDUs for each aisle and prepare to get servers up and running in the new phase.

A little networking and wiring, and D6 Phase 3 is ready for business! In the first picture below, you can see the orange tubes which are primary network drops into the phase … and yes, even those are sealed in the ceiling.

Head over to our The Planet’s Flickr page for a few more pictures of the new data center phase.

-Kevin

I think I’ve officially become a geek. I just got a few of the pictures from the build-out of our newest data center phase, and I’m almost ashamed to admit how excited I was to write this post as an opportunity to showcase them.

The Planet has six data centers between Houston and Dallas, and some have been built out in “phases” or semi-independent sections. This phased construction process allows us to invest our time and money more efficiently because we can create space as we need it, rather than building out an entire data center shell and having to heat/cool/power the excess space that may not actually house servers for months. Following a very welcome ramp up in demand for our dedicated servers and managed hosting, we recognized an impending need for more data center space, so our facilities team fired up their engines and began planning Phase 3 of our D6 data center … and that’s when the fun started.

Jeff Lowenberg, our vice president of facilities, spoke with WHIRtv about the progress we’ve made in improving our data center efficiency over the past year, and the new data center phase is a testament to the work Jeff and his team have done in researching and designing new data center space. With a few of these innovative ideas, we worked directly with our vendors to create custom DC solutions, and the results are beyond ideal … literally: the new phase’s coefficient of efficiency (the total power necessary to operate a data center divided by the power necessary to operate the servers alone) will be approximately 1.5, which is below (better than) the “ideal” ranking of 1.6. With a lower coefficient of efficiency, a greater percentage of our power is running servers rather than heating/cooling/lighting the data center space while maintaining a perfect operating environment for the servers.

Tune in tomorrow for a tour of the new phase’s construction where we’ll “show and tell” HOW this is possible.

-Kevin

P.S. You may have seen that we announced a new data center earlier this week – D7 – to be built out in Plano, Texas, just north of Dallas. We’re preleasing it and expect to have it ready in May, so you can expect to see a bit more info about that space on the blog as that facility takes shape.

Hosting companies often operate their data centers with The Wizard of Oz’s “Pay no attention to the man behind the curtain” mentality, so you never know what the data center operations look like on a day-to-day basis. You might say I’m one of The Planet’s “behind the curtain” employees … not only do I work in one of The Planet’s data centers, I’m also the overnight supervisor. While some of these activities may seem a bit mundane, each is designed to provide our customers with the very best operational environment. The processes and procedures documented here are performed by all shifts in all of our data centers.

Shift Change

Our overnight shift begins daily at 7:00 PM (CDT), with a hand-off of issues from the previous shift which may need special attention. These hand-offs are typically “work already in progress” issues like operating system reloads and a system upgrades.

Once this “shift change” is complete, and the new shift’s operations staff is in place, each member of the team has specific tasks to perform. These tasks include the assessment of scheduled maintenance during the shift, a review of pending customer orders and server upgrades, and response to reboot requests that were submitted during the shift change.

Following these initial checks, we move into normal operational status for a majority of the night. During this period, we monitor and respond to high priority issues (usually in the form of tickets) and undertake any scheduled maintenance work. In addition to this technical work, a staff member will also perform a perimeter patrol every four hours.

Perimeter Patrols

While the name doesn’t sound like much, the perimeter patrol is an integral part of our data center operations designed so our staff can constantly monitor and control the data center’s status and its operational readiness. Each patrol takes around an hour to perform and is a top-to-bottom inspection of our facility and server environment. To better understand the significance of the patrol, take a look at some of the system and facility checks.

- Temperature and Humidity Readings for all Environmental Units

Each phase of our data centers have as many as 18 heating, ventilation and cooling units. These units are operationally independent from the others and use a dual-compressor technology to provide heating, air conditioning, and humidity environmental control. Our data centers are kept at or below 76 degrees Fahrenheit, and if temperatures exceed that threshold on any one unit, redundant units are brought online and a facilities engineer is immediately contacted regardless of the time of day.

- Commercial UPS System Status Displays

Our data center is fully protected by redundant, commercial-grade uninterruptible power systems. In the unlikely event of an interruption to electricity supplied to our facility, these UPS systems provide stop-gap power and automatically signal our generators to start. Within seconds, our generators are running and providing power. In my data center, we’ve got 10 display panels to check for operational readiness on each patrol.

- Generator System Panels and Status Displays

There are two levels of generator monitoring in our data center. The first is a remote set of annunciation panels installed in our monitoring center. These displays provide an instant status of our power generating system. The second and more comprehensive set of screens we check are in our power generation and transfer control room. On those displays, we check important generator operational parameters such as engine coolant temperature, fuel level, and battery voltage. An interesting fact about our generators: the engines are temperature controlled, even when they are not running. If you ever win Jeopardy with that information, I’ll expect a cut of your prize.

- Electrical Power Transfer Switch Status Displays

Each one of our power transfer switches has a detailed display that shows us the status of the power entering our facility, whether backup power is online and available, and other, more detailed parameters. Here, we also check the total amperage in use by the facility for any abnormal variances.

- Life Safety System Status, Piping Pressures, and Fire Suppression Tank Levels

Our data center uses combined methods of fire suppression technology, so we regularly verify their operational status by reading air and water pressures, inspecting the piping for integrity and checking the master status panel in our monitoring center. Our fire suppression system is constantly monitored by our data center staff and an outside monitoring company. In the event of an alarm, emergency units are automatically dispatched to our facility.

- Exterior Doors and Intrusion Detection System Panels

All entry points are physically verified as being closed and locked on a regular basis. We have an intrusion alarm panel in our monitoring center which shows the status of our entire facility which is constantly monitored by data center staff.

- Closed Circuit Video Monitoring System

We employ a comprehensive video monitoring system in our data center with remotely controlled cameras mounted at strategic points both inside and outside of the data center.

- Outdoor Generator Enclosures and Engine Components

On each patrol, we actually open the generator compartments and peer inside at the massive engines which supply our data center with power during utility power outages to check the area around the generator for leaks, debris, and manually verify fuel level indicators. To get a sense of the size of the generator, check out Kevin’s “Data Centric” post.

- Outdoor Air Conditioning Condensers

For each of our heating, ventilation, and cooling systems inside the data center, there is a corresponding condenser outside the facility. We walk by each condenser and make sure the fans are running at optimal performance.

I’d love to sit and chat a little longer, but I’ve got some work to do. Assuming that this post is helpful and informative, I’ll start thinking of a few of the other tasks in the data centers that you might enjoy learning about.

-Eric
H2 Data Center Operations Supervisor (overnights)

As you know, Halloween is known for candy, costumes and sequels. Hence, nothing would be more fitting to follow yesterday’s Spoooooky post than a sequel post with more candy and costumes.

Never to be outdone by the Houstonians, The Planet’s Dallas contingent had their own little hootenanny in celebration of All Hallow’s Eve. In this anthropological investigation of the Dallas festivities, we see several striking similarities.

Apparently, every Halloween event needs two things: A Mario, A Pirate, and A Ninja. You can see Hiram costumed as Mario in this picture, and we already established that you will not be able to catch a ninja on camera, so we just need to have a documented pirate for the party to be a complete success.

Whew! Now that Alicia helped us get those requirements out of the way, we can take a look around at some of the other characters making an appearance.

This is officially one of the easiest “Where’s Waldo?” pictures anywhere. Interviews with Waldo-finders will be featured in success stories on a first-come, first-served basis.

In my humble opinion, Jessica’s vending machine costume was one of the day’s most creative. Unfortunately, the novelty and humor was lost on Thuy, another pirate attendee … Shortly after this picture was taken, the chips and candy were pillaged and plundered. Let that be a lesson to everyone: Don’t take pictures with pirates.

Beehive hairdo? Check. Ridiculous amount of tattoos? Check. Bling? Check. Patrice’s Amy Winehouse got my vote for the best costume in Dallas. She and Adron strike a pose in the picture above.

Do you see Waldo in this picture? He’s a little sneakier… As an interesting side-story, Steve, the dapper “pilot” formerly paid the bills as an airport lounge singer. To give you an idea as to why he was “formerly” an airport lounge singer, we can simply investigate where he got the pilot’s uniform: A commercial pilot walked up to the stage in the middle of Steve’s set and gave him one of his extra uniforms on the condition that Steve never subject a crowd of people to his singing voice again.

Back and by popular demand, I’ve got a new set of pictures for you! About a week ago, I was in Dallas, where I did a little investigatory photo-journalism in our offices and in a couple of our data centers. I could add a disclaimer on the picture quality, but if you’ve endured the last few picture-laden posts, you’re probably desensitized to the blurred focus and poor photo composition.

If you were to drive South on I-35E through Dallas, you’ll see one of The Planet’s offices on your right-hand side just before you got to the home of the Dallas Mavericks and Dallas Stars.

Trendy, contemporary, stylish, post modern? Office buildings like this one make me want to pretend to be an architect (like George Costanza).

Did you think the stylishness was going to be left with just the exterior? You should be embarrassed about being so wrong. On the right side of the hall, you will see a large white paper with hand-written notes. This poster was created at a recent meeting where every manager committed to improving some aspect of our business, and it is displayed prominently to remind our team of the goals we set.

As you can see, we don’t relegate the fluorescent pink ethernet cables to the data centers, but the keen observer will note that the cables didn’t make it too far away from a data center. This conference room (called Asteroids) overlooks a small portion of our D2 data center.

The window in the upper-left corner of this image is the conference room from the previous picture. D2 is one of The Planet’s legacy data centers. As you can see, the central floorspace is occupied predominantly by cages and cabinets with no overhead cabling. How do the cables get run? I thought you’d never ask …

In data center tours of this facility, these plexiglass sections are a big attraction. The cable tray is run under the floor to get the cables to their respective destinations. I mentioned how raised our “raised flooring” is in the “Data Centric” post, but I snapped a perspective picture for you from D6:

Before you cry “Foul!” on this picture, I should explain that the Dr. Pepper bottle in this picture was not opened in the data center … it’s simply one of the most standard heights I had to give a good perspective of the floor’s height.

I am starting to believe that the bright colors used on the cabling in our data centers is meant specifically to keep me from taking a clear picture of a hot row.

I realized after the previous data center post that I didn’t include any pictures of the other side of a server row, so I made sure to include one in this post. Aren’t the blue lights soothing? If you walk down this aisle, you feel like you need to be listening to Blue Train (which happens to be the album playing through my headphones right now).

Here is another beautiful picture of the side of the server that really matters. This Clovertown rack again features the organization I explained previously: every cable is individually numbered and assigned to a specific box to make server identification as quick and easy as possible.

Cut so short? No!!! Well, the mark of a great storyteller is the ability to leave an audience wanting more, so while I have a few more pictures of the backbones of the data centers (power, generators, network, GNOC, etc.), I’ll save those for a rainy day.

- Kevin