Where does a data center go to retire? Wayfair's 2,000 servers are going off to colleges

The road between Union Square and Sullivan Square in Somerville, Massachusetts, just north of Boston, is a somewhat desolate place. 

Union Square has become a bustling hive of high-density construction and hip restaurants while Sullivan Square is an important waystation for people visiting Boston’s brand new Encore Casino in nearby Everett. The road is more reminiscent of the old “Slummerville” moniker that Somerville has tried long and hard to shed, a post-industrial wasteland pocked with independent mechanic shops destined for redevelopment, modernity creeping in from both sides.

It’s the perfect place to put a data center.

In fact, this lonely stretch has been the address of one of the world’s largest home furnishing businesses, or at least its roughly 2,000 servers. A short drive from Wayfair’s downtown Boston headquarters, the facility has played host to the company’s digital operations since 2016, up until last year. That’s when Wayfair completed a two-year migration to Google Cloud.

While we often ponder the future of infrastructure, it’s less common to consider its afterlife.

Traditionally, a company would update and replace its servers over time, as its computing needs evolved and shifted. Now, with many organizations moving much or all of their digital work to the cloud, it’s leading to the wholesale decommissioning of their server fleets, and the sensitive work that comes with that.

So, what happens to those machines once they’ve served their purpose?

Sometimes — after they’re wiped and secured — the servers are sold back to the data center operator to wait for a new client. Or they can be sent to a recycling center and shredded for parts and materials. Less often, they are sold to another organization, restarting the cycle of capital expenditure and cost amortization for someone else.

On rare occasions, they are donated to an institution or nonprofit, where the servers can take on a new life improving the lives of others. It's an act of sustainability as much as charity, keeping valuable technology going, rather than going to the scrap heap.

“As we were disposing of the hardware, we wanted it to be done in an ethical way,” Jen More, a senior engineering manager at Wayfair who helped lead the decommissioning project, said during a recent tour of the data center, standing amongst the no-longer-glowing racks. 

“It was in this process of finding an ethical way to dispose of the servers,” More continued, “that some people within Wayfair reached out and said, ‘Hey, what are you doing with this stuff? Because we have affiliations with these schools and these other programs that could really use the hardware if Wayfair doesn't have a plan for it.’”

In short order, there was the plan.

Giving back to the alma mater

Fred Oettinger, a staff engineer who leads a small group on the Messaging Team at Wayfair, was one of the first to reach out about a possible donation. A 2005 graduate of The Rochester Institute of Technology in northern New York state, Oettinger had been a member of the Computer Science House while at school. The CSH, as it’s known, is something of a cross between a dorm, a computer science lab, and an inventor’s garage located on the third floor of the Nathaniel Rochester Hall. 

When Oettinger heard that the CSH was looking for servers around the same time that Wayfair was preparing to relinquish theirs, he had his eureka moment.

And so, rather than resell or recycle, Wayfair embraced the donation idea of its engineers, along with a desire to encourage the next generation of entrepreneurs and innovators. In addition to roughly 400 servers that went to the CSH, the remaining 1,600 were given to the Massachusetts Open Cloud Alliance, a technology and research consortium run by five of Boston’s top universities.

 “I was thinking we could get five or ten servers for CSH,” Oettinger said in an article announcing the donation, “but then Wayfair’s Data Center Team and Corporate Social Impact Team asked how many hundred they needed.”

According to CSH director Klaus Curde, the group had 15 old Proxmox servers running virtual machines with roughly 24 CPU cores and 72 gigabytes of RAM each. The upgraded servers have 48 CPU cores and 600 gigabytes of RAM each. That’s more than a 20-fold improvement in performance when everything is up and running.

Wayfair’s more than 2,000 server units have found their new purpose: higher education.

Upcycling infrastructure offers new opportunities 

Over the last year, Wayfair worked with Johonnot Technologies, a third-party IT logistics partner to undertake the process of wiping servers, loading them up on pallets, and sending them to their respective academic institutions. 

Rochester and the Massachusetts Open Cloud Alliance already support research into cancer, fusion energy, physics, healthcare, and other groundbreaking areas. With the compute power donated by Wayfair, the potential for scientific breakthroughs by some of the world’s top researchers increases exponentially.

“This is hardware that was going to be recycled, essentially — it's being reused to support a lot of different things,” Michael Daitzman, the head of engineering for the MOC Alliance and also part of the alliance’s leadership, said during the data center visit. “It's a large enough donation that the universities are trying to figure out what is the most effective way to use this.”

The MOC Alliance, as the Massachusetts group is also known, was established in 2013 as a partnership between five of the Boston area’s largest and most prestigious schools: Boston University, Harvard University, Massachusetts Institute of Technology, Northeastern University, and the University of Massachusetts. 

The goal was to build a cloud environment that the institutions could share, allowing researchers, programmers, and scientists access to vital resources to run calculations and experiments, while also testing open-source cloud software. 

“For years, Mass Open Cloud was providing cloud services to various researchers and classes using donated equipment,” Peter Desnoyers, an associate professor at Northeastern and part of the MOC leadership team, said on the tour. “The MOC services validated the demand for cloud offerings, and the donation from Wayfair accelerated the transition to cloud services managed by the Research IT groups at the schools."

Since inception, the MOC Alliance had grown to about 400 machines, most originally donated by companies like Lenovo, Cisco, and the quantitative hedge fund Two Sigma. The alliance runs out of a data center in Holyoke called the Massachusetts Green High Performance Computing Center, a modern 90,000-square-foot building that is a 15-megawatt LEED-certified data center that runs on 100% carbon free power, in part from a nearby hydroelectric plant on the Connecticut River. It is owned and operated by the same five universities.

“This donation is about 1,600 machines,” Daitzman said. “And they are bigger machines and more current. There are some GPUs involved. It’s a huge shift.”

The donation of so much high-end hardware, which Wayfair estimates to be worth a bit over $3.1 million, will turbocharge Mass Open Cloud’s capabilities and accelerate many of its plans to expand services.

“It’s closer to 10 times bigger in terms of capability because the hardware is much newer,” Desnoyers said. “It’s four times as many machines as we have accumulated over the 10 years of the Mass Open Cloud project. And many of the machines are not just newer and faster, but relative to when they were bought, are bigger, with more CPUs, more RAM, and more sockets.”

How to decommission a data center

Few tech companies have the time, people, and expertise to decommission their own data centers. The early days of companies like Google, where all hands — from engineers to marketers — were required to come in and plug in servers on haphazard racks, are long gone. Data centers are now professionally managed, something akin to a highly secure self-storage warehouse.

Migrating from an on-premises data center to the cloud can feel like draining one lake to fill another. Data, applications, containers, workloads, and everything that makes up the digital operation is moved, with great care and meticulous detail, over a period of months or even years. The pools of information are gradually emptied until nothing is left but the silent remains of servers sitting in neat columns inside cages.

With such a large and complex task at hand, Wayfair turned to Johonnot Technologies to take the lead. “If you're doing a decommission of this size, it really helps to have a third party dedicated just for this effort,” More said.

Rochester’s Computer Science House is not a large operation, for example, compared to the scale of a Big Tech implementation. The Wayfair machines weren’t moving into an established data center and CSH did not have the people and expertise to handle the installation and set up of all that new equipment. 

When Matt and Chris Delgaudio, the cofounders of Johonnot Technologies, learned about this, they decided to donate the trucking , with Johonnot delivering the machines themselves following the Somerville decommissioning. The students at RIT then unloaded and installed the machines.

“Data centers are huge, hundreds of thousands of square feet, and there's like four people in them,” Matt Delgaudio said. “We were already going to help them decommission the data center, but when we heard it was a donation, we were just like, oh, this is awesome. This is the ultimate, most pure form of reuse and repurpose.”

Family business meets global business

The Delgaudio brothers named Johonnot after an old, family-owned jewelry store in downtown Boston.  As a full IT asset management and services company they have extensive experience decommissioning data centers, taking on these massive projects several times a year. Johonnot helps manage IT assets, logistics, recovery, and more for companies all across the globe.

Decommissioning a data center requires feats of extreme logistics, inventory management, and of course, security. For Wayfair, the scope of work included thousands of server units, each of which needed to be wiped, unplugged, secured, and then shipped to two different locations over the course of a year. Plus, they housed the entirety of Wayfair’s business operations, including customer data, internal communications, and product inventories; ensuring a complete, total, and secure erasure was essential.

“It’s a daunting task and a task we do enjoy,” Matt Delgaudio said, noting that Johonnot often had more than 10 team members working in the data centers, wiping and moving machines.

Not only were the servers themselves expensive, cumbersome, and as fragile as any other kind of electronics, but the sensitivity of Wayfair's proprietary information meant a complete and total erasure was paramount to protect the business. To make matters even more pressing, Wayfair’s lease of its space in the data center was set to expire in November 2022, putting a hard deadline on the entire operation. 

“Making sure that data is secure, that's our main focus, our number one,” Matt Delgaudio said. “And because they were going to donations, you can't just then remove the drives and shred them there. They need to be reused, and the most efficient way to do that is to erase them with software on site.”

Johonnot wiped every bit of data off every single server, plugging in drives and running a NIST 800-88 software program used for “media sanitization.” This ensures any data stored on the machines is irretrievable. Depending on the server, the program can take up to 35 hours to wipe the machine, going through what’s known as a “clear, purge, and destroy” protocol. It’s a time-consuming and cumbersome process and Johonnot employed teams working 10 to 14 hour shifts for months to get through all of the servers.

Once the wiping process was complete, the machines were unplugged (no simple task, involving miles of networking and power cables) and then palletized. Each piece of equipment had to be logged for tracking purposes and chain of custody. A typical server weighs between 40 and 90 pounds, making the job not just intellectually challenging but also physically demanding as well. 

“It's highly choreographed, there's a lot of moving parts, very complicated,” Matt Delgaudio said. “You have coordination with the data center, coordination with your client, coordination with where they're going. I think that is a pivotal part about this whole thing.”

A complete state change 

The decommissioning of the Somerville data center took more than a year, with servers going to their destinations in waves of deliveries. According to Wayfair’s associate director of public cloud enablement, Jim Richards, the entire migration (including data centers in Washington state and Ireland, as well) involved, “23,000 physical operating instances moved or retired” and “330,000 computer cores moved or retired.” Much of that computing power was part of the donations to the MOC Alliance and Rochester.

“The horsepower they sent out is insane,” Matt Delgaudio said. “This is probably one of the biggest donations I've ever seen.”

By giving such a large amount of computing power to some of the most advanced research universities in the world, the potential benefit of Wayfair’s donation is nearly incalculable.

For instance, given that cloud computing is, by nature, elastic, the MOC Alliance can now shift resources to projects when they are needed. Peak usage often comes when research papers are due, at the end of semesters, so machines can be redeployed to other projects when they need more computing power at other times. If a researcher is running simulations of a molecule or the weather, for instance, computing resources can be diverted to support the project, operating like a high performance computing system.

“Because of the work that we've been doing on building out the elastic infrastructure, once it's in the data center, the ability to constantly reuse the computing power gets to continue because we can move it back and forth between projects that need it,” MOC's Daitzman noted.

Wayfair’s donation, and the ten-fold increase in computing capabilities, will make all of these projects easier to perform and allow the MOC Alliance to expand current and future offerings. Treatments for fighting cancer, or the key to solving nuclear fusion could realistically be run on the hardware now residing at the data center in Holyoke.

“Fundamentally, what the donation does is it completely shifts the kinds of things that we can support,” Daitzman said. “The hardware is new, the hardware is more powerful. It’s industry standard hardware which means that the things that we are building based on OpenShift and OpenStack all are going to work on it. For us, it’s a large enough amount of hardware that it enables a state change in what we are able to do. It’s hugely exciting.” 

For more than 20 years, Wayfair has been furnishing homes around the world, offering more than 15 million products from 11,000 global suppliers across a network of websites — making millions of rooms more welcoming and warm in the process. 

It’s only fitting that the company is now furnishing thousands of servers to a pair organizations that will welcome and energize a new generation of students, academics, scientists, and researchers for years to come.