SCinet Behind the Scenes: The NOC Move

Every year, one of the pivotal movements in SCinet is the NOC move. The NOC (pronounced like the English word knock), or Network Operations Center, is both a working space for all the volunteers who help build, run, and maintain SCinet during the conference, as well as a place to house the bulk of the networking equipment that makes up SCinet.

NOC Building and Staging

Depending on the equipment being deployed that year, the NOC racks consist of seven to nine full-height, four-post equipment racks, plus a rack-sized power distribution unit (PDU). An additional equipment rack is needed for each DNOC (Distributed NOC). For SC25, The NOC design featured eight NOC racks and five DNOCs. These racks contain switches, routers, firewalls, intrusion detection systems, servers, patch panels, fiber taps, power systems, DWDM (dense wavelength division multiplexing) equipment, console servers, and all of the other equipment needed to operate, test, and monitor SCinet’s infrastructure.

SCinet planning takes most of a year, but the build process happens in just a few weeks. Four weeks before the SC conference begins, SCinet volunteers start to gather at the convention center and begin the process of unboxing, inventorying, and racking dozens of pieces of equipment on loan from SCinet contributors. Boxes and shipping materials are carefully stored for returning equipment to these contributors. Once the gear is racked, links can start to be built. Optics are installed, and fiber is patched and run, building the network up bit by bit. In just a week, the NOC transforms from empty racks to the beginnings of a functioning enterprise network.

DNOC (left) and NOC (right) racks in the SC25 staging area, just prior to the NOC move.

Because this work starts four weeks before the conference and three weeks before SC staff takes over the convention center, the NOC cannot be built in place on the exhibit room floor. In those weeks before SC, most convention centers are hosting other events, and the exhibit hall usually is not available. This requires the NOC to be built “back of house,” often in unused or non-public areas of the convention center. Having the NOC in private space allows for it to be built and continuously run and maintained until the SC Conference setup starts.

Building the NOC in an unused part of the convention center also means, at some point, the NOC must be moved. During setup week, the week before the actual conference, the NOC is transported from the staging area to the NOC booth. This involves powering down the NOC racks, disconnecting all power and cross-rack network links, and physically moving the racks to the exhibit hall floor, where they are lifted onto the display stage. Once reassembled, the NOC racks will remain there until the end of the show. Moving the NOC is an intense process and defining moment for every SCinet.

Moving the NOC

The NOC move takes the better part of two days. On Day One, everyone is responsible for saving device configurations and prepping for power-down. Servers and similar systems are shut down. By midday, the power switches are thrown, and the racks are powered down. After running for more than three weeks, the staging area becomes strangely quiet.

Once power is shut down, power and grounding connections are disconnected. Power cables are gathered up. Fiber connections to the racks and rack cross-connections are also disconnected. Wheel locks and leveling feet are disengaged.

The first thing to move is the PDU. The PDU is extremely heavy at 2300 lbs. (1040 kg), somewhat top heavy, and must be moved with great care. To reduce risk, it is picked up via forklift and returned to its transport crate. The entire crate is then slowly moved via forklift to the NOC booth, where the PDU is removed from its crate and positioned. For safety reasons, the PDU is placed in controlled space separate from the NOC racks. The PDU is reconnected to its high-voltage feed lines and powered up, while power feeds are run under the NOC stage, ready for the racks to arrive.

Getting the NOC PDU ready for its move.

NOC Rack 1 (including a 360-degree camera on top) is moved from the staging area to the exhibit hall in preparation for SC25.

Early the next morning, the NOC and DNOC racks are moved. Safety is the highest priority here. Before anything is moved, the day starts with a safety briefing to review processes, plans, and responsibilities. Only select team members are on the floor during the rack move, and a secure path is determined before anything is moved. At SC25 in St. Louis, the path between the staging area and NOC was fairly direct and did not require the use of a cargo elevator. The racks could be rolled on their own wheels with relative ease. Such a direct path is not always a viable option, depending on the SC Conference location.

Once the racks are moved to the area in front of the NOC stage, the most sensitive part of the move begins. Each NOC rack is moved directly in front of the stage before being picked up by a forklift and positioned on it. This is done with all the networking equipment and in-rack connections still in place. As they are positioned, the racks are bolted together to provide alignment and stability. After all the NOC racks have been positioned, the forklift makes a tour of the DNOCs, lifting each rack onto its stage.

SCinet NOC racks are lifted onto the stage before being aligned and bolted together for the remainder of the show. Many thanks to the skilled forklift operators from Freeman convention services.

With the racks moved into their final positions, the process of reconnecting everything begins. Rack PDUs are reconnected to power supply cables. Fiber cross-connects are reestablished between the racks, and equipment is powered back on. Once everything is verified as correct and operational, work can resume. If all goes well, this is done by midday on the second day.

With the NOC moved and reconnected, work can continue configuring and building out the services for the show.

Learn More

The NOC move is not your average day for a network engineer. If this type of challenge sounds interesting, SCinet is always looking for interested volunteers, regardless of skill level.

SCinet consists of a large group of volunteers, on more than a dozen teams, who contribute their time to design, build, and run the SCinet infrastructure during the SC Conference. SCinet provides both production connectivity for conference facilities and exhibitors, as well as a research platform for academic research and development. SCinet itself is also a research platform, providing a testbed for leading-edge production network design and deployment. At SC25, the SCinet team consisted of 215 volunteers from more than 80 organizations in eight countries. The SC25 installation linked together 13.73 terabits per second (13,730 gigabits) of external network connectivity into the convention center.

To learn about how to participate in SCinet at SC26, visit: