In 2011, we founded the Open Compute Project (OCP), a rapidly growing community of engineers and companies around the world whose mission is to foster more openness, more innovation, and a greater focus on scale in the development of computing technologies. Then, in 2013, we launched OCP's networking project, and one of our goals was to bring OCP's mission to the networking industry by disaggregating the hardware and software of networking devices. This allowed us to grow the networking hardware and software ecosystem and enable engineers to build new systems that are more flexible, scalable, and efficient. Over the past few years, we've contributed both networking hardware and software to the networking project: Wedge and Wedge 100, our top-of-rack network switches; and FBOSS and OpenBMC, the software libraries that we use on those switches.
Now, looking back at the first half of 2016, we've been busy. We've already deployed thousands of Wedges into production, and Wedge 100 will go through a similarly rapid deployment as well. Given that, the developer community knows that Facebook network switches have been tested and hardened in our production data centers — thus, starting late last year and into 2016, we've seen several groups and companies start to experiment and develop on top of Wedge. The openness of the hardware and available software building blocks are pushing the networking developer community forward.
Late last year, Big Switch Networks helped start this movement by providing a version of OpenNetworkLinux (ONL) packaged with Broadcom's OpenNSL and Facebook's FBOSS that can be downloaded and loaded onto a Wedge. This allows other developers to start writing software on top of a Wedge and gain experience programming a networking ASIC — all while providing an example of working production code in FBOSS. This ONL-based platform was used by several other companies, outlined below, in their own demos and proof-of-concepts.
At the OCP Summit this March, a number of companies demoed the software that they built on top of Wedge:
- Marcel Neuhausler and Julius Mueller from AT&T Foundry in their talk, "Applications, Microservices, VNF controlled by Top-of-Rack Controller (TORC),” showed how they took advantage of the full microserver in Wedge to allow it to provide higher-layer networking services for the rack. They dubbed this the "Top-of-Rack Controller (TORC)," and it provided resource-efficient and on-demand management of servers and VNFs at the edge of the network by running them on Wedge. At Facebook, we were pleasantly surprised the first time the AT&T team told us that they were running Docker, Mesos, and other open source packages on Wedge. All these packages provide the orchestration, measurement, and state management that the services in the rack need. Marcel and Julius are now starting to investigate whether other services, like PXE/DHCP, encryption, or auditing, are also candidates for the top-of-rack switch.
- Junho Suh from SK Telecom in his talk, "Transforming Networks to All-IT Network with OCP and Open Networking,” showed how they worked with multiple networking stacks on Wedge. They started with the aforementioned ONL+FBOSS combination, but then ported the open-source OpenFlow agent, Indigo, onto Wedge so they could use OpenFlow-based control mechanisms with the Wedge. They even investigated how to make OpenSwitch available on Wedge. Supporting this variety of networking stacks and paradigms is exactly what we had hoped for when we started sharing these technologies.
- Dave Duffy and Mark Shuttleworth from Canonical in their talk, "How app-enabled switches increase innovation for telcos and DC operators," demoed how Wedge could be managed using their open-source server management tools. MAAS was used to deploy the initial software onto the Wedge. After that, Snappy Ubuntu Core was used as the platform for running different networking operating systems (NOS's) as "snaps." For example, FBOSS was loaded as a "snap" onto the Wedge. This demo shows that not only can you treat a switch just like a server for control services (see AT&T's talk above), but you can also treat it like a server from a management perspective. Canonical also demoed this at the recent OCP engineering workshop in Austin.
At Facebook, we've also been demo'ing Wedge as a development platform at a couple of conferences, specifically SCaLE14x and the Embedded Linux Conference. We had a stack of Wedges running all open source software and showing some forwarding work based on open-source FBOSS and open source software.
Wrapping up the first half, at the recent OCP engineering workshop in San Jose, we were excited to share that a new networking startup, SnapRoute, announced and demoed their support of FlexSwitch software on top of both Wedge and Wedge 100 (as well as a Mellanox-based switch, another Broadcom-based switch, and the Barefoot simulator). FlexSwitch is all open-source software, and SnapRoute is contributing FlexSwitch to OCP. This contribution continues the move by the OCP networking project up the stack to higher layer networking software, and FlexSwitch is the first full L2/L3 open source networking stack officially available on top of Facebook Wedge (and Wedge 100, once it's generally available).
FlexSwitch is a package that can be installed on a Wedge running OpenNetworkLinux — see the uname output below:
snaproute@Wedge100:/$ uname -a Linux Wedge100 3.18.25-OpenNetworkLinux #1 SMP Fri May 13 19:10:40 snaproute@Wedge100:/$ snaproute@Wedge100:/$ dpkg -l | grep flexswitch ii flexswitch 1.0.0.104 Stacks> snaproute@Wedge100:/$
Overall, we see a huge opportunity for developers in this open and disaggregated networking ecosystem. And specifically, we can see it growing on Wedge and Wedge 100 — from large ISPs developing their own software, to large open-source projects, such as those from Canonical, to new startups like SnapRoute.
The future of networking is open and disaggregated, and we encourage everyone to try it out on Facebook Wedge and other forthcoming Facebook networking switches.
