vRack 3.0: the OVH private network, intensively redesigned to continue anticipating IT project requirements

As a cloud services provider operating its own datacentres and network, OVH has a privileged position when it comes to observing and anticipating usage in terms of traffic.

For several years now, this position at the heart of digital exchange has established one very clear assessment: that if data traffic is historically client-server (‘north-south’ or vertical, i.e. outgoing traffic from datacentres to the rest of the network), then server-to-server traffic (‘east-west’ or horizontal, i.e. a datacentre's internal traffic) increases exponentially. Sometimes, server-to-server traffic is even higher than client-server traffic! The servers now work in clusters, and there is a continuously growing need for them to exchange data between one another. And they need to do this via a private network, rather than a public one (the internet).

This can be done using n-tier infrastructures, for example, which can be distributed between several sites as part of a disaster recovery or business continuity plan (DRP/BCP), or big data clusters that are powered by constantly increasing data production. More recently, with the rise of the IoT (hardware connected to very limited resources) and microservices architecture (facilitated by containerisation), more and more computing needs to be shifted to datacentres, which in turn requires more data exchange between the servers they house.

These infrastructures reflect the three crucial requirements of today’s IT projects: sufficient computing resources to process increasingly large data volumes effectively, guaranteed high availability for applications, and maximum security for the infrastructure.

To stay ahead of the curve with modern IT project requirements, OVH developed the vRack (virtual rack) technology internally, and deployed it in 2009 to connect machines to one another. vRack technology was far ahead of its time when it was launched. It is used to create a private connection between servers within the same datacentre, and servers housed in other OVH datacentres. And there’s one factor that makes vRack technology stand out: it is based on a physical, dedicated network built by OVH, which means it establishes a server-to-server connection without using the public network! Higher bandwidth, lower latency and increased security are the main immediate benefits. It also simplifies the way that users manage their infrastructure, and offers a high degree of flexibility. For example, if an IP block is attributed to a VLAN, the routing is then carried out dynamically within a private network. This spares admins from having to set up a specific IP address map, particularly as part of an infrastructure that spans across multiple datacentres.

With vRack, users can connect their infrastructure’s building blocks to one another. And this is also the case for different resource types: physical and virtual servers (Public Cloud, Private Cloud), and even the company’s internal resources, with vRack Connect (a connection between your company’s network and your private network at OVH). This enables hybrid cloud solutions to be set up on a massive scale.

More than simply being an additional feature, vRack has now become one of the corner stones for a large number of our customers' infrastructures. OVH first used this technology internally to build a lot of our solutions (Private Cloud, Exchange, Public Cloud, etc.), which proves how stable the technology is. vRack has a crucial role in building infrastructures and it is offered at no extra cost for most of the products it can be used with. Like anti-DDoS protection, vRack is included in the overall price of our solutions, with unlimited traffic volume! And as a customer, you will always get the maximum bandwidth stated in the contract for your solution (or the maximum capacity of the private network adapter integrated into your dedicated server).

As an innovative technology, vRack has been continuously developed and improved over the years by OVH teams. As anticipated when this technology was first deployed, company demand for this type of network has increased and diversified. As a result, there have been various pivotal points in vRack’s development. If it were necessary to work with the network technology that existed in 2009 (not considered for this usage type), the technology has developed a lot, and has experienced the virtualisation revolution.

vRack 1.0 was available in only one region (RBX), and offered a single VLAN, based on its eponymous technology. Concretely, it was a virtual private network created within the public network, and it only connected dedicated servers to one another.

vRack 1.5 required a second network adapter to be added to servers. This time, it was a physical private network that ran parallel to the public network, and it also needed to be deployed within OVH datacentres. To do this, the datacentres also needed to be rewired!

When this step was completed, vRack became a multi-zone network. Customers were able to connect their servers to one another between the four main geographical regions that were offered by OVH at the time: RBX (France), SBG (France), BHS (Canada) and GRA (France). This created a value proposition that was brand new for cloud providers.

This was also the point at which the OVH Private Cloud solution became compatible with vRack. By connecting virtual datacentres and dedicated servers, users were able to get more raw power than they would if they used heavy databases, for example.

vRack 2.0 is closely linked to the adoption of QinQ technology. Among other things, it enabled users to create several VLANs within the same Ethernet frame, using tags. With vRack 2.0, each user could also create multiple VLANs – up to 4,000 – in order to increase the level of isolation on their infrastructure. By segmenting their setup into compartments that were isolated on a network level, it became significantly more difficult to penetrate.

At this stage, vRack was used as a core building block of the OVH Private Cloud solution. Virtual machines (VMs) in the OVH Private Cloud solution use this type of technology to exchange data.

vRack 3.0 is much more than a development. Although it just looks like a simple increase in version number, this is in fact a major pivotal point, from a technical point of view. It has been live for two years now, used for the entire dedicated server product universe (for vRack-compatible dedicated servers), and its architecture as well as the technology used have been thoroughly redesigned.

In contrast to previous versions, vRack 3.0 is no longer based on QinQ technology, and is instead based on VxLAN technology. There were several advantages to this change, the first one being the number of addressable VLANs. Based on the 802.1Q protocol, QinQ does not enable users to address more than 4,096 VLANs natively. By keeping this number, the change to VxLAN technology would increase this theoretical limit to more than 16 million VLANs on a single domain!

Additionally, with the transit of frames directly to level 2 in UDP, this technology offers VLAN segmentation outside of a single Ethernet domain, thus allowing the user to work outside of the domain. Within the context of a virtual datacentre, the VxLAN also considerably extends layer-2 networks, offering more flexibility to admins. It is worth noting that vRack 3.0 works harmoniously with private networks implemented natively in OpenStack Neutron. As a consequence of this, vRack 3.0 private networks that connect OVH Public Cloud instances can be managed from the OpenStack native API! This is a significant degree of integration and transparency, which simplifies the way admins manage a complex infrastructure. It is also worthwhile to note that, contrary to standard market practice, OVH Public Cloud customers are not charged for traffic between datacentres.

In this version, the vRack system architecture and its underlying architectures have been completely redesigned. The aim is to offer a more scalable, resilient service that doesn’t experience the “noisy neighbour effect” (overconsumption of resources by other users).

A few pieces of context are needed to give a better understanding of the scope of this design change, as well as the advantages it offers.

From vRack 1.5 onwards, G5 routers were deployed to act as gateways between OVH datacentres for vRack traffic. It involved the use of high-availability hardware (two remote supervision cards that can switch between one another in the event of any issues), equipped with increased computing power to support high volumes of traffic. They became a requirement because of the centralised infrastructure built for early versions of vRack, with each geographical region equipped with its own hardware. However, hardware has physical limits. Although it was perfectly scaled and robust, this architecture was not infinitely scalable.

FIGURE 1: Overall architecture of vRack 2.0


Conversely, vRack 3.0 was based on a segmented structure, composed of several cells that are distributed. There is no longer a central point or single pathway in each geographical region!

The vRack network multiplies the points of exchange within datacentres, also significantly improving bandwidth and latency. For example, two servers situated within the same datacentre will directly establish a vRack connection, without needing to reach another location.

The segmentation provided by this distributed infrastructure prevents users from experiencing the “noisy neighbour” effect. With general traffic distributed across a virtually unlimited number of exchangers, admins can create a multitude of cells that are independent from one another.

The overall infrastructure also increases resilience, by massively limiting the scope of any potential vulnerabilities on the customer side. Users can fine-tune the way they distribute their virtual machines, by placing them in different zones within the same region, or in the same datacentre, without compromising their entire setup in the event of a cell experiencing downtime (it is worth noting that at OVH, all of the critical components that make up vRack are redundant, in different locations).

FIGURE 2: Overall architecture of vRack 3.0

vRack 3.0 is the result of progressive development. While it was initially based on a rather monolithic architecture, it has now become a full mesh network. In order to continue anticipating its users’ requirements, vRack can now be scaled up easily, to increase its processing capacity, and it can easily be scaled out to guarantee its scalability.