VMware Does Complicated Dance With Open Source

Nov 09, 2012 (05:11 AM EST)

Read the Original Article at

When it comes to thinking about the future data center, it's well understood that servers and storage will function as pools of virtualized resources that can automatically switch from task to task.

Networking is a much harder nut to crack. It remains unvirtualized and has lagged far behind the other two. But without it, we'll never get to the flexible, automated data center of the future envisioned as a "private cloud."

That's why I sat down to talk to Martin Casado, the former Stanford grad student whose PhD thesis turned into the OpenFlow networking protocol. Casado co-founded virtual networking firm Nicira, which soon became the lead contributor to OpenStack's virtual networking project, Quantum. VMware bought Nicira for $1.26 billion in July -- not bad for a young company with 100 employees.

I'd asked Casado to explain how Nicira was going to continue to lead development of virtual networking in OpenStack, since open source OpenStack competes directly with VMware to manage the virtualized part of the data center that's implementing automated self provisioning, elastic expansion and chargeback -- the private cloud part.

[ Want to learn more about why VMware acquired Nicira? See Nicira Acquisition Is VMware's Smartest Move Yet. ]

VMware now has a lot of influence in OpenStack. The Quantum project's technical lead and elected chairman of the development team is Dan Wendlandt, who had been a Nicira designer and software team leader and now is a senior product line manager at VMware. Casado is now chief architect for networking at VMware.

VMware, of course, is the most likely candidate to become the kingpin of proprietary private cloud through its vCloud Director and vCloud Suite. And that makes it OpenStack's chief competitor in establishing private clouds inside the enterprise. After listening to Casado, a virtual networking enthusiast, I felt I understood how these dissimilar pieces -- proprietary product line and open source code contributions -- fit together. But be your own judge.

First, Casado says Quantum is not a particular set of networking features or a new kind of switch/router/controller hardware combination. It is "a framework of open interfaces you use to build up virtual networking for a software-defined data center or a cloud."

Without virtual networking, the virtual machine's connection to a network is buried as a software switch in a hypervisor. That software switch can be made more efficient by offloading its work to a nearby hardware fabric, as HP and Cisco do. But it would be easier if the compute server, storage and networking could all be virtualized upfront as a pool of resources, with shares of capacity "snapped together" when a virtual machine is created. "Quantum provides the virtual network platform and OpenStack provides the harness where all three fit in," said Casado.

The network virtualization platform embeds route-building and capacity-assigning capabilities into a network controller, or distributed controllers, which manage on a dynamic basis the network switches and routers. (Nicira offers commercial software with the name Network Virtualization Platform). A network management console allows instructions to flow from network administrators, or virtual machine administrators, to the controllers. If the size of a virtual server needs to be increased to match its growing traffic load, the network can be increased at the same time.

Both Nicira products and the Quantum framework follow the principles incorporated into the OpenFlow standard, which was developed as a cooperative effort between advanced networking groups at Stanford and Berkeley. (How often does that happen?)

In many settings, the OpenFlow protocol will gradually displace the frequently used spanning tree protocol, which applies a hardwired answer to the question of what route and on what type of network a particular message should travel. OpenFlow requires the network hardware to look more like a blank check than a payroll check. The network controller under OpenFlow will periodically write in the intended recipient and other delineators.

With such a method, IT teams may take many different approaches to their existing network hardware. A super-secure network can be designated and separated from all links to the Internet. High-capacity or low-capacity, high-latency or low-latency networks can be constructed, on top of the existing hardware resources, Casado explained.

Even in a jumble of mixed hardware, each network has its own address space, security configurations, statistics gathering and capacity management, managed by an automated controller, not human hands.

Lew Tucker, CTO of cloud computing at Cisco, has eloquently described to me the possibility that a virtualized network could recognize the nature of the application that's been brought to it, and respond with appropriate services.

Casado agreed that scenario is likely, but he said such an approach would be an engineered subsystem brought to the Quantum framework, not the framework itself. What the Quantum part of OpenStack is attempting to accomplish is putting a programmable, vendor-neutral interface between a human network manager and the network.

"The operational interface to the network has always been a proprietary one for the last 20 years," said Casado. Quantum in OpenStack will provide "a vendor-neutral one. You can slot in whatever network services you want" and even dissimilar services can function alongside each other on the same hardware because they're run through programmatic instructions or rules put into the governing code.

Whoever is managing this programmatic interface will be in a strong position to manage the whole private cloud. If the neutral perspective remains absent, the private cloud will never quite function in the way it was conceived. OpenStack and VMware are both competing to provide the virtualization management platform for the private cloud.

But since VMware joined OpenStack, the benevolent view is that both have reason to support Quantum's development. Without Quantum, OpenStack doesn't offer a complete private cloud because it can't handle virtual networks. In a similar vein, without Nicira, VMware can't build out the software-defined data center and its version of private cloud. The idea of private cloud founders on the customer's lack of virtualized networking. If VMware products don't always serve as the basis for private clouds, it's placing a bet on OpenStack as the runner-up.

Asked about VMware's commitment to OpenStack, Casado said: "It is important that we support our customers deploying on OpenStack and other open source technologies ... We plan to continue the contributions made by Nicira and Cloud Foundry ... add support for customers who choose to run (VMware) vSphere in OpenStack environments, and help shape the future of OpenStack and the value delivered to our customers by being an active participant [in OpenStack]."

Some question how benevolent VMware can afford to be. Boris Renski, co-founder of OpenStack consulting group Mirantis, voted against VMware joining the OpenStack project as a gold member, for which it pays $200,000 a year.

Renski says he has no problem with VMware employees contributing code to Quantum, as Dan Wendlandt does. The problem lies in VMware's "gold" status as a sponsor of the organization and its potential board membership. In an email to me, Renski elaborated:

"Through the very nature of business that VMware is engaged in they are simply unable to 'promote OpenStack software' as that would imply cannibalizing their existing business. For instance, they would never unlock ESX hypervisor to allow for some of the same features that are present in KVM (such as live migration) without forcing people to pay for vSphere. Without live migration capabilities, ESX is not relevant for an OpenStack environment. VMware can't promote OpenStack and compete with it at the same time. If you can't promote it, you can't help the foundation on its mission and you can't be a gold sponsor," Renski wrote.

However, some fear that denying VMware gold sponsor status would prompt VMware to pull Nicira expertise from the Quantum project. Renski thinks that risk isn't great, citing Mirantis' investment in three full-time developers, along with investments by HP, IBM, Red Hat and others.

"This is the beauty of open source," he said in his email. "Nobody, including VMware, can derail OpenStack development velocity."

Jonathan Bryce, executive director, noted that the earliest VMware could take a seat on the board is 2014, so there's plenty of time to watch and see if VMware unfairly pushes its own agenda. "Bad behavior tends to get called out pretty quickly in open source projects," he said.

Bryce also emphasized that not only does VMware have virtual networking expertise but there is also value in getting ESX Server to work with OpenStack, through VMware contributions.

Frank Rego, business development manager at Novell's SUSE Linux unit, offered an additional perspective at Cloud Expo in Santa Clara, Calif., Tuesday. He drew an analogy to Microsoft's opposition to Linux, which suddenly turned around and became Microsoft cooperation with SUSE and its current support for running SUSE Linux virtual machines in the Azure cloud.

A dominant vendor, like Microsoft, "wakes up one morning and realizes the whole world isn't going to be Windows. Then it starts cooperating" with the thing it previously opposed. VMware has had such an awakening, he suggested.

I agree. I think it's also true that VMware, while a strong proprietary company, has never expressed vehement opposition to open source code in the manner that Microsoft once did. It has found ways to cooperate with it in the past. Its Cloud Foundry is all open source.

But the main point, I think, is that virtual networking is the key to the expansion of virtualization in general, and VMware's vision for a software-defined data center in particular. VMware has as much interest as anyone in removing obstacles from the virtual net's path, and that's why it joined OpenStack.