The Virtualization Puzzle – If Servers Were Cars

These first few weeks at GS have given me some food for thought. While I have been bombarded with TLAs and FLAs, (and some SLAs as well), most of which are new to me (I have just crawled out from under the kernel / device driver / firmware rock, you see), the constant running theme in all this crossfire has been cloud, of any and all kinds, OpenSource stacks, and at the root of all of these, hypervisors and virtualization.

I came across this blog Docker and Containers – my IDF14 favorite and one particular point got me thinking. I reproduce the particular statement from the blog verbatim here – “Twenty years ago, the problem of exploiting more performance than one app could use was to put more than one app on the same system.” The blog further goes on to point out that virtualization provided a convenient and relatively secure way of packaging these apps in their own OS container – a VM, thus creating the foundations of today’s cloud.

Rewind to the statement, I pointed out above – Twenty years ago, the problem of exploiting more performance than one app could use was to put more than one app on the same system. Let’s start with a loose analogy of a computer system with an automobile, particularly a car. The processor in the computer system can be likened to the engine in the car, the car’s passengers to the apps that run on the system. It’s immediately obvious that a car carrying only one passenger is inefficient, especially if its capacity is 5 passengers. Similarly, a computer system with a processor capable of supporting multiple apps is inherently inefficient when only one app runs on it.

In the transportation world, carpooling evolved as a technique to better utilize the resources available in a car, just as in the IT world, this was achieved by running multiple apps on the same system. As we all know, carpooling has numerous limitations, chief among them being the unique time and location constraints of each passenger. The corresponding limitation in the IT world is the library and OS dependency of apps the author talks about in the blog to which I referred. But the manner in which these limitations have been overcome is fundamentally different in the automobile world and the IT world, and I will further argue that the automobile industry has historically evolved in a more efficient manner than the IT industry.

In the automobile world, long before carpooling even existed, we had a thriving community of different automobile makers, each designing and building their own engine. Even if we limit the discussion to cars, manufacturers historically have each designed and produced a range of cars, with differing engines, for their customer base. Each car manufacturer has usually 2 or even 3 base engine models with variants for different market segments. This availability of choice in the automobile sector has allowed the market to evolve in a more sensible manner – witness the rise of 2 wheelers in India vis-a-vis the absolute domination of cars in the US and parts of Europe.
If the IT industry had evolved in a similar manner, one would have expected to see a range of general purpose processors, available from different vendors, catering to different market segments and needs. It is possible that market dynamics would have allowed for a processor manufacturer to solve the “exploiting more performance than one app could use” problem, by designing a low cost, minimal featured processor, that was optimized for running a single app or a suite of closely related apps.

Instead, what we got was virtualization, first of the server – its CPU, its NIC, and now of the network – its switches and routers.

The carpooling analogy is imperfect in that carpooling is a relatively late phenomenon and appeared after the automobile industry had already matured and stabilized with numerous key players. The underlying point, I believe, remains valid.

Now virtualization has its clear benefits when analyzed from the standpoint of overcoming the limitations of poor semantics of tightly coupled data – I’m talking for example the longstanding association of the MAC address of a NIC with a server’s identity. But was it really necessary to virtualize the CPU as a way of using its underutilized capacity? Or would it have been better for a processor vendor to design and deliver a processor that addressed this need?

Your thoughts?