Powering up a Private Cloud!

With the proliferation and maturity of Cloud and Virtualization technologies, many industries and the public sector alike are coming to terms with the benefits of migration of applications to virtualized and/or cloud environments. At the same time availability of enhanced enterprise features like HA, scalability, fault-tolerance, etc. on these platforms is further propelling the conviction for adoption. Alongside, one of the most important factors is “cost”! While buzzwords like Public/Private/Hybrid Cloud are no longer unknown, a similar fire is seen in discussions about which model to choose over the others. Private Cloud continues to be adopted more and more as a platform for the more cautious, security-sensitive user, and also because it is a practical choice for those who have invested heavily in their internal infrastructure over the years and don’t want to throw away their existing datacenter hardware.

Private Cloud Computing is among the highest interest areas across all cloud computing according to Gartner, with 75% of respondents in Gartner polls saying they plan to pursue a strategy in this area by 2014. [Forbes]

We at GS Lab strongly believe in these technologies and the value they add to datacenters. In order to optimize infrastructure and to be able to support our customers’ cloud products, we built our internal Private cloud lab for flexible development operations. Multiple options exist such as open-source cloud ecosystems, namely: CloudStack, Eucalyptus, OpenStack and OpenNebula. These are common-place and have the highest brand recognition associated with them. We have run experiments on Eucalyptus and OpenNebula for virtual workload migration. For instance, kPoint (a product incubated by GS Lab) uses OpenNebula private cloud as a test bed for various kPoint releases before going live.

In the last year or so, we built our own private cloud using OpenStack. The choice of the platform was obvious based on popularity & our customers’ adoption of this IaaS cloud computing model. We examined several use cases for building our own private cloud for internal labs such as training, Virtual Desktops for our campus-hires, elastic testbeds for project teams (e.g. usually QA teams require short bursts of resources for testing) and the more challenging scenarios like migration of our IT apps. Within a span of a few months, a team of GS Lab engineers built the infrastructure ground up.

Some of the highlights of our existing OpenStack setup are:

OpenStack version: Essex (upgraded to Folsom recently)

Up time : Nov 12 till date

Deployment Details:

  • 4 high grade hosts running KVM
  • Dedicated controller and storage nodes
  • HA and load balancing for controller and volume nodes (this was planned and nodes were reserved; a lot of research was done for this – Quantum OpenvSwitch based Independent Network deployment)
  • On-demand provisioning – user & infrastructure
  • Support for a range of VM instances: micro/tiny to very large
  • No. of high end instances that can be run: 350 – 400
  • 2 TB storage available for volume creation
  • Customized horizon dashboard for dedicated support page integration (work was already done on this with just integration remaining)

Some interesting use-cases:

GS lab’s internal initiatives, for example Big Data, can use this OpenStack based setup for provisioning Hadoop instances with a view to scale the cluster deployment and satisfy the hunger to crunch growing data, and experiment internally on a private cloud prior to migrating an application to public cloud. A couple of our QA teams provisioned their products to perform various types of testing.

In-house utility:

We internally also developed an image creation utility for OpenStack. We found that not all OS vendors support ready-made template images to be used for OpenStack setup. This small utility was created to solve this problem. The utility was created using existing chef based framework and used lightweight qemu hypervisor internally to generate QCOW2 format OS images. This utility automatically uploads generated images in OpenStack setup through euca clis.

What we look forward to:

  1. With Quantum in place with OpenvSwitch, exploring integration of other switching components likes Cisco Nexus 1000v plugin .
  2. Image migration to/from OpenStack [and CloudStack] to Public Cloud is ongoing at GS Lab for one of our customers. We are also working on providing monitoring and auto-scaling support for OpenStack and CloudStack instances.
  3. Recently we worked on OpenStack’s GUI customization (Horizon), where the customer requirement was to manage their Device’s policy configuration using OpenStack user interface. The UI directly speaks to the Device’s management plane. Going forward, the plan is to have tighter integration with Nova or Quantum components as well.
  4. High Availability & Fault tolerance – these were our top concerns with deployments we did so far. We are looking to support smooth transits across power outages and upgrades as well
  5. 5. With the buzz around CloudStack and one of our key customer being part of the community, we have started experimenting with CloudStack deployments and comparing it with our OpenStack deployment experience. A private cloud with harmonious co-existence of both Cloud platforms within GS Lab is a goal worth targeting.