3 most used VMware ESXServer features

Question: Do you know what are the 3 most used features of VMware ESXServer by a typical IT data center?
Answer: They are vMotion, HA & DRS! Though all of these 3 features are similar in nature, and are likely to have a common code-base, the implementation (customer's use cases) are different. The differences being:

• vMotion is the user initiated live migration of a guest VM across hosts (ESXServers). The guest OS is hot (running) and will be moved to a new host with the most minimal delay. The network & storage connectivity are not lost during this process. The requirements for this to happen are to have a shared storage (NAS, iSCSI or FC) among the two ESXServers where the guest OS resides and access data + common CPU architectures on the hosts. Both hosts need to be on the same virtual center too.
  
• HA involves load balancing of the physical systems to maximize the usage of guest OS instances. If there is higher load, the guest OSes would be distributed uniformly across servers. Conversely, if one or more the ESXServers are under-utilized they would also go to power down mode
• DRS involves kick-starting guest OS instances on another ESXserver in case of server crashes or hardware issues on ESXServer

Both vMotino & DRS requires VirtualCenter, but not HA.

VI4... Just wait... vSphere

The upcoming release of VMware virtual infrastructure product would be called as vSphere, per a VMUG announcement. This would support a maximum of 1TB memory on the host and a max 256GB for guest OS.

By the way what would be the equivalent name of ESXi in this release? Any guesses?

Live VM migration across CPU Platforms with RedHat KVM

Those of who you who are familiar with vMotion know that it doesn't work for cross-platform migration (VM to migrate from an Intel to AMD or vice versa). There is no vendor who can provide this feature, except for RedHat KVM.

The below demonstration shows for the first time live migration of a virtual machine across vendor platforms (Intel to AMD or Vice Versa). Live migration enables the movement of running virtual machines (VMs) from one physical server to another without disrupting service to the end user. This is a great news, where RedHat is leading in the virtualization market.

Note that oVirt/KVM is not yet officially released from RedHat.

Watch the demo on Youtube here.

Hyper-V and Virtual Devices in Windows Guest OS

In the Hyper-V environment, when you do a fresh install of Win2008 guest OS - you will observe that the guest OS automatically gets all the virtual devices giving the 100% portability and the best possible performance. Watch how the device manager shows up on a native Win2008 installation and on a virtual machine installation.

Community benchmarking of VMs - Will it work?

There is a new virtualization community resource, from vKernel, called CompareMyVM. It’s a community of users where virtualization users around the globe can submit and share their VM & virtual hardware sizing information - described as:

CompareMyVM is a community site where you learn how to right size your VMs. Compare your virtual machine resource allocations with that of the community at large. You can compare by application category to see how your peers are allocating resources to gain the most efficiency out of their environments. You can also contribute your VM sizings to the community. Your submissions are anonymous.

Things you can do with CompareMyVM…
* Browse Community Submissions
* Vote Submissions Up or Down
* Edit Submissions
* Submit Your Own VMs

This seems to be addressing the need of a benchmark source for information, highlighting that there is a clear lack of this information analysis to be provided by any other tool.

Who would provide opportunities to improve based on the data submitted?

Virtual Hard Disk in Windows 7 - Game Changing?

Virtual Hard Disk allows you to create, mount, and unmount OS images and Microsoft did a wonderful job by providing a native support in Win7. This allows users to use virtualization on Win7 without any additional software, a turf that VMware cannot play! The guest OS residing in VHD can be booted directly without the host OS. Microsoft have effectively created a "loopback HBA", and the bootloader can use to address VHD's as any regular disk. I used this on my PC and admit that this feature works rock solid on Win7, though still in beta phase.


This could be a game changing technology, since one doesn't need additional software (like VirtulBox or VMware Workstation) to use Virtualization for booting multiple OS. This gives new abilities to test even a native device driver in the guest OS as it works just as the host OS. Realize that Win2008 Server also supports this feature, but the kernel implementations of Win2008 and Win7 are totally different!

In order to use this feature, just know to use the specifics of Win7 VHD commands diskpart, bcedit and you are good to go.

Cloud Storage - QoS & Adoption

I was at the Cloud Computing track of SNIA winter symposium this week, and got a good pulse of the industry happenings. The cloud enablers are gaining momentum, and are geared towards driving the adoption. Similar to what we saw with the adoption of low-cost SATA drives during the 2001 downturn, this year is likely to see a greater adoption of cloud computing to save on capital expenditure. All that – assuming the focus is on QoS!

‘Cloud storage’ data is mostly (~96%) is either the backup/archive or files and only the rest (4%) is of transactional data. The minority transactional data (coming from live applications) is because of the current barriers in ‘cloudifying’ applications as well as cloud storage. Talking purely about cloud storage, the entry barriers are more of the security, SLA compliance, vertical industry needs & transfer times in/out of the cloud.

There is also an unpleasant reality that the ‘unstructured’ data is growing @10% YOY and unless there is a strong audit-ability and quotas on the cloud – your monthly cloud invoices are only going to go up!

Virtualization: If not planned right!

Hadoop & Use Cases

I spent a little time in digging out various use cases for Hadoop. Here are some of the use cases, listed by business use:

• Search: Yahoo, Zvents, Amazon A9,
  
• Log processing: Facebook, ContextWeb, Yahoo, Joost
• Recommendation Systems: Facebook, Amazon (eg, ad placements or site suggestions)
  
• Data Warehouse: Facebook, AOL
• Video and Image Analysis: Eyealike, New York Times

Note that even the search use cases of Yahoo or A9 do not use Hadoop when a user clicks "Search" button, but merely parses the web data and generates the index files on the back-end. Hadoop takes care of the node failures during the processing and provides massive scalability & load balancing.

Performance benchmark comparisions of hypervisors

One of the very few publicly available performance comparision results of Hypervisors (ESXServer, XenServer, Hyper-V, Virtual Iron & SLESXen). Look at the key statistics of bops (Basic Operations Per Second) and Disk I/O Results.

Read Network World for test details & results.

Distributed File Systems - for Virtualization & Clouds

In order to distribute the workload and processing uniformly across the virtualized systems, one would need a different file system - like Hadoop's HDFS or PNFS (Parallel NFS).

Hadoop is a Apache java project that supports data intensive distributed applications. Realize that this only works for offline processing - like logs, search, business intelligence or ad placements. It enables applications to work with thousands of nodes and petabytes of data. The current tested implementations work across 4000 nodes - plan is to make it work for about 15,000 nodes.

Most notable users of Hadoop are Facebook, Yahoo, Joost, Google & Veoh. I will try to list various uses for Hadoop some other day!

Cloud: 'vendor lock-in' Vs 'developer lock-in'

"When times are tough, money is tight. But is the Cloud Computing era of pay-as-you-go there yet?"

Unless you are really careful in cloud adopting life cycle - you are locked-up either with the cloud vendor (eg Amazon AWS) or with your development platform (eg Google Apps). The data portability interfaces are the key factors, not only for the success of Platform as a Service (PaaS, like Google App Engine) or Infrastructure as a Service (IaaS, like Amazon EC2 or S3) but also for the SaaS players. It is imperative that you choose the right design so that you reap the cost benefits and also keep your users happy on a longer-term.

While choosing the cloud models, some of the key factors one needs to look at are:

• Computing model (HTTP requests only, processes, OS etc)
  
• Service creation platform (Python, Ruby, Perl or any)
• File storage model (local, ZFS, block storage, tempFS)
  
• Database storage model (BigTable, SQL, simpleDB)
  
• Network topology (hidden, private, shared networks)
  
• Network protocols (http, tcp, udp)
  
• Network connectivity (hidden, NAT, VPN, L4, L7)
  
• DNS (static, dynamic, fixed)
  

Virtualization: Don't implement it this way!

Data Tsunami: Why do we need a 'cloud' for storage?

Ever thought why do you need a 'cloud' for storage and not manage the data on the internal servers? It is a combination result of people not being able to manage their data (like someone filling up the garage with crap and parking the car outside, even during a snow fall!).

The storage data growth is 10-times every 5 years and an example halo effect is explained here: one employee sending a 1.1MB email to 4 colleagues takes up 51.5MB total storage.

Source: IDC, "The Diverse and Exploding Digital Universe," March 2008

What is about Cloud Computing?

Use cases for cloud computing?

Several people keep asking me what are the use cases for cloud computing. Here are those in my opinion (as applicable to enterprises - large, medium & small).

• Test & Development, R&D, for Engineering organizations (eg Soasta)
  
• Storage Functional Offload (eg TimesMachine)
  
• Batch Process Functional Offload (eg Smugmug)
  
• Workload Augmentation (eg Animoto)
  
• Web Services

Depending on the specific nature of the applications, choose your specific cloud implementations (load balancing, scalability, file systems, network,...) etc