PaaS with choice: Motivation for VMware to acquire SpringSource

VMware announced to buy SpringSource yesterday. SpringSource had been a technology innovator focused on the application-centric areas rather than on the hardware-infrastructure focus. Their employees were thought leaders in Apache Tomcat, Apache http server, hyperic, Groovy and Grails. SpringSource’s obsession has been simplifying and automating the build-run-manage lifecycle that all applications go through, and they have done so by attacking similar pockets of complexity. They bring this complexity-busting focus to several areas… high-productivity developer tools and frameworks, lightweight application server runtimes, and application management and monitoring. The end goal is very similar; attack the time and money spent on application complexity and maintenance tasks, shifting the focus to new and more reliably deployed applications.

VMware feels SpringSource enables an evolutionary path for application developers to reach end goals without requiring complete infrastructure or application rebuilds.

Ultimately for VMware the end goal is on vCloud which is aimed at IT applications, not infrastructure - there is an extensive application flavor for the cloud to provide these key features. SpringSource would definitely enable VMware in preaching the right solutions to VMware's customers:

1. Elasticity: automatically scaling up and down the infrastructure to meet the needs of the application
2. Multi-tenancy: being able to isolate resources and applications from one another in a shared infrastructure
3. Simplified provisioning: Isolate the developer from worrying about how is code gets installed and deployed
4. Self-service: allowing developers to gain access to their development infrastructure at any time, in many cases to circumvent the processes and inefficiencies of their typical IT service request processes.
5. Rapid development: go from code to cloud in a matter of minutes, particularly during the development and test phases
6. Simplified (or invisible) management: PaaS offerings typically have built-in application availability and performance management

Topology for Inter-datacenter VMotion

An often asked question in any of the vCloud discussions is about the probability of vMotion across data centers (over wide area network) - it is not an easy thing considering the latencies. The complications are both in terms of stretching the L2 domain between the sites and the shared storage issue across sites.
What are the business requirements for inter-datacenter vMotion?

1. Load balance/Go-green to save power: “follow the sun” or simply consolidate VMs to conserve power.
   
2. Uptimes during maintenance windows: Have a fallback plan for your applications to be available during datacenter shutdowns
   
3. Disaster Recovery/avoidance: This is an obvious reason, to fallback during DR situations

Compared to #2 and #3, #1 could be higher priority as that also gives a provision to use someone else's (like AWS) data center for peak loads (like month end report generations) instead of building up in-house data centers.

Overall, this initiative would lead to consolidation in data centers slowly and bigger players would benefit. Below is the topology of vMotion that would be demoed at VMworld, jointly by Cisco & VMware.

Cloud Operating System & Google

Where do you think Google is headed with Chrome & Android? I see both as a direct connection to a "Cloud Operating System".

Chrome faciliated faster web browsing, meaning more searches which translate into more ad revenue for Google. Then, the obvious motivation of developing an OS came up to increase the volume of Web activity itself. Google wants everyone to keep searching & searching fast.

The Chrome operating system is said to be released next year, but will it ever lure the mom and pop to move to this new browser? All said & done, Intel would definitely doesn't want this to happen - otherwise, with little of client processing - what is the use of the CPU horse-power?

To completely ignore operating systems in favor of the cloud operating systems might be an efficient route to failure. We keep re-inventing the wheel once every few years, and create technology confusion as well.

Network Port List for VMware products

Here is a list of network ports that are used by VMware products - this is an informal, unpublished list. The details here are all gathered from various product manuals and VMTN and could be updated at VMware's discretion.

VMware Studio 2.0 Beta launch

VMware Studio 2.0 Beta was made publicly available just a few days ago. Note that the name of this product is not yet vStudio!

New features of Studio 2.0 are:

• Build vApps and virtual appliances (with in-guest OS and application components) compatible with VMware Infrastructure, VMware vSphere 4.0 and the cloud
• Support for OVF 1.0 and 0.9
• Available as an Eclipse plugin in addition to the standalone version
• Ability to accept existing, Studio-created VM builds as input
• Support for 32 bit and 64 bit versions of Windows 2003 and 2008 Server, SLES 10.2, RHEL 5.2 and 5.3, CentOS 5.2 and 5.3, Ubuntu 8.0.4.1 and additional operating systems
• Publish patches to update deployed virtual appliances
• Extensible in-guest management framework
• Automatic Dependency resolution
• VMware ESX Server, VMware ESXi, VMware Server 2.0, Server 1.0.4, 1.0.5, 1.0.6, VMware Workstation enabled as provisioning engines
• Infrastructure enhancements in the GUI and builds

Strategies for Virtualization adoption in your company

Me being a virtualization consultant, I keep getting the asked the question of "what should be my strategy for virtualization adoption?". It is obvious for the top management to adopt Virtualization for obvious cost reasons, and these are my thoughts:

Take the first steps of adopting virtualization for all new projects and converting low-risk physical servers (like FTP servers) into VMs.

And then the more important strategies would be:

• Process changes & convincing mid-management to think of "VMs" instead of physical servers while requesting for new server platforms
  
• Build IT SLAs around VMs instead of physical servers
  
• Internal education of virtualization & VMs for application teams - incentivate them by passing on the cost savings
  
• Process setup to migrate physical servers into VMs
• Any other financial aspects involved while converting physical assets into shared virtual assets
• Setting up cost sharing model to share server costs across business units, based on VM usage
  

It is easy for you to achieve about 30-40% of virtualization adoption with minimal effort. After that - it would need someone to sit down and do the above implementations to increase the adoption further.

vSphere to launch on April 21st

VMware to spill the vSphere beans on April 21
Mark your calendars. VMware will launch the next generation of its virtualization platform on April 21, at an event at its Palo Alto headquarters, flanked by partners Cisco and Intel.

...

The hypervisor itself will be 64-bit, will provide support for up to 256 GB of RAM per guest and eight-way virtual symmetric multiprocessing, or SMP. On the management side, it will be possible to cluster the vCenter servers, and users will be able to create and provision virtual machines using new host profiles and guest templates.

...

In addition to core ESX and vCenter products, VMware is also expected to launch several new ancillary products, including the long-awaited VMware Fault Tolerance, and AppSpeed for performance management of applications running within a virtual machine.

Read the details here.

VMware to Manage Virtual Machines from Mobile Phones

Introducing VMware vCenter Mobile Access (vCMA). vCMA allows you to monitor and manage VMware Infrastructure from your mobile phone with an interface that is optimized for such devices. Specifically, it allows you to:

• Search for virtual machines in your data center
• Migrate virtual machines from one host to another using vMotion
• Execute recovery plans using VMware Site Recovery Manager
• Access Scheduled Tasks, Alarms and Events
• And much more...

Do you think if it is just a cool feature or is this really useful for a datacenter class management?

Happy Square Root Day

Today is a "square root day", hope you are enjoying it. The next such day would only be on 4/4/16.

Vendor lock-in, severity across cloud architectures

Several people have asked me in the recent past about the vendor-lock-in issue with various deployment architectures. The below picture should be self-explanatory on the amount of lock-in across SaaS, PaaS and IaaS. The lock-in that I am referring-to here is a combined aggregate of Vendor lock-in + data lock-in + dev environment lock-in + data lock-in.

Java platform as a service for cloud

I used to think about the alternatives for Google platform for the java world and came across CloudFoundry today. They provide the java stack, similar to SpikeSource' LAMP stack, on the cloud. Though the stack is not very flexible for various types of Java apps (4 layer models, server types, EJBs etc) - it works for the common deployments. You just make the .war file and upload it - watch it to get deployed in a couple of minutes.

Microsoft & RedHat sign deal on Virtualization interoperability

Red Hat and Microsoft have signed reciprocal agreements today to enable increased interoperability for the companies' virtualization platforms. Each company will join the other's virtualization validation/certification program and will provide technical support for their mutual server virtualization customers.

Server virtualization is moving towards the commodity model and this is a natural transformation. Crucially, Red Hat's interoperability deal with Microsoft does not include any patent covenants.

Gartner Report - Virtualization to grow 43% in 2009

Cost reduction, resource utilization and management advantages drive market growth - says Gartner Report.

... Global virtualisation penetration is on pace to reach 20 per cent in 2009 from 12 per cent in 2008...

...Virtualisation helps organisations to cut costs, better utilise assets and reduce implementation and management time and complexity, all of which are crucial in this economic environment...

Gartner recommends that vendors take advantage during this disruptive period by introducing leading-edge management tools in support of virtualisation initiatives and ensure that virtualisation-specific management products can integrate within existing management frameworks.

Cloud Computing - Top 10 Obstacles & Opportunities

Quick Preview of Top 10 Obstacles to and Opportunities for Growth of Cloud Computing (from UC Berkeley's Research Publication, Feb 10 2009).

# Obstacle Opportunity
1 Availability of Service Use Multiple Cloud Providers; Use Elasticity to Prevent DDOS
2 Data Lock-In Standardize APIs; Compatible SW to enable Surge Computing
3 Data Confidentiality and Auditability Deploy Encryption, VLANs, Firewalls; Geographical Data Storage
4 Data Transfer Bottlenecks FedExing Disks; Data Backup/Archival; Higher BW Switches
5 Performance Unpredictability Improved VM Support; Flash Memory; Gang Schedule VMs
6 Scalable Storage Invent Scalable Store
7 Bugs in Large Distributed Systems Invent Debugger that relies on Distributed VMs
8 Scaling Quickly Invent Auto-Scaler that relies on ML; Snapshots for Conservation
9 Reputation Fate Sharing Offer reputation-guarding services like those for email
10 Software Licensing Pay-for-use licenses; Bulk use sales

Bespin - Cloud based application development

Mozilla's Bespin, released today, is a cloud based development environment that acts as a collaborative working space. This has HTML5 Canvas, runs the tools in the cloud and pretty response - see the screen capture that I posted here.

There is a very decent editor on the hosted & extensible dev environment. There is also an integrated CLI, emacs and collaboration features.

What is missing are the publishing, caching capbilities, ability to do version control, client-side customizations and the standards enforcements. Hence - but this is just version 0.1

Kidaro - Microsoft Desktop Virtualization

Microsoft released the beta version of a enterprise desktop virtualization, MED-V, based on Kidaro. This would allow execution of any Windows OS applications on Vista.

MED-V is a solution for Application-to-OS incompatibility and accelerates the usage path on Windows Vista and any future OS. You can download the beta of MED-V here.

This is a step towards IT management of OS images across the corporate on any type of hardware (including flash drives) and data management using Active Directory controls.

The picture here would be self-explanatory on the architecture & lifecycle of virtual images:

Best practices for zoning in SAN

Ever wondered if the same best practices of SAN zoning are applicable with or without server virtualization? Let me give some pointers to create a stable, manageable & secure SAN zone.

• Always go for Zoning, even if LUN Masking is being used
• Always implement a default zone
  
• Disable any unused storage ports on the switch to increase security and avoid potential problems
• Use pWWN identification for all Zoning configuration unless D,P identification is required
  
• Create Zoning aliases and names with only as long as required to allow maximum scaling
• Single Initiator Zoning are to be used with separate zones if a HBA is carrying both types of traffic (eg for tape and disk traffic)
• Use accurate Zoning terminology
  
• Describe Zoning by enforcement method and identification type
• Always use the vendor given software to validate the zoning configuration
• Zones should use frame-based hardware enforcement

Business benefits of deploying & central management of OS images

In the first look, it might appear just as any other Desktop Virtualization or Application Virtualization - but the key difference is in the ability to manage OS images and in giving offline access.

The LivePC solution from MokaFive allows to create and distribute controlled OS images across the group with a single click. This helps in identical image replication across users, and control images from a central place.

Business benefits are to user OS images dynamically, work anywhere, and start up in an instant. As with any other virtualization, this technology works onl for x86 OS images. Task based users & test organizations would be the greatest benefactors of this!

Virtualization (in Real-time) for mobile devices

I was always a strong proponent of Virtualization on mobile devices from a criticality perspective (imagine a lost call from your spouse, because of calendar app hanging!). VirtualLogix has now announced a real-time virtualization solution for ARM based SoCs. The picture is self-explanatory on how multiple execution environments run on a single core. The critical phone service runs in a real-time isolated environment, while other apps run in regular OS – Linux/Windows/Symbian.

This solution also guarantees the reliability & performance of critical phone services, while advancing features for other apps. This also provides device management functions eg OS monitoring and automatic restart, allowing the system to be repaired or restored independently of the rich OS.

Dispersed storage - save costs on replication for DR

Did you know that there is an optimized way to achieve storage replication & reliability via "dispersed storage"? Essentially, the data resides behind a gateway which disperses data across physically separated storage units and reassembles it when needed. Based on the solution you use - this would be efficient, flawless & quick.

Traditional replication is achieved by maintaining duplicates of data, at 200% or more storage capacity needs. Attached picture from CleverSafe explains the concept of dispersed storage.

The average storage savings are about 60% with this mechanism for the same reliability.
Even if some of storage servers fail, one can get your entire, uncorrupted and undamaged file back, as long as you have access to a minimum threshold for retrieval [example, if six of 16 slices are down, you can still get the entire file back]. One also doesn't need to worry about the security of individual storage, because the data on any individual server cannot be interpreted!

3 most used VMware ESXServer features

Question: Do you know what are the 3 most used features of VMware ESXServer by a typical IT data center?
Answer: They are vMotion, HA & DRS! Though all of these 3 features are similar in nature, and are likely to have a common code-base, the implementation (customer's use cases) are different. The differences being:

• vMotion is the user initiated live migration of a guest VM across hosts (ESXServers). The guest OS is hot (running) and will be moved to a new host with the most minimal delay. The network & storage connectivity are not lost during this process. The requirements for this to happen are to have a shared storage (NAS, iSCSI or FC) among the two ESXServers where the guest OS resides and access data + common CPU architectures on the hosts. Both hosts need to be on the same virtual center too.
  
• HA involves load balancing of the physical systems to maximize the usage of guest OS instances. If there is higher load, the guest OSes would be distributed uniformly across servers. Conversely, if one or more the ESXServers are under-utilized they would also go to power down mode
• DRS involves kick-starting guest OS instances on another ESXserver in case of server crashes or hardware issues on ESXServer

Both vMotino & DRS requires VirtualCenter, but not HA.

VI4... Just wait... vSphere

The upcoming release of VMware virtual infrastructure product would be called as vSphere, per a VMUG announcement. This would support a maximum of 1TB memory on the host and a max 256GB for guest OS.

By the way what would be the equivalent name of ESXi in this release? Any guesses?

Live VM migration across CPU Platforms with RedHat KVM

Those of who you who are familiar with vMotion know that it doesn't work for cross-platform migration (VM to migrate from an Intel to AMD or vice versa). There is no vendor who can provide this feature, except for RedHat KVM.

The below demonstration shows for the first time live migration of a virtual machine across vendor platforms (Intel to AMD or Vice Versa). Live migration enables the movement of running virtual machines (VMs) from one physical server to another without disrupting service to the end user. This is a great news, where RedHat is leading in the virtualization market.

Note that oVirt/KVM is not yet officially released from RedHat.

Watch the demo on Youtube here.

Hyper-V and Virtual Devices in Windows Guest OS

In the Hyper-V environment, when you do a fresh install of Win2008 guest OS - you will observe that the guest OS automatically gets all the virtual devices giving the 100% portability and the best possible performance. Watch how the device manager shows up on a native Win2008 installation and on a virtual machine installation.

Community benchmarking of VMs - Will it work?

There is a new virtualization community resource, from vKernel, called CompareMyVM. It’s a community of users where virtualization users around the globe can submit and share their VM & virtual hardware sizing information - described as:

CompareMyVM is a community site where you learn how to right size your VMs. Compare your virtual machine resource allocations with that of the community at large. You can compare by application category to see how your peers are allocating resources to gain the most efficiency out of their environments. You can also contribute your VM sizings to the community. Your submissions are anonymous.

Things you can do with CompareMyVM…
* Browse Community Submissions
* Vote Submissions Up or Down
* Edit Submissions
* Submit Your Own VMs

This seems to be addressing the need of a benchmark source for information, highlighting that there is a clear lack of this information analysis to be provided by any other tool.

Who would provide opportunities to improve based on the data submitted?

Virtual Hard Disk in Windows 7 - Game Changing?

Virtual Hard Disk allows you to create, mount, and unmount OS images and Microsoft did a wonderful job by providing a native support in Win7. This allows users to use virtualization on Win7 without any additional software, a turf that VMware cannot play! The guest OS residing in VHD can be booted directly without the host OS. Microsoft have effectively created a "loopback HBA", and the bootloader can use to address VHD's as any regular disk. I used this on my PC and admit that this feature works rock solid on Win7, though still in beta phase.


This could be a game changing technology, since one doesn't need additional software (like VirtulBox or VMware Workstation) to use Virtualization for booting multiple OS. This gives new abilities to test even a native device driver in the guest OS as it works just as the host OS. Realize that Win2008 Server also supports this feature, but the kernel implementations of Win2008 and Win7 are totally different!

In order to use this feature, just know to use the specifics of Win7 VHD commands diskpart, bcedit and you are good to go.

Cloud Storage - QoS & Adoption

I was at the Cloud Computing track of SNIA winter symposium this week, and got a good pulse of the industry happenings. The cloud enablers are gaining momentum, and are geared towards driving the adoption. Similar to what we saw with the adoption of low-cost SATA drives during the 2001 downturn, this year is likely to see a greater adoption of cloud computing to save on capital expenditure. All that – assuming the focus is on QoS!

‘Cloud storage’ data is mostly (~96%) is either the backup/archive or files and only the rest (4%) is of transactional data. The minority transactional data (coming from live applications) is because of the current barriers in ‘cloudifying’ applications as well as cloud storage. Talking purely about cloud storage, the entry barriers are more of the security, SLA compliance, vertical industry needs & transfer times in/out of the cloud.

There is also an unpleasant reality that the ‘unstructured’ data is growing @10% YOY and unless there is a strong audit-ability and quotas on the cloud – your monthly cloud invoices are only going to go up!

Virtualization: If not planned right!

Hadoop & Use Cases

I spent a little time in digging out various use cases for Hadoop. Here are some of the use cases, listed by business use:

• Search: Yahoo, Zvents, Amazon A9,
  
• Log processing: Facebook, ContextWeb, Yahoo, Joost
• Recommendation Systems: Facebook, Amazon (eg, ad placements or site suggestions)
  
• Data Warehouse: Facebook, AOL
• Video and Image Analysis: Eyealike, New York Times

Note that even the search use cases of Yahoo or A9 do not use Hadoop when a user clicks "Search" button, but merely parses the web data and generates the index files on the back-end. Hadoop takes care of the node failures during the processing and provides massive scalability & load balancing.

Performance benchmark comparisions of hypervisors

One of the very few publicly available performance comparision results of Hypervisors (ESXServer, XenServer, Hyper-V, Virtual Iron & SLESXen). Look at the key statistics of bops (Basic Operations Per Second) and Disk I/O Results.

Read Network World for test details & results.

Distributed File Systems - for Virtualization & Clouds

In order to distribute the workload and processing uniformly across the virtualized systems, one would need a different file system - like Hadoop's HDFS or PNFS (Parallel NFS).

Hadoop is a Apache java project that supports data intensive distributed applications. Realize that this only works for offline processing - like logs, search, business intelligence or ad placements. It enables applications to work with thousands of nodes and petabytes of data. The current tested implementations work across 4000 nodes - plan is to make it work for about 15,000 nodes.

Most notable users of Hadoop are Facebook, Yahoo, Joost, Google & Veoh. I will try to list various uses for Hadoop some other day!

Cloud: 'vendor lock-in' Vs 'developer lock-in'

"When times are tough, money is tight. But is the Cloud Computing era of pay-as-you-go there yet?"

Unless you are really careful in cloud adopting life cycle - you are locked-up either with the cloud vendor (eg Amazon AWS) or with your development platform (eg Google Apps). The data portability interfaces are the key factors, not only for the success of Platform as a Service (PaaS, like Google App Engine) or Infrastructure as a Service (IaaS, like Amazon EC2 or S3) but also for the SaaS players. It is imperative that you choose the right design so that you reap the cost benefits and also keep your users happy on a longer-term.

While choosing the cloud models, some of the key factors one needs to look at are:

• Computing model (HTTP requests only, processes, OS etc)
  
• Service creation platform (Python, Ruby, Perl or any)
• File storage model (local, ZFS, block storage, tempFS)
  
• Database storage model (BigTable, SQL, simpleDB)
  
• Network topology (hidden, private, shared networks)
  
• Network protocols (http, tcp, udp)
  
• Network connectivity (hidden, NAT, VPN, L4, L7)
  
• DNS (static, dynamic, fixed)
  

Virtualization: Don't implement it this way!

Data Tsunami: Why do we need a 'cloud' for storage?

Ever thought why do you need a 'cloud' for storage and not manage the data on the internal servers? It is a combination result of people not being able to manage their data (like someone filling up the garage with crap and parking the car outside, even during a snow fall!).

The storage data growth is 10-times every 5 years and an example halo effect is explained here: one employee sending a 1.1MB email to 4 colleagues takes up 51.5MB total storage.

Source: IDC, "The Diverse and Exploding Digital Universe," March 2008

What is about Cloud Computing?

Use cases for cloud computing?

Several people keep asking me what are the use cases for cloud computing. Here are those in my opinion (as applicable to enterprises - large, medium & small).

• Test & Development, R&D, for Engineering organizations (eg Soasta)
  
• Storage Functional Offload (eg TimesMachine)
  
• Batch Process Functional Offload (eg Smugmug)
  
• Workload Augmentation (eg Animoto)
  
• Web Services

Depending on the specific nature of the applications, choose your specific cloud implementations (load balancing, scalability, file systems, network,...) etc