Performance comparison of clouds without moving apps

Working with Srikanth Kandula and Ming Zhang of Microsoft Research in Redmond, WA, Yang and Li have developed a suite of benchmarking tools that make it possible to compare the performance of different cloud platforms without moving applications between them. These tools use algorithms to measure the speed of computation, and shuttle data around to test the speed at which new copies of an application are created, the speed at which data can be stored and retrieved, the speed at which it can be shuttled between applications inside the same cloud, and the responsiveness of a cloud to network requests from distant places.
...

They are building software that captures the performance of an application when running on a local server and then creates a dummy version of it on several clouds to compare how they hold up. The dummy doesn't actually perform the function of the software, but it exerts the same computational, storage, and network demands. "Without actually migrating the app, I could just try out that representation of what it does," says Yang.

http://www.technologyreview.com/computing/25815/page1/

Cloudkick has been acquired by Rackspace!

... as posted in the Cloudkick site ...

Why did Rackspace acquire Cloudkick?

From the beginning, Cloudkick has been obsessed with making life easier for system administrators as they deploy and manage infrastructure and applications on the cloud. In a short time, they established their position as one of the most advanced business-to-business focused start-ups in the cloud computing space and the system administrators tools they provide are recognized as best in the market. As Rackspace continues help customers manage their move to the cloud, Cloudkick will play an important role in improving the Fanatical Support® Rackspace already offers its customers. Rackspace’s mission is to be recognized as one of the world’s great service companies and we believe Cloudkick aligns very well with this mission.

https://www.cloudkick.com/acquisition-faq

Cloud broker - new word?

... One of his biggest concerns now is to make sure the various cloud services work well together—and are compatible with existing systems...

...

In the future, companies may hook up with more than a dozen different cloud services providers. Brokers would serve as intermediaries, offering such services as customization, integration, security, and aggregation. Through 2014, cloud service brokerage will generate more than $5 billion in sales—up from less than $50 million this year—making it the fastest growing area of cloud computing, Gartner said earlier this year.

The need for brokers is compounded by the lack of standards in the cloud services industry, which means that information can't travel between different services without specially written code to translate among them. "If companies integrate it themselves, they have no hope of saving money," says Daryl Plummer, group vice-president of Gartner Research. In general, cloud services appeal to companies because they're less expensive than buying hardware and software.

...

http://www.businessweek.com/technology/content/dec2010/tc2010126_515410.htm

Who Has the Most Web Servers? - May 14 2009

There was a milestone buried deep within the earnings tables in this week’s first quarter results from Rackspace: "the San Antonio company has become one of a select number of companies that have more than 50,000 servers. Rackspace reports that as of March 30 the company’s data centers house 50,038 servers, up from 47,518 at the end of 2008. Of the companies that publicly report their server counts, only European hosts 1&1 Internet and OVH have more than Rackspace. Here’s a look at some of the providers with high server counts, gleaned from public reports and partial data from a recent Netcraft server count report:"

• 1&1 Internet: 55,000 servers (company)
• OVH: 55,000 servers (company)
• Rackspace: 50,038 servers (company)
• The Planet: 48,500 servers (company)
• Akamai Technologies: 48,000 servers (company)
• SBC Communications: 29,193 servers (Netcraft)
• Verizon: 25,788 servers (Netcraft)
• Time Warner Cable: 24,817 servers (Netcraft)
• SoftLayer: 21,000 servers (company)
• AT&T: 20,268 servers (Netcraft)
• Peer1/ServerBeach: 10,277 servers (company)
• iWeb: 10,000 servers (company)

VMware Expands vCloud Portfolio With Zimbra

"Zimbra is a great example of the type of scalable 'cloud era' solutions that can span smaller, on-premise implementations to the cloud. It will be a building block in an expanding portfolio of solutions that can be offered as a virtual appliance or by a cloud service provider. We are excited to welcome the Zimbra team and community to the VMware family."

VMware is obviously looking to build out its stack of cloud infrastructure.

PaaS with choice: Motivation for VMware to acquire SpringSource

VMware announced to buy SpringSource yesterday. SpringSource had been a technology innovator focused on the application-centric areas rather than on the hardware-infrastructure focus. Their employees were thought leaders in Apache Tomcat, Apache http server, hyperic, Groovy and Grails. SpringSource’s obsession has been simplifying and automating the build-run-manage lifecycle that all applications go through, and they have done so by attacking similar pockets of complexity. They bring this complexity-busting focus to several areas… high-productivity developer tools and frameworks, lightweight application server runtimes, and application management and monitoring. The end goal is very similar; attack the time and money spent on application complexity and maintenance tasks, shifting the focus to new and more reliably deployed applications.

VMware feels SpringSource enables an evolutionary path for application developers to reach end goals without requiring complete infrastructure or application rebuilds.

Ultimately for VMware the end goal is on vCloud which is aimed at IT applications, not infrastructure - there is an extensive application flavor for the cloud to provide these key features. SpringSource would definitely enable VMware in preaching the right solutions to VMware's customers:

1. Elasticity: automatically scaling up and down the infrastructure to meet the needs of the application
2. Multi-tenancy: being able to isolate resources and applications from one another in a shared infrastructure
3. Simplified provisioning: Isolate the developer from worrying about how is code gets installed and deployed
4. Self-service: allowing developers to gain access to their development infrastructure at any time, in many cases to circumvent the processes and inefficiencies of their typical IT service request processes.
5. Rapid development: go from code to cloud in a matter of minutes, particularly during the development and test phases
6. Simplified (or invisible) management: PaaS offerings typically have built-in application availability and performance management

Topology for Inter-datacenter VMotion

An often asked question in any of the vCloud discussions is about the probability of vMotion across data centers (over wide area network) - it is not an easy thing considering the latencies. The complications are both in terms of stretching the L2 domain between the sites and the shared storage issue across sites.
What are the business requirements for inter-datacenter vMotion?

1. Load balance/Go-green to save power: “follow the sun” or simply consolidate VMs to conserve power.
   
2. Uptimes during maintenance windows: Have a fallback plan for your applications to be available during datacenter shutdowns
   
3. Disaster Recovery/avoidance: This is an obvious reason, to fallback during DR situations

Compared to #2 and #3, #1 could be higher priority as that also gives a provision to use someone else's (like AWS) data center for peak loads (like month end report generations) instead of building up in-house data centers.

Overall, this initiative would lead to consolidation in data centers slowly and bigger players would benefit. Below is the topology of vMotion that would be demoed at VMworld, jointly by Cisco & VMware.

Cloud Operating System & Google

Where do you think Google is headed with Chrome & Android? I see both as a direct connection to a "Cloud Operating System".

Chrome faciliated faster web browsing, meaning more searches which translate into more ad revenue for Google. Then, the obvious motivation of developing an OS came up to increase the volume of Web activity itself. Google wants everyone to keep searching & searching fast.

The Chrome operating system is said to be released next year, but will it ever lure the mom and pop to move to this new browser? All said & done, Intel would definitely doesn't want this to happen - otherwise, with little of client processing - what is the use of the CPU horse-power?

To completely ignore operating systems in favor of the cloud operating systems might be an efficient route to failure. We keep re-inventing the wheel once every few years, and create technology confusion as well.

Network Port List for VMware products

Here is a list of network ports that are used by VMware products - this is an informal, unpublished list. The details here are all gathered from various product manuals and VMTN and could be updated at VMware's discretion.

VMware Studio 2.0 Beta launch

VMware Studio 2.0 Beta was made publicly available just a few days ago. Note that the name of this product is not yet vStudio!

New features of Studio 2.0 are:

• Build vApps and virtual appliances (with in-guest OS and application components) compatible with VMware Infrastructure, VMware vSphere 4.0 and the cloud
• Support for OVF 1.0 and 0.9
• Available as an Eclipse plugin in addition to the standalone version
• Ability to accept existing, Studio-created VM builds as input
• Support for 32 bit and 64 bit versions of Windows 2003 and 2008 Server, SLES 10.2, RHEL 5.2 and 5.3, CentOS 5.2 and 5.3, Ubuntu 8.0.4.1 and additional operating systems
• Publish patches to update deployed virtual appliances
• Extensible in-guest management framework
• Automatic Dependency resolution
• VMware ESX Server, VMware ESXi, VMware Server 2.0, Server 1.0.4, 1.0.5, 1.0.6, VMware Workstation enabled as provisioning engines
• Infrastructure enhancements in the GUI and builds

Strategies for Virtualization adoption in your company

Me being a virtualization consultant, I keep getting the asked the question of "what should be my strategy for virtualization adoption?". It is obvious for the top management to adopt Virtualization for obvious cost reasons, and these are my thoughts:

Take the first steps of adopting virtualization for all new projects and converting low-risk physical servers (like FTP servers) into VMs.

And then the more important strategies would be:

• Process changes & convincing mid-management to think of "VMs" instead of physical servers while requesting for new server platforms
  
• Build IT SLAs around VMs instead of physical servers
  
• Internal education of virtualization & VMs for application teams - incentivate them by passing on the cost savings
  
• Process setup to migrate physical servers into VMs
• Any other financial aspects involved while converting physical assets into shared virtual assets
• Setting up cost sharing model to share server costs across business units, based on VM usage
  

It is easy for you to achieve about 30-40% of virtualization adoption with minimal effort. After that - it would need someone to sit down and do the above implementations to increase the adoption further.

vSphere to launch on April 21st

VMware to spill the vSphere beans on April 21
Mark your calendars. VMware will launch the next generation of its virtualization platform on April 21, at an event at its Palo Alto headquarters, flanked by partners Cisco and Intel.

...

The hypervisor itself will be 64-bit, will provide support for up to 256 GB of RAM per guest and eight-way virtual symmetric multiprocessing, or SMP. On the management side, it will be possible to cluster the vCenter servers, and users will be able to create and provision virtual machines using new host profiles and guest templates.

...

In addition to core ESX and vCenter products, VMware is also expected to launch several new ancillary products, including the long-awaited VMware Fault Tolerance, and AppSpeed for performance management of applications running within a virtual machine.

Read the details here.

VMware to Manage Virtual Machines from Mobile Phones

Introducing VMware vCenter Mobile Access (vCMA). vCMA allows you to monitor and manage VMware Infrastructure from your mobile phone with an interface that is optimized for such devices. Specifically, it allows you to:

• Search for virtual machines in your data center
• Migrate virtual machines from one host to another using vMotion
• Execute recovery plans using VMware Site Recovery Manager
• Access Scheduled Tasks, Alarms and Events
• And much more...

Do you think if it is just a cool feature or is this really useful for a datacenter class management?

Happy Square Root Day

Today is a "square root day", hope you are enjoying it. The next such day would only be on 4/4/16.

Vendor lock-in, severity across cloud architectures

Several people have asked me in the recent past about the vendor-lock-in issue with various deployment architectures. The below picture should be self-explanatory on the amount of lock-in across SaaS, PaaS and IaaS. The lock-in that I am referring-to here is a combined aggregate of Vendor lock-in + data lock-in + dev environment lock-in + data lock-in.

Java platform as a service for cloud

I used to think about the alternatives for Google platform for the java world and came across CloudFoundry today. They provide the java stack, similar to SpikeSource' LAMP stack, on the cloud. Though the stack is not very flexible for various types of Java apps (4 layer models, server types, EJBs etc) - it works for the common deployments. You just make the .war file and upload it - watch it to get deployed in a couple of minutes.

Microsoft & RedHat sign deal on Virtualization interoperability

Red Hat and Microsoft have signed reciprocal agreements today to enable increased interoperability for the companies' virtualization platforms. Each company will join the other's virtualization validation/certification program and will provide technical support for their mutual server virtualization customers.

Server virtualization is moving towards the commodity model and this is a natural transformation. Crucially, Red Hat's interoperability deal with Microsoft does not include any patent covenants.

Gartner Report - Virtualization to grow 43% in 2009

Cost reduction, resource utilization and management advantages drive market growth - says Gartner Report.

... Global virtualisation penetration is on pace to reach 20 per cent in 2009 from 12 per cent in 2008...

...Virtualisation helps organisations to cut costs, better utilise assets and reduce implementation and management time and complexity, all of which are crucial in this economic environment...

Gartner recommends that vendors take advantage during this disruptive period by introducing leading-edge management tools in support of virtualisation initiatives and ensure that virtualisation-specific management products can integrate within existing management frameworks.

Cloud Computing - Top 10 Obstacles & Opportunities

Quick Preview of Top 10 Obstacles to and Opportunities for Growth of Cloud Computing (from UC Berkeley's Research Publication, Feb 10 2009).

# Obstacle Opportunity
1 Availability of Service Use Multiple Cloud Providers; Use Elasticity to Prevent DDOS
2 Data Lock-In Standardize APIs; Compatible SW to enable Surge Computing
3 Data Confidentiality and Auditability Deploy Encryption, VLANs, Firewalls; Geographical Data Storage
4 Data Transfer Bottlenecks FedExing Disks; Data Backup/Archival; Higher BW Switches
5 Performance Unpredictability Improved VM Support; Flash Memory; Gang Schedule VMs
6 Scalable Storage Invent Scalable Store
7 Bugs in Large Distributed Systems Invent Debugger that relies on Distributed VMs
8 Scaling Quickly Invent Auto-Scaler that relies on ML; Snapshots for Conservation
9 Reputation Fate Sharing Offer reputation-guarding services like those for email
10 Software Licensing Pay-for-use licenses; Bulk use sales

Bespin - Cloud based application development

Mozilla's Bespin, released today, is a cloud based development environment that acts as a collaborative working space. This has HTML5 Canvas, runs the tools in the cloud and pretty response - see the screen capture that I posted here.

There is a very decent editor on the hosted & extensible dev environment. There is also an integrated CLI, emacs and collaboration features.

What is missing are the publishing, caching capbilities, ability to do version control, client-side customizations and the standards enforcements. Hence - but this is just version 0.1