Why 'Private Cloud' Computing Is Real -- And Worth Considering
informationweek.com
To some, it's the same data center strategy by a trendier name. Don't believe them.
By Charles Babcock
InformationWeek
April 11, 2009
Companies that want the benefits of cloud computing services without the risks are looking to create cloud-like environments in their own data centers. To do it, they'll need to add a layer of new technologies--virtualization management, cloud APIs, self-service portals, chargeback systems, and more--to existing data center systems and processes.
Be ready for a debate as you discuss this new way of doing things. Just the term "private cloud" irks some computer industry veterans, who argue that cloud computing by definition is something that happens outside of your data center, or that the technologies involved in private clouds have been around for years, or both. Even some of my InformationWeek colleagues pooh-pooh private clouds. "Nothing new under the sun," scoffed one editor.
It's true that no single piece of an internal cloud architecture looks like breakthrough technology; it all looks deceptively familiar. I would argue, however, that private clouds represent a convergence of tech trends holding great promise for enterprise computing. Private clouds are a more powerful combination of modular commodity hardware that can be sliced and diced into many small pieces, with networking and storage that can be dynamically allocated through preset policies.
A virtualization management layer treats that whole set of technologies as a combined resource, while Internet networking and Web services allow us to interact with the cloud from any location. We can create new services out of existing ones hosted in the cloud and run user workloads at the click of a button. End users were far removed from the old mainframe and Unix server data center; with clouds, the business user can become king. Creating a private cloud will take considerable IT skill, but once one is built, authorized business users will be able to tap that computing power without a lot of know-how.
The Department of Veterans Affairs has deployed a small internal cloud. It wanted an early-warning system that could analyze data from its 100-plus clinics and hospitals and spot outbreaks of infectious diseases, and it had to do so on a tight budget. The project, dubbed the Health Associated Infection and Influenza Surveillance System, was built on six standard blade servers with converged network and storage I/O. The CPUs can be managed individually or as a virtualized whole, with workloads shifted and capacity summoned as necessary.
The six-blade system runs Egenera's cloud management software, PAN Manager, which manages I/O, networking, and storage for the servers as a logical set. It can execute several applications, while always having enough horsepower to do its main job. The system's Dell blades and storage can be virtualized as a pooled resource in such a way that processing power can be devoted quickly to the VA's cloud, its highest-priority task. In many ways, the VA's new system anticipated Cisco's recently introduced "unified computing" platform, a virtualized, multiblade server chassis with converged I/O that Cisco touts as just the thing for cloud computing.
Some see a hard line between the public clouds operated by Amazon Web Services, Google, and Microsoft and mixed-use corporate data centers. Such a line used to exist between proprietary enterprise networks and the Internet, too. Yet internal intranets gradually offset some of the functions of enterprise networks because they were patterned on TCP/IP and, thus, were compatible with the Internet surrounding them. Standard TCP/IP ultimately replaced proprietary networks, and the Internet began to function as an extension of corporate networks.
A similar phenomenon could, and probably will, happen with cloud computing. If efficient external clouds such as Amazon's Elastic Compute Cloud are based on a few standards, why can't data centers start to be built out as internal clouds that more closely resemble them? And once the two start to match up in architecture, what's to prevent a workload in one from being exported to the other?
That's the concept known as a hybrid cloud--part public cloud service, part internal cloud--and Bob Muglia, president of Microsoft's server and tools division, expects many companies to move in this direction. "All of our customers will have Windows servers on premises and, over time, add usage of cloud services," he says.
But Muglia adds that hybrid clouds will be "super hard" to pull off when they involve applications that require true cross-cloud integration, not simply moving a virtualized application from a private cloud to a public cloud. "The hard part is moving all of the services attached to that workload," he says. Muglia's group hopes to solve that problem by incorporating technologies developed for Microsoft's Windows Azure cloud operating system into Windows Server, so that the two environments will not only resemble each other but also work together. For Microsoft, however, that work all lies ahead.
Stephen Brobst, CTO of data warehouse provider Teradata, foresees other complications. For instance, while it's technically feasible to run data warehouses in public clouds, there are privacy and governance concerns that make it almost inconceivable to do so with personal data, he says. The Health Insurance Portability and Accountability Act, Sarbanes-Oxley, and the credit card industry's PCI standard put stringent controls on personal data. Running a data warehouse on an internal cloud gets around those issues. Teradata customer eBay runs a 5PB data warehouse internally, adding 40 TB a day, on a grid of x86 servers.
Internal clouds can help companies in several ways. Computing "elasticity" is one of the biggies. If there's a spike in demand on IT systems--the launch of a new product line involving manufacturing, marketing, and distribution channels, for example--they can scale to meet it. If a retailer expects spikes in its e-commerce system during a promotion, a marketing manager, or the IT liaison to the marketing team, could book an internal cloud to handle the spikes, holding other jobs in abeyance temporarily while also billing that cost to the promotion.
That brings us to the realm of self-service portals, metering, and chargeback systems needed to make it possible to dole out IT resources on demand, measure consumption, and allocate expenses with increased granularity. The best way to set up such a system is with the virtual lab manager products that software developers use to provision servers, says Forrester Research analyst James Staten. VMware's vCenter Lab Manager, Citrix Systems' Lab Manager, and Surgient's Virtual Automation Platform all come with self-service portals.
Start Planning
Even if you aren't implementing an internal cloud right away, consider a pilot project. "Enterprises should start training now to take advantage of cloud computing two years from now," says Staten.
Anyone thinking about internal cloud architecture needs to start with commodity x86 servers; the more similar they are in configuration, the easier it will be to manage them in a cluster or grid. (Look no further than Amazon, Google, and Microsoft for evidence that look-alike x86 servers are a staple of cloud architectures.) Indeed, migrating virtual machines can be done only between servers that use essentially the same chip in the CPU; otherwise, x86 instruction sets can vary in minute ways that will throw off VMware's VMotion or other live migration managers. Most companies don't have this homogeneous environment today.
In most cases, private cloud designers also will need to implement a virtualization management layer that goes beyond what they already have in place. Virtualization isn't a requirement for private clouds. PAN Manager is one example where workloads can be moved around without hypervisor software. But in most cases, virtualization and internal clouds will go hand in hand. And it's hard to gain the efficiencies needed and manage the scale involved without full-fledged virtualization management along the lines of VMware's vCloud or Citrix Essentials for XenServer and Hyper-V, or DynamicOps' Virtual Resource Management.
Rule of thumb: If you can master virtualization in the data center, you'll master the private cloud.
Many enterprises are marching toward private and hybrid clouds by following in VMware's footsteps. VMware this year is bringing out elements of what it calls the Virtual Data Center Operating System, or VDC-OS, which it illustrates in a diagram as hovering over both an internal and external cloud. Ultimately, VMware's goal is to let customers use VDC-OS to manage x86 servers and related storage as combined resources and move virtualized workloads between internal and external clouds.
The capabilities to provision, monitor, and move virtual servers around already are part of VDC-OS, but other private-cloud tools are still missing. One is vCloud, which VMware describes as an "initiative" rather than product. Part of its purpose is to let private cloud users get services from external clouds. Toward that end, VMware is establishing links with services providers, such as Melbourne IT, Savvis, SunGard, and Terremark.
They'll do that via the vCloud API. If VMware can gain broader acceptance for its API, it will be much easier to send a workload to an external cloud or enable a workload in an external cloud to tap into services that are part of the enterprise infrastructure. At this point, the vCloud API is in limited release.
Longer term, CEO Paul Maritz says VMware is seeking to give customers a single management interface with which they could run VMs in private or public clouds. Whether VMware can live up to these expectations probably won't be known for 18 months to two years.
Citrix's Lab Manager, NetScaler, and other products also can be used to build private clouds, along with its Workflow Studio for orchestrating resources. Like VMware, Citrix is working on a set of APIs that will work with its WANscaler, a product for speeding application delivery over a wide area network, to bridge the gap between enterprise and external clouds. Those still-to-come APIs promise to let customers move VMs and application resources between on-premises and external clouds.
Sun Microsystems recently introduced a private cloud platform--the Sun Open Cloud Platform--and public cloud services called Sun Cloud that are based on a set of open APIs. But Sun has been in tenuous negotiations to be acquired by IBM, and it isn't clear what IBM's plans would be for a Sun-centric cloud.
Users of the Amazon EC2 cloud might want to monitor the Eucalyptus open source project, which is creating a set of open APIs that closely mimic Amazon's and could be used to build private clouds that function in a way similar to EC2. Eucalyptus APIs could be used to summon the equivalent of EC2's Simple DB database or S3 storage services. An application built using such interfaces could be readily adapted for export to Amazon's cloud.
Prepare
A private cloud will help IT teams get ready for private-public hybrid clouds in the future data center Another problem private clouds must address is the need for a shared underlying storage file system. Without it, a set of VMs can't be treated as an elastic resource. VMware implements VMotion migration of VMs by imposing its own storage file system on a portion of the customer's disk arrays. Citrix works with Veritas and other storage vendors to ensure that its live migration feature will work.
Without a shared file system, the VM, when it moves to a new physical server, will not only leave the CPUs and memory behind in favor of a new machine's, it will also migrate away from its assigned storage and not be able to retrieve its pre-move data.
Egenera came up with its own solution to this problem outside of the fractious storage industry, where vendors have never agreed on a set of common standards. In the Dell PAN system, the Egenera software assigns each workload with a unique storage identifier, regardless of whether it's running in a VM or directly on a physical blade, says James Yaple, CTO of the Department of Veterans Affairs' data center operations. If a blade fails or a VM is moved to a different blade, the workload's storage identifier moves with it, providing a path back to the pre-move data.
Internal clouds aren't just a more efficient way of maintaining old data center practices. "You have to rethink the processes of how you did things before," says Jerry McLeod, VP of product management at cloud workload configurer FastScale Technology. Instead of having a system administrator configure each server, virtual machine configuration needs to take place based on a few reference images that will be widely used in your company.
In a traditional data center, a network administrator maps the addition of a new server to the network, assigning it switch and router resources, then a security and compliance administrator checks the configurations and installs any additional protections needed for the new server. With an internal cloud, those three tasks can be collapsed into one--the creation of a VM that's met with the approval of all three. IT departments need to put work into the process of constructing VMs so that can be accomplished in an automated fashion without disrupting IT operations or creating security risks or data privacy breaches, McLeod says.
App Tuning
After some established applications are running in an internal cloud, IT managers will start thinking about how to architect future apps for the cloud.
With external clouds, choices are limited as to what you can run. Google's App Engine now runs applications that must be written in Python 2.5. If you go to an external provider, such as Salesforce.com, you'll be able to use your internal data and customize the application, but you'll have to work in Salesforce's proprietary Visual Force language or the only other language it supports, Adobe's Apex.
Microsoft is seeking to broaden the selection to its popular Visual Studio and .Net technologies in its upcoming Azure cloud, but cloud developers will be restricted to Microsoft products and services, which may or may not work if they try to coordinate them with internal clouds.
With internal clouds, there's the prospect that in-house developers will continue to use their favorite development environment and code management system to produce cloud applications. They also will have access to the custom code and in-house services of the enterprise, provided they've been designed to be consumed as services. Internal clouds are about accessing and using services, and in that sense, are a specific execution environment for service-oriented architecture systems.
Likewise, instead of adjusting to Amazon's virtual file format, called the Amazon Machine Image, private cloud administrators can work with the virtualized files they're most comfortable with in their own data centers. Tools are being built to convert such files into the format used by an external cloud. RPath's rBuilder online tool, for example, packages AMI virtual appliances for shipping off the Amazon cloud.
If you succeed in building an internal cloud, the newly empowered end user is likely to find ways to combine services and generate applications that IT hadn't considered. Yes, a flurry of end user activity will multiply VMs and increase data center complexity. But if it also begins to increase revenue, by helping business units execute on new opportunities faster, private clouds could become a sought-after IT delivery model in record time.
The prospect that internal creativity will one day unite with well-designed external cloud services is so tantalizing that skeptics of private clouds won't hold sway for very long.
_____
Reasons To Build Private Clouds:
Lower Capacity
Pooling resources will let companies reduce computing capacity by giving higher-priority tasks power during peaks
Reduce Overhead
x86 servers and related resources in a virtual data center can be managed as a unit
___________________
GE Puts 'Private' Cloud Computing To The Test
informationweek.com |
