eucalyptus: delivering a private cloud

3
COMPUTER 102 INTERVIEW Published by the IEEE Computer Society 0018-9162/11/$26.00 © 2011 IEEE Eucalyptus: Delivering a Private Cloud I n an interview conducted by Dejan Milojicic, editor in chief of Computing Now (computing now.computer.org), Rich Wolski, CTO of Eucalyptus Systems, gives an overview of the services and applications his company provides. The extended transcript of that conversation, along with the audio podcast, is available at www.computer. org/portal/web/computingnow/videos/ trendwars. DEJAN MILOJICIC: What was your original motivation for starting Eucalyptus? RICH WOLSKI: Eucalyptus started in my research lab at the University of California, Santa Barbara, where I’m a computer science professor, currently on leave working on the commer- cialized version of this open source project. It was originally designed to solve a distributed computing problem in which we were attempting to link together the National Science Foun- dation’s supercomputer centers, Amazon Web Services (AWS), and sev- eral university sites where graduate students and postdoctoral research- ers were working on the project. We had a legacy, large-scale HPC science code that we were trying to run in Amazon that had already been ported to the supercomputer centers, and we needed to find a quick way to get the same version of the code to work at several disparate university sites. The fastest way that we could think of to do this was to build an emulator for AWS so that the version that was running inside Amazon could quickly be ported to other nonstandard uni- versity-supported environments. That emulation layer, the set of Web ser- vices that could fool the science code into believing it was Amazon, became Eucalyptus. MILOJICIC: Compatibility with AWS was one of the key design choices. What were the other choices and tradeoffs you had to make while you were designing and implementing the system? WOLSKI: The science was being driven by this notion of doing a hybrid. The other design criteria included porta- bility to a variety of infrastructures that had different levels of manage- ment and different technology life cycles. As you can imagine, university datacenters that are accessible to researchers vary widely in the equip- ment that’s available. To run the experiment, we needed Eucalyptus to be able to exist in a variety of envi- ronments with equal functionality. We designed it with a great deal of attention to portability and to what we call agnosticism for the underly- ing technologies. We considered it necessary for Eucalyptus to be able to manipulate all of the things that make a datacenter a datacenter. That made the system design very specific, because we couldn’t depend on any given functionality being in place at any one time. The other design tradeoffs cen- tered around the actual model. We spent a lot of time studying how clouds implement the nature of the software abstractions. We found that these abstractions were very much driven by the e-commerce model, the notion that users would interact with the system transac- tionally and that service would be asynchronous. That’s true for cloud abstractions as well. Thus, we knew going in that it had to be an e-commerce style of interaction and that this interaction had to be completely generic with respect to the infrastructure it was going to run on. MILOJICIC: You said you started with high-performance computing types of applications and HPC users, but today, you have a large variety of users out- side HPC. Please describe the kind of users you currently have. Dejan Milojicic, HP Labs Rich Wolski, Eucalyptus Systems An interview with CTO Rich Wolski provides an overview of the Eucalyptus open source cloud computing platform.

Upload: rich

Post on 22-Sep-2016

217 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Eucalyptus: Delivering a Private Cloud

COMPUTER 102

INTERVIEW

Published by the IEEE Computer Society 0018-9162/11/$26.00 © 2011 IEEE

Eucalyptus: Delivering a Private Cloud

In an interview conducted by Dejan Milojicic, editor in chief of Computing Now (computing now.computer.or g ), R ich

Wolski, CTO of Eucalyptus Systems, gives an overview of the services and applications his company provides. The extended transcript of that conversation, along with the audio podcast, is available at www.computer. org/portal/web/computingnow/videos/ trendwars.

DEJAN MILOJICIC: What was your original motivation for starting Eucalyptus?

RICH WOLSKI: Eucalyptus started in my research lab at the University of California, Santa Barbara, where I’m a computer science professor, currently on leave working on the commer-cialized version of this open source project.

It was originally designed to solve a distributed computing problem in which we were attempting to link together the National Science Foun-dation’s supercomputer centers, Amazon Web Services (AWS), and sev-eral university sites where graduate students and postdoctoral research-ers were working on the project. We had a legacy, large-scale HPC science code that we were trying to run in Amazon that had already been ported

to the supercomputer centers, and we needed to find a quick way to get the same version of the code to work at several disparate university sites.

The fastest way that we could think of to do this was to build an emulator for AWS so that the version that was running inside Amazon could quickly be ported to other nonstandard uni-versity-supported environments. That emulation layer, the set of Web ser-vices that could fool the science code into believing it was Amazon, became Eucalyptus.

MILOJICIC: Compatibility with AWS was one of the key design choices. What were the other choices and tradeoffs you had to make while you were designing and implementing the system?

WOLSKI: The science was being driven by this notion of doing a hybrid. The other design criteria included porta-bility to a variety of infrastructures that had different levels of manage-ment and different technology life cycles.

As you can imagine, university datacenters that are accessible to researchers vary widely in the equip-ment that’s available. To run the experiment, we needed Eucalyptus to be able to exist in a variety of envi-ronments with equal functionality.

We designed it with a great deal of attention to portability and to what we call agnosticism for the underly-ing technologies. We considered it necessary for Eucalyptus to be able to manipulate all of the things that make a datacenter a datacenter. That made the system design very specific, because we couldn’t depend on any given functionality being in place at any one time.

The other design tradeoffs cen-tered around the actual model. We spent a lot of time studying how clouds implement the nature of the software abstractions. We found that these abstractions were very much driven by the e-commerce model, the notion that users would interact with the system transac-tionally and that service would be asynchronous.

That’s true for cloud abstractions as well. Thus, we knew going in that it had to be an e-commerce style of interaction and that this interaction had to be completely generic with respect to the infrastructure it was going to run on.

MILOJICIC: You said you started with high-performance computing types of applications and HPC users, but today, you have a large variety of users out-side HPC. Please describe the kind of users you currently have.

Dejan Milojicic, HP Labs

Rich Wolski, Eucalyptus Systems

An interview with CTO Rich Wolski provides an overview of the Eucalyptus open source cloud computing platform.

Page 2: Eucalyptus: Delivering a Private Cloud

103APRIL 2011

WOLSKI: Our competition is pri-marily VMware. When we ta lk to customers, they’re sometimes trying to decide between VMware and Eucalyptus.

On the partnership side, we’ve been working with several compa-nies. Canonical, the company that manages the Ubuntu Linux distribu-tion, has been a long-term partner of ours. We’ve released Eucalyptus in Ubuntu for a couple of years now. Canonical has its own technology ecosystem built around Eucalyptus called the UEC (Ubuntu Enterprise Cloud). We work closely with them to make sure that the ecosystem and platform are production qual-ity, and they have been for some time.

We also partner with Red Hat. There’s a lot of back and forth with the Linux community. Red Hat and Eucalyptus recently established a partnership, and we’ll be doing some technology development with them as well.

We also get some components of the cloud space from the AWS eco-system. A good example of that is RightScale, a cloud dashboard. Right-Scale customers who are using AWS today can quickly build hybrid clouds by simply pointing their RightScale installation to a Eucalyptus installa-tion that runs on-premises.

We also have partnerships in the datacenter management space. A good example of that is Puppet, which is a great cloud-compatible system for distributing collections of Linux boxes inside datacenters.

MILOJICIC: What’s next for Eucalyp-tus? Do you have a roadmap for R&D productization?

WOLSKI: Near term, we’re focusing on providing more enterprise-qual-ity features in the platform. For example, we’re going to develop a high-availability version of Euca-lyptus that provides hot failover of internal components.

We’re also going to enhance the accounting system so that customers can perform hierarchical account del-egation and integration using existing identity management systems.

Longer-term R&D, we believe that hybrid clouds—the combination of the on-premises platform and the public platform—are the future. Much of our R&D is focused on the ability to link Eucalyptus on-premises with a public cloud to make the seamless use of both as easy as possible.

MILOJICIC: In your opinion, what were the key enablers for the suc-cess of Eucalyptus? If you had a magic wand, what would you wish for Eucalyptus?

WOLSKI: Key enablers were high-quality open source for e-commerce technology. It’s not an area I had worked in substantively before the Eucalyptus project, and I was really impressed by how high- quality and enterprise-ready many open source Web 2.0 technologies were.

The magic wand I would like to wave would bring about a happy marriage between the Linux pack-aging and distribution rules or conventions and the way Java-based e-commerce technologies are pack-aged and distributed. In many ways, they’re incompatible, and that incom-patibility makes it difficult for us to package and distribute Eucalyptus as a Linux-support platform, given

WOLSKI: Since the science project reached its conclusion, most of the Eucalyptus users have been more commercia lly focused or open source users who are interested in clouds. It’s a new application devel-opment paradigm. Many Eucalyptus users are building new applications that increasingly take advantage of the cloud-provisioning semantics.

Many commercia l users are adopting a self-service, scalable, transactional model as a datacenter management technology. There are many ways to manage datacenters, and the e-commerce self-service nature of cloud computing is appeal-ing for IT management and IT operations, and production users are gravitating toward Eucalyptus for that type of application.

MILOJICIC: What specific cloud ser-vices and applications are running on Eucalyptus?

WOLSKI: A great example is USAs-pending.gov, a fairly large-scale website operated by the US govern-ment that tracks how the stimulus money is being spent, where it’s being spent, and so on. The Web services that are serving that traffic are running inside Eucalyptus virtual machines or VMs.

This is a good example of what the early applications looked like: very network-facing, very Web-services oriented. Eucalyptus is useful for this type of application because it’s possible to dynamically change the infrastructure footprint in response to the load. To run a legacy version and then upgrade, it’s possible to run them side by side. The cloud is a very powerful model for deploying Web services.

MILOJICIC: Eucalyptus has achieved considerable popularity, but are there any alternatives to Eucalyptus? Who do you consider your competi-tion? Who do you consider partners, enablers?

Many commercial users are adopting a self-service, scalable, transactional model as a datacenter management technology.

Page 3: Eucalyptus: Delivering a Private Cloud

COMPUTER 104

INTERVIEW

computing, especially now that new cloud providers are emerging?

WOLSKI: There are two ways to look at this. Many people worry about whether there should be a cloud API standard, but my view is slightly different. If there are going to be standards, they should center around cloud federation. The standards should focus on cloud interoperabil-ity, not the API.

If we had cloud interoperability standards, the APIs might gravitate toward the interoperability standards, and in fact, there would just be one API. But where standards seem to make the most sense is where we’re trying to interconnect things.

This essentially takes a page out of the experience with the Internet, where the standardization efforts have been very successful in making it possible for two disparate com-puter systems to interoperate. I think that’s going to be true for clouds as well. Independent of what the local APIs are, the standardization effort is going to have the most impact where we find ways for two different clouds or two different sets of abstractions to interoperate.

Dejan Milojicic is a senior researcher and director of the Open Cirrus Cloud Computing testbed at HP Labs. He has worked in the areas of operating systems, distributed systems, and service management for more than 20 years and is an IEEE Fellow, ACM dis-tinguished engineer, and member of Usenix. Milojicic received a PhD from the University of Kaiserslautern. Con-tact him at [email protected].

Rich Wolski is the CTO and cofounder of Eucalyptus Systems and a professor of computer science at the University of California, Santa Barbara. He’s also a strategic advisor to the San Diego Supercomputer Center and an adjunct faculty member at the Lawrence Berkeley National Laboratory. Wolski received a PhD from the University of California, Davis. Contact him at [email protected].

We’ve been trying to put the open source features into the small amount of proprietary software and deliver it in an enterprise setting in a way that allows it to be managed according to best practices for enterprise infra-structure management.

Community is very important. We get a tremendous benefit from our community. Although we do receive code contributions, they’re not the only valuable contribution. We also benefit from quality assurance. Many in our community test the code con-tributions to make sure that they work, and we benefit greatly from this form of documentation. They do a much better job of describing what the system does, how to install it, and the tips and tricks than we do. The community enhances the open source and the product offering tremendously.

MILOJICIC: If you could afford to start over and redesign Eucalyptus from scratch, would you do it the same way? Did you learn from any mis-takes that you made?

WOLSKI: The methodology would be the same. That methodology was to begin by studying the paradigm at hand and try to understand what differentiates it from previous distrib-uted computing paradigms.

What I wouldn’t do—and this is more about the tension between an open source project and a research project—is use the research devel-opment life cycle to drive the open source project.

Very quickly, a tension developed between the extremely goal- and deadline-driven needs of the research project and the legitimate requests and complaints that the open source community had about the code that was being produced for that project. In retrospect, I would have sequenced those.

MILOJICIC: What are the most critical standards that are needed in cloud

that it was built using open source e-commerce technology.

MILOJICIC: Going forward, how do you see this happening? Are you going to feed certain features back into the open source version? It’s almost an ecosystem that is sensitive to perturbations. How are you going to manage it?

WOLSKI: We have managed that up to this point, and we’ll do so going forward, continuing to issue new releases of open source Eucalyptus with additional features.

Our policy so far has been to issue features in open source that are endemic to the cloud abstrac-tion. If we see a feature that must be part of the platform and it’s a cloud abstraction—it’s scalable, can be credentialed, is transactional, and so on—it absolutely goes in open source.

Nokia Inc. has the following open position in

Sunnyvale, CA

SENIOR NETWORKOPERATION ENGINEER

Design network infrastructure for new internet service based on marketing & eng’g requirement; work with Windows & Linux system administration; DNS, NIS, & NFS protocols; Linux & Windows virtualiza-tion; & other duties/skills required. [Job ID: NOK-SV11-SNOE]

Mail resume to: Nokia Recruiter,3575 Lone Star Circle, Ste. 434

Ft. Worth, TX 76177 & note specific Job ID#.