nebula james williams

Download Nebula james Williams

Post on 14-May-2015

3.779 views

Category:

Technology

3 download

Embed Size (px)

TRANSCRIPT

  • 1.NASA NebulaPast, Present, and Future
    A Story in Three Parts
    James F. Williams
    CIO, NASA Ames Research Center
    April 26, 2011

2. One of the first cloud computing platforms built for the Federal Government by the Federal Government
Publicly launched IaaS with the White House as the first customer in production
Basis of OpenStack Compute, aka Nova
There are over 300 users across nine NASA Centers + JPL + HQ
NASA Nebula Cloud Computing
3. Lots Been Said About Nebula.some of it crazy
What do you get when you combine cloud computing and data center containers? You get NASAs Nebula, the space agencys new data powerhouse, which provides on-demand computing power for NASA researchers.
http://www.datacenterknowledge.com/archives/2009/12/02/nasas-nebula-the-cloud-in-a-container/
The world will end if we dont do this.
Former NASA Center CIO
Will create world peace.
Future Miss America
Putting the Space in Rackspace
Start-up Co-founder
This will can help solve NASAs real compute issues
New NASA Center CIO
Because we need more developers
Well known cloud architect
4. Tell you the unofficial Story of how NASA Nebula started..from my perspective.I was there for most of it.Ive blacked out for some of it.
Where we are today and what NASA is doing
Our vision for NASA Nebula for the future
Getting Past the Hype
5. NASA Nebula, Part 1
6. 1000s of other NASA websites
"A long time ago in a US Agency far, far away.... (well, DC is far)"
www.nasa.gov
Photo Credit:www.starwars,com
Photo Credit:www.starwars,com
7. Why wouldnt they join NASA.gov?
www.nasa.gov
Perception:
Control
Issues with CMS
Flexibility
Cost
Photo Credit:www.starwars,com
8. Theres a better way to do this
Problem:How do we get these web developers to stop building out their own sites?
Solution:Give Developers a better alternative to status quo
NASA.net was born
9. NASA.net
Setting:Basement of NASA Ames Research Center (ARC) Building 200, in an old conference room
Imagine:Small team of developers working on Platform as a Service
Code hosting
Continuous integration
Bug Tracking
Best Practices in code development
Making unicorns happy across NASA
10. But after working on Platform as a Service prototypes
Learned in order to run a web application framework properly as a service, we need elastic infrastructure
11. Over Indian Food in Mountain View
Joshua McKenty pitched the cloud idea to us.I just ate curry.
We decided build out an IaaS capability just to support NASA.net
Chris didnt pay for lunch.
Photo Credit:http://www.tandooribistrosj.com/
12. Problem
Us:Thanks!We need funding..
Everyone:Great Idea!
13. 14. Why?
FY09:
Line item for cloud?
FY06:
Plan FY09 Budget
How do we fund IT innovation?
15. At the same time, White House cloud first initiative was gaining traction
Federal CIO, VivekKundraevangelized the idea of Cloud Computing
In coordination with the data center consolidations, agencies should evaluate the potential to adopt cloud computing solutions by analyzing computing alternatives for IT investments in FY 2012. Agencies will be expected to adopt cloud computing solutions where they represent the best value at an acceptable level of risk.http://www.whitehouse.gov/sites/default/files/omb/assets/memoranda_2010/m10-19.pdf
16. A team said No
They just said No.
A wise woman said. Drop back.Punt.And wait to get the ball back.
17. We didnt have any money but we had popsicle sticks and string so we started anyway.
Josh liked Indian food and was friends with Jesse
These guys were the first IT hippies I met.I gave them free headbands.
There eventually was a guy named Vish(from Iowa? Really?) and someone from ZZTop(Hes got cloud, and he knows how to use them..)
I bought cases of Red Bull to cover up the fact I didnt have money for chairs.
But ARC is creative..
18. USASpending.gov
Then the White House wanted to use our cloud.Then they came to ARC and some NASA officials were there...
Somehow we got a container and a couple $$s to do something but still no money for chairs.
Lots of catsno milk. Herding was a problemEnter Soo and Ray and WilliamFinally a semblance of a team that started to make things happen.
19. Then things started to happen
20. Massive Technical Challenges
)
Ran into every problem you can think of:
Hardware, Database, Software, Client management
Details:
Jumbo frames causing VMs to kernel panic
2 minute network separation between cloud controller & VMs caused controller to decide to terminate instances
Prepare your container for the cloud it rains
Pieces of their really cloud ready.Indexes didnt fit in RAM (even with 96GB allocated to VMs)
21. Weekend Hack-a-Thon
Decided to spend the weekend hacking a new open source cloud controller framework. This became Nova
Some thoughts:Monolithic is bad each component should scale independently
APIs are good you shouldnt have to use web UI to configure
Simple things should be simple, hard things possible
The hard parts (hypervisor, storage system, networking) is done by others.Build a cloud the same way the way you build a scalable web application
Nova has been empowering NASA users for a year this May.
22. Launched USAspending.gov on5/21/2010
23. OpenStack History
2010
2005
March
May
June
July
Rackspace Cloud developed
Rackspace Decides to Open Source Cloud Software
NASA Open Sources Nebula Platform
OpenStack formed with contributions from Rackspace & NASA
Inaugural Design Summit in Austin
24. Rackspace Called NASA
Rackspace:
Wow. Can we meet your team of 400 developers?
NASA:
Sure. We got 8 developers.And not all of them full time.
25. Why not a NASA-Driven Foundation?
Resources
Expertise
Focus
Not NASAs mission
26. NASA Nebulas contributions to OpenStack aligns with the Administrator's strategic goals
Facilitate the success of a viable commercial space industry to provide assured U.S. access to low Earth orbit for cargo and crew and acquire, mature, and infuse commercial capabilities across all NASA activities
Promote enhanced cooperation with international, industry, other U.S. government agency, and academic partners in the pursuit of our missions.
26
26
27. NASA Nebula + OpenStack supports the Agencys Goals:
Goal 6: Share NASA with the public, educators, and students to provide opportunities to participate in our mission, foster innovation and contribute to a strong National economy
28. NASA Nebula, Part 2
29. Developers from all over the world want to contribute code because they want to be part of the space exploration.
Overheard at the OpenStack Design Summit, My code could be part of NASA Nebula.This is as close to being astronaut I am ever going to get!
Overheard at Design Summit
30. IaaS Status as of the start of FY11

  • Nebula is maturing from an Agency innovation project to a new OCIO service offering

31. In April 2010, the Alpha phase began with 20 users began 32. Even though weve been concentrating on ARC and GSFC,Beta closed with 240 IaaS users across 9 Centers, HQ,JPL, and NSSCthrough word of mouth 33. Nebula requires OCIO sponsorship to expand and become an institutionalized service for the entire AgencyGRC: 4
HQ: 8
GSFC: 104
MSFC: 2
ARC: 111
LaRC: 1
JSC: 4
JPL: 3
KSC: 1
NSSC: 2
SSC: 1
30
34. Our major challenge
35. How do we get here?
36. Requirements*
Current Options*
* Requirements and Options documented in over 30+ interviewswith Ames scientists as part 2009 NASA Workstation project.
37. Offer scientists services to address the gap
Desktop
TARGET COMPUTE PLATFORM
Server-based compute resources
Excellent example of how OCIO-sponsored innovation can be rapidly transformed into services that address Agency mission needs
High-end Compute
Vast Storage
High Speed Networking
Super Computer
34
38. ROI and ARC Case Study
2,600 Civil Servants and Contractors
Ames Research Center
*2,600 Servers

  • Non-enterprise applications

600 servers in traditional data center environments
1,000 under the desk desktops and workstations being used as additional compute resources
1,000 servers or desktops/ workstations being used as servers in lab environments
CHALLENGES

  • Dedicated use (non-shared)

39. **Underutilized (average of 15% utilization) 40. Inefficient space and power use 41. Numerous security plans 42. Significant system administration expense*Number of servers:Estimate based on data collected from NASA Workstation Project, inventory of Ames institiional data center, review of Ames IP address allocation, and consultation with Ames Network Engineers.
**15% utilization based on two reports from Gartner Group, Cost of Traditional Data Centers (2009), and Data Center Efficiency (2010).
35
43. ROI and ARC Case Study
POWER:Computers typically require 70% of their total power requirements to run at just 15% utilization.
*15% utilization based on two reports from Gartner Group, Cost of Traditional Data Centers (2009), and Data Center Efficiency (2010).
36
44. ROI and ARC Case Study
2,600 Servers

  • Non-enterprise applications

=
High-end Compute
Vast Storage
High