automating oracle database deployment with amazon web services, fabric, and boto
DESCRIPTION
Have credit card, need database? In this talk, I'll show you how to deploy your own Oracle 11gR2 sandbox with a single keystroke (and I don't mean RDS). Along the way, we'll learn about Infrastructure-as-a-Service with boto, provisioning tools like fabric, and Oracle response files. When we're done, we'll have a repeatable, ten-minute process that can deliver a server as cheap as $5/day or as powerful as 40k IOPS and 2.6GB/s throughput. More importantly, we'll understand what the big deal about IaaS and automated provisioning really is, and how enterprise products like Oracle can still fit comfortably in the space.TRANSCRIPT
SEMOPFeb. 12 t h, 2013
AUTOMATING ORACLE DATABASE DEPLOYMENT
WITH AMAZON WEB SERVICES, FABRIC, AND
BOTO
Michael J Bommarito II CEO, Bommarito Consulting, LLC Email: [email protected] Web: http://bommaritollc.com/ LinkedIn: http://linkedin.com/in/bommarito
Expert infrastructure, development, and data services.
INTRODUCTION
© Bommarito Consulting
CIO, Solid Logic Technology, Inc. Web: http://solidlogic.com/
Large, labor-intensive software and infrastructure projects.
GOALS
DEFINE - Understand the myriad of acronyms and cryptic phrases: IaaS, DevOps, automated deployment…
DESIGN - Understand how the requirements fi t together: IaaS provider and/or private cloud Boto Fabric Oracle
EXECUTE – Run and modify a basic deployment example.
EXTRAPOLATE – Discuss possibilities for real business needs.
© Bommarito Consulting
DEFINE
The Cloud is dead. Long live the Cloud!
Note: I try to use “cloud” as little as possible in this discussion.
What do we need to cover (in increasing diffi culty)? Automated deployment Virtualization Automated provisioning Configuration management Infrastructure-as-a-Service (IaaS) Programmable infrastructure (“infrastructure as code”) DevOps
© Bommarito Consulting
DEFINE
Automated deployment: Process-driven build of software, OS and up. Examples: Oracle response fi les, RHEL kickstart, Puppet, that
maze of bash or perl scripts.
Virtualization: Abstract raw compute and storage for more flexible, effi cient utilizations. Hardware: Xen/KVM, ESx, Hyper-V, Oracle VM (Xen), VirtualBox “Storage:” SAN, DFS, S3, OpenStack Swift, Storage-as-a-Service
Automated provisioning: Process-driven build of compute, storage, and network resources. Examples: Norton Ghost, vSphere/VMware Automation, xen-api,
AWS APIs
© Bommarito Consulting
DEFINE
Configuration management: Process-driven management of software (apt, yum) and configuration (/etc/, Windows Registry). Examples: Puppet, Chef, Fabric(~), CFEngine, and, again, that
maze of bash or perl scripts. Unlike automated deployment, this is stuff you live with and
change every day. Some tools, like Puppet, can gracefully span automated
deployment and configuration management.
Infrastructure-as-a-Service (IaaS): Compute, storage, and network virtualization, but with someone else running the data center and virtualization operations. Many providers offer some form of IaaS/cloud/virtualization, but
very few add real cost-adjusted value.
© Bommarito Consulting
DEFINE
Programmable infrastructure (“infrastructure as code”): When your IaaS has a featureful API and unbounded capacity relative to your needs. Never ask a hardware rep about lead times. Never ask a technician when a box will be cabled. Never ask a sysadmin when your server will be ready. Never pay for unutilized, depreciating assets.
DevOps: Collaborative, process-driven approach to managing infrastructure and software. Think Toyota Production System + “cloud” for infrastructure and
ALM. Reduce cost of failure and probability of failure. Make ITIL and CMMI easy (…well, easier)! Honest truth: More of a culture than a methodology.
© Bommarito Consulting
DESIGN
Phew. Enough theory and philosophy. Let’s take inspiration from the previous slides and actually build something to solve a real problem:
“I want an Oracle database server. I want it now. I want it to look the same every time, but be easily configurable if I do want it to change. I only want to pay for it when I use it. Did I mention now?”
Components: IaaS: Amazon Web Services Automated Provisioning: boto/Fabric Automated Deployment: Fabric
© Bommarito Consulting
DESIGN
IaaS: Amazon Web Services Fully programmable infrastructure. Plug in credit card and email, download credentials.
Automated Provisioning: boto/Fabric Provision network, storage, and compute resources for server. Python + configuration
Automated Deployment: Fabric Update host and ensure Oracle prereqs met. Install software. Configure listener and database. Python + configuration
© Bommarito Consulting
DESIGN
What does basic configuration look like? Configuration
fabricrc: Basic fabric configuration for users, SSH config, behavior
hosts.txt: Hosts managed by Fabric infrastructure. python-requirements.txt: Required packages for deployment
server. yum-requirements.txt: Required packages for target servers.
Templates database.rsp: Oracle database software response fi le dbca.rsp: Oracle database response fi le netca.rsp: Listener response fi le public-yum-el5.repo: Yum repo config, e.g., RHEL5U9
© Bommarito Consulting
DESIGN
What does Boto look like?
© Bommarito Consulting
DESIGN
What does Fabric look like?
© Bommarito Consulting
DESIGN
Workflow Overview1. Provision (boto)
1. Create security group.2. Create storage mapping.3. Create instance.
2. Deploy (Fabric)1. Resize root volume (not required by newer OS images).2. Disable software firewall (no routing/QoS, use AWS hardware).3. Upgrade base OS packages and reboot.4. Enroll system in public OEL yum and install oracle-validated.5. Setup basic Oracle DB and OFA requirements.6. Install database software and run post-install scripts.7. Create a listener.8. Create a database.
© Bommarito Consulting
EXECUTE
© Bommarito Consulting
EXECUTE
And an aside on cloud performance: don’t let anyone tell you that I/O isn’t available on public cloud.
hi1.4xlarge instances: ORION benchmarks at 1.8GB/s large reads, 40k IOPS, stable <1ms latency.
Provisioned IOPS: dynamically provisioned storage that can be configured to provide guaranteed IOPS.
RAID/LVM: just as viable in the cloud as with DAS/NAS. Application design: Was I/O the bottleneck because it
had to be or because your application was designed when memory was expensive?
© Bommarito Consulting
EXTRAPOLATE
Let’s go back to our motivating problem:
“I want a ________ server. I want it now. I want it to look the same every time, but be easily configurable if I do want it to change. I only want to pay for it when I use it. Did I mention now?”
How many times have you heard something like this need? Development for testing, QA, skunkworks. Sysadmins for testing, QA, skunkworks. Sysadmins for HA, DR. Management for compliance, cost, and urgency.
© Bommarito Consulting
EXTRAPOLATE
Signs you might be a good candidate: Large number of similar assets. Large number of repetitive tasks (or large sysadmin team). Audit requirements around deployment and ongoing configuration
changes. Unpredictable, variable, or low utilization, e.g., uncertain usage,
bursty usage, or overcapacitization. Bottlenecking in the dev->sysadmin->QA workflow.
Signs you might not be a good candidate: Tasks are not repeated enough relative to complexity/cost to
amortize setup costs. Total number of assets is low. Predictable, high utilization. Low-to-no compliance environment.
© Bommarito Consulting
Bommarito Consulting Blog Oracle ORION I/O benchmark results for AWS EC2 hi1.4xlarg
e instance typeSoftware:
Fabric Boto Puppet Chef
Businesses Amazon Web Services Rackspace Microsoft Azure
Fabric and Boto for Managing EC2. Graham Poulter.
REFERENCES
© Bommarito Consulting
Michael J Bommarito II CEO, Bommarito Consulting,
LLC Email:
[email protected] Web: http://bommaritollc.com/
THANKS!
You can get these slides on my blog – http://bommaritollc.com/blog/.
© Bommarito Consulting