4.16.12 oraclevm3 topten ioug

1 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Top 10 Tips to Accelerate Oracle VM Deployments

Greg King, Senior Best Practices Consultant, Oracle VM Product Management

Honglin Su, Director of Product Management, Oracle VM, Oracle


The following is intended to outline our general product direction. It

is intended for information purposes only, and may not be

incorporated into any contract. It is not a commitment to deliver

any material, code, or functionality, and should not be relied upon

in making purchasing decisions.

The development, release, and timing of any features or

functionality described for Oracle’s products remain at the sole

discretion of Oracle.


Oracle’s Virtualization Strategy Application-Driven Virtualization • Virtualization offers key cost savings benefits

– Server consolidation

– Lower energy, facility and labor costs

• Oracle’s virtualization strategy offers much more

– Most comprehensive desktop to data center product portfolio

– Application-centric: Integrated management of the full hardware and software stack from

servers and storage to the end-user device

– Total solution support

• Virtualization that makes software easier to deploy, easier to manage, and easier to

support

– Not Virtualization for Virtualization’s Sake


Oracle VM Server Virtualization and Management • Oracle VM Manager & Enterprise Manager

– Manage hundreds- or thousands of servers

– Central Java management server

• Web browser-based management console

– Advanced virtualization management including Dynamic Resource

Scheduling (DRS), HA, Dynamic Power Management…

• Oracle VM Server for x86

– Installs on “bare-metal” servers in about a minute

– Guest operating systems:

• Solaris, Linux, & Windows

– Support for paravirtualized- and hardware-virtualized kernels &

drivers


Oracle VM 3

Oracle VM

Servers

Storage Repository

Server

Pool 1

Storage Repository

Server

Pool 2

Top 10 Tips to Accelerate

Oracle VM Deployments


A Roadmap to Success

• Tip 1: Review and understand the roadmap

• Tip 2: Make hardware less stressful

• Tip 3: Plan your platform

• Tip 4: Plan your network

• Tip 5: Plan your storage

• Tip 6: Faster deployments with Oracle VM Templates

• Tip 7: Gather all the required software

• Tip 8: Name your objects for clarity

• Tip 9: Create simple Oracle VM validation guests

• Tip 10: Validate every step of the way


TIP 1: REVIEW AND UNDERSTAND THE ROADMAP Keep from straying off the road…


The Roadmap: pre-implementation

• Planning

– Understand the infrastructure

– Plan your network

– Plan your storage

– Plan your guests

• Preparation

– Prepare your infrastructure

– Prepare your platform

– Validate your platform


The Roadmap: implementation

• Oracle VM Server Pool

– Prepare Oracle VM server pool resources

– Create Oracle VM server pool

– Validate Oracle VM server pool

– Configure Oracle VM Server Update Manager

– Update your Oracle VM Servers

– Import templates and ISO images

– Edit imported templates and ISO images to customize them for your environment

• Oracle VM Validation Guests

– Create Oracle VM validation guests

– Validate Oracle VM validation guests

• Oracle VM Production Guests

– Create Oracle VM “production” guests with basic networking and storage in place

– Validate Oracle VM “production” guests

– Configure databases and applications

– Complete a final validation of entire Oracle VM server pool


TIP 2: MAKE HARDWARE LESS STRESSFUL Make life easy on yourself…


Install hardware in like locations

NET 3SER MGT NET MGT NET 2NET 1NET 0OK

0 1

0PCIe

1PCIe

2PCIe

~AC

OK

~AC

OK


0 1

0PCIe

1PCIe

2PCIe

~AC

OK

~AC

OK


0 1

0PCIe

1PCIe

2PCIe

~AC

OK

~AC

OK

If a quad port card is in PCI

slot 2 here…

Then make sure the other servers

have them in the same slot

• Although Oracle VM can handle inconsistent hardware configurations – humans can’t


Cable I/O & network consistently


0 1

0PCIe

1PCIe

2PCIe

~AC

OK

~AC

OK


0 1

0PCIe

1PCIe

2PCIe

~AC

OK

~AC

OK


0 1

0PCIe

1PCIe

2PCIe

~AC

OK

~AC

OK

If these ports are connected

to 10.11.12.0/24 subnet…

Then make sure these are all

connected to 10.11.12.0/24 also

• Although Oracle VM can handle inconsistent hardware configurations – humans can’t


TIP 3: PLAN YOUR PLATFORM Everything begins with a solid infrastructure…


Servers

mymanager

mypool3 mypool4 mypool1 mypool2

Oracle VM Manager on Bare Metal 1. Install 64-bit Oracle Linux 5.6 + 2. Install Oracle VM Manager

Oracle VM Manager on VM image 1. Create an 64-bit Oracle Linux 5.6+ VM under

Oracle VM 2. Install Oracle VM Manager on VM image (Refer to Installation and Upgrade Guide for details)

Oracle VM Servers Oracle VM Servers Oracle VM Manager

PS

TOP

10 2

345

6 7DVD

Sun Fire X4170

OK

OK OK OK

OKOKOK OK OK


Other servers you will need…

mymanager

Other uses for the management server 1. You will need a YUM repo for Oracle VM server update manager (patches from here) 2. You will need a repository for ISOs and VM templates (import templates from here) 3. You will need a SE or EE database if more than a few Oracle VM guests

Oracle VM management server

PS

TOP

10 2

345

6 7DVD

Sun Fire X4170

OK

OK OK OK

OKOKOK OK OK

SE or EE database YUM repo ISO/Template repo

Any other servers in your environment Or Use

Oracle VM Manager


TIP 4: PLAN YOUR NETWORK Take time to design…


Assign IPs for Oracle VM Server Pool (simple network)

Host Name OVM Network IP Netmask Gateway Description

mymanager Public 10.11.12.10 255.255.255.0 10.11.12.1 Public Oracle VM Manager

mypool1 Public 10.11.12.100 255.255.255.0 10.11.12.1 Public Server Pool VIP (master server)

myserver1 Public 10.11.12.101 255.255.255.0 10.11.12.1 Public access to myserver1 (OVM mgmt)

myserver1-ovm OVM

management

10.11.13.101 255.255.255.0 Private network OCFS2 heartbeat, live migration and storage


myserver2-ovm OVM

management

10.11.13.102 255.255.255.0 Private network OCFS2 heartbeat, live migration and storage


Assign IPs for Oracle VM Server Pool (a more complex network)


mymanager Public 10.11.12.10 255.255.255.0 10.11.12.1 Public Oracle VM Manager

mypool1 Public 10.11.12.100 255.255.255.0 10.11.12.1 Public Server Pool VIP (master server)


myserver1-ovm OVM heartbeat 10.11.13.101 255.255.255.0 Private network OCFS2 heartbeat

myserver1-mig OVM migration 10.11.14.101 255.255.255.0 Private network for OVM live migration

myserver1-nfs OVM storage 10.11.15.101 255.255.255.0 Private admin scripts, tools & backups (pool FS & repositories if using NFS)


myserver2-ovm OVM heartbeat 10.11.13.102 255.255.255.0 Private network OCFS2 heartbeat

myserver2-mig OVM migration 10.11.14.102 255.255.255.0 Private network for OVM live migration

myserver2-nfs OVM storage 10.11.15.102 255.255.255.0 Private admin scripts, tools & backups (pool FS & repositories if using NFS)


Assign IPs for Oracle VM Guests

- a more complex network with Real Application Clusters (RAC)

guests


myrac1 Public 10.11.12.122 255.255.255.0 10.11.12.254 Public IP for RAC cluster/SCAN

validation1 Public 10.11.12.133 255.255.255.0 10.11.12.254 Public IP for validation guest

validation1-vip Public 10.11.12.233 255.255.255.0 10.11.12.254 Public VIP for validation guest

validation1-rac RAC management 10.11.15.133 255.255.255.0 Private IP for validation guest

validation2 Public 10.11.12.134 255.255.255.0 10.11.12.254 Public IP for validation guest

validation2-vip Public 10.11.12.234 255.255.255.0 10.11.12.254 Public VIP for validation guest

validation2-rac RAC management 10.11.15.134 255.255.255.0 Private IP for validation guest

myguest1 Public 10.11.12.131 255.255.255.0 10.11.12.254 Public IP for RAC node 1

myguest1-vip Public 10.11.12.231 255.255.255.0 10.11.12.254 Public VIP for RAC node 1

myguest1-rac RAC management 10.11.15.131 255.255.255.0 Private IP for RAC node 1

myguest2 Public 10.11.12.132 255.255.255.0 10.11.12.254 Public IP for RAC node 2

myguest2-vip Public 10.11.12.232 255.255.255.0 10.11.12.254 Public VIP for RAC node 2

myguest2-rac RAC management 10.11.15.132 255.255.255.0 Private IP for RAC node 2


TIP 5: PLAN YOUR STORAGE Take time to design…


Server pool file system and repositories

mypool4

mypool3

mypool2

mypool1

Oracle VM Server Pools Storage

NFS, iSCSI or FCP mypool4 only




Pool FS mypool4

Repository mypool4

Pool FS mypool3

Repository mypool3

Pool FS mypool2

Repository mypool2

Pool FS mypool1

Repository mypool1


Server pool shared repositories

mypool4 Pool FS mypool4

Repository mypool4

Repository shared1


Repository mypool3

Repository shared1


Repository mypool2

Repository shared1


Repository mypool1

Repository shared1

NFS Templates/ISOs for all server pools

Oracle VM Server Pools Storage





LUN numbering/naming for Oracle VM Guests

Description Qty LUN Size Unit Number Presentation OVM Disk Name

OVM server myserver1

Local SAN boot disk

1 9g 0 myserver1 NA


Local SAN boot disk

1 9g 0 myserver2 NA


Local crash dump disk

1 100g 1 myserver1 NA




Server pool

Shared Pool File System

1 20g 2 myserver1

myserver2

OVM3SP PoolFS

Server pool

Shared Guest repository

1 500g 3 myserver1

myserver2

OVM3SP Repo1

RAC guests myguest1 & myguest2

Shared ASM disks for PeopleSoft data

5 75g 51,52,53,54,55 myserver1

myserver2

MYRAC1-DG1-PSoft1

MYRAC1-DG1-PSoft2

MYRAC1-DG1-PSoft3

MYRAC1-DG1-PSoft4

MYRAC1-DG1-PSoft5


Shared ASM disks for Flash Recovery

4 75g 61,62,63,64 myserver1

myserver2

MYRAC1-DG2-Flsh1

MYRAC1-DG2-Flsh2

MYRAC1-DG2-Flsh3

MYRAC1-DG2-Flsh4


TIP 6: FASTER DEPLOYMENTS WITH ORACLE VM

TEMPLATES

A most important aid for stable deployments …


Oracle VM Templates

• Templates are pre-configured virtual machines (VMs)

containing software products that are ready-to-go

• Download Oracle VM Templates for many Oracle products

– Oracle Applications (EBS, JD Edwards, PeopleSoft, Siebel, etc.)

– Oracle Middleware

– Oracle Database & Real Application Clusters (RAC)

– Oracle Solaris

– Oracle Linux


Oracle VM Templates are quick

• Just import a template into the Oracle VM Manager

• Customize the template

• Create a Oracle VM Guest from the template

• Start the Oracle VM Guest

• Answer a few questions about networking

• Log into the Oracle VM Guest and start working


An example of using templates…


Create an Oracle VM Guest…


The Oracle VM Guest is created…


Customize the Oracle VM Guest…


Start the Oracle VM Guest…


Answer a few questions…


TIP 7: GATHER ALL THE REQUIRED SOFTWARE Get everything you need…


Make sure you have everything…

Description Download URL Purpose

Download Oracle Linux 5.7 https://edelivery.oracle.com/oraclevm under the “Oracle Linux Release 5 Update 7

Media Pack for x86_64 (64 bit)” link

This will be used to install Oracle Linux onto a bare

metal server or VM guest where the Oracle VM

manger application will eventually be installed (DO

NOT install on a server or VM guest that will be part

of your server pool)

Download “Oracle VM Manager 3.0.3” installer ISO https://edelivery.oracle.com/oraclevm This file will be copied to the /tmp directory of the

above server or VM guest you installed Oracle

Linux 5.6

Download “Oracle VM Server 3.0.3 for x86_64 (64

bit)” ISO

https://edelivery.oracle.com/oraclevm This will be used to install Oracle VM Server onto

bare metal servers that will be part of your server

pool

Download “Oracle Linux 5 Update 6 template - PV

x86_64 (64 bit)”

https://edelivery.oracle.com/oraclevm under the “Oracle VM Templates for Oracle

Enterprise Linux 5 Media Pack for x86_64 (64 bit)” link

Upload to the document root of a web server or

anonymous FTP server within your own intranet.

This will be used to create the Oracle VM guests

used for testing.

Download Oracle RAC VM Template 11.2.0.3 (two

parts)

https://edelivery.oracle.com/oraclevm under the “Oracle VM Templates for Oracle RAC

11gR2 Media Pack for x86_64 (64 bit)” link

Upload to the document root of a web server or

anonymous FTP server within your own intranet.

This will be used to create the Oracle VM guests

myguest1 & myguest2.

https://edelivery.oracle.com/oraclevm








TIP 8: NAME YOUR OBJECTS FOR CLARITY Leave yourself clues…


Name your networks so they are obvious…


Edit templates after import to customize


Use LUN naming/number to advantage…

Description Qty LUN Size Unit Number Presentation OVM Disk Name


Local SAN boot disk

1 9g 0 myserver1 NA


Local SAN boot disk

1 9g 0 myserver2 NA







Server pool

Shared Pool File System

1 20g 2 myserver1

myserver2

OVM3SP PoolFS

Server pool

Shared Guest repository

1 500g 3 myserver1

myserver2

OVM3SP Repo1


Shared ASM disks for PeopleSoft data

5 75g 51,52,53,54,55 myserver1

myserver2

MYRAC1-DG1-PSoft1

MYRAC1-DG1-PSoft2

MYRAC1-DG1-PSoft3

MYRAC1-DG1-PSoft4

MYRAC1-DG1-PSoft5


Shared ASM disks for Flash Recovery

4 75g 61,62,63,64 myserver1

myserver2

MYRAC1-DG2-Flsh1

MYRAC1-DG2-Flsh2

MYRAC1-DG2-Flsh3

MYRAC1-DG2-Flsh4


An example for renaming disks… mystorage> lun show -m

LUN path Mapped to LUN ID Protocol

-------------------------------------------------------

/vol/mypool1/myrac1-dg1-psoft1 myserver1f 51 FCP

myserver2f 51 FCP


myserver2f 52 FCP


myserver2f 53 FCP


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f

[root@MyServer1 ~]# multipath –ll /dev/mapper/360a9800056724433565a6948696e4873 360a9800056724433565a6948696e4873 dm-0 NETAPP,LUN

size=12G features='1 queue_if_no_path' hwhandler='0' wp=rw

`-+- policy='round-robin 0' prio=4 status=active

`- 0:0:0:51 sdc 8:32 active ready running


[root@MyServer1 ~]#


Find LUN number using WWID on server… mystorage> lun show -m


-------------------------------------------------------


myserver2f 51 FCP


myserver2f 52 FCP


myserver2f 53 FCP


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f






[root@MyServer1 ~]#


Find corresponding LUN on storage… mystorage> lun show -m


-------------------------------------------------------


myserver2f 51 FCP


myserver2f 52 FCP


myserver2f 53 FCP


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f






[root@MyServer1 ~]#


Use the name assigned to LUN on storage… mystorage> lun show -m


-------------------------------------------------------


myserver2f 51 FCP


myserver2f 52 FCP


myserver2f 53 FCP


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f






[root@MyServer1 ~]#







[root@MyServer1 ~]#

Rename the disk in Oracle VM Manager mystorage> lun show -m


-------------------------------------------------------


myserver2f 51 FCP


myserver2f 52 FCP


myserver2f 53 FCP


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


myserver2f


TIP 9: CREATE ORACLE VM VALIDATION GUESTS Don’t start creating your production guests yet…


An Oracle VM validation guest…

• Use an out-of-the-box Oracle VM Template

• Use an Oracle VM Template with an OS you are very familiar

• Configure all networking your actual guests will have

• Configure all storage your actual guests will have

• No other custom changes or user accounts – keep it simple!!

• No databases or applications

• Use to test live migration, network and storage connectivity

• Delete these “validation” guests just before going production


TIP 10: VALIDATE EVERY STEP OF THE WAY The key to ensuring success…


Validate the platform

Test Name Purpose/Expected Result Test

Manager ping test Confirm that the Oracle VM manager can be pinged

from each Oracle VM server. Use the hostname,

not IP address to ensure hostnames resolve

correctly.

From each Oracle VM server:

ping <ovm manager>

Server ping test Confirm that the Oracle VM manager can ping each

Oracle VM server. Use the hostnames, not IP

addresses to ensure hostnames resolve correctly.

From the Oracle VM management server:

ping <ovm server>

Agent password test Confirm that the ovs-agent password is the same on

each Oracle VM server.

Log into each Oracle VM server as root as run the following commands…

Preexisting OCFS2 test Only needed when Fibre Channel disks are

presented to the Oracle VM Servers and only if the

server is not being added to an existing server pool

There should not be any OCFS2 file systems from

previous clusters or installation attempts.

Log into each server as root and run the following command…

Core dump test Confirm that crash dumps are enabled and working Log into each server as root and run the following command…


Validate the Oracle VM server pool


Pool FS verification Verify that the Pool File System is mounted to each

Oracle VM server

Log into each Oracle VM server as root and run the following command…

OVM VIP ping test You should be able to ping the Oracle VM virtual IP

for the master server role from the Oracle VM

manager. This is critical since this is the only means

for Oracle VM Manager to communicate with the

Oracle VM servers.

Log onto the Oracle VM management server as root and run the following…

VLAN ping test You should be able to ping the hostname/IPs

assigned to each VLAN segment that was used to

create an Oracle VM network within the Oracle VM

Manager

Log into myserver1 as root and ping the following hostnames…

Bond failover test Ensure that the second interface associated with

bond0 continues network connectivity in the event

that the first interface fails.

This test scenario assumes eth0 and eth2 are slaves

for bond0 – use appropriate interface names for your

environment.

Log into a server that has an hostname/IP on the same subnet being tested on

bond0 of each Oracle VM server. Run a continuous ping on the hostname/IP...


Validate the Oracle VM guests


Live migration test This ensures that the validation guests live migrate without

interruption of connection and verifies all networking and

storage is

Log into a validation guest and start a continuous ping between the Oracle VM

Manager and the validation guest. Monitor the ping results to ensure there

are...

Maintenance test This ensures Oracle VM guests are live migrated from a

Oracle VM Server once the server has been put into

maintenance mode

All of the validation guests should migrate to other Oracle VM

Servers in the pool and then remain running after all the

guests have been migrated

Put a single Oracle VM Server into maintenance mode using the Oracle VM

Manager.

Crash recovery test Initiate a kernel panic on each Oracle VM server. This will

validate that Oracle VM guests will automatically restart on

another Oracle VM server (not migrate) and that the Oracle

VM server pool will recover.

It should take about one to two minutes for failed Oracle VM

guests to start on other servers in the pool.

Keep in mind that the guests will stop running on the failed

server and you will lose access to the guests – this is

expected.

Reset all Oracle VM guests back to the Oracle VM servers they were originally

running. Then log into the first server as root and execute the following…

VNC console test This should already have been completed when you

configured the Oracle VM test guests. The console is needed

No further testing necessary


An example for testing bond failover


Ping myserver1 from myserver2… [root@myserver2 ~]# ping myserver1

PING myserver1m.ovmsand.oracle.com (10.11.12.101) 56(84) bytes of data.

64 bytes from myserver1m (10.11.12.101): icmp_seq=1 ttl=64 time=0.169 ms























Get current active device on myserver1…

[root@myserver1 ~]# cat /proc/net/bonding/bond0

Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)

Bonding Mode: fault-tolerance (active-backup)

Primary Slave: eth0 (primary_reselect always)

Currently Active Slave: eth0

MII Status: up

MII Polling Interval (ms): 250

Up Delay (ms): 500

Down Delay (ms): 500

Slave Interface: eth2

MII Status: up

Link Failure Count: 0

Permanent HW addr: 00:04:23:bf:62:96

Slave queue ID: 0


MII Status: up


Permanent HW addr: 00:14:22:16:65:5a

Slave queue ID: 0

[root@myserver1 ~]#


Bring down eth0…

[root@myserver1 ~]# ifdown eth0

[root@myserver1 ~]#


Get new current active device on myserver1…

[root@myserver1 ~]# cat /proc/net/bonding/bond0

Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)

Bonding Mode: fault-tolerance (active-backup)

Primary Slave: None

Currently Active Slave: eth2

MII Status: up

MII Polling Interval (ms): 250

Up Delay (ms): 500

Down Delay (ms): 500


MII Status: up


Permanent HW addr: 00:04:23:bf:62:96

Slave queue ID: 0

[root@myserver1 ~]#


Ensure ping is still going…


























Bring up eth0 and cancel ping…

[root@myserver1 ~]# ifup eth0

[root@myserver1 ~]#


ORACLE INFORMATION RESOURCES Get more help…


Learn More

• Oracle Virtualization: http://oracle.com/virtualization

• Oracle VM OTN Page:

– http://www.oracle.com/technetwork/server-storage/vm/overview/index.html

• Oracle VM Wiki, blogs and forum:

– Oracle VM Wikis: https://wikis.oracle.com/display/oraclevm/Home

– OTN Forum: https://forums.oracle.com/forums/forum.jspa?forumID=823

– Oracle’s Virtualization Blog: http://blogs.oracle.com/virtualization

• Social Media

– Twitter: http://twitter.com/orcl_virtualize

– Facebook: http://www.facebook.com/OracleVirtualization

http://oracle.com/virtualization

http://www.oracle.com/technetwork/server-storage/vm/overview/index.html




https://wikis.oracle.com/display/oraclevm/Home

https://forums.oracle.com/forums/forum.jspa?forumID=823



http://blogs.oracle.com/virtualization

http://blogs.oracle.com/virtualization

http://twitter.com/orcl_virtualize

http://www.facebook.com/OracleVirtualization


Q&A

4.16.12 oraclevm3 topten ioug

Documents