xen summit 2010 - globo.com

case studyIaaS using XCP and XAPI

marco sinhorelivirtualization project leader

Monday, May 17, 2010

• Internet branch of Globo organizations

• Biggest media conglomerate in Latin America and 4th around the world

• Largest vertical web portals in Brazil

• More than 150 million page views per day

• Average of 27 million unique visitors in February 2010

• More than 1500 physical servers in the production datacenter

• More than 60 Gbps internet link

• Considered benchmark for high-volume web distribution

• Responsible for the largest simultaneous video streaming audiences in the country

about globo.com


• Automate existing manual workflow between teams’ processes

• Keep existing automated systems used for infrastructure setup

• Identify new infrastructure processes to be automated

• The IaaS needs to fit the company and not the reverse

making decisions


new infrastructure processes

Switches

Virtual MachinesPhysical Servers

LoadBalancers

Monitoring

NetworkAPIcore

Core applicationXAPI

REST

REST

Deploy

All infrastructure processes are integrated in a single core application responsible for the flux organization. Each process has an API controlling its physical/logical/virtual resources.


• Relationship between server and switch ports

• needed for adding tagged vlan into switch port trunk when a VM is installed or started

• Switch communication by SNMP or SSH

• check/add/remove/list vlans in switch ports

• Load-balancer communication by API or SNMP or SSH

• management of virtual IPs

• Users profiles

• implement policies for human or system users

networkAPI


• The monitoring is a part of the VM life-cycle

• when the VM is started all information about the VM is included in the monitoring system and then enabled

• when the VM is stopped or is tagged as “maintenance”, the monitoring is disabled

• if the VM is removed, the VM monitoring is removed as well

monitoring


• Used for internal application deployment

• Install all dependencies needed in OS

• Setup the virtual machine for applications

• Policies for users and groups

• Dev/QA/Staging/Production environment

deployment


• Pooling

• One place to get status and setup all pool virtual resources

• Easy API

• Fast to program and maintain

• Many client bindings for many flavors

• Paravirtualization

• Large and devoted community

motivation to use XCP


• Django application running on Apache

• Users access front-end and authentication

• Unifies all others IaaS components

• Defines all workflow and system interactions

IaaS front-end component


• Django application running on Apache

• VNC console proxy encapsulated with token to identify client

• Translates REST calls from front-end application to native xapi calls

• All calls to XCP pools are only natively accessible from the integration component

IaaS xapi integration component


• Hosts

• PXE boot

• answerfile for unattended installation

• rewrite /etc/firstboot.d files to not change the static IPs

• VM

• kickstart for unattended installation integrated into system

• static IP configured by system inserting information on Linux kernel cmdline

host and VM installation


system view (traditional)


system view (logical)


benefits of this new approach


• Defines the workflow between teams

• Integrated with ticket system to request an human action

• Keeps the same flux previously used when the actions weren’t automated

• Checklist of each item to release the resources

workflow


user profile

• Each profile has a group of actions in the system

• Some profiles inherit group actions of others


user profile


user profile

Project administrator


user profile


roadmap• Deployment (Integrate with the IaaS core to tag the deployment in projects)

• DNS (Manage the DNS using an API)

• Monitoring (rich details)

• XCP

• Permit many VMs to use the same shared storage block in web farms

• Master recovery

• Virtual Machine HA


thanks

Questions?


xen summit 2010 - globo.com

Technology

monitoring system

vm monitoring

ticket system

system interactionsmonday

system usersmonday

frontend application

iaas core

system static ip