self-service virtual environments at deutsche telekom ... ?· self-service virtual environments at...

Download Self-service virtual environments at Deutsche Telekom ... ?· Self-service virtual environments at Deutsche…

Post on 13-Sep-2018




0 download

Embed Size (px)


  • Self-service virtual environments at DeutscheTelekom based on OpenStack

    Alexander StellwagDeutsche Telekom AGProducts & Innovation

  • Agenda Introduction Short History Of Wolke-7 Planning and Setup Virtual Private Environments Version 2: havana and neutron Lessons Learned Q&A

  • Introduction Alex Stellwag 43 Years Started at Deutsche Telekom

    in 2001 Cloud Architect since 2011 OpenStack since Diablo

  • Short History of Wolke-7 Started back in late 2011 after major reorganization to gain knowledge of

    OpenStack Dec 2012: initiated two basic use cases

    Plain IaaS, configurable via EC2-API Fully managed environments (PaaS)

    Both available from one cloud environment Internal customers only, no public services Core principles

    Stay open source Build over partnering

  • Intranet Infrastructure Supermicro servers (2UTwin and 2U 12 Disk) Ubuntu 12.04 LTS (Kernel 3.2 and 3.11) OpenStack packages from UCA Ceph packages from Inktank I/S automation via FAI1 and puppet Monitoring via nagios / check_mk


  • Some Figures 12 compute nodes with 2x Xeon X5650 (6 cores, 2.67 GHz,

    Intel HT enabled), 96 GB Ram, 2x 250 GB SSD each 3 Ceph nodes with 2x Xeon E5-2630, 64 GB Ram, 2x 240 GB

    SSD + 10x 2TB S-ATA (OSDs) each. ==> ~60 TB net 2x Cisco Nexus 5548 for tenant VLANs and floating IPs 2x Cisco 48 port 1 GE for administration

  • Planning and Setup Setting up OpenStack is quite painless thanks to the official docs We needed a complete DC

    Location Connectivity Basic I/S services (DNS, Mail, Proxy) Automation services (FAI, puppet)

    Complex Setup due to internal security requirements Adapting the puppet modules for our needs took a while

  • Virtual Private Environments I Simple applications (apache, mysql, tomcat)

    Provided by user-data Published on project website User has to copy & paste them into dashboard Designed to fulfill security requirements Full self-service via OpenStack dashboard Users have full control and responsibility

  • #cloud-config

    # this cloud config code installs apache2-worker webserver# and changes several configuration directives to already meet# some GIS requirements# after customization of the webserver the requirements have to be# checked agein

    # supported OS: Ubuntu 12.04

    # addtional packages to be installedpackages: - apache2 - libapache-mod-security

    # commands to modify the default installationruncmd: - a2dismod cgid # req 7 - sed -i 's/\(.*Alias \/\(cgi-bin\|doc\)\)/#\1/; //{s/^/#/}; //,/Options /{s/\(.*\n\t\t\tdeny from all\n\t\t \ /}' /etc/apache2/sites-available/default # req 9,11,12,13,16,18 - sed -i 's/^/#/' /etc/apache2/mods-available/alias.conf # req 16 - sed -i 's/^\s*ServerTokens .*/ServerTokens Prod/' /etc/apache2/conf.d/security # req 19 - service apache2 restart # restart webserver to apply changes

    Virtual Private Environments II

  • Virtual Private Environments III Complex environments (currently CI/CD for Java only)

    Jenkins, git, tomcat Set up via bash and python scripts Not yet integrated in dashboard, ordered via e-mail Fully managed by Test & Development team

  • Version 2: havana & neutron Started in 10/2013 New location, so again we needed to setup a complete DC Ready for production in May 2014 Currently looking for dedicated operations team Heat installed and enabled but Virtual Environments not ported

    over yet. Users may use their own stack templates

  • Infrastructure Changes No dedicated network node, all agents distributed amongst

    compute nodes Multiple L3 and DHCP agents per L3 network Custom python script to check availability of agents and

    reschedule if necessary Works, but we have no long-time experience Currently looking into Icehouse, but migration has issues

    (mainly OS-wise)

  • New Hardware 11 compute nodes with 2x Xeon E5-2660 v2 (10 cores, 2.2

    GHz, Intel HT enabled), 128 GB Ram, 2x 400 GB SSD each 8 Ceph nodes with 2x Xeon E5-2620 v2 (6 cores, 2.1 GHz, Intel

    HT enabled), 32 GB Ram, 2x 100 GB SSD + 10x 4TB S-ATA (OSDs) each. ==> ~312 TB net

    2x Alcatel Lucent OS 6900-T40 for VxLAN tunnels and floating IPs

    2x Alcatel Lucent OS 6850 for administration

  • Lessons Learned

    I/S deployment is hard even if you start on a green field Deploying OpenStack without the ability to fix the code is

    difficult (and sometimes frustrating) HA for neutron is a complex beast Deploying multi-tier applications without heat is no fun

  • Lessons Learned II

    Manage internal stakeholder for proper funding Many customers don't know the difference between

    virtualization and cloud computing Change management program needed because cloud impacts

    95% of operations roles For large and inhomogeneous teams SCRUM is not the right

    project management method

  • Q & A

    Questions ?

    Folie 1Folie 2Folie 3Folie 4Folie 5Folie 6Folie 7Folie 8Folie 9Folie 10Folie 11Folie 12Folie 13Folie 14Folie 15Folie 16