Download - AMS-IX provision system
AMS-IX provision system
Maksym Tulyuk, [email protected] event, Amsterdam, 22nd May 2014
Agenda
1. Definition of the provision system
2. Evolution of the provision system
3. Achievements and plans
Definition of provision system
Maksym Tulyuk, [email protected] event, Amsterdam, 22nd May 2014
Provision system as Black Box
Customerrequests a link
AMS-IX provides access to a
VLAN
AMS-IXprovisio
nsystem
• link is a physical port(s) or rate-limit on a partner’s port• in 95% cases VLAN is Internet Peering VLAN
Inside The Black Box
reservation
assignment
testing
enabling
customer’s request
signed documents
link is up
router is ready
Deeply inside The Black Box
Complexity inside each step!
Evolution of provision system
Maksym Tulyuk, [email protected] event, Amsterdam, 22nd May 2014
Provision 4 years ago• customer: sends email to MR team
• MR: copy&paste information into my.ams-ix and RT, and sends email to NOC
• NOC: copy&paste to browser to find free port
• NOC: copy&paste to XML (internal database)
• NOC: copy&paste to console to configure port
• NOC: copy&paste to email to send reserved port to MR team
• MR: copy&paste into ...
“Issues” with old systems
• each engineer did it in his/her own way
• a NOC engineer made a lot of errors
• it took a lot of time and energy
• it required a lot of the engineers
Improvements
• Internal procedures
• Scripts
• Email templates in my.ams-ix.net
Procedures
• Procedure = best practice
• Write down existing processes
• Constantly check and improve them
Scripts
• “One step" scripts: change configuration, find free resources (port, IP address), etc
• “Do-it-all” scripts: replacement of procedures
• Require careful thinking
• Have a lot of checks
Email templates
• Replacement of copy&paste operations
• Integration with my.ams-ix and our database
• Flexibility via simple programming language
Historical perspective
• Write down best practices as procedures
• Replace the procedures with scripts
• Generate emails via templates
Achievements and plans
Maksym Tulyuk, [email protected] event, Amsterdam, 22nd May 2014
Fixing the “issues”
• his/her own way
• a lot of errors
• a lot of time and energy
• a lot of the engineers
• standardisation
• no errors
• minimum and predictive time
• no engineers involved
Scalability
• Four new sites in Amsterdam: Equinix 3, EvoSwitch, Telecity 5, Terremark
• Four new exchanges: CW, EA, HK, NY
Some numbers
Achievements
• Free up engineers
• Speed up provision
• Scale up for new sites and exchanges
Plans
• customer can add, remove and change mac addresses
• customer can configure BGP sessions with route-servers and our router
• partners can modify virtual links
Even more ideas
• automatic testing and enabling
• automatic addition of mac addresses, BGP sessions, IPv6 addresses, etc
• automatic operations (show logs, port status, etc)
Summary
• Simple concept of provision: request from customer -> link from AMS-IX
• From procedures to scripts
• Automatic emails and processes
• Scalability and growth
Big thanks!
• m6_provision: Martin Pels
• my.ams-ix: Martin Lyden, Petros and Peter provision scripts: Gloria
• XML: Steven
• NOC and MR teams: feedback