using and building open source in google corporate engineering - justin mcwilliams

Open Source inGoogle Corporate Engineering

Justin McWilliams

Corporate ITWorkstations & WiFi

Video Conferencing &Telecommunication

VPN & Remote Access Helpdesk / Tech Support

Servers & Networking

Calendar & Productivity

Mobile Devices Email

Google CorpEng Challenges

CorpEng not only runs, but develops most solutions!

● Video conferencing, complete with fleet management● Mac OS X software deployment client and server● Full disk encryption recovery key escrow solution● Binary whitelist approval workflow system● Employee performance review and promotion system● Interview scheduling, feedback, and hiring decision solution● Financial and expense reporting tools● Compensation review/display● International payroll calculation● Much, much more

Most of the above are built on Google App Engine, and some are open sourced.

Why does CorpEng Open Source?

Why does Google Open Source?

Why does anyone Open Source?

"Open" more than just "Source"

OSS widely used within CorpEng● Need a high-performance, distributed in-memory object caching system?

○ Memcached

● Java date and time classes causing pain?○ JodaTime

● Want a library or framework to help with testing?○ Mockito, EasyMock, Machinist, UnitRecord, FlexMock, pymox

● Developing a web application?○ Django, GWT, Spring, Struts, Closure, jQuery, AngularJS, ...

● Can't write in binary?○ Java, Python, Ruby (on Rails), Go, etc.

What's interesting about these?

AngularJSClosureFlexMock

GinGuiceGSONGWT

Keyczaripaddr-pyoauth2clientpymox

Example OSS use within CorpEng● AutoTest

○ Automated testing and releasing of our customized Ubuntu distribution

● CodeMirror (JavaScript)○ Used to build an internal application dev tool for a browser-based IDE

■ Not Google Collide, which also uses CodeMirror

● FreeSWITCH○ Used to build our custom call center telephony routing system

● Munki & Puppet○ Mac OS X systems administration tools

+ =

Simian

Cauliflower Vest

CorpEng Open Source Solutions

Simian Software mgmt for one of the largest Mac deployments in the world

Cauliflower Vest Enterprise-class FileVault 2 recovery key escrow solution

Ganeti Cluster-based virtualization management software

CloudCourse LMS for enterprise course scheduling and management

CorpEng OSS Releases / ContributionsGORD Platform agnostic RPC daemon for interaction with Microsoft SMS/SCCM

Compliance Management Automated maintenance, analytics and reporting of compliance data

AppReduce* Enterprise software license tracking and employee transparency tool

Contributions to non-Google OSS: Firebug, FreeJ, Objectify, PyActiveResource, PyCups, Rails, ..., Debian, Ubuntu, ...

CorpEng Open Source Release ProcessAkin to Android, not Chromium; develop internally then release large dumps, because....

Links! Questions?CorpEng Open Source Projects

● Simian http://code.google.com/p/simian○ Munki http://code.google.com/p/munki

● Cauliflower Vest http://code.google.com/p/cauliflowervest● Ganeti http://code.google.com/p/ganeti● Compliance Management http://code.google.com/p/compliance-management

Misc● Make Open Easy http://code.google.com/p/make-open-easy

CorpEng Videos & Presentations:● Google on Google, I/O http://goo.gl/ovJ2D● Coding For Cloud, I/O http://goo.gl/nt6Ox● Corp Apps on AppEngine, I/O http://goo.gl/m1XzN● Life@Google IT Teams http://goo.gl/7cYSf