devops goes mobile - jax 2014 - jesper richter-reichhelm

Jesper Richter-Reichhelm (@jrirei) | Wooga

DevOps goes Mobile

November 2013

!? … ……

!

November 2013

…!?

…… …

…

“In recognition of your incredible efforts and achievements, I’m happy to announce that we’re extending the Thanksgiving holiday this year.”

- Tim Cook

November 2013

!?… … …

… ……

Crashing 15%

Not affected 50%

Old version 35%

>200,000 users affected

November 2013 (on web)

• 5 hours crashing, not 5 days! • 9 days live earlier

• 2009 focus on Facebook • 2011 first iOS game • 2013 mobile first strategy

Wooga

“You build it, you run it. - Werner Vogels

• Agile admins • Faster releases • Virtualization • Automation tools

Some say DevOps is about

• Big Picture Mindset • Learning together • Reality Feedback • => Collaboration

I say it’s about

Mobile is differentespecially for apps

DevOps is different

for Mobile

Problems & Solutions

Mobile Network

• use async. communication • use compression • batch of requests • monitor network differences

Unreliable network

• queue monitoring calls locally • queue server updates locally • store state local

Offline mode

Runs on a Device

WORD

• consistency problems • merge/conflict handling • versioning of data • esp. tricky when allowing usage of

multiple devices

Local storage

• Different mindsets of mobile devs • Reproduce error vs. log lookup

• Using an error analytics service • not only crashes • must work offline! • no OS related crashes • just memory warning

Log file access

Deploy w/o Control

Jenkins

• Live with it • Use online config • AB testing • balancing changes • careful of what is allowed • use local storage to cache

Apple Test Cycle

• Only pulls • Users can deactivate

No push updates

0,0%

20,0%

40,0%

60,0%

80,0%

4.4.2 4.4.1 4.4 4.3.x 4.2.x 4.1.x 4.0.x 3.x 2.x 1.x

5% 3%

0,0%

20,0%

40,0%

60,0%

80,0%

2014 ... ... ... ... ... ... 2013 2012 2011

5% 3%

• Only pulls • Users can deactivate • Or force users to update in app • check server for newer version • Online config helps a lot

No push updates

WORD

Canary Testing

Wooga Solutions

Cross Platform

Test Pool

• No manual builds • Keep dSYM file • Jailbreaked iPhone • Copy live to staging

Lessons learned

WORDSimple Backend Services

Key Value Store

Config Service

AB Test Service

• It’s hell - live with it • form factor, OS version

• Restrict devices (camera, min. OS) • cross platform dev. • many local devices • Apple Enterprise acc.

• complex tool chain

• It’s hell - live with it • form factor, OS version

• Restrict devices (camera, min. OS) • cross platform dev. • many local devices • Apple Enterprise acc.

• complex tool chain

Error Analytics

0

25.000

50.000

75.000

100.000

Users Errors

Version 1 Version 2

0

1.250

2.500

3.750

5.000

Users Errors

Version 1 Version 2

0%

2,5%

5%

7,5%

10%

Users Errors Affected Users

Version 1 Version 2

SDK

Wrap up

Mobile is differentespecially for apps

DevOps is different

for Mobile

Dev + Ops is needed

Continuous Delivery is good

Be able to react

always

Questions?@jrirei

http://wooga.com/jobs