scaling product experimentation

#atlassiansummit

ZANE SALIM • SENIOR TECHNICAL PM • TWITTER • @ZANES

Scaling Product Experimentation

with JIRA and Confluence

Technology is disrupting companies faster than ever

Companies that stop innovating fizzle… fast

How are you innovating?

1. Our approach @twitter

2. Challenges at scale

3. How we scaled

4. What got better

5. Final takeaways

1. Our approach @Twitter

@Twitter we innovate through experimentation

Anyone can launch an

experiment

Unleashes creativity

Drives user insights at scale

100s every quarter

Experimentation is the new black

… and the old black

Both refinement and exploration

Example: which drives more clicks?

Example: which drives more retention?

2. Challenges at scale

Visibility

Learning

Collaboration

Rigor

3. How we scaled

COMMON JIRA WORKFLOW FOR COLLABORATION

Lifecycle of an Experiment

Lifecycle of an Experiment in JIRA

Lifecycle of an Experiment in JIRA

KNOWLEDGE MANAGEMENT IN

CONFLUENCE

Confluence: Knowledge Management

Confluence: Experiment Brief Templates

Confluence: Experiment Brief Templates

Experiment Brief Experiment Write-up Retrospective Analysis

Describe the problem Describe the experiment Present retrospective analysis

Define a hypothesis Present the results Present the results

Design the experiment Offer take-away’s Offer take-away’s

Confluence: Reviewed Experiments & Learning

HYPOTHESIS TREES

Visualizing Focus Areas & Insights

4. What got better

Doubled completed experiments

Reduced cycle time up to 50%

Learning org smarter experiments

5. Final takeaways

Experimentation drives innovation

Collaboration & learning essential

Smart use of tools can make you 4x more effective

Thank you!