engineering career ladders as cultural manifesto // flatiron health (firstmark's code driven)

© 2016 Flatiron Health, Inc.

Engineering Ladders as Your Cultural Manifesto

Code Driven, Feb 18th 2016

Gil ShklarskiVP of Technology [email protected]

Maayan RothEngineering [email protected]

mailto:[email protected]





who am i

Used to optimize and scale

Now I optimize and scale


Flatiron Health

250+ employees including:14 Medical oncologists and

oncology nurses6 Practice administrators3 Clinical oncology pharmacists

80+ Engineers

Key stats:Founded: 2012Fundraising to date: $313MM

Our Mission: To serve cancer patients and our customers by dramatically improving

treatment and accelerating research

Our client Network: 230 Cancer Clinics2,000 Clinicians 1M Active cancer patients


Flatiron Health

Largest real-world oncology data source


Today

High growth startupsEngineering cultureCareer ladders

- Mechanics- Lessons from Flatiron Health


High growth startups

What your friends

think you do

What your mother

thinks you do

What society

thinks you do

What your clients

think you do

What your friends

think you do

What I think I do


Engineering @ High growth startups

- (a lot of) opportunities for individuals to step up- opportunities to decide how engineers do things- opportunities to decide what we value in

engineering as a team- opportunities to decide what we value about an

engineer’s contribution

Now, realistically, when you grow quickly: ‘s/opportunities to decide/lack of clarity about/g’



t

n_hired_engineers



t

n_engineers


Engineering Culture

How engineers do things:

- How we make things- How we operate things- How we function as a

team

“How we engineer”


Formalizing Engineering Culture

Source: First Round Review, “How I Structured Engineering Teams at LinkedIn and AdMob for Success”, Kevin Scott (SVP engineering and operations at LinkedIn)



Formalizing an engineering culture is important beyond the value it provides to the engineering team.

It helps engineers explain how we are going to help the company win.

Otherwise, - Non leverageable feature work - Unsustainable architectures. - Inevitable team burnout.



2.5 years and 30 software engineers in...

Time to talk about the careers of our engineers- Changing the world, business path, technological

curiosity is a lot- Not enough, though - need to define what is a

promotion

Career Ladder


A career ladder

- Articulation of a career path- Career path is divided into levels you climb

(‘ladder’)- Specification of a set of desired engineering

behaviors we expect at each level - The same behaviors (‘swim lanes’) are present at

each level, with progression by levels- Separate ladders exist for individual contributors

and people managers


A career ladder

Level 1 2 3 4 5 6

Swim Lane 1

Swim Lane 2

Swim Lane 3


We haven’t invented the concept

Our ladder is inspired by ideas our engineers learned from:

but we customized the contents to our engineering challenges.

Special thanks to Camille Fournier, sharing ‘s ladder in their blog. It is inspired by ‘s ladder...


This sounds so formal, do we need this?

A definition for career growth:- Personal guide- Meaningful discussions / feedback - Clear expectations setting

Clarity → Accountability

Consistency, fairness and clarity in:- Performance standard- Compensation (formulaic compensation FTW)- Titles if exist


Flatiron’s Engineering IC Career Ladder

Specification of a set of desired engineering behaviors we expect at each level

Ladder competencies are split into 3 swimlanes:- TS: Technical Skill, Knowledge, and Expertise- GSD: Impact, Reliability, and Execution- C: Contributions to Eng and Flatiron Culture

This covers an important chunk of ‘how we engineer’.


Ladder sub-categories:TS1: Coding and testing

TS2: Architecture and design

TS3: Production and deployment

Manifesto items covered:- How we make things: SCM, Testing, Code Review, ...- How we operate things: On-call, deployment, access to

production- How we Function as a team: Collaboration, high bus factor

Technical Skill, Knowledge, and Expertise (TS)Technical Skill, Knowledge, and Expertise (TS)


TS / Coding and Testing: Sample Rubrics

E3 E4TS1: Writes correct, well-tested, and clean code with minimal guidance.

Consistently follows best practices within minimal prompting through code reviews. Can design and execute on more complex testing independently, such as integration testing.

TS1: Leaves code and tests in substantially better shape than before. Contributes to overall quality and stability of Flatiron codebase.

Enforces best practices in code reviews. May contribute to creating or updating style guides and establishing best practices.

TS2: [Design and architecture] TS2: [Design and architecture]

TS3: [Production and deployment] TS3: [Production and deployment]


TS/ Design and Architecture: Sample Rubrics

E3 E4TS1: [Coding and testing] TS1: [Coding and testing]

TS2: Makes small design decisions independently, making appropriate tradeoffs between simplicity/readability and performance.

TS2: Code and designs avoid premature optimizations, and are extensible and reusable. Provides detailed technical feedback on the designs of others. Has a comprehensive understanding of a substantial component of the Flatiron technology ecosystem.

TS3: [Production and deployment] TS3: [Production and deployment]


TS / Production and Deployment: Sample Rubrics

E3 E4TS1: [Coding and testing] TS1: [Coding and testing]

TS2: [Design and architecture] TS2: [Design and architecture]

TS3: Trusted to participate in on-call/Tier 3 support rotations and/or deploy code to production with minimal guidance and supervision. Capable of debugging production bugs with help.

TS3: Trusted to participate in on-call/Tier 3 support rotations and/or deploy code to production, or modify production databases, independently.


Ladder sub-categories:GSD1: Execution and delivery, independence

GSD2: Planning and estimation

GSD3: Technical communication

Manifesto items covered:- How we make things: Deliver incremental value, ...- How we operate things: Access to production, ...- How we Function as a team: Documentation, ...

Impact, Reliability, and Execution (GSD)


GSD / Execution and Delivery, Independence: Sample Rubrics

E3 E4GSD1: Delivers on larger features within an existing product or pipeline, which may be vaguely scoped or designed, with minimal guidance. Knows when and who to ask for help in order to get themselves unblocked.

GSD1: Drives, executes, and delivers on work that spans a complete product component or subsystem, from design through successful deployment. Able to unblock themselves independently in most cases.

GSD2: [Planning and estimation] GSD2: [Planning and estimation]

GSD3: [Technical communication] GSD3: [Technical communication]


GSD / Planning and Estimation: Sample Rubrics

E3 E4GSD1: [Execution and delivery, independence]

GSD1: [Execution and delivery, independence]

GSD2: Breaks commits or tickets into appropriately-sized pieces with guidance. Makes and communicates accurate time estimates for individual tickets.

GSD2: Breaks stories into appropriately-sized commits, to facilitate incremental progress and effective code review. Makes and communicates accurate time estimates for own work, potentially spanning multiple sprints. Contributes to roadmap, planning and estimation for their team.

GSD3: [Technical communication] GSD3: [Technical communication]


GSD / Technical Communication: Sample Rubrics

E3 E4GSD1: [Execution and delivery, independence]

GSD1: [Execution and delivery, independence]

GSD2: [Planning and estimation] GSD2: [Planning and estimation]

GSD3: Writes first design documents and solicits and integrates technical feedback from peers. Contributes to documentation and processes that make the team more effective (e.g. on-call playbooks).

GSD3: Consistently communicates technical decisions through design docs, tech talks, and the wiki. Ensures high bus factors through documentation and training of apprentices.


Ladder sub-categories:C1: Cross-functional or domain expertise

C2: Recruiting

C3: Mentoring and onboarding

Manifesto items covered:- How we make things: - How we operate things: - How we Function as a team: A bunch here

Contributions to Eng and Flatiron Culture


C / Cross-Functional or Domain Expertise: Sample Rubrics

E3 E4C1: Understands the clinical or customer facing implication of their features. Effectively accepts guidance and information from team members in other disciplines (e.g. Product, Accounts) or members of other engineering teams.

C1: Able to explain domain or customer-facing implications of their project to other engineers and provide context to new team members. Communicates effectively across functions, is able to work well with Product, Clinical, Accounts, Medical Informatics, etc., as necessary.

C2: [Recruiting] C2: [Recruiting]

C3: [Mentoring and onboarding] C3: [Mentoring and onboarding]


C / Recruiting: Sample Rubrics

E3 E4C1: [Cross-functional or domain expertise]

C1: [Cross-functional or domain expertise]

C2: Contributes to hiring “sell” efforts (e.g. reverse interviews, attending recruiting events or meetups).

C2: Trained and certified as an interviewer (pending enough time in company/industry). Actively participates in software engineering interview process.

C3: [Mentoring and onboarding] C3: [Mentoring and onboarding]


C / Mentoring and Onboarding: Sample Rubrics

E3 E4C1: [Cross-functional or domain expertise]

C1: [Cross-functional or domain expertise]

C2: [Recruiting] C2: [Recruiting]

C3: N/A C3: Successfully mentors a new hire or hosts an intern. Contributes to onboarding through e.g. owning a bootcamp session, creating bootcamp tickets, or improve onboarding documentation. Increases engineering team knowledge through presenting at Engineering or company Knowledge Sharings, or participating in intergrouplets.


Engineering Ladder as a living document

- Deep retro after each performance cycle- We didn’t get it fully right in the first review cycle- Two examples:

- Initial version adopted the term ‘system ownership’ - everyone wanted to own a system (contradicts our bus-factor-value…)

- Initial version focused too much on career-growth by increasing systems breadth. Second version added acknowledgement for going deep, still doesn’t capture complexity and friction of some of our systems. → 3rd generation coming


Engineering Ladder as a living document

- Ladders are a very “big hammer” - An amazing tool for keeping people accountable - Need to be careful not to incentivize the wrong

behaviors- As a performance standard - living document is ok,

but need to converge quickly- Build your ladder not too early, but not too late- Running a Perf process using ladders:

- Ladders proved to be an advocacy tool for managers- Helped provide feedback that is specific and actionable


Key Takeaways

(I hope that)- You understand what is a career ladder- You care more about career ladders and about the

performance process in your team- You see the connection between formalizing

engineering cultures and a career ladders- You care more about documenting your

engineering culture...


Questions?


KEEP CALMAND

FIGHT CANCER

Shameless plug: We are hiring! Drop me a note at [email protected] or visit http://flatiron.com


http://flatiron.com/careers