Bell Laboratories 1
Organizational Patterns
James O. Coplien
Bell Laboratories
Naperville, Illinois, USA
JaCC Software Developers
16 September 1999
11:00 - 12:30
Bell Laboratories 2
Organizational Patterns
James O. Coplien
Bell Laboratories
Naperville, Illinois, USA
IIT Colloquium Series
Tuesday, February 8, 2000
M-50 Auditorium
Bell Laboratories 3
Overview
What are organizational patterns? Methodology Our model of organizational change Using patterns for change management The patterns themselves can be found
athttp://www.bell-labs.com/cgi-user/OrgPatterns/OrgPatterns
Bell Laboratories 4
What is a pattern?
A solution to a problem in a context Architectural patterns ideas first published by
Alexander in 1977 Look at issues of structure, not just parts Build on proven practice, not “promising theories” Adopted by the software community starting in 1990;
started gaining critical mass by 1993 Today, widely used as a design tool, especially in the
OO community
Bell Laboratories 5
What are organizational patterns?
Solutions to organizational problems in a context First appeared in the Alexander + software context at
PLoP in 1994 (Coplien, Whitenack); received with some skepticism
Now, a growing body of knowledge
Or, a construct from anthropology, Kroeber:» Universal patterns: transcend cultures» Systemic patterns: have a common root in an ancient
culture» Total culture patterns: give a culture its identity
Patterns define culture
Bell Laboratories 6
Why do we care about organizational patterns?
Because process-based approaches have serious limitations that hark back to the era of the industrial revolution
Because experience with project management, technology transfer, etc. show software to be a primarily social activity
Because of the tie to cultures, and the study of cultures is about social activity
Because we should care about human comfort in addition to product quality
Because they work!
Bell Laboratories 7
The Pasteur Process Research Project at Bell
Labs
Goal: To know how to design highly productive software development organizations
Premise: Organization drives process, which drives productivity
Strategy: Correlate properties of organizations with order-of-magnitude productivity
Technique: Patterns
Bell Laboratories 8
A Theory of Organizational Change
The idea of an organization as an anthropomorphic entity» Some are intelligent: good reactions, learn from mistakes» Some are stupid: react in insane ways, don’t learn
Organizations can learn! Organizations can’t be taught Therefore, learning is experiential
» Experience is a hard teacher
Can be helped with empirical foundations and introspection
Bell Laboratories 9
Empirical foundations
“Pattern Mining” Finding recurring structures and practices in
healthy organization To a degree, seeking the absence of these
structures in dysfunctional organizations A soft science Akin to anthropology and its many branches
Bell Laboratories 10
Organizational Pattern Mining
Based on social network analysis with informal extensions
Organizational role-play to gather social network data
Dyadic and triadic sociometric data Uses CRC cards Light facilitation Half-day exercise
Bell Laboratories 11
CRC Cards: Classes, Responsibilities, and
Collaborators
Subsystem coord.
Validate MR listsBuild group products
Administer ENVY Resolve logical deps.
Subsystem coord.Change committee
DesignersSystem test
Bell Laboratories 12
Work-life Role-Play
Identify project roles Study subjects play roles Development scenarios drive role-play Capture interaction & coupling on CRC cards Social Network Analysis Tools
» Organization Structure Visualization» Organizational Metrics
Capture Trends as Generative Patterns
Bell Laboratories 13
Distilling the Patterns
Sociometric analyses» Pasteur analysis tool» Analytical studies of data
Catalog sociograms, sociomatrices, sociometric data
The pattern must recur The pattern must solve a problem Document in pattern form
Bell Laboratories 14
QPW as a Subject
Fourth generation product in one generation Remarkable press reviews The most effective organization we have found
» Productivity» Quality» Interval» Market
Used to calibrate our “spectrum” Analysis done at my request I am not a QPW user
Bell Laboratories 17
Organization Metrics
0123456789
10
2 3 4 5 6 7 8 9
Average Collaborators Per Role
Frequency
QPW
Bell Laboratories 18
Writers’ Workshops
A forum to review patterns Borrowed from the creative literature community Designed to protect the dignity of the author Several basic steps:
» Moderator introduces author» Author reads the work» Author becomes a fly on the wall» Someone summarizes the pattern» The strengths are emphasized» Suggestions for improvement» Author asks for clarification» Thank the author
Bell Laboratories 19
Using the Patterns Let dysfunctional organizations become
exposed to the patterns to see what is possible
Use organizational role-play as an introspection tool
Collect and process sociometric data from the role play
Present the data to the organization for a second round of introspection—a “mirror”
Learning and growth build on the insights
Bell Laboratories 20
The Organizational Role-Play
Same technique as used for pattern mining Largely self-running Role-based
» Useful in most development organizations
Actor-based» Useful for diagnosing specialized team problems» “Ethnological” studies
Bell Laboratories 21
Facilitation Guidelines Involve everybody Don’t try solving other peoples’ problems
» Other organizations must solve their own problems
» Interface issues can be dealt with in another forum» Interfaces generally involve politics
Don’t look for “the problem”» Problems are systemic» Solutions are generative» Let the patterns cause the “aha”s
Bell Laboratories 22
More principals of organizational change
Happens bottom-up, perhaps with the top-down support of a sponsor or patron
Patterns are closely tied to value systems—which must be elicited bottom-up
Organizational change means building a new culture: reverse anthropology» New language
» New normative behaviors
» New values
» New symbols
» New stories (patterns)
» New rituals
Bell Laboratories 23
What makes organizations excel?
No silver bullet Well-known productivity enablers A little luck The right balance of techniques to make
a working social (and economic and technical and...) system
Bell Laboratories 24
Communication is Key
The “Buffalo Mountain” pattern Decentralized, yet a graceful distribution Almost fully connected Anti-schmisogenetic
» No splinter groups» No time-serial sub-processes» No “subroutines”
Bell Laboratories 25
Communication Intensity
0102030405060708090
100
0 20 40 60
Number of Roles
% Saturation
QPW
Bell Laboratories 26
Size
Easy to socialize the architecture Self-directing
» Correlates to Gerry Weinberg’s SEI/CMM parallel
» The true hallmark of a mature organization Divide and conquer Essentially large organizations need
hierarchy
Bell Laboratories 27
Architect Also Implements
Problem: Preserving the architectural vision through to implementation
Context: An organization of developers that needs strategic technical direction.
Forces: Totalitarian control is viewed by most development teams as a draconian measure. The right information must flow through the right roles. The architect is one of the few people, or may be the only person, with the big picture. Big picture knowledge is important even during implementation, especially when using iterative development. But the architect usually isn’t around during implementation.
Solution: Beyond advising and communicating with developers, architects should also participate in implementation.
Resulting Context: A development organization that perceives buy-in from the guiding architects, and that can directly avail itself of architectural expertise. The architect learns from implementation constraints and may modify the architecture accordingly.
Rationale: Vitruvius notes: “...[A]rchitects who have aimed at acquiring manual skill without scholarship have never been able to reach a position of authority to correspond to their pains, while those who relied only upon theories and scholarship were obviously hunting the shadow, not the substance. But those who have a thorough knowledge of both, like men armed at all points, have the sooner attained their object and carried authority with them.” Larry Constantine emphasizes the value of “street creds” in an architect.
Bell Laboratories 30
Distribute Work Evenly
Like QPW adjacency graph Sociometric analysis: low graph
centrality Can be remedied by load leveling,
staffing, splitting roles, combining roles
Bell Laboratories 31
Organization Metrics for Distribute Work Evenly
05
1015202530354045
0 2 4 6 8
Communication Intensity Ratio
Number of Roles
QPW
Bell Laboratories 32
Personnel Issues
Personal excellence and integrity Highly specialized Balance of teamwork (communication) and
individuation (specialization, personal sign-offs)
How does one assemble such a group?1. Magic: A secret that only Philippe knows
2. Outside interests? (Gabriel)
3. Self-selecting Team (another pattern)
Bell Laboratories 33
Engage Quality Assurance
QA is tightly coupled in the social network
Large beta program–customer engagement
Developers reserve early verification to themselves
QA first-in, last-out
Bell Laboratories 34
Reward Excellence
A sense of contribution» Reinforced in project signoffs» Reinforced in reward system
Financial incentives» No first-hand insights into QPW» Controversial, but claims demonstrable results
Two cultures with extravagant rewards:1. The U. S. West Coast
2. Financial trading
Overcomes fear of change
Bell Laboratories 35
Process?
The ability to introspect well The ability to self-manage Organizational structure over task
sequencing Effective meetings, and lots of them
Bell Laboratories 36
Conclusion
Organizational patterns are:» Patterns that capture the “architecture” of
organizations» In anthropology, the fabric of social structure
Role-playing data gathering and introspection Organizational change is generative Organizations can use patterns as exemplars
for improvement A rich body of literature exists and is growing