enhancing configuration management workspaces with information of parallel activities topic proposal...

Enhancing Configuration Management Workspaces

with Information of Parallel Activities

Enhancing Configuration Management Workspaces

with Information of Parallel Activities

Topic Proposal

Anita Sarma

October 2005

2

A Typical Development ScenarioA Typical Development Scenario

CMrepository

Pete’s workspace

A

Ellen’s workspace

DC E

3


CMrepository

Pete’s workspace

A

Ellen’s workspace

DC E

Pete and Ellen modify different artifacts with no dependencies

No conflicts

4


CMrepository

Pete’s workspace

A

Ellen’s workspace

DC E

Ellen starts to modify artifact “C”

No conflicts

C

5


CMrepository

Pete’s workspace

A

Ellen’s workspace

DC E

Pete and Ellen modify entirely different files with no dependencies

No conflicts

C

No conflicts

Direct conflicts

Pete and Ellen concurrently modify the same artifact “C”

lines 20-40 changed

lines 5-10; 30-40 changed

6


CMrepository

Pete’s workspace

A

Ellen’s workspace

DC E


No conflicts

C

No conflicts

Direct conflicts

Pete modifies artifact “B” on which E depends

B

7


CMrepository

Pete’s workspace

A

Ellen’s workspace

D


No conflicts

C C

No conflicts

Direct conflicts

Pete and Ellen modify the same file

B E

No conflicts

Direct conflicts

Indirect conflicts

Pete modifies “B” that affects “E”; Ellen modifies “E” in parallel

signature of interface I1 changed

Changes to method body that calls

interface I1

8

GoalGoal

Help mitigate the impact of direct and indirect conflicts

9

Traditional CM ApproachesTraditional CM Approaches

Coordination mechanism

Directconflicts

Indirect conflicts

Pessimistic

(RCS, DSEE)

Locking before changes are made

Avoided, at the expense of project delays

Not addressed

Optimistic

(CVS, ClearCase)

Automated merging after changes have been made

Resolved, except for overlapping changes

Not addressed

10

Field StudiesField Studies

Perry, et al. – 1994– Number of conflicts proportional to amount

of parallel development Grinter – 1995

– Use information from CM systems to pace their development

De Souza, et al. – 2003– Send email detailing changes and their

expected effects before check-in

11

Inferences from Field StudiesInferences from Field Studies

Conflicts regularly occur and affect productivity – (Perry, et al.,1994)

Developer responses:– Actively obtain information from the CM systems –

(Grinter,1995) – Pace their development to avoid performing

conflict resolution – (Grinter,1995) – Send extra information to enable detection of

effects of changes on other artifacts – (De Souza,2003)

– Place their work in the context of others’ changes – (De Souza,2003)

12

Inference – Informal CoordinationInference – Informal Coordination

CMrepository

Pete’s workspace

CBA

Ellen’s workspace

CED

Informal coordination conventions

13

HypothesisHypothesis

Providing information of parallel development activities and their effects enables developers to:– place their work in the context of others’– self-coordinate their actions– reduce the magnitude and occurrence of

direct and indirect conflicts

14

Inference – Informal CoordinationInference – Informal Coordination

CMrepository

Pete’s workspace

CBA

Ellen’s workspace

CED

Informal coordination conventions

15

More efficient and effectivecoordination

Proposed Solution – Overview Proposed Solution – Overview

CMrepository

Pete’s workspace

CBA

Ellen’s workspace

CED

Enhanced workspace Enhanced workspace

16

Proposed Solution – Context SettingProposed Solution – Context SettingTraditional CM

approaches

Problems – late conflict detection

Proposed solution – early conflict detection

CM infrastructure – Private workspaces– Well structured coordination protocols

Isolated workspaces Insulated workspaces

“Pull-based” “Push-based”

Information only at specific synchroniza-tion points

Relevant real-time information provided continuously

Syntactic correctness– Pessimistic locks– Automated merge tools

Overlapping direct conflict not resolved

Identify and characterize direct conflicts through severity analysis

Semantic correctness– Continuous builds– Integration testing

Indirect conflicts undetected until build, test, or even deployment stage

Identify and characterize indirect conflicts through impact analysis

17

Overarching Plan of AttackOverarching Plan of Attack

Research question: Does information of parallel activities and their effects provide an effective context in which to place one’s own work?

Validation: – Implicit validation from existing field studies– Evaluation results:

Usability case studies – observations on how users monitor information provided and interact with each other

Conflict detection case studies – analysis of the number and magnitude of conflicts detected in a real-life project

18

Research Questions Research Questions

1. How can a tool provide this information in a usable, scalable, and effective manner?

2. To what extent can this information affect self-coordination?

3. To what degree can self-coordination reduce the occurrence of conflicts?

4. To what degree can self-coordination reduce the magnitude of conflicts?

19

Approach embodied by workspace awareness prototype – Palantír

ApproachApproach Provide information of parallel activities continuously

at real time Identify direct and indirect conflicts Provide metrics to denote the size and effect of

conflicts Allow detection of conflicts earlier while changes are

still in progress Build on existing CM infrastructure Present information through visualizations that are

unobtrusive and contextualized Filter events based on relevance

20

Palantír ArchitecturePalantír Architecture

Visualization

Extractor

Internal State

Palantír Client

Visualization

Extractor

Internal State

Palantír Client

Event Database

Palantír Server

BootstrapCapture

Workspace Wrapper Workspace Wrapper

CM System

CM Server

Repository

The Eclipse Platform

Event Listeners

CM Plug-in

Workspace


Event Listeners

CM Plug-in

Workspace

Pete’s Workspace Ellen’s Workspace

21

Palantír Client – VisualizationsPalantír Client – Visualizations

22

Severity Analysis Severity Analysis

Severity Analysis – The amount (size) of change between two versions of an artifact– Artifact severity: the percentage of lines of code

that have changed Example: 25 lines on 100 = 25%

– Directory severity: ∑Actual artifact severities

∑Possible artifact severities Example: dirA – 12.5%

dirB – 25% foo.c – 50 lines of 100 changed – 50% bar.c – 0 lines of 100 changed – 0% dirC – 0% head.c – 0 lines of 100 changed – 0%

23

Impact Analysis – Remaining ResearchImpact Analysis – Remaining Research

Impact of changes – “The effect of changes on my current workspace”

Requirements– Identification of levels of analysis

Local workspace level Remote workspace level Repository level

– Identification and presentation of effects of indirect conflicts Outgoing conflicts – artifacts that affect others Incoming conflicts – artifacts that are affected

– Identification and implementation of metrics to denote the size of “effects” of a change

24

Impact Analysis – Local Workspace Impact Analysis – Local Workspace

CMrepository

Pete’s workspace

CBA

Ellen’s workspace

CED

Local workspace level

Effect of local changes on local workspace (A → B)

Pete’s view

25

Impact Analysis – Remote Workspace Impact Analysis – Remote Workspace

CMrepository

Pete’s workspace

CBA

Ellen’s workspace

CED


Remote workspace level

Effect of changes in remote workspace on local workspace (E → A)

Pete’s view

26

Impact Analysis – Repository Impact Analysis – Repository

CMrepository

Pete’s workspace

CBA

Ellen’s workspace

CED


Remote workspace level

Repository level

Effect of changes in repository on local workspace (E → A)

E

A

Pete’s view

27

Impact Analysis – Outgoing/Incoming ConflictsImpact Analysis – Outgoing/Incoming Conflicts

CMrepository

Pete’s workspace

CBA

Ellen’s workspace

CED

Outgoing conflicts

Incoming conflicts

Outgoing conflicts – artifacts that affect others (E)Incoming conflicts – artifacts that are affected (A,B)

Pete’s view

28

Options for Impact AnalysisOptions for Impact Analysis

Program analysis– Dependency analysis– Call graphs– Abstract syntax trees

Eclipse IDE capabilities– Refactoring – Continuous build capabilities

XML tags – Java-XML, Castor, Apache (XML Beans)

…

29

Evaluation – Criteria for SuccessEvaluation – Criteria for Success

1. Usability study RQ#1: tool provides information in a usable, scalable,

and effective manner RQ#2: information of parallel activities increases self-

coordination RQ#3, #4: reduction in the occurrence and magnitude

of direct and indirect conflicts

2. Conflict detection RQ#3: identification of potential direct and indirect

conflicts RQ#4: calculation of magnitude of direct and indirect

conflicts

30

Evaluation 1 – Usability StudyEvaluation 1 – Usability Study

Five or more confederate case studies– At least one without Palantír

Subjects:– Undergraduates, graduates, volunteer programmers– Role: Complete given programming assignment in teams

within the time limit Confederate:

– Fellow graduate students, research group members– Role: Act as a team member, but run predefined scripts that

lead to conflicts in the assignment The subject is not aware of the identity and intentions

of the confederate

31

Usability Studies – Goals Usability Studies – Goals

How subjects monitor and interact with Palantír

Whether subjects monitor conflict warnings How subjects interact with their partners on

detecting potential conflicts Identify differences (if any) in coordination

efforts:– Only direct conflicts– Direct and indirect conflicts

Conduct structured interviews

32

Evaluation 2 – Conflict DetectionEvaluation 2 – Conflict Detection

Create light-weight Palantír client– Use in research group, open-source and

commercial companies– Capture relevant Palantír events onsite

and investigate them later offsite– Compare direct conflict detection with SCM

merge conflicts– Compare indirect conflict detection with

build logs, bug trackers, build managers,…

33

TimelineTimelineFall 05: Preliminary research on Impact Analysis (Oct – Dec 05)

Investigate and design Impact Analysis metrics (Oct – Dec 05)

Restructure Palantír events (Nov – Dec 05) Winter 06: Build Impact analysis tool / algorithm (Jan – Mar 06)

Create Palantír client wrapper (Feb – Mar 06)Spring 06: Test Impact analysis tool (Apr 06)

Reconstruct Palantír to be plug-in oriented (May 06) Integrate Impact analysis component with Palantír (May 06) Evaluate Palantír with Severity and Impact (Jun 06) Release Palantír client wrapper (Jun 06)

Summer 06: Collect data from Palantír client wrapper (Jul – Sept 06) Create scenario for evaluation (Jul – Sept 06)

Fall 06: Schedule user case studies (Jul – Sept 06) Analyze data from Client wrapper (Oct – Dec 06)

Winter 07: Conduct user case studies (Jan –Apr 07) Dissertation writing (Jan – Mar 07)

Spring 07: Evaluate scenario case studies (Apr – May 07) Complete dissertation (Apr – Jun 07)

34

ContributionsContributions

A coordination approach that …– enhances traditional CM workspaces with information of

parallel activities– allows placement of one’s work in the context of others– enables early detection of potential conflicts

…leads to reduction in occurrence and magnitude of direct and indirect conflicts

This approach embodied by prototype – Palantír – Mechanisms for detection of potential conflicts– Metrics to denote magnitude of conflicts– Set of visualizations with varying degrees of obtrusiveness– Easy adoption into current practices

35

Questions???Questions???

36

Directory Severity ChoicesDirectory Severity Choices1. Directory Severity = max {Artifact severities}

– Grabs attention: max {75, 0, 0} = 75– Multiplicity has no effect: max {75, 25, 50} = 75

2. Directory Severity =

– Every artifact has an equal weight– Can degrade to zero when a majority of the artifacts have severity 0,

pushing the combined severity to < 1:

3. Directory Severity =

– Does not degrade to 0 with many 0 severity artifacts– Additional artifacts with smaller severity lowers directory severity:

∑Actual artifact severities

∑Possible artifact severities

1+0+0

100+100+100

∑severity>0Actual artifact severities

∑severity>0Possible artifact severities

75100

= 75, but75+25+25

100+100+100 = 42

= 0

37

Examples of Indirect ConflictsExamples of Indirect Conflicts

Interface signature changed– Return type– Input parameters

Object type changed – Return type– Input parameters

The behavior of the method changed Global variable setting within the method changed The order of the input parameter changed, while

keeping their types constant The interface/method deleted from the class Splitting the module on which another depends into

two or more parts

38

Usability Case Study Set Up Usability Case Study Set Up

Programming assignment– Java programming language– 10-15 classes– 1.5 – 2 hour long sessions

Direct conflicts Indirect conflicts– 1 (C) + 3 (S) = 4– 2 (C) + 5 (S) = 7– 3 (C) + 7 (S) = 10

– 2 (C) + 5 (S) = 7– 3 (C) + 7 (S) = 10– 3 (C) + 7 (S) + 5 (V) = 15– 3 (C) + 7 (S) + 10 (V) = 20

Legend:C – Confederate; S – Subject; V – Virtual

39

Work Thus FarWork Thus Far

Identified set of events to represent development activities

Integration with three stand alone CM systems (RCS, CVS, JSVN)

Workspace monitor Integration with Eclipse IDE (Eclipse-CVS, Subclipse) Set of four visualizations Severity analysis and directory severity analysis Robustness against loss of connectivity Ongoing work with project management visualization Ongoing data collection from software archives

40

Future WorkFuture Work

Use insights into workspace activity and development patterns– Workspace management– Project management

Create tools to facilitate coordination across teams– Abstraction of development activities to be useful for

gatekeepers Trust based CM Towards continuous coordination Coordination support for all phases of the software

life cycle

41

Continuous Build SystemsContinuous Build Systems

Apache gump– Builds and compiles software against the latest

version– Detects potentially incompatible changes very

soon after a check-in– Notifies the project team of failed builds

Tinderbox provides information of– What platforms has built successfully– Which platforms failed and how they failed– State of the files that made up the failed build

42

An Alternative ViewAn Alternative View

Continuous coordination,collaborative architecture,

seamless development environments,

Asynchronous communicationAccess to common set of artifacts,

isolated workspaces and version control

Parallel development,roles and access rights

Passive awareness ofdevelopment activities

and developers, manageinformation overload

Task allocation and assignment

Communication archivalalong with artifacts

Communication

Collocation benefits todistributed development

Organizational memory ,knowledge acquisition and

dissemination, social navigation

Prescribed and defined coordination support

Artifact Management Task Management

Advanced conflictdetection

Bas

ic F

unct

iona

lity

Rig

id P

roce

ssIn

form

atio

n D

isco

very

Info

rmat

ion

Pro

visi

onIn

tegr

ated

Ope

ratio

ns

Fine grained versioning,conflict resolution

Instant Messaging,monitoring changes

to artifacts

43

Back to Configuration ManagementBack to Configuration Management










Communication


Organizational memory,knowledge acquisition and





Bas

ic F

unct

iona

lity

Rig

id P

roce

ssIn

form

atio

n D

isco

very

Info

rmat

ion

Pro

visi

onIn

tegr

ated

Ope

ratio

ns



to artifacts

44

A Glimpse into the FutureA Glimpse into the Future










Communication


Organizational memory,knowledge acquisition and





Bas

ic F

unct

iona

lity

Rig

id P

roce

ssIn

form

atio

n D

isco

very

Info

rmat

ion

Pro

visi

onIn

tegr

ated

Ope

ratio

ns



to artifacts

45

Restructure PalantírRestructure Palantír

Restructure Palantír events– To make them generic– To make it easier to query and parse

Restructure Palantír architecture– To make it plug-in oriented– To make it more modular– To enable users to experiment with different

versions of Palantír components (wrappers, analyzers, visualizations)

– To encourage open source community developers to contribute

46

Workspace WrapperWorkspace Wrapper

Tasks– Intercept workspace activity (both CM commands

and local operations)– Interpret workspace activity– Determine whether the activity is relevant to Palantír– If relevant, gather information regarding the activity– Transmit the event to the Palantír server

Unique per CM system Unique per editor Is and should remain unobtrusive

Visualization

Extractor

Internal State

Visualization

Extractor

Internal StateEvent Database

Palantír Server

BootstrapCapture



Event Listeners

CM Plug-in

Workspace

CM SystemCM Server

Repository


Event Listeners

CM Plug-in

Workspace

Palantír Client

47

Palantír ServerPalantír Server

Tasks– Stores events from all workspaces– Maintains the order of the events– Bootstraps new clients and clients on a

reconnect

Common for all SCM systems Is unobtrusive

Visualization

Extractor

Internal State

Visualization

Extractor


Palantír Server

BootstrapCapture



Event Listeners

CM Plug-in

Workspace

CM SystemCM Server

Repository


Event Listeners

CM Plug-in

Workspace

Palantír Client

48

Palantír ClientPalantír Client

Internal State– Maintain overview of activities in both the local and

remote workspaces Extractor

– Allow users to configure the events to be viewed Set of workspaces Set of authors Set of artifacts …

Visualizations– Organize and display events

Visualization

Extractor

Internal State

Visualization

Extractor


Palantír Server

BootstrapCapture



Event Listeners

CM Plug-in

Workspace

CM SystemCM Server

Repository


Event Listeners

CM Plug-in

Workspace

Palantír Client

49

Integration ExperienceIntegration Experience

Standalone wrappers– RCS (pessimistic)– CVS (optimistic)– Subversion (optimistic)

Development editor – Eclipse – Eclipse-CVS– Subclipse

50

Event Type Indication

Populated Artifact has been placed in a workspace

Unpopulated Artifact has been removed from a workspace

Synchronized Artifact has been synchronized with repository

ChangesInProgress Artifact has changed in a workspace

ChangesCommitted Artifact has been stored in repository

ChangesReverted Artifact has been returned to its original state

Added New artifact has been added to workspace

Removed Artifact has been removed from workspace

Renamed Artifact has been renamed

Moved Artifact has been moved from one artifact to another

Metrics Severity / impact of changes to an artifact

List of EventsList of Events

51

Typical Sequence of EventsTypical Sequence of EventsPessimistic Populated ChangesInProgress MetricsChanged MetricsChanged … ChangesCommitted ChangesInProgress MetricsChanged MetricsChanged … ChangesCommitted UnPopulated

Optimistic Populated ChangesInProgress MetricsChanged MetricsChanged … ChangesCommitted ChangesInProgress MetricsChanged MetricsChanged … ChangesCommitted UnPopulated

enhancing configuration management workspaces with information of parallel activities topic proposal...

Documents

conflicts c slide

ellens workspace dce

ellens workspace dce

ellens workspace dce

b slide

ellens workspace d pete

checkin slide

typical development