1

Are Happy Developers more Productive?

The Correlation of Affective States of Software Developers and their self-assessed Productivity

Daniel Graziotin, Xiaofeng Wang, Pekka AbrahamssonFree University of Bozen-Bolzano

PROFES 2013, 12-14 June, Paphos, Cyprus

2

BackgroundTheory, Related Work, Constructs, Hypotheses

Research MethodologyResearch Design, Analysis Method

ResultsDescriptive Statistics, Hypotheses Testing

DiscussionImplications, Limitations, Future Studies

IntroductionMotivation, Research Questions 1

2345

3

To improve Software Productivity and Quality, focus

on people(Boehm, 1990)

4

“People trump Process”(Cockburn & Highsmith, 2001)

5

How to verify this claim?

6

How to focus on people in SE Research?

Picture Credits

7

Little is known on the productivity of individual programmers(Scacchi, 1995) Picture Credits

8

Human Factors, multidisciplinary

Picture Credits

Intellectual Activities

9

Software Developmentis Cognitive(Khan et al., 2010)

Picture Credits

10

Affective StatesEmotions, Moods,

FeelingsManagement and PsychologyInfluence on:

• cognitive activities• working behaviors• Productivity(Ashkanaasy et al., 2002; Fisher et al., 2004; Ilies et al., 2002; Miner et al., 2010) Picture Credits

Photo

11

.. collect psychometrics(Feldt et al., 2008)

.. focus on Affective States of Software

Developers(Shaw, 2004 & Khan, 2010) Picture Credits

PhotoSoftware Engineering

Empirical studies should..

12

Research Question

‣ How do the affective states related to a software development task in foci influence the self-assessed productivity of developers?

13

BackgroundTheory, Related Work, Constructs, Hypotheses

Research MethodologyResearch Design, Analysis Method

ResultsDescriptive Statistics, Hypotheses Testing

DiscussionImplications, Limitations, Future Studies

IntroductionMotivation, Research Questions 1

2345

14

Background Theory

‣ Emotions• states of mind raised by external stimuli,

directed toward the stimulus by which they are raised (Plutchik et al., 1980)

‣ Moods• individual feels “good” or “bad”;

“likes” or “dislikes” what is happening around (Parkinson et al., 1996)

‣ No agreement in literature‣ Affective States: umbrella term

Defining Affective States

15

Background Theory

‣ Real-time positive Affective States are positively correlated with real-time performance of workers (Fisher et al., 2004)

‣ Affective States of Software Developers dramatically change during a period of 48 hours (Shaw, 2004)

‣ Positive Affective States have positive impact on software developers debugging abilities (Khan et al., 2010)

Affective States and their Impact on Workers

16

Background Theory

‣ Discrete Approach (Plutchik et al., 1980)

• Basic set of unique Affective States• Example: interested, excited, guilty, upset

‣ Dimensional Approach (Russel, 1980)

• Major dimensions- Valence- Arousal- Dominance

Categorize Affective States

17

Background Theory

‣ Questionnaires, surveys‣ Self-assessment Manikin (SAM) (Bradley, 1994)

• Assess Affective States triggered by stimulus• Pictorial (universal)• Likert-items

Measure Affective States

18

SAM - Valence

19

SAM - Arousal

20

SAM - Dominance

21

Background Theory

‣ SAM items range: [1,5]‣ Not stable across persons‣ Stable within persons

• Conversion to Z-scores

• Dimensionless, range: [-3,+3]

Measure Affective States

22

BackgroundTheory, Related Work, Constructs, Hypotheses

Research MethodologyResearch Design, Analysis Method

ResultsDescriptive Statistics, Hypotheses Testing

DiscussionImplications, Limitations, Future Studies

IntroductionMotivation, Research Questions 1

2345

23

Research Methodology

‣ Repeated measurements design‣ Natural Settings‣ Participants work on their software project for

90 minutes‣ Affective States and Productivity self-

assessed each 10 minutes‣ Productivity Likert item [1,5]

Research Design

24

Research MethodologyResearch Design

Pre-task Interview Development

Post-task InterviewQuestionnair

e

Researcher

25

Research Methodology

‣ Sam and Questionnaire on tablet device‣ 4 “taps” to complete the survey

• Limit context switch‣ Pre- and post-task interviews‣ In-field observations

Data Collection

26

Research Methodology

‣ Repeated measurements of individuals‣ Data Dependencies

• Participant• Time per participant

‣ Anova does not fit (Gueorguieva et al., 2004)

‣ Linear mixed-effects models

Analysis Procedure

Research Methodology

‣ Linear Mixed Effects Model • Fixed effects• Random effects

27

Analysis Procedure

Random variables

Parameters fixed value

KnownRegressors

I.i.d random error terms

28

BackgroundTheory, Related Work, Constructs, Hypotheses

Research MethodologyResearch Design, Analysis Method

ResultsDescriptive Statistics, Hypotheses Testing

DiscussionImplications, Limitations, Future Studies

IntroductionMotivation, Research Questions 1

2345

29

Results

‣ 8 Participants (mean age 23.75)• 4 Professional Software Developers• 4 Students

‣ 8 Projects• Work-related• Course-related

Descriptive

30

ResultsDescriptive

id role project task p. lang. p.langexp.

taskexp.

P1 PRO

Data collection

for hydrological

defense

Module for data

displayingJava HIG HIG

P2 PROResearch

Data Collection &

Analysis

Script to analyze

dataPython LOW HIG

P3 PROHuman

Resources Manager for

a School

Retrieval and display of DB data

Java HIG HIG

P4 PRO Metrics Analyzer

Retrieval and sending of metrics

C++ HIG HIG

P5 STU Music EditorConversion

of music score to pictures

C++ LOW LOW

P6 STU Code EditorAnalysis of Cyclomatic Complexity

C++ LOW LOW

P7 STU CADSingle-lined

labels on objects

C++ LOW LOW

P8 STU SVG Image Editor

Multiple objects on a

circle or ellipse

C++ HIG HIG

31

ResultsProductivity and Valence

32

ResultsProductivity and Arousal

33

ResultsProductivity and Dominance

34

ResultsFirst Observations

‣ For all participants• Strong variations of Productivity and Affective

States• In 90 minutes of time

35

ResultsHypotheses Testing

‣ Linear Mixed Effects Model with R (lme4.lmer)

• productivity ~ fixed + random• Fixed: • Random:

36

ResultsHypotheses Testing

Fixed Effect Value Sum Square F-value Upper p-value (64 d.f.)Lower p-value (48 d.f.)

Deviance Explain. (%)

valence 0.10* 7.86 19.10 0.000 0.000 12.39

arousal 0.07 0.00 0.00 0.950 0.950 0.00

dominance 0.48* 7.44 18.07 0.000 0.000 11.71

time 0.00 0.11 0.26 0.614 0.615 0.17valence:time 0.04 0.35 0.84 0.363 0.364 0.54

arousal:time -0.03 0.45 1.09 0.300 0.301 0.71

dominance:time -0.01 0.06 0.15 0.699 0.700 0.10

37

Results

‣ Scalar random effects for participants: [-0.48, 0.33];

‣ Time Random effects estimated to 0.

Hypotheses Testing

38

BackgroundTheory, Related Work, Constructs, Hypotheses

Research MethodologyResearch Design, Analysis Method

ResultsDescriptive Statistics, Hypotheses Testing

DiscussionImplications, Limitations, Future Studies

IntroductionMotivation, Research Questions 1

2345

39

Discussion‣ Support for

• positive correlation with valence, dominance and productivity

‣ No support for• positive correlation with arousal and productivity• interaction between affective states and time.

‣ The model • Estimates valence to 0.10, dominance to 0.48 (Z-

scores)• Explains 25.62% of productivity deviance

- 12.39% for valence and 11.71% for dominance.

Implications

40

Discussion

‣ Attractiveness perceived towards the development task (valence)

‣ Perception of possessing adequate skills (dominance)

‣ Almost the same explanation power‣ The productivity was self-assessed by

“deltas” of the previous input and the expectation of the task

Implications

41

Discussion

‣ This work• Provides basic theoretical building blocks on

researching the human side of software construction.

• Performs empirical validation of psychometrics and related measurement instruments in Software Engineering research.

• Introduces rarely employed analysis methods

Implications

42

Discussion

‣ Limited Number of Participants, task duration• Background Skills balanced• All 72 measurements are valuable• Still typical number of participants and

measurements (Vickers, 2003)

Limitations

43

Discussion

‣ Use of self-assessed productivity• Software metrics difficult to be employed• Productivity still open problem• Self-assessed productivity consistent to objective

measurements of performance (Miner, 2010)

Limitations

44

Discussion

‣ Student employment• Next generation of software developers

(Kitchenham et al., 2002)• Close to the actual population

(Tichy, 2000)

‣ Individuals working alone• Control Purposes• Limit network of affective states

Limitations

45

Are Happy Developers

More Productive?‣ Towards “Yes, they are”‣ Definitive Answer

• Multidisciplinary theories• Validated instruments• Open Mind

Picture Credits

46

Future Studies

‣ Same programming task ‣ Software teams‣ New understanding of software development.‣ Traditional software productivity metrics‣ Mood induction techniques

47

Software developers are unique human beings. • Perception of development life-cycle• Cognitive activities affect

performance

New understanding of software development.

48

Thank you for your attention

Daniel Graziotindaniel.graziotin@unibz.it

49

References

Boehm, B.: Understanding and Controlling Software Costs. IEEE Transactions on Software Engineering 14(10), 1462–1477 (1990)

Cockburn, A., Highsmith, J.: Agile software development, the people factor. IEEE Computer 34(11), 131–133 (2001)

Scacchi, W.: Understanding Software Productivity. Advances in Software Engineering and Knowledge Engineering 4, 37–70 (1995)

Khan, I.A., et al.: Do moods affect programmers’ debug performance? Cognition, Technology & Work 13(4), 245–258 (2010)

50

References

Ashkanasy, N.M., Daus, C.S.: Emotion in the workplace: The new challenge for managers. The Academy of Management Executive 16(1), 76–86 (2002)

Fisher, C.D., Noble, C.: A Within-Person Examination of Correlates of Performance andEmotions While Working. Human Performance 17(2), 145–168 (2004)

Ilies, R., Judge, T.: Understanding the dynamic relationships among personality, mood, and job satisfaction: A field experience sampling study. Organizational Behavior and Human Decision Processes 89(2), 1119–1139 (2002)

51

References

Miner, A.G., Glomb, T.M.: State mood, task performance, and behavior at work: A within persons approach. Organizational Behavior and Human Decision Processes 112(1), 43–57 (2010)

Shaw, T.: The emotions of systems developers. In: Proceedings of the 2004 Conference on Computer Personnel Research Careers, Culture, and Ethics in a Networked Environment, SIGMIS CPR 2004, p. 124. ACM Press, New York (2004)

Russell, J.: A Circumplex Model of Affect. Journal of Personality and Social Psychology 39(6), 1161–1178 (1980)

Tichy, W.: Hints for reviewing empirical work in software engineering. Empirical Software Engineering 5(4), 309–312 (2000)

Plutchik, R., Kellerman, H.: Emotion, theory, research, and experience. Academic Press, London (1980)

52

ReferencesRussell, J.: A Circumplex Model of Affect. Journal of Personality and Social Psychology 39(6), 1161–1178 (1980)

Bradley, L.: Measuring emotion: the self-assessment semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25(1), 49–59 (1994)

Kitchenham, B.A., et al.: Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering 28(8), 721–734 (2002)

Gueorguieva, R., Krystal, J.H.: Move over ANOVA: progress in analyzing repeated measures data and its reflection in papers published in the Archives of General Psychiatry. Archives of General Psychiatry 61(3), 310–317 (2004)

Robinson, G.K.: That BLUP is a Good Thing: The Estimation of Random Effects. Statistical Science 6(1), 15–32 (1991)

Vickers, A.J.: How many repeated measures in repeated measures designs? Statistical issues for comparative trials. BMC Medical Research Methodology 3(22), 1–9 (2003)