Motivation in Software Engineering: A Systematic Review Update

A. César C. França, Tatiana B. Gouveia, Pedro C. F. Santos, Celio A. Santana, Fabio Q. B. da Silva

Center of Informatics Federal University of Pernambuco

Recife, Brazil cesarfranca@gmail.com, {tbg2,pcfs, casj, fabio }@cin.ufpe.br

Context

“Motivation”...

• …is believed to be a source of many benefits for projects in general – Performance, productivity, retention, etc.

• …has more than one hundred different theoretical definitions

• ...has been studied in software engineering since the 80´s

(Beecham, et al., 2008)

( Golembiewski, 2000)

(Toledo & Unger, 1980

Background

Motivation in Software Engineering: A Systematic Review Beecham et al (2008)

From Beecham et al (2008)

1980 ... 2006

peer reviewed studies

92 RelevantPapers

Based on the guidelines presented by Kitchenham (2004).

Background

Motivation in Software Engineering: A Systematic Review Beecham et al (2008)

Some conclusions...

• Motivation is context dependent and varies from one engineer to another.

• The most commonly cited motivator was the job itself– but they found fewer work on what it is about the job that Software Engineers find motivating.

• Surveys often aimed at how Software Engineers feel about “the organization”– rather than “the profession‟.

• There is no clear understanding of the Software Engineers´ job, – what motivates Software Engineers,

– how they are motivated, or

– what are the outcomes and benefits of motivated Software Engineers.

Our Objectives

(1) update data available on motivation in software engineering;

(2) compare our results with the original study.

Pre-conditions: • Same research questions• Same data sources*• Same search string**• Same selection criteria• Same quality assessment form• Same extraction and synthesis procedure

* We did not have access to 1 of theoriginal data sources (Inspec). On theother hand, we added 4 alternativedata sources.

** Our search string was based on the same keywords of the Original Study, although we did not compose specific strings to match each research question. Instead, we compose only one generic string

Following the Original Study protocol, available at Beecham et al. (2006)

Review Method

Review Method

Automated Search: StringString Validation:

(1) All seven papers published in 2006 found in the Original Study were retrieved;

(2) We previously selected a set of five known papers, published after March 2006t, they were all retrieved;

(3) we run our string again, with no time restrictions, and we looked into our results for a set of 45 randomly selected studies from the original study. All papers were retrieved.

Our search string was based on the same keywords of the Original Study, although we did not compose specific strings to match each research question. Instead, we compose only one generic string

Review Method

Automated Search: Data SourcesAdded Sources ( )

4,723 papers 46 papers

SD, CSX, COMP

IEEE, GS, ISI

ACM, JSTOR, SL

SCP

We did not have access to 1 ofthe original data sources (Inspec). On the other hand, we added 4 alternative data sources.

Review Method

Manual Search

1811 papers 44 papers

Search strategies to increase coverage (Google Scholar):

(1) We snow-balled for all papers in which the Original Study is cited (SB OS)

(2) We snow-balled for all the papers in which one or more of the 92 primary studies of the Original Study are cited (SB PS).

Manual Search

1. Search Results6534 papers

2. Filtering based on Title and Abstract: 391 papers

3. Removing duplications292 papers

4. Inclusion/Exclusion criteria53 papers

Study Selection

Same of the Original Study

Review Method

+

+ +

Experienced in SLRSpecialist in Motivation

Quality Assessment

Review Method

Carried out by 2 researchers and

double checked by a third one

Clarifying itens...

+ +

Experienced in SLRSpecialist in Motivation

Same of the Original Study

Review Method

Extraction & Synthesis1. Syntactical data were extracted:

– title, authors, year, publisher, source type, study type, keywords, geographical location of subjects, type of subjects

2. passages which answered each research question were transcribed

3. an open coding procedure

4. similar codes were condensed according to themes categories as in an axial coding procedure

– We used the same categories of the Original Study whenever made sense

5. frequencies of citation for each category were accounted

Results

Sources

Original Study Our Study

• ACM SIGMIS Computer Personnel Research Conference: 8 papers

• 15 distinct conferences

• 23 distinct Journals

1980 2006 2006 2010

Results

Study Type

Original Study Our Study

1980 2006 2006 2010

Results

Temporal Distribution

Original Study Our Study

1980 2006 2006 2010

Results

Countries

Original Study Our Study

1980 2006 2006 2010

ResultsRQ1: What are the characteristics of Software Engineers?

Observations:

• Sixteen papers provided answers to RQ1

• Three new characteristics were found in the present study

– Ch. 17: Competent in Management

– Ch. 18: Flexible/Team Worker

– Ch. 19: Have fear of punishment

• These characteristics do not consistently describe software engineers.

– Several characteristics seem to be conflicting

e.g. Introverted x Team Worker.

– The Original Study reached similar results.

• Two characteristics described in the original study do not appear in the present study

– Ch.7: Need for involvement in personal goal setting

– Ch. 13: Marketable

ResultsRQ2: What (de)motivates Software Engineers to be more

(less) productive?

Observations:

• Twenty five papers provided some answer to RQ2

• Eight New motivators were found– M. 22: Team quality

– M. 23: Creativity/Innovation

– M. 24: Fun (playing)

– M. 25: Professionalism (high professional environment)

– M. 26: Having an Ideology

– M. 27: Non-financial benefits (availability of perks)

– M. 28: Penalty Policies

– M. 29: Good relationship with users/customers

• Two motivators described in the original study do not appear in the present study

– M.19: Appropriate working conditions

– M.21: Sufficient resources

ResultsRQ2: What (de)motivates Software Engineers to be more

(less) productive?

Observations:

• Only seven papers helped on finding de-motivators.

• The only new de-motivator that appeared in this study was D. 16 Task Complexity

ResultsRQ3: What are the external signs or outcomes of (de)motivated

Software Engineers?

Observations:

• Eight papers provided answers to RQ3.

• Papers about retention not always present it as an outcome of motivation.

• Only two external signs were added to the original list.

– Ext. 7: Organizational Commitment

– Ext. 8: Benevolence

• Four external signs described in the original study do not appear in the present study

– Ext.2: Project Delivery Time

– Ext.4: Budgets

– Ext5.: Absenteeism

– Ext.6: Project Success

ResultsRQ4: What aspects of Software Engineering (de)motivate Software

Engineers?

Observations:

• Six papers provided answers to RQ4.

• Only three (de)motivating aspects were added to the original list.

– Asp. 10: Creativity

– Asp.11: Relationship with users/customers

– De-asp.2: Boredom (repetitive tasks)

• Five motivating aspects described in the original study do not appear in the present study

– Asp.3: Change

– Asp.5: Benefit

– Asp.6: Science

– Asp.7: Experiment

– Asp.9: Lifecycle

ResultsRQ5: What models of motivation exist in Software Engineering?

Observations:

• Sixteen papers provided answers to RQ5.

• The MOCC model of motivation in software engineering (Sharp, et al., 2009), constructed based on the findings from the original study, has been cited twice.

Discussion

General Observations:

• The overall understanding of how software engineers are actually motivated does not seem to have significantly advanced in the last five years

• There is no consensus about the characteristics of software engineers

• Motivation may be a key for retaining people

• Evidence about the actual effect of any motivator on the expected outcomes is generally weak.

• High presence of studies on Open Source Software (OSS) projects,

• Motivation x Agile Development

(antecedent or consequence?)

Lessons Learned

• Researches in software engineering does not always seem to be careful with the “motivation” construct.

• Having access to a complete and detailed review protocol is extremely important!

• The quality assessment we carried out is focused on the quality of the report, rather than properly on the strength of the evidence .

• The complexity of the human-technological setting of software engineering projects makes the construction of evidence indeed hard .

• It is very important to project managers be aware of the specific characteristics of the individuals in their context.

Conclusion

Limitations• Even though some adaptations were necessary

to use the Original Study protocol, we were as careful as possible

• The amount of published work on motivation in software engineering is still wispy, which may provide only a partial view of the sought potential answers

• the USA still leads the amount of research carried out on motivation on software engineering

Future Work• Future research should focus on a deeper

analysis of the relationship between motivators and outcomes, in order to provide more reliable results

• Qualitative research could also contribute to the development of a clear and grounded understanding of motivation in software engineering

We are currently running a qualitative ethnographic study in three distinct organizations, in pursuit of answers to

some of the same questions explored here.

Motivation in Software Engineering: A Systematic Review Update

A. César C. França, Tatiana B. Gouveia, Pedro C. F. Santos, Celio A. Santana, Fabio Q. B. da Silva

Center of Informatics Federal University of Pernambuco

Recife, Brazil cesarfranca@gmail.com, {tbg2,pcfs, casj, fabio }@cin.ufpe.br

Motivation in Software Engineering: A Systematic Review Update by A. César C. França, Tatiana B. Gouveia, Pedro C. F. Santos, Celio A. Santana, Fabio Q. B. da Silva is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Based on a work at Evaluation and Assessment in Software Engineering, Durham, 2011.

Permissions beyond the scope of this license may be available at http://www.haseresearch.com