ESEM 2014 10/15/2015

Paper #86 1

A Controlled Experiment to Evaluate the

Effects of Mindfulness in Software

Engineering (SE)

Beatriz Bernárdez, Amador Durán,

José A. Parejo & Antonio Ruiz-Cortés

(University of Seville)

1

Index of contents

• Goal

• Mindfulness

• Experiment description

• Analysis of results

• Threats to validity

• Conclusions and future works

2

ESEM 2014 10/15/2015

Paper #86 2

Goal

To present a controlled experiment carried out for

evaluating the effects of mindfulness in:

• quality

• performance

of conceptual models developed by our students.

3

• Is good practicing mindfulness for software engineers?

Mindfulness: at the begining

– Movement began in the 60/70 in the USA.

– Young people, looking for something new, went to

India attracted by the Eastern philosophy.

4

ESEM 2014 10/15/2015

Paper #86 3

– Jon Kabat--Zinn (biologist at a hospital in Massachusetts)

began using Mindfulness to patients with chronic

diseases.

– He developed MBSR (Mindfulness Based Stress

Reduction)

– Germer & Siegel (psychologists) in Boston studied the

effects of mindfulness.

5

Mindfulness: at the begining

Formal mindfulness*

Meditation

• A mental exercise of deep contemplation, accompanied by:

– Silence

– Relaxation

– Breath control

"Positive Psychology Therapies", Antoni Adsera. 3 Temes Publishers. 9th Edition, 2012. 6

ESEM 2014 10/15/2015

Paper #86 4

Formal mindfulness

Step by step

7

Step What to do

1 Sit up right.

2 If there is a limit in time, set an alarm.

3 Focus on various parts of your body in a

conscious way.

4 Take three slow, deep breaths.

5 Begin to breathe normally, but with your

focus on your breathing.

6 If thought comes, you experience it and

gently let it go.

7 Enjoy this mental state.

8 Return to contact with external reality.

Informal mindfulness

8

* Source: http://www.recursosdeautoayuda.com/

*

ESEM 2014 10/15/2015

Paper #86 5

Informal mindfulness

9*Vicente Simón. Course of Mindfulness. Seville, 2013 October.

Calm the mind

See with clearness

Make the right decision

What is going to happen?– Mindfulness is now subject of scientific studies.

10

ESEM 2014 10/15/2015

Paper #86 6

Why Mindfulness?• Neurological

– It diminishes the activity of DMN (Default Mode Network).

– It increases the activity of left hemisphere (positive

thoughts)

– It develops a new Default Network, with greater self-

awareness and a tendency to daydream less.

11

Why Mindfulness?

• Cognitive

– Working memory capacity

– Creativity

– Awareness

– Reduction of mind

wandering

• Affective

– Compassion

– Empathy

– Assertiveness

– Emotional intelligence

– … 12

ESEM 2014 10/15/2015

Paper #86 7

Why Software Engineers?

13

“Computer science should be more

directly devoted to making the world a

better place. ”

B.Shneiderman

• Because…

– In 1980, the concept

Software Psychology

appeared in term of “the

study of human

performance in using

computer and

information systems”

Why Software Engineers?• Because…

– They are generally introverts and very technical.

– They need to empathize with clients’ needs (social

interaction).

14

ESEM 2014 10/15/2015

Paper #86 8

Why conceptual modelling?

• Because

– It’s a difficult task for our students.

– Mental clearness is necessary.

– Measuring conceptual modelling is less subjective than

measuring requirements.

15

Experiment description (GQM)

Analyze the practice of mindfulness

For the purpose of evaluating its effects

With respect to the performance of students developing conceptual modelling, and the quality of the developed models

From the point of view of the experimenters

In the context of sophomore of the degree in SE (Introduction to Software Engineering & Information Systems, ISEIS) at the University of Seville.

16

ESEM 2014 10/15/2015

Paper #86 9

Pre– exercise

(ERASMUS)

Intuitive-Idea Experiment

17

Post– exercise

(End of Degree Project)Cross– Training

/ four weeks

Group 2: Public Speaking

Group 1: Mindfulness

Variables

x1,..,xn y1,..,ymExperimental process

•Cross-Training

•Effectiveness

•Efficiency

•Pre/Post Exercise

Two levels:

Pre: ERASMUS

Post: EoDP

18

Students advancedin SE knowledge

Two levels:

Mindfulness

Public Speaking

ESEM 2014 10/15/2015

Paper #86 10

Dependent Variables

• Dependent variables

– Effectiveness measured as:

������������� �����������������

������������� �����������������

– Efficiency measured as:

N�����������������������������

Time

19

Experimental Design

As consequence… 2x2 mixed factorial design

Cross-training

Mindfulness Public Speaking

Exercise Pre (ERASMUS) G1 G2

Post (EoDP) G1 G2

20

ESEM 2014 10/15/2015

Paper #86 11

Experimental Design*

N O X O

N O O

N: Nonrandom, O: Observation, X: Treatment

*D. T. Campbell and S. Julian. Experimental and Quasi–Experimental Designs for Research. Wadsworth, United States, 1963.

21

Experiment planning

22

T

ESEM 2014 10/15/2015

Paper #86 12

Analysis of effectiveness

23

1,0

0,8

0,6

0,4

0,2

0,0

G1 (Mindfulness) G2 (Public Speaking)

Effectiveness (Pre-Exercise)

Effectiveness (Post-Exercise)

8

Analysis of effectiveness

24

,700

,650

,600

,550

,500

,450

,400

Pre-Exercise(ERASMUS)

Post-Exercise(EoDP)

Mindfulness (G1)

Public Speaking (G2)

Cross-Training

ESEM 2014 10/15/2015

Paper #86 13

Analysis of effectiveness

ANOVA for mixed—factorial design

25

Analysis of efficiency

26

1,0

0,8

0,6

0,4

0,2

0,0

G1 (Mindfulness) G2 (Public Speaking)

10

11

1031

8

Efficiency (Pre-Exercise)

Efficiency (Post-Exercise)

ESEM 2014 10/15/2015

Paper #86 14

Analysis of efficiency

27

Pre-Exercise(ERASMUS)

Post-Exercise(EoDP)

Mindfulness (G1)

Public Speaking (G2)

Cross-Training

,400

,200

,300

,500

Analysis of efficiency

ANOVA for mixed—factorial design

28

ESEM 2014 10/15/2015

Paper #86 15

Threats to validity

• Conclusion

– The size of the sample

• Internal

– Maturation -> Knowledge in SE

– Selection -> Non random assignment

• Construct

• External

– Materials

– Subjects

29

Conclusions

• Mindfulness students obtained similar results in less time.

• The number of mindfulness sessions could have been

increased.

• The practice of mindfulness was very interesting for our

students.

• Random assignment could be considered in future

replications.

30

ESEM 2014 10/15/2015

Paper #86 16

Future work

• Several replications of experiment

• Change the object of experiment

– Technological problems

– Elicitation/negotiation requirements

• Develop a grown—personal program based on mindfulness

to software organizations.

31

*

Thank you very much,

32* Words cloud generated using the text of the paper by http://www.wordle.net developed by Jonathan Feinberg

ESEM 2014 10/15/2015

Paper #86 17

Analysis of effectiveness (Details)

• Both groups have a statistically better effectiveness after

training than before training, measured by the percentage of

semantic expressiveness achieved.

• However, the interaction between Pre/Post–Exercise and

Cross–Training is not significant at the 0:01 level. This means

that, although the mean of effectiveness varied significantly

for both exercises, the effect of the Cross–Training is not

necessarily linked to these differences.

33

Analysis of efficiency (Details)

• This analysis revealed a significant effect ( alpha = 0:01) for

Pre/Post–Exercise, i.e.

– both groups had a statistically better efficiency after

training than before training, as measured by the semantic

expressiveness achieved per unit of time.

• The interaction between Pre/Post–Exercise and Cross–

Training is also significant.

– This means that, although the mean efficiency varied

significantly for both exercises in both groups, the practice

of mindfulness is linked to a higher improvement in the

efficiency.

34