is it safe to use wsjf for prioritisation in financial ...1372030/fulltext01.pdf · trying to scale...

IN DEGREE PROJECT INDUSTRIAL MANAGEMENT,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2019

Is it SAFe to use WSJF for prioritisation in financial software development?A case study of prioritisation needs at a Swedish bank

GUSTAV DAHLSTRÖM

JESPER ROBERTSSON LUND

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF INDUSTRIAL ENGINEERING AND MANAGEMENT

This page is intentionally left blank

Is it SAFe to use WSJF for prioritisation in financial software development?

A case study of prioritisation needs at a Swedish bank

by

Gustav Dahlström Jesper Robertsson Lund

Master of Science Thesis TRITA-ITM-EX 2018:XYZ

KTH Industrial Engineering and Management Industrial Management

SE-100 44 STOCKHOLM

Är det SAFe att använda WSJF för prioritering inom finansiell

mjukvaruutveckling?

En fallstudie av prioriteringsbehov hos en svensk bank

av


Examensarbete TRITA-ITM-EX 2018:XYZ KTH Industriell teknik och management

Industriell ekonomi och organisation SE-100 44 STOCKHOLM

Master of Science Thesis TRITA-ITM-EX 2019:393

Is it SAFe to use WSJF for prioritisation in

financial software development?


Approved

2019-06-15 Examiner

Pernilla Ulfvengren Supervisor

Matti Kaulio Commissioner

Contact person

Christer Berg

Abstract As a response to digitalisation and new niche competitors, incumbent banks tries to increase their flexibility by adopting agile principles at their development departments. However, the flexibility of the agile departments is constrained by the flexibility of the rest of the organisation, which are still working according to the waterfall method. To address this problem, several incumbent banks are focusing towards scaling their agile work to the entire organisation. The case company for this study is a bank trying to scale their agile work by using the Scaled Agile Framework (SAFe). Nonetheless, there is scepticism within the bank whether Weighted Shortest Job First (WSJF), the prioritisation model used in SAFe, is applicable in financial software development. The available case studies are often conducted by consultancy firms implementing the framework and are focused towards the positive effects, excluding challenges or adaptions made for specific industrial contexts. This study contributes to this gap in the literature with empirical findings by assessing the adaptability of WSJF and its ability to satisfy the prioritisation needs of financial software development. Moreover, empirical data is used to formulate recommendations for how WSJF can be adapted and integrated with the current processes of financial software development.

To operationalise the study, the following research question was formulated: "How could WSJF be modified to better suit financial software development?". To answer upon this, a qualitative case study employing semi-structured interviews was conducted. The semi-structured interviews were used to identify the prioritisation needs at a selected case department. The needs were used to formulate prioritisation requirements that were structured according to an analytical framework. The framework was used to evaluate how many of the requirements WSJF were able to satisfy. Moreover, the analytical framework was used as a guideline in formulating the recommendations of modifications to better satisfy the prioritisation needs.

The results showed that no single set of needs is able to conclude the prioritisation needs of all organisational levels. Employees on an operational level have a more practical approach towards prioritisation and requires a model that is easy to use and time efficient. Meanwhile, the strategic level requires a model with high precision that could estimate the value of deliveries monetarily, allowing them to track the realisation of deliveries in the financial figures. WSJF are able to satisfy the needs of the operational level but fails to satisfy the needs of the strategic level. Derived from the obtained needs of both levels, the following two modifications are recommended: Establish a uniform value definition that are used by the entire organisation and Implement a prioritisation model that can combine monetarily CoD at a strategic level with relative CoD at an operational level. By implementing these changes the percentage of satisfied requirements increased from 50% to 89% on a strategic level and from 85% to 90% on an operational level. Resulting from this, the conclusion is that WSJF can be modified to better suit financial software development by implementing the recommended changes formulated in this report.

Key-words: Prioritisation, WSJF, CD3, SAFe, Agile, Cost of delay.

Examensarbete TRITA-ITM-EX 2019:393

Är det SAFe att använda WSJF för prioritering inom finansiell mjukvaruutveckling?


Godkänt

2019-06-15

Examinator

Pernilla Ulfvengren

Handledare

Matti Kaulio Uppdragsgivare

Kontaktperson

Christer Berg

Sammanfattning De traditionella storbankerna försöker implementera agila arbetssätt inom deras utvecklingsavdelningar för att öka flexibiliteten. Detta är ett svar på en ökad digitalisering och nya nischade konkurrenter. Dock är flexibiliteten av de agila arbetssätten begränsad av flexibiliteten av den övriga organisationen, som fortfarande jobbar enligt vattenfallsmodellen. För att adressera denna begränsning fokuserar bankerna på att skala upp det agila arbetet så att det täcker hela organisationen. I denna studie studeras en av dessa banker, som har valt att använda sig av det agila skalningsramverket Scaled Agile Framework (SAFe). Trots valet av SAFe finns en viss skepticism huruvida den medföljande prioriteringsmodellen Weighted Shortest Job First (WSJF) lämpar sig för finansiell mjukvaruutveckling. De studier som finns tillgängliga kring SAFe och WSJF är till stor del framtagna av konsultfirmor vars uppgift är att implementera SAFe. Således är dessa rapporter fokuserade på de positiva effekter som uppnås, samtidigt som utmaningar och anpassningar för det specifika industriella kontexten utelämnas. Detta skapar ett gap inom litteraturen, till vilket denna studie ämnar att bidra med empiriska resultat från en undersökning av WSJFs förmåga att tillfredsställa och anpassas efter prioriteringsbehoven inom finansiell mjukvaruutveckling. Därutöver ligger de empiriska resultaten till grund för rekommendationer gällande hur WSJF bättre kan anpassas och integreras inom finansiell mjukvaruutveckling.

För att operationalisera studien formulerades följande forskningsfråga: "Kan WSJF modifieras för att lämpa sig bättre inom finansiell mjukvaruutveckling?". För att besvara detta genomfördes en kvalitativ fallstudie där semi-strukturerade intervjuer användes för att identifiera prioriteringsbehoven av en utvecklingsavdelning hos den undersökta banken. Behoven användes för att populera ett analytiskt ramverk, som sedan användes för att utvärdera hur många prioriteringskrav som WSJF lyckas tillfredsställa. Dessutom agerade ramverket som guide i framtagandet av rekommenderade modifikationer, för att öka antalet krav som WSJF tillfredsställer.

Resultatet indikerar att ingen enskild uppsättning av behov täcker samtliga prioriteringsbehov i organisationen. Anställda på en operationell nivå har en praktisk syn på prioritering och efterfrågar en modell som är tidseffektiv och enkel att använda. Anställda på en strategisk nivå kräver en prioriteringsmodell med hög precision som kan uppskatta värdet av en leverans monetärt, då detta möjliggör en uppföljning av hur leveransvärdet realiserats genom de finansiella siffrorna. WSJF tillfredsställer prioriteringskraven från den operationella nivån medan den misslyckas med att tillfredsställa hälften av kraven från den strategiska nivån. Utifrån de prioriteringskrav som samlats in rekommenderas följande två modifikationer: Fastställ en gemensam värdedefinition som används i hela organisationen och Implementera en prioriteringsmodell som kombinerar monetärt värde på en strategisk nivå med relativt värde på en operationell nivå. Genom att implementera dessa modifikationer ökar antalet tillfredsställda krav från 50% till 89% på en strategisk nivå och från 85% till 90% på en operationell nivå. Utifrån detta är slutsatsen att WSJF kan modifieras för att bättre lämpa sig för finansiell mjukvaruutveckling. Nyckelord: Prioritering, WSJF, CD3, SAFe, Agil, Cost of delay.

Contents

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Problematisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Purpose and Research Questions . . . . . . . . . . . . . . . . . . . . . 31.4 Scientific Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . 31.5 Industry Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . 41.6 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.7 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.8 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Literature Review 62.1 Agile Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2 Scaled Agile Framework . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 Using Cost of Delay in Prioritisation . . . . . . . . . . . . . . . . . . 9

2.3.1 Weighted Shortest Job First . . . . . . . . . . . . . . . . . . . 102.3.2 Cost of Delay Divided by Duration . . . . . . . . . . . . . . . 112.3.3 Qualitative Cost of Delay Using Urgency and Value . . . . . . 122.3.4 Useful Tools for Calculating Cost of Delay . . . . . . . . . . . 15

2.3.4.1 Assigning Cost of Delay to Enablers . . . . . . . . . 162.3.4.2 Minimal Viable Product . . . . . . . . . . . . . . . . 16

2.4 User-Centered Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.5 Situational Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.6 Summary of Literature . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3 Method 213.1 Model of Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.1.1 Work Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.1.2 Analytical Framework . . . . . . . . . . . . . . . . . . . . . . 22

3.2 Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.3 Pilot Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.4 Main Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.5 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.5.1 Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.5.2 Observation Methodology . . . . . . . . . . . . . . . . . . . . 303.5.3 Literature Search and Review . . . . . . . . . . . . . . . . . . 313.5.4 Document Study . . . . . . . . . . . . . . . . . . . . . . . . . 32

Contents

3.6 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.7 Reliability, Validity and Research Ethics . . . . . . . . . . . . . . . . 34

4 Empircal Findings 364.1 Introduction of the case company . . . . . . . . . . . . . . . . . . . . 364.2 Needs and Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.2.1 Value-based Prioritisation . . . . . . . . . . . . . . . . . . . . 384.2.2 Difficult Valuation Cases . . . . . . . . . . . . . . . . . . . . . 40

4.2.2.1 Regulations and Compliance Risk . . . . . . . . . . . 414.2.2.2 Operational Risk . . . . . . . . . . . . . . . . . . . . 414.2.2.3 Enablers . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.2.3 Monitoring Delivered Value . . . . . . . . . . . . . . . . . . . 424.2.4 Roles of Prioritisation . . . . . . . . . . . . . . . . . . . . . . 434.2.5 Easy to Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5 Analysis and Discussion 465.1 Prioritisation Needs of Financial Software Development . . . . . . . . 46

5.1.1 Functional Perspective . . . . . . . . . . . . . . . . . . . . . . 485.1.2 Environmental Perspective . . . . . . . . . . . . . . . . . . . . 495.1.3 Organisational Perspective . . . . . . . . . . . . . . . . . . . . 505.1.4 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.1.5 Types of Projects . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.2 Assessment of WSJF in Financial Software Development . . . . . . . 515.2.1 Functional Perspective . . . . . . . . . . . . . . . . . . . . . . 535.2.2 Environmental Perspective . . . . . . . . . . . . . . . . . . . . 545.2.3 Organisational Perspective . . . . . . . . . . . . . . . . . . . . 545.2.4 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555.2.5 Types of Projects . . . . . . . . . . . . . . . . . . . . . . . . . 565.2.6 Summary of WSJF Evaluation . . . . . . . . . . . . . . . . . . 56

5.3 Recommended Prioritisation Approach for Financial Software Devel-opment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575.3.1 Uniform Value Definition . . . . . . . . . . . . . . . . . . . . . 575.3.2 Combining Monetary CoD with WSJF . . . . . . . . . . . . . 59

5.3.2.1 Integration Between WSJF and CD3 . . . . . . . . . 595.3.2.2 Recommendations for getting to Monetary Value . . 61

5.4 Assessment of WSJF with Modifications for Strategic Prioritisation . 63

6 Conclusion 676.1 Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

6.1.1 Academic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 696.1.2 Industrial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

6.2 Limitations and Future Research . . . . . . . . . . . . . . . . . . . . 70

APPENDICES I

A Interview Guide I

List of Figures

2.1 Illustration of SAFe (Scaled Agile 2018). . . . . . . . . . . . . . . . . 92.2 The WSJF formula (Scaled Agile 2018). . . . . . . . . . . . . . . . . 102.3 Components of CoD used in WSJF (Scaled Agile 2018). . . . . . . . . 102.4 The iterative valuation process using 5-Whys (Arnold & Yüce 2013). 122.5 Illustration of Cost of Delay x Urgency (Arnold 2017). . . . . . . . . 132.6 Urgency profile for a short benefit horizon and with a reduced peak

due to late delivery (Arnold & Yüce 2013). . . . . . . . . . . . . . . . 142.7 Urgency profile for projects with a long-life and reduced peak due to

late delivery (Arnold & Yüce 2013). . . . . . . . . . . . . . . . . . . . 152.8 Urgency profile for projects with a long-life and a peak unaffected by

delay (Arnold & Yüce 2013). . . . . . . . . . . . . . . . . . . . . . . . 152.9 Illustration of the UCD workflow. . . . . . . . . . . . . . . . . . . . . 17

3.1 Illustration of how the UCD approach is used to answer the researchquestions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.2 Workflow of the study. . . . . . . . . . . . . . . . . . . . . . . . . . . 253.3 Illustration of the funnel model. . . . . . . . . . . . . . . . . . . . . . 303.4 Illustration of the procedure for analysing qualitative data by Miles

& Huberman (1994). . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.1 Value distribution of a development team at Maersk, investigated byBlack Swan Farming (Arnold & Yüce 2013). . . . . . . . . . . . . . . 39

5.1 Illustration and formula for calculating enabling value. (Based ontheories from Wright (2018)). . . . . . . . . . . . . . . . . . . . . . . 58

5.2 Illustration of how monetary CoD interacts with WSJF. . . . . . . . 605.3 Illustration of combination of CD3 and WSJF. . . . . . . . . . . . . . 615.4 Urgency profile for ideas with a long-life and a peak unaffected by

delay (Arnold & Yüce 2013). . . . . . . . . . . . . . . . . . . . . . . . 625.5 Urgency profile for for regulatory projects with a deadline. . . . . . . 63

List of Tables

3.1 Analytical framework. . . . . . . . . . . . . . . . . . . . . . . . . . . 233.2 Interviews and the observation conducted in the pilot study. . . . . . 263.3 Interviews of the main study. . . . . . . . . . . . . . . . . . . . . . . 283.4 Summary of documents used to conduct this study. . . . . . . . . . . 32

5.1 Populated evaluation framework on an operational level. . . . . . . . 475.2 Populated evaluation framework on a strategic level. . . . . . . . . . . 485.3 Assessment of WSJF on an operational level. . . . . . . . . . . . . . . 525.4 Assessment of WSJF on a strategic level. . . . . . . . . . . . . . . . . 535.5 Distribution of satisfied, unsatisfied and partly satisfied needs on an

operational and a strategic level. . . . . . . . . . . . . . . . . . . . . . 565.6 Unsatisfied needs with recommended solutions. . . . . . . . . . . . . . 575.7 Assessment of modified WSJF from a strategic perspective. The re-

quirements affected by the recommendations are highlighted in bold. . 645.8 Distribution of satisfied, unsatisfied and partly satisfied needs on a

strategic and operational level before and after modifications. . . . . . 65

1Introduction

This chapter aims to introduce the reader to the study by presenting its backgroundas well as presenting the problem. This is followed by the purpose and the researchquestions which forms the foundation for this study. Lastly, the contribution, delim-itations and limitations of the study are presented.

1.1 BackgroundThe software development industry becomes more complex for each day, the indus-try is fast changing and the customers demand a higher level of customisation, atthe same time as the products become more complex and harder to specify. Thiscreates a business climate that challenges the industry to be efficient and flexible,forcing its actors to change and adapt their processes to stay competitive (Stober &Hansmann 2010). Many attempts to address these challenges have been made but itwas not until 2001, when seventeen software developers met at the Snowbird ski re-sort in Utah and formulated the agile manifesto, a wide-spread change was initiated.The agile manifesto promotes: Individuals and interactions over processes and tools;Working software over comprehensive documentation; Customer collaboration overcontract negotiation; Responding to change over following a plan (Beck et al. 2001).The manifesto led to the development of a handful of frameworks, including famousframeworks such as Scrum and Extreme Programming.

The agile principles were quickly embraced by software development companies andhave since then spread to other industries where software development partly con-stitutes to the main business. The Swedish financial sector is one example werethe business climate has moved in the same direction as the software developmentindustry. With its niche competitors utilising digitalisation to gain competitive ad-vantages, the incumbent banks realise the risk of having a technical debt and fallingbehind in technology (Vives 2017). This has led to a search for methodologies thatreplace the traditional waterfall model, that is used for technical development withinthe incumbent banks today. With a replacement of the waterfall model, the banksaim to increase their efficiency and ability to respond to changes in customer be-haviour. With the search for alternative methodologies, many of the incumbentbanks have transitioned their software developing departments to work in accor-dance with the agile principles. However, the banks are often large and consists of

1

1. Introduction

multiple divisions, making the flexibility of the agile frameworks constrained by theflexibility of the rest of the organisation. To address this problem, many banks searchbeyond traditional agile frameworks and attempt to scale their agile work through-out the organisation. The banks’ interest in scaling the agile work becomes clearwhen browsing the website of two of the larger frameworks for scaled agile work, theScaled Agile Framework (SAFe) and Large Scale Scrum (LeSS). The SAFe websiteboasts with case studies from Standard Bank, Nordea, Deutsche Bank and CapitalOne and the LeSS website does the same with Bank of America and JP MorganChase (Scaled Agile 2019, The LeSS Company 2019).

1.2 ProblematisationThe scaling of agile work has been successful in many organisations, leading to re-sults such as shorter lead-times, reduced costs and higher productivity (Scaled Agile2019, The LeSS Company 2019). The success stories from SAFe implementationshave drawn the attention of many banks, such as the case company for this study,towards SAFe. The case company is one of the larger full-service banks in Sweden.The bank has already begun with a small scale implementation of SAFe, where afew departments have been working according to SAFe for a few months.

Even though the implementation of SAFe at the case company has already started,there is some scepticism among its employees of how well the SAFe, and especiallythe integrated prioritisation model Weighted Shortest Job First (WSJF), is suitedfor the financial industry. WSJF prioritises the backlog based on the Cost of Delay(CoD) of each feature. Simply, this means that the prioritisation is based upon anassessment of what feature leads to the highest loss in potential value if not beingdeveloped immediately. The scepticism arise from the fact that the financial indus-try is highly regulated and the incumbent banks often have a large technical debtcompared to its niche competitors. This makes regulatory projects and technicalenablers, such as updates of old systems, two common types of projects in financialsoftware development. Today, most of the regulatory projects are imperative andthe system updates are seen as necessary development costs and does not have avalue connected to them. The lack of a clear business value makes it challenging toestimate the CoD of these types of projects. Consequently, several employees at thecase company question if WSJF could be used in this context or if it is developedwith more traditional software development in mind, where a larger share of theprojects are connected to business development. Other concerns among the employ-ees regarding WSJF are the usage of relative numbers with unclear scales and thecombination of different units, such as time and business value. Resulting from this,the case company wants to investigate if WSJF satisfies the prioritisation needs offinancial software development or if the model needs customisation.

Even though there is a significant number of reports from case studies where SAFeand WSJF have been implemented, no report presents the challenges faced or adap-tions being made for the specific industry context. A possible reason for this mightbe that the case studies are conducted by the founders of SAFe or by consultancy

2

1. Introduction

firms implementing it. Hence, one can question the bias of the authors and the objec-tivity of the reports. This creates a lack of independent studies, creating problemsfor companies to assess the applicability of using WSJF in their specific context.Thus, there is a need for an independent study investigating how well WSJF, in itsoriginal form, adapts to industry specific challenges. This study attempts to addressthe gap in the literature by investigating the industry-specific prioritisation needsin financial software development and assess the adaptability of WSJF.

1.3 Purpose and Research QuestionsThe purpose of this study is to assess the adaptability of WSJF by investigating itsability to satisfy the prioritisation needs of financial software development. More-over, empirical data will be used to formulate recommendations for how WSJF couldbe adapted and integrated with the current processes of financial software develop-ment.

From a sustainability perspective, the purpose of this study is to improve the eco-nomic sustainability of the financial industry by enlighten the value of regulatoryprojects and technical enablers. This would allow the industry to focus its re-sources to maximise its value efficiency. Additionally, the study aims to achieve animproved social sustainability by increasing the transparency of value in the organ-isation. This would facilitate the involvement of the employees in the prioritisationprocess and thereby give them increased influence in the planning of their own work.

To fulfil the purpose of the study, the following research questions (RQs) have to beaddressed:

Main RQ How could WSJF be modified to better suit financial software develop-ment?

Sub RQ1 What are the prioritisation needs of financial software development?

Sub RQ2 How does WSJF satisfy the needs of prioritisation in financial softwaredevelopment?

1.4 Scientific ContributionThis study will contribute to the scientific field of prioritisation by conducting anindependent study of the prioritisation model WSJF, investigating its adaptability.Moreover, the study will contribute with empirical findings of the prioritisation needsof the financial software development industry.

3

1. Introduction

1.5 Industry ContributionThis study will contribute to the financial industry by collecting empirical data ofthe prioritisation needs of financial software development. This information willbe used to formulate recommendations of how WSJF can be adapted in such anindustry.

1.6 DelimitationsThis study is delimited to focused on prioritisation in financial software developmentwith an underlying focus of prioritisation in a scaled agile environment. Moreover,the study covers prioritisation on a team level and higher, excluding prioritisationmade on an individual level.

Due to the geographical location of the case company, the data collection is geo-graphically limited to Stockholm, Sweden. Additionally, the selection of case com-pany restricts the coverage of the study to full-service banks.

1.7 LimitationsDue to the competitive nature of the software development industry, companies maybe unwilling to share their experience of prioritisation in a scaled agile environment,especially where any modifications to the original model have been made. Conse-quently, the sourcing of interviewees may be limited to the employees of the casecompany. Furthermore, the study is limited by the previous strategic decisions ofthe case company to scale their agile work by using SAFe. Consequently, the studyinvestigates prioritisation in an organisation with the target of scaling their agilework, limiting the study to only investigate prioritisation where a scaling decisionhas already been taken.

As the study is conducted as a Master thesis, the study is obligated to follow a timerequirement determined by the supervising institution. The time for the study isdetermined to five months, limiting the possibility to test the results of the study ina practical environment. Resulting from this, the testing of the recommendationsformulated in this report will be limited to theoretical testing and discussions withemployees.

1.8 Thesis OutlineChapter 1: Introduction. This chapter aims to introduce the reader to the studyby presenting its background as well as presenting the problem. This is followed bythe purpose and the research questions which forms the foundation for this study.Lastly, the contribution, delimitations and limitations of the study are presented.

4

1. Introduction

Chapter 2: Literature Review. This chapter introduces the literature thatforms the theoretical foundation of this study. Initially, information regarding ag-ile principles and SAFe are presented. This is followed by a review of the currentprioritisation literature. Finally, the chapter presents frameworks that are used tooperationalise the study.

Chapter 3: Method. This chapter begins with an introduction of the work pro-cess and the analytical framework used to structure the study. This is followed bythe research design and work procedures of the pilot and the main study. Lastly,the data collection and analysis methods are presented followed by a discussion ofthe reliability, validity and research ethics of the study.

Chapter 4: Empirical Findings. This chapter presents the context in which thedata has been collected followed by the empirical findings from the interviews andthe observation. The empirical findings are grouped into common themes to obtaina better overview of the main topics.

Chapter 5: Analysis and Discussion. The following chapter is structured basedon the research questions. Initially, an analysis of the empirical data is used to an-swer upon Sub RQ1. The answers are then used to assess the suitability of WSJFand thereby answer upon Sub RQ2. The findings from the two Sub RQs are thenused to answer the main RQ.

Chapter 6: Conclusions. This final chapter concludes the thesis and provides aconcrete answer to each of the research questions. This is followed by an explanationof the implications of the thesis from an academic and industrial perspective. At last,the limitations of the study are presented in combination with recommendations forfurther research.

5

2Literature Review

This chapter introduces the literature that forms the theoretical foundation of thisstudy. Initially, information regarding agile principles and SAFe are presented. Thisis followed by a review of the current prioritisation literature. Finally, the chapterpresents frameworks that are used to operationalise the study.

2.1 Agile PrinciplesTo better grasp the current prioritisation model and prioritisation needs at the casedepartment, their current work procedures must be understood. The case depart-ment works according to the agile methodology Scrum. Hence, literature regardingthe agile mindset and Scrum play an important role in mapping processes at thecase department.

As described in the background for this study, the agile Manifesto was formulatedby seventeen people at The Lodge at Snowbird ski resort in 2001. The manifestoincludes 12 principles, handling subjects such as how to treat the firm’s customers,employees, methods, etc. These principles support the values of the agile mindset,which are compiled into the following four values:

• Individuals and interactions over processes and tools.• Working software over comprehensive documentation.• Customer collaboration over contract negotiation.• Responding to change over following a plan (Beck et al. 2001).

Agility permeates the four values, where the values focus on collaborations and trustrather than comprehensive plans. The agile principles should not be seen as a setof tools, instead it should be seen from a broader perspective where the tools actas support in a comprehensive mindset. To have the right mindset is equally, if notmore, important than implementing the right tools (Measey 2015).

Besides being adopted by the traditional software developing companies, agile method-ologies have been adopted by the financial sector. One example of the agile adaptionin the financial sector is the adoption by Capital One, they began their agile im-plementation in 2004. The implementation was driven by the desire to reduce the

6

2. Literature Review

time to market of their IT projects. Other arguments for an Agile transformationare an increasing competitive IT landscape, increased efficiency and to sustain therevenue growth in a more digital market (Silva & Doss 2007). However, there aresome scepticism in taking the leap towards working agile. Agile software develop-ment promotes working software over comprehensive documentation, which is theopposite of the requirements in the strictly regulated financial sector. Furthermore,the industry tends to stick to the traditional waterfall method due to the largeamount of regulation within the industry. This is a result of the beliefs that thewaterfall method is more predictable than the agile mindset (Ihme 2013). Despitethese doubts, the trend indicates an increased adaption of the agile mindset in thefinancial sector.

Many banks work according to the agile framework Scrum. Scrum is together withExtreme Programming, of which Scrum overtook the dominant role in 2004, thetwo most used agile frameworks. Scrum is an incremental framework building uponthree pillars of control: Transparency, Inspection and Adaption. The work is di-vided into shorter increments, sprints, that often have a duration between one weekand one month. Scrum is built around three roles, which are briefly describes below:

• ScrumMasterThe ScrumMaster have a servant-leadership approach for the developmentteam. The role of the ScrumMaster is to support the development team whileprotecting it from external distractions.

• Product ownerThe product owner prioritises and defines the product backlog. The Productowner determines what needs to be done and how to maximise the value de-livery.

• Development teamThe development teams are self-organised and cross-functional with a mix ofcompetences. The teams determine how to best realise the requirements ofthe current sprint. (Measey 2015)

2.2 Scaled Agile FrameworkThe agile principles are clearly defining how agile teams should work. However,it has been criticised for not describing how multiple teams could collaborate inlarge projects or how to involve product portfolio management (Brenner & Wun-der 2015). In many organisations, the development teams are working according toScrum while the rest of the organisation often use the traditional waterfall modelfor project planning and budgeting (Dingsøyr et al. 2014). This creates a frictionbetween the agile teams and the waterfall organisation. To address the challenges oflarge scale projects, frameworks like LeSS and SAFe have emerged, trying to applythe agile methodologies throughout entire organisations. Since the case companyhas decided to use SAFe to guide their agile scaling process, it is important for this

7


study to understand its main principles.

Dean Leffingwell, the creator of SAFe, attempts to describe how the agile method-ologies can be adopted in an entire organisation by combining existing lean and agileprinciples. In SAFe, large projects are divided into epics, capabilities, features andstories. Epics are used on the highest level and are a container for large solutionsthat requires financial approval before implementation. Epics are further brokendown to capabilities which often cuts across more than one value stream. A capabil-ity is sized to fit multiple features that contains a benefit hypothesis and acceptancecriteria to fulfil stakeholder needs. Lastly, stories are used at the lowest level andcontains a short description of desired functionality. (Knaster & Leffingwell 2017)

SAFe consists of processes for team, program, large solution and portfolio level. Theteam level uses the traditional principles of Scrum, working in 2-week sprints. Atthe program level, SAFe extends the ideas of Scrum to a higher level, but instead ofworking in 2-week sprints, the program level works in Agile Release Trains (ART),composed of 5 sprints. Each ART consists of multiple teams making deliveries intothe ART. The five sprints make up a 10-week period, called a Program Increment(PI). Before each PI, there is a two-day planning event where all teams, togetherwith the business owners, are planning what features to include in the upcoming5 sprints. After each PI, there is another event including all teams and programstakeholders, called Inspect and Adapt (I&A). At the I&A, the teams have demos ofthe current state of the solution, a quantitative follow up of actual delivered valuecompared to planned value and a retrospective of the completed work. The largesolution level includes economical frameworks and processes to coordinate multipleARTs. At the highest level, the portfolio level, SAFe has a focus on optimising valuestreams to help management prioritising the epics and features to be distributed tothe ARTs. (Knaster & Leffingwell 2017) Figure 2.1 shows an illustration of SAFe.

8


Figure 2.1: Illustration of SAFe (Scaled Agile 2018).

2.3 Using Cost of Delay in PrioritisationTo better understand WSJF, one must understand its foundational mindset. WSJFbuilds upon the concept of Cost of Delay (CoD). Hence, literature regarding CoDare needed to better evaluate the SAFe prioritisation model in a financial softwaredevelopment context.

CoD is an efficient mindset in communicating how the outcome of a project is effectedby time. In simple terms, CoD indicate how much value leaks over a predefined pe-riod of time. This could for example be in terms of missed revenue or higher costsdue to a delay. CoD is expressed in the unit $/time, where time is a predefinedinterval, often selected to one week, month or year. Arnold & Yüce (2013) describesthe concept of CoD as following:

"The Cost of Delay of a feature is the value that could be generated overa given period of time, if it was available immediately."

When calculating the CoD, one must assess how much value being missed by nothaving the feature ready when needed (Arnold & Yüce 2013). An example of thiscould be a feature that increases efficiency by automatising administration tasks.The feature makes it possible to automatise the work of two full-time employeesand the CoD is evaluated per month. Consequently, the CoD for the feature is the

9


cost of the two full-time employees per month.

In the book "The Principles of Product Development Flow: Second GenerationLean Product Development", Reinertsen (2009) describes CoD as "The one thing toquantify" and explains that only 15% of the product managers are aware of CoD.Reinertsen continues by explaining that the value estimations can vary by a factorof 50 between members in the same team. Hence, value estimations by intuition area poor way to estimate CoD and instead he argues that it should be calculated.

2.3.1 Weighted Shortest Job FirstSAFe use WSJF for prioritisation, making use of the concept of CoD. WSJF isused to valuate and prioritise features before every PI planning, to ensure that thebacklog is updated and the most valuable features are included in the upcomingsprints. The WSJF formula, shown in Figure 2.2, is composed by dividing the CoDwith the job duration (Knaster & Leffingwell 2017).

Figure 2.2: The WSJF formula (Scaled Agile 2018).

In WSJF, the CoD consists of three parameters: User-Business Value, Time Critical-ity and Risk Reduction/Opportunity Enablement, see Figure 2.3. The user-businessvalue corresponds to what the user wants and the revenue impact on the business.The Time criticality represents how the value decay over time, is there a potentialpenalty or other consequences if we delay? The third element, risk reduction/oppor-tunity enablement corresponds to what else the feature might do for the business.Does it reduce a risk, or will it open up new business opportunities? (Knaster &Leffingwell 2017)

Figure 2.3: Components of CoD used in WSJF (Scaled Agile 2018).

The valuation of each component is not made in absolute numbers, instead it isbased on a relative comparison of other items in the backlog, using a Fibonacci

10


scale (Scaled Agile 2018). The feature with the highest perceived business value isassigned the highest number in the business value component and so on. This makesthe model simple and effective to use for backlog prioritisation.

The job duration can be difficult to determine, especially early on in the projectplanning phase when the knowledge regarding who is going to perform the job andthe capacity allocations of the teams are limited. WSJF suggests that story pointsare a good proxy for job duration (Knaster & Leffingwell 2017). Story points area relative measure that describes how much functionality a team is able to deliver.The definition of a story point is unique to the specific team and is based on thecomplexity of work, the team’s skill and level of experience (Moreira 2013).

Even though WSJF has received positive feedback for bringing a value focus into theprioritisation process, the model has been criticised by parts of the agile communityfor:

• Having unclear definitions of its CoD components.• Combining monetary value measures with other unit measures, such as time

criticality and risk.• Having relative numbers which makes it hard to quantify the value outcome

(Arnold 2014).

The SAFe framework sees the relative numbers as an advantage, since the featuresare only compared against each other rather than given their own value. This keepsthe valuation process relatively quick and simple. However, Arnold & Yüce (2013)argues that putting an absolute monetary value on each component in a projectbacklog can have even higher effects on value outcome.

2.3.2 Cost of Delay Divided by DurationCost of Delay Divided by Duration, also known as CD3, is a prioritisation methodthat makes use of monetary CoD in the prioritisation process. In CD3, the featurewith the highest CD3-value will deliver the highest value per time unit. The CD3-value is calculated by dividing the CoD with the duration of the feature. The resultindicates what feature to be developed to reduce the total CoD as fast as possible.This method is effective when the development capacity is limited and the value ofthe features needs to be compared (Arnold & Yüce 2013).

In CD3, the CoD is often broken down into four types of benefits. These are: IncreaseRevenue, Protect Revenue, Reduce Costs and Avoid Costs. To get to these values,Lean methods such as 5-Whys are commonly used. The method simply encouragesits user to ask "why?" until the benefits are identified and categorised (Arnold &Yüce 2013). The iterative process is illustrated in Figure 2.4.

11


Figure 2.4: The iterative valuation process using 5-Whys (Arnold & Yüce 2013).

When the benefits are identified and categorised, Arnold & Yüce (2013) suggesta two-stage approach for estimating the value of the feature. The first step is tounderstand what effects being connected with the feature. For example, if old datacan be migrated to a modern system, the dependency of the old system is removedand the licence can be terminated. Hence, the CoD of the project is the cost of thelicence for the older system. If the effects are hard to estimate, the second step isto investigate the cost of alternatives. An example of this is when a feature enablesautomation of a manual process. The CoD for this process can be estimated to thealternative cost for the feature, which in this case could be equal to the cost of thecurrent manual work.

When the CoD is estimated, the development team is estimating the developmenttime of the feature. The duration could either be estimated in absolute numberssuch as weeks or in relative numbers such as story points. The CD3 method isindependent on what unit being selected as long as the duration of all the comparedfeatures are estimated using the same unit. Once the CoD and the duration of thefeature is estimated, the feature is assigned a CD3-score and could be compared toother features in the backlog.

2.3.3 Qualitative Cost of Delay Using Urgency and ValueAn alternative solution to WSJF, handling the critique of mixing units, was devel-oped by Arnold (2017), making use of CoD and urgency. He is clear that these twofactors are not additive but that they are dependent on each other. The simplestway to illustrate this dependency is by a 3x3 matrix with urgency on the x-axis andvalue on the y-axis, see Figure 2.5. This is a qualitative approach and the scales forvalue and urgency are suggested as follows:

12


Urgency• ASAP — The highest level of urgency. If we do not deliver this now, the

value will decrease or someone else will get there before us.• Soon — The middle level of urgency. If we do not deliver soon, the value will

start to decrease.• Whenever — The lowest level of urgency. The total value is not really af-

fected by delay.

Value• Killer — The highest level of value. If we do the project, we make a "killing"

and if we do not, we will be "killed". Very few items should reach this level.• Bonus — The middle level of value. Could be things that we want to tell the

customers or even make a press release about.• "Meh" — The lowest level of value. This is the work that are valuable and

worth doing but not the things that get customers to excited.

Figure 2.5: Illustration of Cost of Delay x Urgency (Arnold 2017).

Derived from the matrix, an item in the upper right corner has a high CoD while anitem in the lower left corner has a low CoD. Naturally, you begin with tasks fromthe upper right corner and work your way down to the lower left. However, oneimportant thing to remember is that this scale is dynamic, meaning the perceptionof value and urgency can change over time. Hence, the matrix should be updatedregularly to ensure the right tasks are included in the upcoming sprint. According toArnold (2017), the method helps to improve prioritisation, enables better decision-making and creates discussions about value.

13


To get a deeper understanding of how urgency affects the cost of delay, Arnold &Yüce (2013) have developed three different urgency profiles. These profiles are help-ful to understand the life-cycle of benefits and the effects of late deliveries, whichare the two main things to consider when prioritising.

The first urgency profile is for projects with a short life-cycle and where the peakis affected by delay, see Figure 2.6. This is a common case in consumer electronicswhere the demand quickly ramps up and after a while it becomes standard foreveryone and hence the value of the project decreases (Arnold & Yüce 2013).

Figure 2.6: Urgency profile for a short benefit horizon and with a reduced peakdue to late delivery (Arnold & Yüce 2013).

The second urgency profile is for projects with a long life-cycle and where the peakis affected by delay, see Figure 2.7. This is often referred to as first mover advantage.If you are too late to market, it will be difficult to gain market shares. An exampleof this is PC operating systems, in which the market consists of a few main players(Arnold & Yüce 2013).

14


Figure 2.7: Urgency profile for projects with a long-life and reduced peak due tolate delivery (Arnold & Yüce 2013).

The third urgency profile is for projects with a long life-cycle and where the peakis unaffected by delay, see Figure 2.8. This is the easiest profile to calculate cost ofdelay. This could be an automation of a process or cost-cutting where the benefitof doing so is the same regardless of when it is done (Arnold & Yüce 2013).

Figure 2.8: Urgency profile for projects with a long-life and a peak unaffected bydelay (Arnold & Yüce 2013).

2.3.4 Useful Tools for Calculating Cost of DelayBesides the established theories regarding CoD, complimentary theories that oftenare mentioned in the CoD context are included in the literature review. The compli-mentary theories of interest for this study handled enablers and project break down.These theories are presented in Section 2.3.4.1 and 2.3.4.2.

15


2.3.4.1 Assigning Cost of Delay to Enablers

Some projects do not deliver direct value to the business, instead they enable forfuture value capturing (Scaled Agile 2018). These projects are called enabler projectsand are often quite difficult to valuate in terms of CoD. This could be projectslike transferring an old system written in COBOL into a new platform written inJava or Python, enabling for development of future features. The transfer itselfdo not add any business value but the system upgrade has enabled for future valuecapturing. Wright (2018) has developed an approach for assigning value to enablers.His approach is to assign the enabler a portion of the value it has enabled. The sizeof the portion is based on the enablers size in comparison to the enabled work. Forexample, if an enabler takes two months to develop and it has enabled value for aproject that takes six months to develop, then the enabler would get 25% of thetotal value since the total developing time is eight months.

2.3.4.2 Minimal Viable Product

Minimum Viable Product (MVP) is not a prioritisation tool itself. However, theconcept of MVP is used in the agile methodology and is an efficient mindset tobreak down projects into working prototypes. This concept is utilised when break-ing down projects into components in Scrum and SAFe. Consequently, theoriesregarding MVP are important to increase the understanding of how agile depart-ments break down and prioritise their projects.

MVP was defined by Frank Robinson in 2001 as a concept to obtain a high amountof customer validated learning to a minimal effort (Lenarduzzi & Taibi 2016). TheMVP is a product, developed with minimal possible effort, that offers a working so-lution in accordance with the customer needs. This enables the customer to interactwith the product and the actual user behaviour can be tracked. The MVP bridgeknowledge between the developers and customers to minimise the risk of developinga faulty product. Hence, the risk and cost of a project could be reduced by breakingdown projects into MVPs, as new or changed requirements are discovered in an earlyphase of the project development (Duc & Abrahamsson 2016).

2.4 User-Centered DesignAs one part of this study consists of formulating recommendations for how the pri-oritisation model could be improved, it requires continuous interactions with theemployees of the case department. Hence, a User-Centered Design approach (UCD)is used to guide this iterative process. Jokela et al. (2003) has identified four generalprinciples characterising a UCD:

• The active involvement of users and a clear understanding of user and taskrequirements.

• An appropriate allocation of functions between users and technology.• Iteration of design solutions.

16


• Multi-disciplinary design.

The UCD approach is based on an iterative mindset, in which the interaction be-tween the researcher and user steers the direction of each iteration (Jokela et al.2003). The activities of the UCD approach are illustrated in Figure 2.9.

Figure 2.9: Illustration of the UCD workflow.

The iterative part of the model contains of four steps. The first step is to know theuser, specify how the product will be used and in which context it will be used. Inthe second step, the information gathered in the first step is used to formulate userrequirements. Step three is to design the actual product based on the requirementsfrom the previous step. Then the solution needs evaluation to test if it meets theuser requirements, this is done in the fourth and last step. If the solution does notmeet the requirements, the process starts all over again from step one. This iterativecycle is performed together with the users until the designed solution meets the userrequirements. (Jokela et al. 2003)

2.5 Situational AnalysisStill & Crane (2017) recommends the usage of situational analysis in conjunction

17


with the UCD approach to understand how a product interact with the user in a realworld scenario. The situational analysis helps to assess the user needs and how theyare affected by their surroundings. This model is useful for this study as it guidesthe work to systematically categorise the prioritisation needs of financial softwaredevelopment. The situational analysis helps to understand the context and whatneeds that are related. The analysis takes a holistic approach and includes both theperspective of the users and other influencing factors. The different perspectives ofthe analysis are defined by Still & Crane (2017) as following:

• Functional analysis: What is the product expected to do?

• Environmental analysis: In what context will the product be used and howcould it be integrated in the user environment?

• Organisational analysis: How is the product design affected by the specificsof the organisation, its budget and target?

• Competitor analysis: What is the unique selling point of the product? Doesit exist products competing with yours?

• Materials analysis: How is your product affected by the choice of material?

• Content analysis: What content does the product have to process to satisfythe user needs?

By investigating each perspective of the situation, the researcher is able to under-stand the needs from all the client perspectives. Hence, the researcher is able tocomplete an assessment of all the clients needs and process them into product re-quirements that satisfies the client (Still & Crane 2017).

2.6 Summary of LiteratureAs the study intend to understand the prioritisation needs at a case departmentthat is working according to the agile framework Scrum, it is crucial for this studyto include the fundamentals of it. Scrum is built around the three roles Scrum Mas-ter, Product Owner and Development Team. These people work together in 2-weeksprints and deliver value continuously. The product owner has a central role forthis study, as its tasks include backlog prioritisation and maximisation of the valuedelivery.

Besides understanding the current work method, one must understand in what direc-tion the case department want to change. As the case department aims to scale theiragile work with guidance of SAFe, literature about SAFe helps to better understandpotential future prioritisation needs. SAFe consists of processes for team, program,large solution and portfolio level. The team level uses the traditional principles ofScrum, working in 2-week sprints. At the program level, SAFe extends the ideas of

18


Scrum to a higher level, but instead of working in 2-week sprints, the program levelworks in agile release trains (ART), composed of 5 sprints. Each ART consists ofmultiple teams making deliveries into the ART.

WSJF builds upon the concept of CoD. CoD indicate how much value leak over apredefined period of time. In other terms, CoD attempts to assess how much valuebeing missed by not having a feature ready when needed. In SAFe, CoD is utilisedin prioritisation through the prioritisation model WSJF. However, WSJF have asimpler approach towards CoD and defines it by three components: User-BusinessValue, Time Criticality and Risk Reduction/Opportunity Enablement. These com-ponents are set by using a Fibonacci scale and the values are relative the other itemsin the backlog. To calculate the WSJF score these parameters are summarised andthen divided by job duration. However, WSJF has received some critique for havingrelative numbers, having vague definitions of the components of CoD and mixingdifferent units, such as time criticality and risk.

Another method, also built around the concept of CoD, is CD3. This prioritisationapproach is an absolute prioritisation approach, meaning that the effects of the fea-ture are estimated to its actual realisable value instead of putting it in relation toother features in the backlog. The CD3-value is calculated by dividing a monetaryCoD with the duration of the feature.

Within the theme of CoD, Arnold & Yüce (2013) developed the method CoD xUrgency. This is a relative prioritisation approach built on the correlation betweenCoD and urgency. The concept is to categorise the value and the urgency of eachitem and fit them in a 3x3 matrix with urgency on the x-axis and value on they-axis. Consequently, the items with high priority are placed in the upper rightcorner and the items with low priority are placed in the lower left corner. To geta better understanding of how urgency affects CoD and to simplify the urgencycategorisation, (Arnold & Yüce 2013) developed three types of urgency profiles forthe most common types of projects. These urgency profiles illustrate how the CoDof a feature change over time and are useful to have in mind when valuating andprioritising tasks.

The literature study stretched beyond the well-established prioritisation models andincluded practices that could enhance the performance of a prioritisation model inthe financial context. The identified practises handle the assignment of value toenabler features and how work can be broken down into MVP’s. When assigningvalue to enablers, Wright (2018) suggests a practise that compare the size of theenabler with the size of the project(s) it enables. This ratio is then used to assignthe enabler with a portion of the total value equal to its share of the total size.The MVP mindset challenges the development team to deliver a working productthat satisfies the customer needs with minimal effort. By quickly develop a workingproduct, the customer is able to interact with it and changes in the requirementscould be identified in an early stage, reducing the risk of extra work due to thedelivery of a faulty product.

19


As the investigation of prioritisation needs at the case department is similar to thetype of investigation conducted in product development and design, literature inthat field of study contributed to the work procedure of the study. The UCD ap-proach is divided into four general principles, where the two initial steps focus onidentifying the context and needs of the user. In the third step, the design solutionis produced. The design solution is then evaluated against the requirements of theuser in the fourth step. This process is iterated until a solution satisfies all the userrequirements. Extending from the UCD approach, literature regarding the usage ofsituational analysis, to evaluate the needs of the users and their surroundings, wasinvestigated. The situational analysis investigates the requirements from the follow-ing perspectives: Functional, Environmental, Organisational, Competitor, Materialsand Content. By using the situational analysis, the researcher are able to obtain abroad perspective of the needs and formulate them into requirements of the client(Still & Crane 2017).

20

3Method

This chapter begins with an introduction of the work process and the analytical frame-work used to structure the study. This is followed by the research design and workprocedures of the pilot and the main study. Lastly, the data collection and analysismethods are presented followed by a discussion of the reliability, validity and researchethics of the study.

3.1 Model of AnalysisThe model of analysis had the purpose to structure the work process and to stan-dardise the analysis. Moreover, the model of analysis included a framework thathelped to categorise the prioritisation needs, standardised the evaluation processand enabled a fair comparison of how well different prioritisation models satisfiedthe needs.

3.1.1 Work ProcessThe study could be compared with a product development project, where the userneeds are identified, converted into requirements and used to develop a product.Similar to product development, the result of the study was highly dependent onthe users´ involvement to identify their needs. Hence, the study was structuredsimilar to a product development process, where a UCD approach was used as aguideline.

The analysis was structured in four phases: analysis of the needs, specification ofrequirements, development of the model and evaluation of the results. As indicatedin Figure 3.1, the research questions were answered in different phases of the UCDapproach. Sub RQ1 was answered in the second step of the UCD cycle. In this stepthe needs of the case department were collected and specified. In first iteration afterall needs were collected, WSJF was assessed. This was done by using WSJF as inputinstead of designing a solution in the third step. This made it possible to evaluatehow well WSJF satisfied the requirements in the fourth step, which answered SubRQ2. As WSJF showed to be insufficient, the UCD iterations continued with rec-ommendations based on the needs and input from employees. The iterative processcontinued until the recommended modifications met the user requirements and the

21

3. Method

main RQ could be answered upon. By following the UCD structure, the quality ofthe findings were improved, as an active involvement of the users enhanced theirknowledge regarding the research and thereby their ability to contribute with moredetailed input.

Figure 3.1: Illustration of how the UCD approach is used to answer the researchquestions.

3.1.2 Analytical FrameworkTo structure the analysis of how well WSJF and the recommended modificationsserved the requirements of prioritisation in financial software development, the per-spectives presented by Still & Crane (2017) were used as a foundation in the struc-turing of an analytical framework. The framework divides the analysis into sixdifferent perspectives, which together gives a holistic perspective of the needs of theuser and its surrounding. For this study, the analytical framework was tweaked tosuit the evaluation of a prioritisation model. The resulting framework is presentedin Table 3.1.

22

3. Method

Table 3.1: Analytical framework.

Analytical Method Evaluated Prioritisation ModelFunctional Yes/No/Partly

Requirement 1 :Requirement 2 :Requirement 3 :Environmental Yes/No/PartlyRequirement 1 :Requirement 2 :Requirement 3 :Organisational Yes/No/PartlyRequirement 1 :Requirement 2 :Requirement 3 :

Metrics Yes/No/PartlyRequirement 1 :Requirement 2 :Requirement 3 :

Types of projects Yes/No/PartlyRequirement 1 :Requirement 2 :Requirement 3 :

For the model to better suit an evaluation of prioritisation models, three modifi-cations were made to the framework. As prioritisation models are often developedand used internally, it is not exposed to the same type of external competition as amany other products might be. This reduces the need of analysing the prioritisa-tion model’s competitive landscape. Hence, as a first modification, the competitoranalysis was excluded from the study. The second modification was a changed focusfor the material analysis. The material analysis in the original framework focus onthe need of tangible material, which is small or non-existent in prioritisation model.Instead, the material analysis focused on the need of intangible material, which forthis study was the need of prioritisation metrics. As the content of a prioritisationmodel consists of the projects being prioritised in it, it is important to analyse towhat extent it can be generalised to suit different types of projects. Consequently, asa third modification, the content analysis focused on analysing what type of projectsthat can be prioritised in the model.

The requirements were based upon the user needs obtained from the analysis in thefirst step of the UCD process. The requirements were included in the analyticalframework under its corresponding perspective. The requirements were determinedcontinuously throughout study and conclusions were drawn based upon the finalversion of the analytical framework.

23

3. Method

3.2 Research DesignDue to the novel nature of the research area and lack of a precise problem definition,the study had an exploratory research approach. This approach was suitable as itgave flexibility and adaptability to revise the researchable problem throughout thestudy (Denscombe 2010). By having an exploratory approach, significant findingsinfluenced the direction of the study and increase its scientific relevance (Blomkvist& Hallin 2015). To allow for a broad perspective of the researched area, an inductiveapproach was selected where no predefined hypothesis was tested(Denscombe 2010).By having an inductive approach, the study intended to contribute with a deeperunderstanding of the research area, rather than resulting in a single defined truth(Neuman 2011).

As the study was conducted at a company with an interest of solving their firmspecific problem, the research had an action science approach. Action science havea two-sided research purpose: solving the problem for the client and contributingto science (Collis & Hussey 2013). Hence, the study was conducted in collaborationwith the case company to share competences as well as presenting the firm spe-cific findings in a way that is simple to understand for its employees (Gummesson2000). As an effect, the practices and models are presented in a simple mannerfor a maximum chance of a successful implementation. To achieve understandableresults, the study had a participative inquiry approach when formulating suggestedimprovements for the resulting model. The participative inquiry approach involvesthe participants in the research and thus enables them, in cooperation with the re-searchers, to steer the research in a direction where it becomes beneficial both forthe firm and science (Traylen 1994).

The study was divided into a pilot and a main study. The purpose of the pilotstudy was to clarify the research problem as well as increase the understanding ofthe current prioritisation process and strategic direction of the case department.The pilot study had a qualitative approach and sourced information from unstruc-tured interviews, an observation and literature. Empirical data provided insightsspecific for the case company, while the literature provided a general understandingof scientific field of prioritisation. The purpose of the main study was to understandthe prioritisation needs of the department, assess the suitability of WSJF and for-mulate recommendations of how the model could be adapted to better satisfy theneeds. The main study employed a focused literature review and semi-structuredinterviews, in which the findings from the pilot study helped to formulate themes.

The data analysis was structured according to a data analysis procedure developedby Miles & Huberman (1994). This procedure guided the analysis through the threesteps of data reduction, data displays and conclusion and verification. The proce-dure was suitable for this study as it allowed for data analysis of multiple types ofqualitative data, as the study utilised data collected from interviews, literature andan observation. To ensure the validity of the collected data, it was validated bytriangulation where data from the different sources were compared.

24

3. Method

The study was conducted from the beginning of January 2019 to early June 2019 asa Master thesis for KTH, Royal Institute of Technology, and a major Swedish bank.The workflow of the study is illustrated in Figure 3.2.

Figure 3.2: Workflow of the study.

3.3 Pilot StudyAs the prioritisation model originally used by the case company was developed in-house, the experience of using it was tied to a few key stakeholders. Hence, theinitial phase of the study consisted of mapping these stakeholders and understandhow the prioritisation model was put to practise. Furthermore, the target for theprioritisation work was vague, hence a pilot study was needed to define the prioriti-sation needs of the department and understand the future target for prioritisation.The initial phase of the pilot study consisted of interviews with key stakeholders anda literature study, including literature regarding different prioritisation approachesand agile methodologies. Additionally, documentation of internal processes providedby the case company was examined. The stakeholders and internal documentationhelped to explain the internal prioritisation processes and their experience of usingthem. The literature provided general knowledge in the scientific field and func-tioned as a foundation for the empirical part of the pilot study. Furthermore, thepilot study provided useful information to narrow down the research problem andfunctioned as a foundation from where the research questions and purpose could beconstructed.

The interviews with the stakeholders of the case department revealed that the casecompany is in a transition process towards implementing SAFe and some depart-ments of the company were already working in accordance with SAFe. The casedepartment wanted to investigate if WSJF was suitable for their needs. To adaptthe study for this finding, SAFe literature was included in the literature review andan observation of one department using WSJF was conducted. The observation wasconducted during a two-day PI-planning event. To follow up on the observations,unstructured interviews were conducted with relevant stakeholders from the same

25

3. Method

department.

The initial interviewees for the pilot study had the positions of Head of Business De-velopment and Head of Quantitative Development. They were selected due to theirknowledge within the existing prioritisation processes and the banks overall strategy.Additionally, they had a wide network of contacts and could therefore recommendother persons with relevant knowledge. To understand the strategic direction of thebank, the Program Manager for developing the future operational strategies wasinterviewed. The Agile Coach and Metric Responsible are working at a departmentthat is using WSJF for prioritisation and could therefore contribute with experienceand knowledge from using WSJF. To better understand the prioritisation needs ofthe case department, interviewees with a practical experience of prioritising workwere needed. Hence, a Product Owner and a Chief Backlog Owner from the casedepartment were interviewed. Information about the interviews are summarised inTable 3.2. In addition to contributing with information, the interviews from thepilot study were a powerful tool in the selection of interviewees for the main study,as the interviews generated recommendations of other knowledgeable people thatcould be interviewed.

Table 3.2: Interviews and the observation conducted in the pilot study.

Title Code Date DurationHead of Quantitative Development A 2019-01-18 2 hAgile Coach B 2019-01-25 45 minMetric Responsible C 2019-01-31 50 minProgram Manager, Central Operational Coordination D 2019-02-06 1 h 20 minChief Backlog Owner E 2019-02-06 45 minProduct Owner F 2019-02-07 45 minHead of Business Development G 2019-02-12 45 minObservation of PI Planning Event H 2019-01-23 2 full days

3.4 Main StudyWith the pilot study as a foundation, the main study had a focused approach oninvestigating the expressed needs of prioritisation at the case department. Themain study was divided into four phases: identify and understand the prioritisa-tion needs, formulate prioritisation requirements, compare and evaluate how wellWSJF satisfies the needs and requirements and if phase three indicates a mismatch,make recommendations for how the prioritisation model could better satisfy therequirements. The purpose of the first phase was to identify what prioritisationmeasures, roles and practises that satisfied the needs of the case department. Withthe needs identified, the analytical framework could be populated with requirementsand the suitability of WSJF at the case department could be assessed. This was

26

3. Method

done by theoretically assessing how many of the requirements that could be satisfiedby using WSJF. The theoretical assessment utilised a combination of theories fromliterature and empirical data to compare the capabilities of the investigated modelwith the requirements defined in the analytical framework. As WSJF showed to bedeficient in satisfying the requirements of the case department, recommendations ofmodifications were formulated. The recommendations were formulated through aqualitative analysis where the unsatisfied requirements formed a lens, from whichthe literature and empirical data were viewed. With the unsatisfied requirements,empirical data and literature as input, the recommendations for how to modify theprioritisation approach to better satisfy the requirements were developed throughdiscussions between the researchers and iterative prototyping with input from theemployees at the case department.

The information sources for the main study were a focused literature review andsemi-structured interviews at the case department. As a general literature studywas conducted in the pilot study, specific literature related to the research ques-tions were examined in the main study. This included literature focusing on theconcept of CoD, prioritisation models utilising CoD and prioritisation models usedwith SAFe. Moreover, the literature review sourced information and tools used tostructure the study, including analytical frameworks and work procedures. However,there are only a few publications investigating prioritisation methods in scaled agileorganisations and none of them were conducted in a financial software developmentcontext. Thus, the study had to largely rely on primary empirical data gatheredfrom the interviews.

From the pilot study, the employees affected by the prioritisation process could bedivided into three groups. These groups included employees: working based onprioritisation outcome, working with prioritisation or deciding the strategic targetof prioritisation. Besides the three groups, employees working with regulations ortechnical enablers were highlighted as important for the study as they experiencedproblems to estimate the value of their deliveries. To get a coverage from multiplestakeholders, semi-structured interviews were conducted with a mix of employeesfrom different parts of the organisation.

To get insight of the prioritisation need from the employees working based on theprioritisation outcome, a Scrum Master for a development team at the case depart-ment was interviewed. The Scrum Master is working close to the team and thereforehas knowledge of the prioritisation needs of the daily operations. To obtain the pri-oritisation needs of the employees working with prioritisation, a Backlog Owner anda Product Owner were interviewed. These interviewees are working with prioriti-sation on a daily basis and have knowledge of the current process. Furthermore,they contributed with requirements that were favourable but not satisfied by thecurrent prioritisation model. The Head of Quantitative Development, Head of Busi-ness Development and the Program Manager were interviewed to gather informationregarding the strategic target of prioritisation at the case company. The ProgramManager is working on a strategic level with how the case company should define

27

3. Method

value and use it in prioritisation and follow-up. The two heads are working withhow the processes of the department could align with the strategic target of the casecompany and hence know how the departments are affected by the strategic initia-tives. Representatives from the Quantitative Development team were interviewed inthe case study as they are sceptical towards the usage of relative numbers in WSJFand their insights helped to understand the scepticism towards WSJF that exists atthe case company. The Head of Group Regulatory Office, the Group Manager andthe employee at operational risk were selected to source information regarding reg-ulations and risks. These interviewees are prioritising projects connected to eithercompliance or operational risk on a daily basis and could explain their needs andchallenges. Lastly, the employee from Treasury Systems has many technical enablerprojects and contributed with experience of prioritising and valuating this kind ofproject. All the interviewees included in the main study are presented in Table 3.3.

Table 3.3: Interviews of the main study.

Title Code Date DurationHead of Business Development G 2019-02-28 45 minBacklog Owner I 2019-03-06 45 minProduct Owner F 2019-03-06 45 minProgram Manager, Central Operational Coordination D 2019-03-12 65 minHead of Quantitative Development A 2019-03-19 45 minQuantitative Development, Chief + 2 Developers J 2019-03-26 45 minHead of Group Regulatory Office K 2019-03-27 45 minScrum Master L 2019-04-04 25 minGroup Manager, Transaction Reporting M 2019-04-10 45 minTreasury Systems N 2019-04-12 45 minOperational Risk O 2019-04-18 60 min

3.5 Data CollectionThe data collection of this study sourced information from: interviews, internaldocumentation, literature and an observation. The collection processes are explainedin Section 3.5.1-3.5.4 below.

3.5.1 InterviewsArksey & Knight (1999) suggests that interviews should not be seen as one singletype of method. Instead, it should be seen as a collection of methods where itsquality is dependent on the researchers´ ability to design the interviews to fit thespecific study. Hence, the researcher needs to give its full thought on what goals tobe achieved and what type of interview that suits the study. In general, there arethree types of interviews: structured, semi-structured and unstructured. Easterby-Smith et al. (2012) suggest that unstructured or semi- structured interviews are

28

3. Method

appropriate when:

• it is necessary to understand the personal constructs (sets of concepts or ideas)used by the interviewee as a basis for his or her opinions and beliefs

• the purpose is to develop an understanding of the respondent’s ‘world’ so thatthe researcher might influence it (for example through action research)

• the logic of a situation is not clear• the subject matter is highly confidential or commercially sensitive, or there

are issues about which the interviewee may be reluctant to be truthful.

As the study was conducted at a case company with the purpose of solving a vaguelyformulated problem within their natural setting, unstructured and semi-structuredinterviews were the favourable options. Blomkvist & Hallin (2015) argues that un-structured interviews are preferably used in the initial phase of a research project toobtain knowledge within the research area, allowing for refinements to the problemformulation, research questions and purpose of the study. Accordingly, unstructuredinterviews were used in the pilot study to define the research problem and facilitatethe generation of research questions. The main study had the purpose to furtherinvestigate the findings and highlighted research areas obtained from the pilot study.Thus, semi-structured interviews were used as it allows for more detailed discussionswithin the specific areas (Blomkvist & Hallin 2015).

For both the pilot and main study, the interviewees received an invitation alongwith a brief explanation of the study by email. This information was sent one weekin advance, allowing the interviewees to gather their thoughts and prepare for theinterview. The interviews were conducted face-to-face at the location of each in-terviewee. This was advantageous since questions were often complex and requiredthe interviewee to elaborate their answer. Both researchers were present during theinterviews and took turns in leading the discussion and taking notes. To ensurethat no information was overseen or misinterpreted, the notes were complementedwith an audio recording. Immediately after the interview, the notes were discussedbetween the researchers to ensure conformity. If conformity was reached, the noteswere merged and archived to later be analysed. Otherwise, the interviewee was con-tacted for clarification.

The major difference between the unstructured interviews in the pilot study andthe semi-structured in the main study was the predefined structure of the inter-views. The unstructured interview had no predefined questions or topics but wasconstrained to the theme of prioritisation. The semi-structured interviews had amore structured approach where several topics and key questions were predefined.Example of discussed topics were purpose, goals and requirements of prioritisation,current problems and key measurements. The full interview guide for the semi-structured interview can be found in Appendix A. To enhance the flexibility andpossibility to steer the interview based on the interviewee’s expertise, the semi-structured interviews were structured according to the funnel model. The funnelmodel divides the interview into different phases, where the initial questions are of a

29

3. Method

probing kind and gets more specific in later phases of the interview, as illustrated inFigure 3.3 (Bender 2005). As the semi-structured interviews were more constrainedto a specific topic than the unstructured interviews, the interviewee received a morecomprehensive explanation of the study and could therefore be more specific duringthe interview and their preparation work.

It is hard to in advance decide the correct number of interviewees needed to answerthe research questions. If the sample size is too small, the risk of having corruptdata due to personal biases is high. Meanwhile, a large sample might aggravatean in-depth analysis due to large amount of data (Bryman & Bell 2011). Slevitch(2011) argues that the sample size is of less importance in qualitative studies. In-stead, the selection process and the accuracy in selecting interviewees are of higherimportance. To ensure the quality of the interview samples, snowball sampling wasused for the pilot and main study. The principle of snowball sampling is to beginwith a small sample of interviewees and having them recommending other personsthat could be of relevance for this study. The idea of snowball sampling is that aknowledgeable interviewee, within the studied phenomenon, is likely to recommendother knowledgeable persons within the studied field. By using snowball sampling,the sampling process gets streamlined as the researcher only need to find a few ini-tial interviewees of importance (Bryman & Bell 2011).

Figure 3.3: Illustration of the funnel model.

3.5.2 Observation MethodologyObservation is a methodology to systematically observe and analyse peoples be-haviours and actions in a natural setting. The observation method is suitable whenexamining phenomena that occur in peoples day-to-day work (Blomkvist & Hallin2015). The observation method was used in this study for collecting data regarding

30

3. Method

the WSJF prioritisation method, used at a department at the case company. Theobservation was conducted during a two-day PI-planning event. This event waschosen for the observation as it is a key event in SAFe and WSJF and it gave a goodoverview of its processes and how they were implemented in a financial softwaredevelopment environment. Prior to the event, literature regarding SAFe and PI-plannings were examined to increase the understanding of the planning procedure.The literature helped to narrow down the PI-planning into a few areas of interestfor the study, resulting in the following focus areas:

• The structure of the SAFe PI-planning• Team discussions regarding task selection and size approximation• Key persons and roles• Discussions about the value components of WSJF

At the end of the first observation day, the focus areas were scrutinised and refinedto resolve unclarities and highlight areas that were not treated the first day. Todocument the observation, notes were taken by both researchers. The notes werecompared and discussed both during and after the event.

3.5.3 Literature Search and ReviewThe literature search and review were divided into two phases, one general literaturereview in conjunction with the pilot study and a more specific literature review inconjunction with the main study. The first phase of the literature review contributedwith information that enhanced the understanding of common prioritisation modelsand agile methodologies. The second phase had a focused scope where models thatcould not be used in a scaled agile context or did not have a value focus were disqual-ified. This phase of the literature review sourced information regarding the conceptof CoD and models utilising CoD. Moreover, the second phase of the literature re-view helped to operationalise the study by sourcing information of frameworks andwork procedures suitable for identifying needs and formulating user requirements.

The databases used for the literature review were the Royal Institute of Technology’ssearch engine PRIMO and Google scholar. The articles were to a large extentpeer-reviewed research articles published in well renowned journals. Hence, theircredibility can be considered high. Due to the topicality of SAFe and CD3, thenumber of publications were limited and the literature study had to stretch outsideof academia. In this case a high credibility was ensured by sourcing informationfrom official websites or authors with a strong connection to the frameworks. Aselection of keywords from the search are: Prioritisation, Agile, SAFe, WSJF, Costof Delay, CD3, User need analysis and Evaluation framework. Additionally, thecitations in relevant articles were used to find more articles. To avoid a lock-ineffect, the keywords and combinations of keywords were updated throughout thestudy.

31

3. Method

3.5.4 Document StudyThe document study could confirm findings from the interviews and hence it in-creased the credibility of the findings. The studied documents were internal docu-ments describing the internal work processes and the strategic initiatives at the casecompany. The documents were useful to get an understanding of how their currentprioritisation process looked like and the targets of prioritisation at the case depart-ment. The internal documents were provided by employees at the case department.A summary of the documents is presented in Table 3.4 below.

Table 3.4: Summary of documents used to conduct this study.

Name of document Brief description of thecontent

In text reference

Central strategic initiatives This document describesthe strategic targets of thebank with a focus on digital-isation, efficiency improve-ments and measures formonitoring delivered valuein the organisation.

(CSI, 2019)

Operational plan at the casedepartment

This document describesthe aims for the coming yearat the case department, to-gether with a description ofthe work methods, focus ar-eas and relevant KPIs.

(DOP, 2019)

Agile teams at the case de-partment

This document describesthe agile teams of the casedepartment and the keyroles in their agile context.Further, there is a brief de-scription of the agile princi-ples used at the case depart-ment, such as sprint plan-ning, backlog refinementsand sprint retrospectives.

(Agile teams, 2019)

3.6 Data AnalysisThe data in this study were sourced from interviews, internal documents, literatureand observation. To make the data comparable between the different sources, it hadto be compiled to a common form. To preserve the quality and comparability ofthe data, the processing of the data had to be identical between the data sources.

32

3. Method

Hence, the data processing and analysis followed a systematic procedure developedby Miles & Huberman (1994) that has the ability to process and analyse multipletypes of qualitative data. The procedure includes three simultaneous activities: Re-duction of the data, displaying the data and drawing conclusions and verifying theconclusions. The workflow of the procedure is illustrated in Figure 3.4.

Figure 3.4: Illustration of the procedure for analysing qualitative data by Miles& Huberman (1994).

In the data reduction phase, the collected data was analysed individually by bothresearchers to minimise personal bias. As some interviews could be archived forup to one month before being analysed, audio recordings were used to refresh theresearchers minds to minimise data loss and to ensure data quality. The researcherssummarised the collected data to a number of themes that represented its contentand the opinion of the interviewee. When all data was summarised, the researchersdiscussed their findings and compared themes. Further, a triangulation approachwas applied to enhance the accuracy and authenticity of the collected data (Den-scombe 2010). This was conducted by comparing data collected from interviews,internal documents, literature and an observation and having each source verifyingthe validity of each other. This was especially efficient in the case where themesdid not match between the researchers. After this process, the themes either gotdiscarded or accepted to be formulated as a requirement in the analytical frame-work. Before a requirement could be displayed, its underlying type of need had tobe considered and mapped to a perspective in the analytical framework.

33

3. Method

3.7 Reliability, Validity and Research EthicsTo evaluate the scientific quality of this study, the reliability and validity had to beconsidered. During the research design, this was done by following the four qualitycriteria developed by Yin (2017):

• Construct validity — Correct operational measure for concepts.• Internal validity — Is the research done right?• External validity — Establishing the domain for generalisation.• Reliability — Repeatability of operations of the case study.

The pilot study helped to ensure the construct validity of the study as it providedinsights of what competences were needed to source the prioritisation needs of finan-cial software development. These insights helped to formulate the three categoriesof employees, as presented in Section 3.4, that had to be interviewed to cover theentire prioritisation process and additional employees that could provide specificchallenges for the industry. Snowball sampling was used to select interviewees asthe employees of the case company obtain a higher knowledge of the distribution ofcompetence. Hence, they could recommend the most competent people within eachrequested category. Additionally, the key informants from each category reviewedthe draft of the report ensuring the collected empirical data was correct and thatno misinterpretations were made. The internal validity was enhanced by the modelof analysis, which structured the data analysis. The UCD approach and the ana-lytical framework were useful tools to iteratively find patterns and systematicallymake conclusions from the collected data. Besides the models helped to maintaina holistic view of prioritisation, reducing the risk of variables being overseen. Ad-ditionally, the validity of the findings was enhanced by the usage of multiple datacollection methods, as it made possible to have a data triangulation approach. Theexternal validity is not practically tested in this study. Instead, the generalisabil-ity of the study is based on the assumption that the Swedish incumbent banks aresimilar to a large extent when it comes to organisational structure, technical debts,number of regulatory projects etc. With this assumption, one can argue for the find-ings to be applicable to the other incumbent banks in the financial software industry.

In general, qualitative case studies are difficult to replicate. A reason for this isthe difficulty of repeating the interviews and observations, as both employees andbusinesses have a tendency to change over time. Additionally, there is an uncer-tainty of to which extent the researchers affect the outcome of the interviews andobservations due to personal bias (Roulston 2010). The usage of established modelsand frameworks helped to strengthen the reliability of the study. To further enhancethe reliability, all modifications to these models were described to preserve the re-peatability. However, one can question how the snowball sampling is affecting therepeatability as the recommendations of interviewees are dependent of the specificindividual.

To ensure that the interviews were conducted in an ethical manner, four principal re-

34

3. Method

quirements developed by the Swedish Research Council were used (Vetenskapsrådet2018):

• Interviewees should be informed about the purpose of the study.• Interviewees should agree to participate in the study.• The gathered data should be treated confidentially and the identity of the

interviewee should not be revealed.• The collected data should be used for the specific purpose stated before the

interview.

One week prior to the interviews, the interviewee received an invitation via e-mailwhere the purpose of the study was explained along with the topics for the inter-view. This ensured that the interviewee had time to prepare for the interview. Theinterviewees were informed that their participation in the study was anonymous,which encourages the interviewees to give more truthful answers.

35

4Empircal Findings

This chapter presents the context in which the data has been collected followed by theempirical findings from the interviews and the observation. The empirical findingsare grouped into common themes to obtain a better overview of the main topics.

4.1 Introduction of the case companyTo enhance the generalisability of the study, this section introduces the empirical en-vironment in which the study was conducted. The case company is one of the largerincumbent banks in Sweden with national coverage. Besides the Swedish market,the bank has business in neighbouring countries. The case department works withmaintenance and development of trading software and have a development approachwhere most of the developers are located in the same physical location. The teamswork according to the agile method Scrum and divide their work into 2-week sprints.After each sprint, the delivery is presented and new features from the backlog areadded to the upcoming sprint.

The competitive market of the case company is challenging due to strict regulationsand the introduction of new niche banks, utilising digitisation to take market sharesfrom the incumbent banks. This is an effect of the technical debt of the incumbentbanks, where their technical platforms are lagging behind the ones of the nichecompetitors’, that are developed with the latest technology. The challenging markethas led to a need of renewing the organisation and the case company is lookingtowards scaling its agile work. The case department has not initiated their scalingprocess but is looking towards scaling in the near future, where SAFe looks like themost probable choice.

4.2 Needs and RequirementsThe current prioritisation model at the case department is dependent on a few stake-holders and their ability to prioritise the backlog based on their knowledge aboutthe different projects. As all the clients believe that their projects are of the highestpriority, it is up to the Backlog Owner to estimate what projects delivering the mostvalue and prioritise the backlog based on that estimation. Hence, the prioritisation

36

4. Empircal Findings

process is dependent on a Backlog Owner and its ability to understand the value ofeach project. The dependability on the Backlog Owner’s knowledge makes it hardto define the process, as the prioritisation happens in their heads. This make itdifficult to standardise the prioritisation process, resulting in a lack of documenta-tion. However, in an attempt to generalise the process, Interviewee D describes theprocess as following:

1. An idea occurs and is added to the list of projects waiting to be estimated.2. The costs and effects of the project is estimated.3. A pre-study is conducted.4. If the pre-study is positive, a more detailed study is conducted.5. If the idea show to be profitable, a project is initiated.

Interviewee D continues by explaining how the strategic decisions regarding perfor-mance monitoring may affect the current prioritisation model. The major differenceis that the bank will focus towards a value-based monitoring approach where thedelivered value will be monetarily measured.

In general, the interest and knowledge regarding prioritisation processes were high.This might be a result of the ongoing agile scaling process in other parts of the casecompany and the open discussions regarding future strategies and how to monitordelivered value. The high knowledge resulted in interesting discussions and inputsregarding the current prioritisation model and how it could be improved. Besideshaving knowledge about the case department’s prioritisation model, a few intervie-wees have knowledge about the WSJF prioritisation model. This is due to the factthat it is used by other departments at the case company that already works ac-cording to SAFe. Hence, this model is seen as a probable alternative to the currentprioritisation model at the case department and employees in managerial positionshave begun to investigate and evaluate the possibility of using WSJF.

The interviewees who already had knowledge about WSJF consider it to be a goodapproach for bringing a value focus into the prioritisation process. However, some ofthem feel a need to modify the model before implementing it at the case department.The most common concerns are:

• usage of relative numbers• combination of different units such as time criticality, business value and risk• subjective estimations of the scores.

The usage of relative numbers is criticised as it assigns value to a feature in the back-log relative others rather than the value of the actual feature. This would, accordingto Interviewee G, complicate the monitoring of delivered value as the monetary valueof a feature gets detached from its prioritisation value. Similar problems are identi-fied by Interviewee F, who believes that the combining of different units will detachthe prioritisation value from the monetary value. As an effect, the delivered valuecannot easily be traced in the financial figures and its realisation is difficult to verify.

37


The interviewees from Interview J are concerned that having a value based approachbased on relative numbers would hide the assumptions made and hence reduce thetransparency.

Besides the concerns related to WSJF, Interviewee F highlights the importance ofhaving a flexible model that fits all types of projects in the bank. Interviewee Ffurther explains that the projects could be divided into three main types:

• Business development projects: This type of project is the most straight-forward to assign a value as it could be directly connected with the business.The development of a new user interface is an example of as business devel-opment project.

• Regulation projects: The regulation projects are often driven by externalforces and usually have a deadline connected to it. The projects are ofteninitiated due to changes in regulations and the business impact is hard toestimate. The unclear business impact and deadlines challenge the backlogowners to prioritise this type of projects in a backlog with different types ofprojects.

• Enabling projects: The business value of an enabling project is often hardto quantify as it delivers indirect value to the business. Indirect value doesnot appear in the financials but might have large impact for future value gen-eration due to the enablement of more lucrative projects. A software updatethat enables future features is one example of an enabling project.

From analysing the interviews, the needs and requirements could be divided intofive main themes. These themes are used to categorise the empirical data. Thesethemes are presented in the sections 4.2.1-4.2.5.

4.2.1 Value-based PrioritisationDiscussions regarding value-based prioritisation were already common at the casedepartment before this study was initiated. Interviewee G had announced a targetof delivering business value at twice the rate within two years. This target wasconfirmed by internal documentation (DOP, 2019). Additionally, Interviewee G em-phasises the importance of understanding and being able to measure what valuecurrently being delivered. This would enable the case department to prioritise themost valuable tasks and thereby deliver more value without increasing the workload.Interviewee G suggests that business value should be measured and prioritised interms of monetary value, to achieve a higher understanding of how each feature inthe backlog relates to the delivered value. The desire to measure delivered valuemonetarily is shared by Interviewee D, who works with strategic questions on acentral level. Interviewee D explains that a monetary prioritisation approach wouldsimplify the reporting to management as it aligns with how management follows-up performance. Therefore, Interviewee D deem that a future prioritisation model

38


should define and prioritise value based on the same definition as the management.The management have decided to monitor value based on the following four cate-gories: increased revenue, reduced cost, saved time and risk impact (CSI, 2019).

Another department that promotes the usage of monetary values in prioritisationis the department of quantitative development. From Interview J, it appears thatthe department prioritises monetarily using the prioritisation approach CD3. Dueto the early stage of the implementation, they are not able to prove that CD3 hasimproved their delivered value. However, using CD3 has put more focus on un-derstanding what projects delivering the most value and consider the value of eachfeature when prioritising the backlog. The value focus has also led to discussionsregarding why a project generates value. One of the interviewees described it asfollows:

"To say that it is hard to put monetary value on projects is a bad ap-proach. If you cannot connect the value to your business, why should youdo the project?" (Interview J).

The interviewees continue by explaining that even though the monetary valuationprocess might take some extra time, it is worth the effort due to the higher con-centration of value in the highest prioritised projects. To emphasise the statement,the interviewee referred to an investigation conducted by Arnold & Yüce (2013) andpresented the value distribution displayed in Figure 4.1.

Figure 4.1: Value distribution of a development team at Maersk, investigated byBlack Swan Farming (Arnold & Yüce 2013).

39


Interviewee F has a positive view on the monetary prioritisation approach used bythe quantitative development team and can relate to similarities with the currentmodel used to investigate of a new project should be initiated. This investigationassigns the project qualitative and quantitative effects, that are used to calculatethe feasibility of the project. However, this mindset is only applied when consideringnew projects and Interviewee F believes that the same mindset is applicable in theoperations, if the value of the project could be broken down into smaller components.Interviewee N adds that such as solution requires a high level of maturity amongbacklog owners and the clients.

4.2.2 Difficult Valuation CasesFrom conducting interviews, it became clear that the difficulty of assigning a mone-tary value to projects vary depending on the client. The teams working close to theend-customers can easily monitor the effects of each delivery, as the delivery is oftenconnected to a monetary effect. Meanwhile, teams working with compliance, IT-infrastructure or operational risks are often disconnected from the profit-generatingbusiness and therefore experience challenges assigning monetary values to their de-liveries. Additionally, several interviewees highlight the difficulties of assigning amonetary value to enabling projects. These projects are seen as a cost and theirmonetary effects are tied to future projects that are enabled by the enabling projects.To cope with these difficult valuation cases, the quantitative development team havedeveloped a general approach which they presented during interview J. The generalapproach could be divided into three questions as follows:

1. Can a monetary value easily be assigned to the activity?2. What is the alternative cost?3. What happens if we do nothing?

If the answer is yes to question one, no further work needs to be done and themonetary value can be assigned to the activity. If the answer is no, consider thealternative costs. What would it cost to hire a consultant or do the work manu-ally, instead of automatising it? Often, by considering alternative solutions, onecan obtain a wider perspective of what value an activity contributes with. If thealternative cost is unclear, the value can be estimated by considering what wouldhappen if nothing is done. What is the effect of not doing the activity, will it resultin a penalty or would a potential income be missed? These kinds of questions aregood for questioning the reason for the activity. If the value of an activity cannotbe understood, should the activity be done?

Regulations, risks or/and enablers are to a large extent the challenging aspects whenprioritising based on value. Hence, the interviews were focused around these areasand the findings within each area are presented in Sections 4.2.2.1-4.2.2.3 below.

40


4.2.2.1 Regulations and Compliance Risk

At the case company, risks are separated into compliance risks and operational risks.Compliance risks are connected to regulations while operational risks are connectedto errors that could occur in the daily operations. As of today, most developmentrelated to regulatory projects/activities are imperative in the bank, according tointerviewee F. When a compliance deficiency is detected, the affected team must, incombination with compliance, determine a limit including a plan for what actions totake and when the regulation must be met. Interviewee F further explains that thisprevents a value focused prioritisation for the case department, as the backlog getscrowded with imperative activities. Regulatory projects seldom have an estimatedvalue connected to them, as compliance do not want to speculate in the value ofbeing compliant when grading the importance of the projects. Instead, they usefour categories to categorise a regulation based upon its severity. The categoriesof severity are: Critical, Major, Moderate and Minor. The categorisation is basedupon factors such as the type of regulation, impact, awareness of its effects and howthe business must be adapted to achieve compliance. Another reason for using thisapproach over a value-based is that many regulations are new and hence no data ofhistorical effects are available. The Head of Group Regulatory Office, IntervieweeK, exemplifies the uncertainty of estimating the cost of not being compliant by thefollowing statement:

"We do not attempt to read between the lines in the regulatory texts tobetter estimate the monetary effect of a fine that nobody previously re-ceived (Interviewee K)."

The unwillingness of assigning a monetary value to compliance projects is questionedby the quantitative development team, from Interview J. They deem that everythingthat brings value to the business can be assigned a monetary value and that thissituation is a good example of where one have to adapt a broader perspective andlook at the alternatives. In the case of valuating regulatory projects, they suggestinvestigating how it alternatively could be solved or what would happen if nothing isdone? An example of an alternative cost could be to investigate the cost of manuallyreview reports instead of developing an automated solution.

Interviewee M explains that a large share of their backlog consists of regulatoryprojects, which restricts their ability to have a value-based prioritisation approach.Hence, value estimations on regulatory projects would facilitate their ability to fullyprioritise based on value.

4.2.2.2 Operational Risk

Another type of project that appeared as challenging when estimating business valueis projects related to operational risks. Interviewee O explains that they make use ofa categorisation system when evaluating operational risks. This system differs fromthe compliance approach by utilising two categories instead of one to evaluate therisk. The two categories are impact and probability. The impact of an operational

41


risk is estimated by investigating its average impact and potential maximal impact.The impact is graded on a scale from 1 to 5, where 5 indicate extreme damage and1 negligible damage. Additionally, each level is connected to a predefined monetarycost that is individually assessed for each department. The probability is also gradedon a scale from 1 to 5 and defines the probability as the frequency of occurrences.Level 5 contains events that occurs at minimum 5-6 times a year or more frequentlywhile level 1 includes events happening every 30 years or even less frequently.

Interviewee O is positive towards applying a more monetary focus and states thatthe operational risks can be assigned a monetary value. However, this would requirea better cooperation between the risk team and knowledgeable people from theoperational teams. This positivity towards measuring operational risks monetarilyis shared by Interviewee F who believes that a monetary approach will make theoperational risks more concrete and closer to the operations.

4.2.2.3 Enablers

The last challenging valuation case that appeared in the interviews is how to assigna value to enabler projects. An enabler project does not directly add value to thebusiness. Instead, its value is the enablement of future projects that can generatevalue. Hence, it is problematic to compare enabler projects with business devel-opment projects which leads to challenges in a value-based prioritisation model, asenabler projects seldom generate value and falls short in comparison to businessdevelopment projects. Interviewee F explains that enabler projects often get lowerpriority in comparison to business development projects as it is often seen as a costrather than looking at its potential value generation via future projects. Intervie-wee F believes that understanding the value of an enabler project will be especiallyimportant in a value-based prioritisation approach as it will be strictly value max-imising, leaving less room for prioritisations based on gut feeling.

For enablers, the quantitative development team considers alternative costs whenestimating a monetary value. Two examples of alternative costs could be hiringconsultants instead of developing and learn themselves or buying entire systems tocope with future applications instead of adapting existing systems. One doubt withusing monetary estimations for enablers is raised by interviewee G who is concernedthat potential symbolic values for enablers would confuse the reporting. As enablersseldom delivers value that instantly can be realised in the organisation, IntervieweeG is worried that it will become hard to differentiate delivered value that shouldbe realised in the organisation and what values that are symbolic and product ofprioritisation.

4.2.3 Monitoring Delivered ValueInterviewee I explains that it is hard to trace delivered and received value on a teamlevel. This is believed to be the effect of having low traceability of who is receiv-ing delivered value when a project is completed. When a project is completed bythe development teams, the operational line manager contacts the receiver of the

42


project and verifies that it satisfies the requirements. However, there is no follow-upwhether or not the effects of the project have been realised. This is something thatInterviewee D, together with the management, sees as a current weakness in theorganisation and wants to have more strict policies on how the realisation of valueshould be followed-up. Interviewee D sees a need for a follow-up system that isbased on strong connections between developer and receiver of value to ensure thatthe estimated value of an ordered project is realised at the receiving end. Further,Interviewee D sees a need for following-up delivered and realised value based onthe same four categories of value that is reported to management, as mentioned inSection 4.2.1.

Interviewee G is aware of the plan to monitor performance based on the manage-ment’s four categories of business value. On a general basis, he is positive towardsthe approach and believes that using monetary measures will increase the trans-parency of what is being delivered. However, one concern is how the quality of thedeliveries will be assessed and how to avoid that the quality of each delivery getssacrificed for the desire to deliver value at a higher pace. Interviewee G exemplifiesthis with projects of poor quality receiving value without having the delivered valuereduced when errors connected to the low quality occurs.

At the departments working according to SAFe, they have dedicated meetings tofollow-up deliveries. Interviewee B describes that they have I&A-meetings aftereach program increment, where the teams and stakeholders meet to discuss theoutcome of the PI and how well it satisfies the expectations. The stakeholderscollaborate with the agile teams to rate the deliveries based on their actual businessvalue and how it compares towards the estimated value. However, Interviewee Dis critical about that the compared values are based upon the CoD components ofWSJF, as they are based upon subjective estimations. Consequently, the realisedvalue cannot be confirmed by analysing the financial figures of the receiving end.Besides following the SAFe textbook, the department has started to develop theirown measures to monitor the performance of the ART. Interviewee C has developedsystems for measuring cycle times of features in the ART and satisfaction of usingSAFe among the participants. The cycle time is measured by measuring for howlong a feature is placed in different phases of the backlog, such as development,testing and implementation. This gives insight on potential bottlenecks and thethroughput time of a typical feature. This is something that the employees of thecase department also want to implement.

4.2.4 Roles of PrioritisationTo a large extent, the prioritisation process at the case department follows the gen-eral principles of Scrum. The Product Owner decides on a high level which featuresto include in a product backlog by having a business oriented approach. Then theBacklog Owner, together with the team, pull items from the product backlog intothe team backlog. However, these roles are merged into one at the case departmentby giving the Backlog Owner a more strategic perspective (Agile teams, 2019). On

43


an operational level, the Backlog Owner prioritises the features, from high to lowpriority, to decide which features to include in the upcoming sprint. The team de-cides, from the top of the backlog, how many of the prioritised features they arecomfortable that they are able to complete in one sprint. The teams do not usuallyallocate themselves 100% of their available time, instead they make sure to leaveroom for unexpected events, bugs and maintenance.

In the parts of the case company that already work according to SAFe, the EpicOwner ensures that the effects of an epic have been estimated. In cases where theEpic Owner cannot estimate the effects on its own, the owner should ensure thatthe parts of the company affected by the epic contribute with their estimations ofthe effects. According to Interviewee B, this approach ensures that epics will alwaysbe assigned a value. The approach used by the SAFe part of the case company issimilar to the one that is used by the quantitative development team that prioritisesusing monetary CD3. They ensure that a monetary effect is assigned to each featurein the backlog by having the person bringing the feature responsible for its valueestimation. If any member of the team adds a new feature to the backlog, they mustestimate its monetary effect either by doing it themselves or request an estimationfrom the receiving end, if the effects are realised somewhere else.

According to Interviewee L, a concern with changing towards a value-based priori-tisation model is that a large share of the additional evaluation work might end upon the development teams. As of today, the developers are part of the prioritisationprocess by estimating the development time of different activities in the backlog.Interviewee L is content with this responsibility as the developers know the timerequirements best. However, he is hesitant towards adding more responsibilities tothe teams as it takes time from developing.

4.2.5 Easy to UseOne shared opinion among the interviewees is that a prioritisation model has to beeasy to use. This was especially emphasised by the interviewees that works in thedaily operations as they want to focus on development rather than prioritisation.They emphasise that the best prioritisation model is the one being used rather thanthe one with the most functionality. Interviewee L is positive towards changing theexisting prioritisation model to get a more standardised work process. This wouldreduce the dependency of individuals and simplify the comparison of activities inthe backlog. For a successful implementation of a new prioritisation model, Inter-viewee L suggests to initially implement a basic version and, if necessary, add morefunctionality to suit the specific needs.

Interviewee A and E promotes the usage of visual flowcharts to guide the prioriti-sation process. Flowcharts are commonly used at the case company to guide workprocesses and hence they believe that a visual tool could be efficient to guide theprioritisation process. Interviewee G further believes that an implementation of amonetary prioritisation model can be used to boost the performance by displaying

44


the progress of the sprint on TV-screens at the case department. The monetary fig-ures would be used to display how much value that has been delivered in each sprintand the potential value of the current one. Within the same theme, Interviewee Fpromotes a prioritisation model where the output could be used with their currentagile planning software to rank and highlight the most important activities in thebacklog.

45

5Analysis and Discussion

The following chapter is structured based on the research questions. Initially, ananalysis of the empirical data is used to answer upon Sub RQ1. The answers arethen used to assess the suitability of WSJF and thereby answer upon Sub RQ2. Thefindings from the two Sub RQs are then used to answer the main RQ.

5.1 Prioritisation Needs of Financial Software De-velopment

One major finding from investigating the needs and requirements of prioritisation atthe case department is that the needs differ between people working on a strategicand operational level. The strategic level is defined as the employees with a highlevel responsibility, ensuring that a system of products work together and are tak-ing decisions on an overarching level. Employees on the strategic level often havemanagerial positions. The operational level includes employees working close to aproduct, with the focus on developing and maintaining their specific product. Em-ployees on the operational level have a more specific focus and are seldom involvedin the strategic decision-making of the case company. In general, employees on astrategic level demand a monetary perspective on prioritisation, which would facil-itate the monitoring of delivered value. On an operational level, one of the mostimportant needs are that the model should be easy to use, allowing for more time todevelop software. Resulting from the difference in needs, two sets of requirementswere formulated. The requirements of each level are mapped to a correspondingperspective in the analytical framework, as shown in Table 5.1 and Table 5.2. Theanalysis of the needs from each perspective of the analytical framework is presentedin sections 5.1.1 – 5.1.5.

46

5. Analysis and Discussion

Table 5.1: Populated evaluation framework on an operational level.

Operational levelFunctionalIndependent of specific individualsEncourages value discussionsClear definition of business valueCommon base for prioritisation regardless of project typeScalability in functionalityAbility to break down value in project componentsEasy to useClear responsibility for project valuationPossibility to integrate with agile planning softwareEnvironmentalApplicable in a scaled agile environmentAbility to prioritise regulated businessesOrganisationalSame measures for prioritisation and follow-upInvolvement of client and developer in the prioritisation processMetricsCycle timeSatisfaction of work methodDelivered value (simple)Accuracy of predicted valueTypes of projectsBusiness development projectsRegulation projectsEnabling projects

47


Table 5.2: Populated evaluation framework on a strategic level.

Strategic levelFunctionalIndependent of specific individualsEncourages value discussionsClear definition of business valueCommon base for prioritisation regardless of project typeScalability in functionalityDeliveries traceable in financial figuresEnvironmentalApplicable in a scaled agile environmentAbility to prioritise regulated businessesOrganisationalShared value definition throughout the organisationSame measures for prioritisation and follow-upComparable results within the organisationMetricsCycle timeSatisfaction of work methodDelivered value (monetary)Accuracy of predicted valueTypes of projectsBusiness development projectsRegulation projectsEnabling projects

5.1.1 Functional PerspectiveWhen defining the functional requirements, the empirical data was isolated to un-derstand what the prioritisation model is expected to do. The first functionality isdefined from the dependency concerns expressed by the interviewees. The modelcurrently used at the case department relies on few numbers of stakeholders thathave knowledge regarding the projects and how the backlog should be prioritised.This dependency creates a risk due to the concentration of knowledge and decisionpower. To minimise this risk, a prioritisation model needs to be independent ofspecific individuals and be transparent with the assumptions made in the decision-making. Hence, the need to reduce the dependency of specific individuals forms thefirst functional requirement, for both the operational and the strategic level.

Several interviewees indicate that the current prioritisation model is lacking a cleardefinition of value and that the delivered value from each sprint is relatively un-known. This model is challenged by the case department’s target of delivering valueat twice the rate. The target is set to stimulate discussions regarding why projectsare developed and what value they contribute with to the organisation. Besideschallenging the existing model, the target is a good indicator of the general direc-

48


tion in which the bank aims to change. As the general strategy of the case companyis to stimulate value discussions at all levels of the organisation, this can be con-sidered a prioritisation requirement for both the operational and strategic level. Tobetter stimulate value discussions and enhance the understanding of value at thecase company, clear value definitions are required.

In the future, the management want to have better follow-up routines to ensurethat the estimated value of a project actually is getting realised in the operations.They aim to do this by comparing the estimated quantitative effect of a project withthe financials of the department receiving it, to investigate if the effects have beenrealised. Hence, the strategic level requires the value estimations to be traceable inthe financial figures.

From the interviews, three project types emerged: Business development, Regula-tion projects and Enabling projects. The specific challenge with each type and howto cope with these are further disused in Section 5.1.5. Nevertheless, a functionalrequirement of the prioritisation model, emerging from both the strategic and op-erational level, is the ability to handle different types of project and to comparethem. For the prioritisation model to be able to compare and prioritise all types ofprojects, the model needs a common base on which it evaluates projects. Moreover,the operational level requires a prioritisation model that is able to break down thevalue estimation into smaller components, such as features. This is required as itallows the teams to maintain a value focus when prioritising the team backlog. Fur-thermore, this makes it possible to prioritise the features individually, regardless ofits corresponding project.

As the value based mindset is new for the case department and still under devel-opment, the moat for implementing changes to a prioritisation model needs to below. An operational requirement is that an initial value based prioritisation modelneeds to be kept simple to understand and implement, to allow as much time aspossible for software development. Meanwhile, the model needs to be flexible andhave the ability to change along as the value mindset becomes more accepted inthe organisation. Hence, a functional requirement expressed from both levels, is thepossibility to scale the functionality of the prioritisation model to adapt for futureprocesses to better estimate value. Another functional requirement expressed bythe operational level, related to have a model that is easy to use, is the possibilityto integrate the model into their agile planning software. This would simplify thecalculations and hence facilitate the prioritisation and planning process.

5.1.2 Environmental PerspectiveTo understand the environmental requirements of a prioritisation model suitable forfinancial software development, one must first understand the larger trends in thebusiness. As many articles indicate, the financial sector has fallen behind as newtechnology developed (Vives 2017). The effect is a mix of old and new systems andtraditional and agile practises. To address this problematic mix, the case company

49


plan to scale up their agile work and adapt its traditional methods to be integratedwith the new work procedure. This will be done using the scaling framework SAFe,as it builds upon their current Scrum process. Hence, a prioritisation model needsto be applicable in this context.

The second environmental requirement, for both levels, is an effect of the highlyregulated financial environment. The strict regulatory landscape challenges the de-veloping teams as they have to choose between business development and regulatoryprojects. As of today, the compliance department does not estimate the businessvalue of the regulatory projects. This makes it difficult for the development teamsto choose business developing projects over regulatory projects as they cannot esti-mate the value of the regulatory projects. Resulting from this, regulatory projectsget a higher priority due the uncertainty of the consequences of not being compliantand thus business development risks falling behind. For a prioritisation model tobe successful in financial software development, it needs to be able to separate theimperative regulatory projects from the ones that should be compared with businessdevelopment projects and be prioritised based on its business value.

5.1.3 Organisational PerspectiveThe management monitor how much value each department delivers by screeningtheir reported numbers. This makes it possible for the management to better guidethe focus of the operations in the direction of the organisational strategy. However,the process of obtaining the reported numbers is time-consuming. As a result, boththe strategic and operational level wants the prioritisation to be made on the samedefinition of value as the monitoring. By aligning the value definition used in priori-tisation at an operational level and the measures used to track performance, boththe management and the operational level could benefit from time savings. Addi-tionally, sharing the value definition between prioritisation and follow-up increasethe transparency of a teams´ performance, as the expectations of its deliveries be-comes clear for both management and the team itself. Furthermore, the strategiclevel requires a shared value definition that allows them to make a fair performancecomparison between different departments in the organisation. This requires a valuedefinition that includes several perspectives and does not solely compare what de-partment that generates the most cash.

Due to the information asymmetry between the clients and the development teams,where the client has more information regarding the end usage of the product and thedevelopers having information of its underlying technical specification, inputs fromboth sides are seen as a necessity to estimate the business value on an operationallevel. Hence, the involvement of stakeholders from both the receiving and developingend is an organisational requirement for a value-based prioritisation model on anoperational level.

50


5.1.4 MetricsThree common metrics for measuring performance are required from both the strate-gic and operational level. These metrics are cycle time, satisfaction of the workmethod and accuracy of predicted value. The requirement of a metric for cycle timeis based on the need to detect bottlenecks and understand the flow of the develop-ment process. To improve the estimations of how much the development teams areable to deliver in one sprint, the accuracy of the estimations first has to be measured.Besides to improve the estimations, the metric can also be used to benchmark howwell the teams performed compared to the target of the sprint. Due to this, bothlevels require a metric that evaluate the accuracy of predicted value, by comparingthe difference between estimated and actual value delivery.

Besides these metrics, both levels expressed a need for measuring delivered value.However, they have different requirements for how this should be measured. Theoperational level demand a simple solution that is quick to calculate, where it ispossible to compare different features in the backlog. The strategic level requires amonetary value measurement, as it facilitates a follow-up on how the delivered valueis realised in the organisation.

5.1.5 Types of ProjectsThere are mainly three types of projects identified in the bank. These are businessdevelopment projects, regulatory projects and enabling projects. Both the opera-tional and strategic level requires these project types to be handled in a prioritisationmodel.

The business development projects are in general the easiest to valuate as the valueare often directly connected to effects in the business, commonly in the form of anincreased income or reduced cost. However, both levels, experience difficulties whenattempting to compare a business development project with other type of projects,as the department lack a common base for prioritisation. Most of the regulatoryprojects are marked as imperative and therefore always receives a high priority.This means that regulatory projects often get prioritised over potential high valuebusiness development projects. For enabling projects, one of the biggest concernsis that the enablers get a low priority as its value often is difficult to visualise orconnect to the business. This can, in many cases, lead to that the backlog is crowdedwith low prioritised enablers lagging behind. Consequently, a prioritisation modelthat can handle these types of projects in a smooth and simple way would facilitatethe prioritisation work at the department significantly.

5.2 Assessment of WSJF in Financial SoftwareDevelopment

To assess the suitability of using WSJF in the organisation, the analytical frame-works from the operational and the strategic level were populated with "Yes", "No"

51


or "Partly" based on the empirical findings. The populated analytical frameworksare illustrated in Table 5.3 and 5.4. These results are further explained and analysedfrom each perspective in sections 5.2.1 – 5.2.5 below.

Table 5.3: Assessment of WSJF on an operational level.

Operational levelAnalytical Method WSJFFunctionalIndependent of specific individuals PartlyEncourages value discussions YesClear definition of business value PartlyCommon base for prioritisation regardless of project type YesScalability in functionality PartlyAbility to break down value in project components YesEasy to use YesClear responsibility for project valuation YesPossibility to integrate with agile planning software YesEnvironmentalApplicable in a scaled agile environment YesAbility to prioritise regulated businesses YesOrganisationalSame measures for prioritisation and follow-up YesInvolvement of client and developer in the prioritisation process YesMetricsCycle time YesSatisfaction of work method YesDelivered value (simple) YesAccuracy of predicted value YesTypes of projectsBusiness development projects YesRegulation projects YesEnabling projects Yes

52


Table 5.4: Assessment of WSJF on a strategic level.

Strategic levelAnalytical Method WSJFFunctionalIndependent of specific individuals PartlyEncourages value discussions YesClear definition of business value NoCommon base for prioritisation regardless of project type YesScalability in functionality PartlyDeliveries traceable in financial figures NoEnvironmentalApplicable in a scaled agile environment YesAbility to prioritise regulated businesses YesOrganisationalShared value definition throughout the organisation NoSame measures for prioritisation and follow-up NoComparable results within the organisation NoMetricsCycle time YesSatisfaction of work method YesDelivered value (monetary) NoAccuracy of predicted value PartlyTypes of projectsBusiness development projects YesRegulation projects YesEnabling projects Yes

5.2.1 Functional PerspectiveFrom a functional perspective, WSJF satisfies or partly satisfies all the expressedneeds from people working on an operational level, while the specific needs of thestrategic level are unsatisfied. WSJF partly satisfies the need of being independentof specific individuals for both levels. This is because the valuation in WSJF issubjective, meaning different people can get completely different valuation of thesame item and hence you are dependent on that the same person makes the esti-mations. However, the fact that the scale is based upon CoD means that a valueis assigned, making it is easy for anyone to prioritise among the items in the backlog.

When it comes to having a clear definition of business value, operations are partlysatisfied while management are unsatisfied with how WSJF defines its CoD. Thiscould be explained by the vague explanation of the components used to calculate theCoD in WSJF. The vague definition of the components raises questions regardingif they are weighted equally or if some components are worth more. For manage-ment, the vague definition in combination with the usage of relative and subjectivenumbers to valuate the items in the backlog makes it difficult to know the actual

53


business value of a project. However, on an operational level, these definitions ofbusiness value are often acceptable, since developers are not really interested in theexact value. They only require an indication of what projects being most importantat the time as they are more interested in doing what they are good at, namelysoftware development.

Regarding the scalability in functionality, both levels are partly satisfied. It is pos-sible to add or modify WSJF to further suit the specific needs of the department.However, even if additional functionalities are added, such as other project typesor more detailed risk estimations, the subjectivity of the WSJF score restricts theprioritisation to be based on subjective estimations.

The specific operational requirements, i.e. the ability to break down value intoproject components, easy to use, clear responsibilities for project valuation and pos-sibility to integrate the prioritisation model with the departments agile planningsoftware are all satisfied by WSJF. SAFe has a clear structure for how projectsshould be broken down into smaller parts by using epics, capabilities, features andstories. For each of these levels of a project, there are clear responsibilities for whois responsible for making value estimations. Additionally, the usage of relative num-bers in WSJF makes the model relatively simple to use as the valuations are maderelative to the other projects in the backlog, allowing for more time to be allocatedto software development. To further facilitate the WSJF process, it is possible tointegrate the WSJF calculations in the agile planning software used at the depart-ment, making the value estimations and WSJF scores visible for all team members.

The strategic level is unsatisfied with WSJFs ability to trace the deliveries in thefinancial figures. As WSJF use relative values based on a Fibonacci scale, it isdifficult to know the monetary value from each delivery and trace its effects in thefinancial figures of the receiving end.

5.2.2 Environmental PerspectiveFrom the environmental perspective, the strategic and the operational level have thesame requirements. These are that the prioritisation model should be applicable ina scaled agile environment and have the ability to function in a regulated business.WSJF satisfies both these requirements. This is a result of WSJF being developed bySAFe to function in a scaled agile context. Additionally, the CoD components usedin WJSF are formed to cover multiple project types, including business development,regulatory and enabling projects.

5.2.3 Organisational PerspectiveFrom the organisational perspective, the operational level is satisfied with havingthe same measures for prioritisation and follow-up while the strategic level is un-satisfied. In SAFe, operations are prioritising based on the WSJF scores and arefollowing up their work on the same basis. However, management are not inter-

54


ested in the relative numbers used to prioritise in WSJF, they want to know howmuch value that has been delivered in monetary terms and therefore requires theoperations to report to them in monetary terms. Consequently, there is a coherencyin prioritisation and follow-up on an operational level that breaks in the transitionbetween the operational and strategic level. Hence, the requirement for a coherencybetween prioritisation and follow-up is satisfied on an operational level and unsat-isfied on a strategic level.

An expressed need by operations is an involvement of the client in the prioritisationprocess. This need is satisfied by WSJF as SAFe has clear descriptions of when theclient and developers should interact. Prior to the PI-plannings, the Business Own-ers are involved in the valuation process, providing inputs from the business pointof view. The business owners are also present at the PI-plannings to participate indiscussions and answer questions regarding the business value.

The specific strategic organisational needs are unsatisfied by WSJF. These are hav-ing a shared value definition throughout the organisation and have comparable re-sults within the organisation. The strategic level defines value in terms of moneyaccording to the strategic targets of the case company, while WSJF uses a relativescale without a clear definition of value. The fact that the valuation is subjectiveand relative in WSJF means that the WSJF scores are not comparable throughoutthe organisation, as different people may have valuated the same activity differently.

5.2.4 MetricsThe cycle time, satisfaction of work method and accuracy of predicted value arealready measured at the departments using WSJF. The cycle time is measuredthrough the agile planning software by measuring how long a feature stays in eachstage of the development process, such as testing and implementation. The satisfac-tion of the work method is measured by using surveys after each PI. The accuracy ofpredicted value is also measured after each PI by gathering the client and developerto compare the actual delivered value with the planned value. At this meeting, theclient evaluates if the delivery meets the expectation. The cycle time and satisfac-tion of work method are developed internally and could be used regardless of whatvalue definition being used. Hence, these two metrics satisfies the needs of both thestrategic and operational level. The measuring of the accuracy in predicted value is abuilt in measure in SAFe and relies on the usage of the subjective WSJF score. Thissatisfies both levels need to measure the accuracy of the value predictions. However,management use costs and potential incomes when evaluation if a project shouldstart or not. For that reason, they prefer to follow-up the accuracy on the samemonetary basis, which is not possible with the measure included in SAFe. Hence,this measure only partly satisfies the needs of the strategic level.

As WSJF assigns the projects with a subjectively decided value, the amount ofdelivered value is measured on the same basis. As the approach of assigning theWSJF value is simple, it satisfies the needs of the operational level. For this level,

55


the possibility to make a relative comparison between the features and trying tomaximise the delivered value is more important than on what basis the score ismade. The strategic level is stricter to measure the delivered value in monetaryterms, as they need to follow-up if the delivered value is realised in the organisationand report to financial stakeholders, which requires reporting in monetary terms.For this reason, the WJSF approach to measure delivered value does not meet therequirements of the strategic level.

5.2.5 Types of ProjectsWSJF is designed to be flexible in the feature evaluation process and includes com-ponents that corresponds to all three types of projects at the case company. As thethree components are summarised into a common score, the three types of projectscould be compared without considering the project type. However, WSJF gener-alise that all three components are weighted equally which could be questioned if italways mirrors the reality or the organisational strategy.

5.2.6 Summary of WSJF EvaluationThe distribution of satisfied, unsatisfied and partly satisfied needs on the operationaland strategic level are summarised in Table 5.5 below.

Table 5.5: Distribution of satisfied, unsatisfied and partly satisfied needs on anoperational and a strategic level.

Level #Yes #No #Partly

Operational 17 (85%) 0 (0%) 3 (15%)

Strategic 9 (50%) 6 (33%) 3 (17%)

As indicated in Table 5.5, WSJF satisfies 17 out of 20 of the operational require-ments and partly satisfies three of them. The three requirements that are partlysatisfied are: independence of specific individuals, clear definition of business valueand scalability in functionality. These are connected to the subjective usage of CoD.However, the subjective approach is essential to keep the model simple and flexible.The employees of operational level clearly stated in the interviews that they want tospend time on development rather than prioritising and therefore requires a simplemodel. With this input, the conclusion is that WSJF is suitable for prioritisationon an operational level even though three requirements are left partly satisfied.

For the strategic level, the number of satisfied requirements is 9 out of 19. Sevenof the requirements are unsatisfied and three are partly satisfied. Many of theunsatisfied requirements are related to the ability to follow-up and compare valuethroughout the organisation. The target of WSJF is to quickly get an indicationof how to maximise the delivered value and how the teams perform. The strategiclevel wants to analyse the performance of the organisation as precise as possible and

56


thereby WSJF falls short in precision. Consequently, the conclusion is that WSJFfails to satisfy the requirements of the strategic level.

5.3 Recommended Prioritisation Approach for Fi-nancial Software Development

Due to the unsatisfied needs of the strategic level, this part of the study focuses onhow WSJF better could satisfy the prioritisation needs of the strategic level. Todo this, the underlying trend of the unsatisfied needs has to be investigated. Thetwo main themes of the unsatisfied needs are related to the definition of businessvalue and how management can ensure that the effects of a delivery is realisedat the receiving end. These needs are similar to the ones described in literatureamong the authors arguing for using a monetary CoD over a relative CoD (Arnold& Yüce 2013), which aligns with the managements´ willingness to use a monetaryCoD. To satisfy the prioritisation needs of the strategic level, two recommendationsare formulated. The recommendations are usage of a uniform value definition and aprioritisation model that can combine the usage of monetary CoD and WSJF. Table5.6 presents what unsatisfied needs that correspond to each recommendation.

Table 5.6: Unsatisfied needs with recommended solutions.

Strategic Need Corresponding recommendationClear definition of business value Uniform value definitionDeliveries traceable in financial figures Combined usage of monetary CoD and

WSJFShared value definition throughout theorganisation

Uniform value definition

Same measures for prioritisation andfollow-up

Uniform value definition & Combinedusage of monetary CoD and WSJF

Comparable results within the organi-sation

Uniform value definition

Delivered value (monetary) Combined usage of monetary CoD andWSJF

5.3.1 Uniform Value DefinitionThe management is sceptical about the vague formulation of business value in WSJFand requires a clear value definition to evaluate if a project aligns with the bank´soverall strategy. Moreover, a uniform value definition would improve the inclusion ofemployees in the prioritisation process, as the value of each project becomes clearer.As a result, employees could better understand how the strategic level defines valueand start to question or motivate the priority of projects where their valuation dif-fers. Hence, the first recommendation for how to better adapt WSJF to the needsof the strategic level is to define a uniform business value that is shared between

57


management and operations.

The suggested components for CoD are based upon the value definition presentedby Interviewee D, who suggests that value is defined as: increased revenue, reducedcost, saved time and risk impact. Even though the time savings and risk impactcomponents could be translated to an increase in revenue or reduced cost, the com-ponents are introduced to simplify the evaluation of features and to clarify whatcorresponding effects that are expected to be realised. This is due to the fact thatthe effects of these two components are dependent on how management chose torealise them. However, it is important not to calculate the effects twice. As anexample, a cost reduction related to a time saving cannot be registered as both acost saving and a time saving.

The four value components are complemented by a fifth component, enabling value.The enabling value is a product from producing features that enables retrieval ofbusiness value in the future. The enabling value is calculated according to the con-cept developed by Wright (2018). To clarify the concept of calculating enablingvalue, it has been illustrated in a formula and an illustration, presented in Figure5.1. The purpose of adding the enabling value to the value definition is to reduce thevolatility in value deliveries and to enable a fair comparison when monitoring howmuch value a team delivers. If enabling value is neglected, the effects of enablingfeatures will not show until its enabled projects are realised and have delivered value.This concentrates the value deliveries and leads to a high volatility when monitor-ing delivered value. Furthermore, the monitoring process might indicate that teamsunderperform during sprints that contains many enabling features. This is solvedby the enabling value, as a high enabling value of the sprint indicate that the teamshave developed many enablers. However, the enabling value is only a symbolic valuethat is used for prioritisation and reporting value outcomes of a sprint. As an effectof the symbolic value for the enablement of future development, the actual effects ofthe enabler could not be realised and traced in the financial figures. Consequently, itshould be seen as an efficient tool to use in prioritisation when comparing enablingprojects and business development project and not being used to control that theeffects of the development is realised in the operations.

Figure 5.1: Illustration and formula for calculating enabling value. (Based ontheories from Wright (2018)).

58


5.3.2 Combining Monetary CoD with WSJFThe study indicates a mismatch in value focus between the operational and strategiclevel at the bank. The strategic level requires a monetary value connected to theprioritisation and the ability to trace deliveries to the financial figures. Meanwhile,the operational level is satisfied with the prioritisation model as long as it encour-ages value discussions and allow projects to be compared with other projects in thebacklog. To address this mismatch, a recommendation is to complement WSJF witha monetarily focused prioritisation model for strategic prioritisation, advantageouslyCD3 as it also is based upon CoD. This would enable prioritisation to be made inmonetary terms at a strategic level using CD3 and in relative numbers on an op-erational level using WSJF. By using both models, the monetary prioritisation andmonitoring needs of the strategic level are satisfied as well as the need of having asimple model at the operational level.

The two models will act as complements and intercept on the MVP level of a project,where CD3 will be used on higher levels of prioritisation and WSJF will be usedon lower level prioritisation. The MVP level is selected as the interception pointas it is considered the natural level to which monetary value could be broken down(Duc & Abrahamsson 2016). This is also emphasised as a good interception pointby Interviewee F. At the interception point, the monetary CoD is translated to theWSJF definition of CoD.

5.3.2.1 Integration Between WSJF and CD3

The interception point for monetary CoD and relative CoD must be at a level thatenables a translation from a CD3 prioritisation score into a WSJF prioritisationscore. This requires a high enough level, where the effect of a delivery could be tracedand assigned a monetary value. However, the lower the level for the translation, themore detailed information regarding how value is distributed in the organisationcould be obtained. With this balance in mind, the recommended interception pointis set to the MVPs at the epic level in SAFe. The epics are made up of a numberof MVPs and are broken down to smaller portions as illustrated in Figure 5.2. AsMVPs are working prototypes, their effects are easier to track compared to smallerproduct elements, such as features that only contributes to a small portion of theworking software. An additional benefit of MVPs being a working prototype is thatthe quality could be controlled before the development teams can consider the valueto be delivered. Hence, the worry of interviewee G, that quality gets sacrificed forthe desire to deliver value at a higher pace is addressed.

59


Figure 5.2: Illustration of how monetary CoD interacts with WSJF.

Figure 5.3 illustrates how the value is broken down from a project level to smallerportions. The effects of the projects are assigned a monetary value through tradi-tional project budgeting models accompanied by the guidance of the uniform valuedefinition. Once the value of the project is calculated, it can be converted to a CoD,using the urgency profiles and recommendations formulated in Section 5.3.2.2. TheCoD can then be converted into a CD3 value by estimating the project duration anddivide the CoD with the estimated duration. The CD3 value could then be usedto prioritise among the projects to identify the one that deliver the most value pertime unit. When a project is selected, it is broken down into epics, consisting ofseveral MVPs. Each epic is assigned a share of the total project value. The epicconsists of MVPs, which are assigned their portion of the epic value. The MVP isthe smallest element to be assigned a monetary value as it is the smallest elementwhere the delivery can be implemented and its effects can be confirmed. Once aMVP is delivered, the value is reported to management, that follow-up on deliveriesto ensure that its effects are realised by the receiver.

Once an epic is selected for development, it is broken down into features, or capabil-ities and then features in full/large solution SAFE, where WSJF is used to prioritiseamong them and ensure that the features with the highest CoD is developed first.The WSJF definition of CoD is modified to better match the value definition ofthe management. Instead of dividing the CoD into business value, time criticalityand risk reduction/opportunity enablement, the CoD consists of two components:business value and time criticality. The time criticality is the same as defined inWSJF to capture the urgency. The business value consists of the five componentsdescribed in Section 5.3.1 to get a uniform value definition. These components are:increased revenue, reduced cost, saved time, risk impact and enabling value. Asthe risk impact and enabling value is included in the value definition, the risk re-duction/opportunity enabler component of WSJF is dropped. Once the modified

60


WSJF score is calculated, the features with the highest score can be included in theupcoming PI.

Figure 5.3: Illustration of combination of CD3 and WSJF.

5.3.2.2 Recommendations for getting to Monetary Value

The process of valuating project monetarily and break them down into MVPs can, insome cases, be rather difficult. It is important to understand the effects connectedto a project in order to estimate a fair value of it. The valuation is an iterativeprocess and as described by (Arnold & Yüce 2013), the 5-Whys are a suggested toolto use, encouraging the users to ask "why" until the benefit is identified. To furtherfacilitate the valuation process, a three-step guide inspired by the quantitative de-velopment team at the case company is a suggested tool to use. The guide is dividedinto three questions one should ask when valuating a project:

1. Can a monetary value easily be assigned to the activity?2. What is the alternative cost?3. What happens if we do nothing?

If the answer to question one is yes, it means the benefit is identified and the valuecan be assigned. If the answer is no, proceed to step two. A good way to stimulatethe valuation mindset is to analyse the alternative costs of the project. This can forexample be to investigate the cost of doing a task manually instead of automatisingit. The alternative cost gives a fair indication of the monetary value of the project.If there is no obvious alternative cost, then one can investigate what would happenif nothing is done? Does it generate a fine or an increased operational risk etc. If thevalue cannot be identified after this step, an appropriate question is if the projectshould be done at all?

61


Another useful tool to facilitate the process of valuating projects is the usage of ur-gency profiles. The urgency profiles are helpful to understand how the value changesover time and the effect of delivering late. From the interviews, it was derived thatthe most common urgency profiles in the financial industry are business developmentprojects with a peak unaffected by delay and for regulatory projects. For businessdevelopment projects, the urgency profile developed by (Arnold & Yüce 2013) forfeatures with a long-life and a peak unaffected by delay is used due to its simpleapproach to CoD, see Figure 5.4. The delay cost for projects with this urgencyprofile is generated by multiplying the delay with the monetary effects.

Figure 5.4: Urgency profile for ideas with a long-life and a peak unaffected bydelay (Arnold & Yüce 2013).

For the regulatory projects, no existing urgency profiles are handling this type ofCoD. Consequently, an additional urgency profile was developed based on empiricaldata sourced from the interviews. The urgency profile supports the need of cal-culating the CoD for regulatory projects with a deadline, after which the impactof a delay applies. The impact is often a fine or a ban, which in most cases havemonetary effects connected to it. To compare regulatory impacts with the CoD of abusiness development project, which accumulates the longer the project is delayed,the impact of a regulatory project has to be converted to a CoD. In the urgencyprofile developed for regulatory projects, the impact is converted to a dynamic CoDthat increase as the time to deadline gets shorter. The CoD reaches its maximumvalue when the time to deadline is equal to the estimated development time, mean-ing that the project will be late and the full impact will apply if the project do notget initiated immediately. Resulting from this, the CoD for regulatory projects isdynamic and the project becomes worth developing only when its CoD exceeds theCoD of other projects or features in the backlog. The urgency profile and formulafor calculating CoD for regulatory projects are illustrated in Figure 5.5.

62


Figure 5.5: Urgency profile for for regulatory projects with a deadline.

By implementing a uniform value definition and the recommendations for a valuefocused prioritisation model, the value of regulatory projects and technical enablerswill become clearer. By understanding the value of these projects instead of relyingon human intuition to prioritise them, these projects could be compared with regularbusiness development projects. Consequently, the company can focus its resourcestowards the projects generating the most value and thereby maximise its valueefficiency. By developing the most value-generating projects, a higher economicsustainability could be achieved.

5.4 Assessment of WSJF with Modifications forStrategic Prioritisation

The final step in formulating the recommendations for how WSJF better could sat-isfy the needs of prioritisation in financial software development is to benchmarkthe recommendations against the needs for the strategic and operational level, pre-sented in the analytical framework. The only operational requirement affected bythe recommendations is the need for a clear definition of business value. This isaffected by the bundling of the risk and enabling value into the business value com-ponent and the adding of the saved time measure. The change clarifies how thebusiness value is defined and hence, the change satisfies the need of a clear defini-tion of business value. As this is the only operational requirement affected by therecommendations, this section will focus on the changes affecting the strategic level.On a strategic level, several needs that previously were unsatisfied are satisfied bythe recommended changes to the model. The requirements that change as an ef-fect of the recommendation are highlighted in bold in Table 5.7 and discussed below.

63


Table 5.7: Assessment of modified WSJF from a strategic perspective. Therequirements affected by the recommendations are highlighted in bold.

Strategic levelAnalytical Method CD3 + WSJFFunctionalIndependent of specific individuals PartlyEncourages value discussions YesClear definition of business value YesCommon base for prioritisation regardless of project type YesScalability in functionality PartlyDeliveries traceable in financial figures YesEnvironmentalApplicable in a scaled agile environment YesAbility to prioritise regulated businesses YesOrganisationalShared value definition throughout the organisation YesSame measures for prioritisation and follow-up YesComparable results within the organisation YesMetricsCycle time YesSatisfaction of work method YesDelivered value (monetary) YesAccuracy of predicted value YesTypes of projectsBusiness development projects YesRegulation projects YesEnabling projects Yes

Two of the strategic requirements were to have a clear definition of business valueand that the value definition should be shared throughout the organisation. Theseneeds are satisfied by the implementation of the five value components: increasedrevenue, reduced cost, saved time, risk impact and enabling value. This definitionis implemented at all levels of the organisation and its components are used by themanagement to follow-up delivered value. Additionally, this definition of businessvalue is used for prioritisation at both the strategic and operational level. Thismeans that the need of having the same measures for prioritisation and follow-up issatisfied. A uniform value definition throughout the organisation makes it possibleto compare results and delivered value across the entire organisation, as the priori-tisation and follow-up are made on the same premises.

One of the requirements demanded by several interviewees working at a strategiclevel was the ability to measure delivered value monetarily. As the strategic levelmostly are involved in prioritising on a project level, this need could be consideredsatisfied by using monetary CD3 for prioritisation of projects and epics. Besidessatisfying the need to measure delivered value in monetary terms, the monetary

64


prioritisation model satisfies the strategic need of being able to trace deliveries tothe financial figures. When a MVP is ready for delivery, the expected effects arereported and the management can follow-up on how well the effects are realised inthe organisation. This enables the management to compare the expected effectswith the realised effects, making it possible to calculate the accuracy of the pre-dicted value. Moreover, the monitoring of performance in monetary terms allows acomparison of performance between different departments in the organisation. Thiscomparability is enhanced even further by the uniform definition of business value.

By having a shared value definition between the operational and strategic level, theprioritisation on the operational level could be aligned with how the managementevaluate their performance. This creates a better understanding of the expectationsof the teams and the two levels could work together in maximising the same valuestream. It also improves the managements understanding for deliveries that do notcontribute with a monetary effect, such as enablement and some regulatory projects.

To benchmark the effects of the recommendations, the ability to satisfy the strate-gic level’s prioritisation requirements are compared between WSJF and the versionmodified according to the recommendations. An updated distribution of the satis-fied, unsatisfied and partly satisfied requirements for the strategic and operationallevel before and after the recommended modifications are presented in Table 5.8.

Table 5.8: Distribution of satisfied, unsatisfied and partly satisfied needs on astrategic and operational level before and after modifications.

Level #Yes #No #Partly

Operational - Original WSJF 17 (85%) 0 (0%) 3 (15%)

Operational - With modification 18 (90%) 0 (0%) 2 (10%)

Strategic - Original WSJF 9 (50%) 6 (33%) 3 (17%)

Strategic - With modification 16 (89%) 0 (0%) 2 (11%)

As indicated in Table 5.8, all the requirements that were unsatisfied by WSJF aresatisfied by the recommended prioritisation approach. The ability to measure the ac-curacy of delivered value changed from being partly satisfied to being fully satisfied.This is due to the fact that the delivery of an epic can be traced in the financials ofthe receiving department and its actual effects can be quantified. The requirementsfor a model that are independent of specific individual and scalable in functionalityare still partly satisfied due to the usage of relative value on an operational level.The usage of relative numbers on an operational level makes the prioritisation offeatures dependent on the value perception of the individuals and the scalability infunctionality is still restricted to having a subjectively estimated value. However,the usage of relative numbers at an operational level is a necessity to keep the modelsimple, that was a highly emphasised requirement by the operational employees inthe interviews. They consider their value contribution to be highest when developing

65


and do not want to spend more time on prioritisation. As a compromise, the needsfor a model that are independent of specific individual and scalable in functionalityare left partly satisfied. As the number of satisfied needs increased from 50% to 89%on a strategic level and from 85% to 90% on an operational level while both thenumber of unsatisfied and partly satisfied needs decreased, the conclusion is that therecommended approach better satisfies the needs of financial software developmentthan WSJF.

66

6Conclusion

This final chapter concludes the thesis and provides a concrete answer to each ofthe research questions. This is followed by an explanation of the implications of thethesis from an academic and industrial perspective. At last, the limitations of thestudy is presented in combination with recommendations for further research.

As a response to the trend of scaling the agile methodologies using SAFe in thefinancial industry, a qualitative case study was conducted with the purpose of in-vestigating the prioritisation needs in financial software development. These needswere used to assess how well WSJF is able to satisfy these needs. The qualitativeanalysis sourced empirical data collected from interviews, internal documents andan observation. The study was structured by one main RQ with two underlying subRQs, serving as a foundation to answer the main RQ. A conclusion of the findingsis presented hereunder.

Sub RQ1 What are the prioritisation needs of financial software development?

A major finding from investing the prioritisation needs is that no single set of needsare able to conclude the prioritisation needs of all organisational levels. Even thoughsome of the needs are shared on all levels in the organisation, they differ betweenthe strategic and the operational level of the organisation. Consequently, the needshad to be divided into two sets, needs of the strategic and operational level.

The needs of the strategic level are characterised by their focus on a monetary per-spective of prioritisation. The general trend of the strategic needs is to have a cleardefinition of business value and processes that are coherent throughout the organisa-tion. This is connected to their willingness to have a precise monitoring and abilityto track the organisation’s performance from the financial figures. Furthermore, thestrategic level has a higher focus on comparability than the operational level. Theoperational level has a more practical approach to prioritisation, which shows intheir prioritisation needs. They require a prioritisation model that is easy to useand requires a minimal amount of time, allowing for more development time. Theyshare the need for a value focus with the strategic level. However, a monetary ap-proach is not a necessity for the operational level. Instead, they want the ability toquickly understand what features in the backlog that bring the most value to thebusiness and give these the highest prioritisation. The full sets of requirements are

67

6. Conclusion

presented in Table 5.1 and 5.2 in Chapter 5. The findings from Sub RQ1 was usedas input to Sub RQ2, which investigated the following:

Sub RQ2 How does WSJF satisfy the needs of prioritisation in financial softwaredevelopment?

To evaluate how well WSJF satisfies the needs of prioritisation in financial soft-ware development, the needs obtained from the empirical study formed two setsof requirements, one set for the operational level and one for the strategic level.These sets were compared with the capabilities of WSJF. The capabilities of WSJFwere obtained through literature, the observation conducted at a PI planning andinterviews with the employees of a department utilising WSJF in their operations.The comparison reveals that WSJF satisfies 85% of the operational prioritisationrequirements and 50% of the strategic prioritisation requirements. The remainingoperational requirements are partly satisfied by WSJF. However, WSJF fails to sat-isfy 33% of the strategic requirements. This leads to the conclusion that WSJFin general satisfies the operational requirements while failing to satisfy the strate-gic requirements. When the two Sub RQs were answered, they formed a platformfrom which the main RQ could be solved. The main RQ was formulated as following:

Main RQ How could WSJF be modified to better suit financial software develop-ment?

As Sub RQ2 indicates, WSJF does not fully satisfy the needs of both the strategicand operational level in financial software development. Hence, the model couldbe modified to better satisfy the industry. Derived from the needs obtained whenanswering Sub RQ1, two recommendations for improvements are suggested. Thetwo recommendations are:

• Establish a uniform value definition that are used by the entire organisation.

• Implement a prioritisation model that can combine monetarily CoD at a strate-gic level with relative CoD at an operational level.

For the uniform value definition, the following value components are suggested forfinancial software development: increased revenue, reduced cost, saved time, riskimpact and enabling value. These could be used with monetary value using the pri-oritisation model CD3 at a strategic level and with relative values using WSJF forprioritisation at an operational level. The epic level is a suggested interception pointfor the translation from monetary value into relative value. This point is natural forthe translation as the epic consists of several MVPs, which is the lowest level whereworking software is delivered. This makes the value of each delivery easier to trackand confirm. The two recommendations are integrated with the WSJF prioritisationmodel and were theoretically benchmarked towards the needs obtained in Sub RQ1.The benchmark indicates a significant improvement in satisfied needs for the strate-gic level and a marginal improvement for the operational level. The percentage of

68

6. Conclusion

satisfied requirements increased from 50% to 89% on a strategic level and from 85%to 90% on an operational level. Meanwhile, the percentage of unsatisfied and partlysatisfied requirements decreased for both levels. Resulting from this, the conclusionis that WSJF could be modified to better suit financial software development byimplementing the recommended changes formulated in this report.

6.1 ImplicationsThis section describes the implications from an academic and industrial perspec-tive. The academic implications refer to the theoretical usage of the findings, whilstthe industrial implications refer to the practical application of the findings in thefinancial software industry.

6.1.1 AcademicThe result of this study indicates that there are different needs and requirements ofprioritisation between employees working on an operational and strategic level. Asthis difference in needs are present for prioritisation, it is likely that similar differ-ences are present in other areas affected by a scaled agile implementation. Conse-quently, this study aims to highlight the importance for an academic researcher toconsider both the needs from a strategic and operational perspective when research-ing the field of scaling agile. Furthermore, this study contributes with empirical dataof the needs and requirements of prioritisation in financial software development.

In the present literature, there are several studies indicating the positive effects ofusing monetary value for prioritising backlogs. However, none of the papers describesthe processes or challenges of calculating the monetary value. This study aims tocontribute to the existing body of knowledge by presenting challenging valuationcases and proposals for how these could be approached.

6.1.2 IndustrialFrom a financial software development perspective, one of the main takeaways fromthis study is the different needs of prioritisation on an operational and a strategiclevel. The different needs are something that needs to be taken into considerationwhen trying to implement activities that involves both the strategic and operationallevel. An additional takeaway is that it is fundamental to have a coherent definitionof value throughout the organisation before attempting to scale the agile work. Thisensures an alignment between management and operations and the focus could besteered towards a coherent value stream. Without a common definition of value thevalue estimations will be inaccurate, regardless if an absolute or relative prioritisa-tion model is used. Besides aligning the organisation, the uniform value definitionimproves the transparency of the management’s expectation. This improves socialsustainability for the employees as they have a better understanding of why certainactivities are higher prioritised and can affect the decision-making.

69

6. Conclusion

The study also aims to improve the economic sustainability of the financial sectorby contributing with recommendations for how to better understand the value ofthe industry´s common types of projects. By increasing this understanding, thecompanies could focus their resources on doing the most value generating tasks.This would reduce waste and increase the delivered value, without increasing theworkload.

6.2 Limitations and Future ResearchIn this section, the limitations of the work and recommendations for future researchare presented.

• Due to the competitive landscape of the software development industry andthe race towards digitisation in the financial sector, the study experiencedproblems to source empirical data outside of the case company. Consequently,the results could only by validated at the case company and its generalisabil-ity builds on the assumption that other incumbent banks are structured in asimilar way. Hence, empirical data from other incumbent banks are needed tofurther confirm the generalisability of this study.

• As the study was conducted as a master thesis, its duration was limited tothe time requirement set by the supervising institution. Due to the time lim-itation, the testing of the prioritisation recommendations was limited to atheoretical benchmark. To better understand the practical effects of the rec-ommendations, a future study is needed where the recommended changes areimplemented and tested in a real-world scenario.

• The study focused on investigating how well WSJF suited financial softwaredevelopment and therefore had the underlying focus of how well a relativevalue approach suited the financial sector. An interesting research approachwould be to investigate how well a monetary prioritisation approach satisfiesthe needs, to investigate if this approach reaches the same conclusion. Thiswould allow for a triangulation of the conclusion reached in this study.

• The gap in prioritisation needs between operational and strategic levels areobserved and used when formulating the recommendations for how to adaptthe prioritisation model. However, this study does not explain why this gapoccurs and hence a future research could be to further investigate reason be-hind this gap.

70

Bibliography

Arksey, H. & Knight, P. (1999), Interviewing for social scientists: an introductoryresource with examples, Sage Publications.

Arnold, J. (2014), ‘Safe and weighted shortest job first (wsjf)’.URL: https: // blackswanfarming. com/ safe-and-weighted-shortest-job-first-wsjf/[Acessed 2019-05-20]

Arnold, J. (2017), ‘Qualitative cost of delay’.URL: http: // blackswanfarming. com/ qualitative-cost-delay/ [Ac-cessed: 2019-04-29]

Arnold, J. J. & Yüce, Ö. (2013), Black swan farming using cost of delay: Discover,nurture and speed up delivery of value, in ‘Agile Conference (AGILE), 2013’,IEEE, pp. 101–116.

Beck, K., Beedle, M., Van Bennekum, A., Cockburn, A., Cunningham, W., Fowler,M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R. et al. (2001), ‘Manifestofor agile software development’.

Bender, J. R. (2005), Reporting for the Media, NY: Oxford Press, New York.

Blomkvist, P. & Hallin, A. (2015), Method for engineering students: Degree projectsusing the 4-phase Model, Studentlitteratur AB.

Brenner, R. & Wunder, S. (2015), Scaled agile framework: Presentation and realworld example, in ‘2015 IEEE Eighth International Conference on Software Test-ing, Verification and Validation Workshops (ICSTW)’, pp. 1–2.

Bryman, A. & Bell, E. (2011), Business research methods, 3. ed. edn, Oxford Uni-versity Press, Oxford.

Collis, J. & Hussey, R. (2013), Business research: A practical guide for undergrad-uate and postgraduate students, Macmillan International Higher Education.

Denscombe, M. (2010), Good research guide : for small-scale social research projects,Open up study skills, 4. ed.. edn, Open Univ. Press, Maidenhead.

Dingsøyr, T., Moe, N. B., Tonelli, R., Counsell, S., Gencel, C. & Petersen, K. (2014),Agile Methods. Large-Scale Development, Refactoring, Testing, and Estimation:XP 2014 International Workshops, Rome, Italy, May 26-30, 2014, Revised Se-lected Papers, Vol. 199, Springer.

71

https://blackswanfarming.com/safe-and-weighted-shortest-job-first-wsjf/

http://blackswanfarming.com/qualitative-cost-delay/

Bibliography

Duc, A. N. & Abrahamsson, P. (2016), Minimum viable product or multiple facetproduct? the role of mvp in software startups, in ‘International Conference onAgile Software Development’, Springer, pp. 118–130.

Easterby-Smith, M., Thorpe, R. & Jackson, P. R. (2012), Management research,Sage.

Gummesson, E. (2000), Qualitative methods in management research, Sage.

Ihme, T. (2013), ‘Scrum adoption and architectural extensions in developing newservice applications of large financial it systems’, Journal of the Brazilian Com-puter Society 19(3), 257–274.URL: https://doi.org/10.1007/s13173-012-0096-0

Jokela, T., Iivari, N., Matero, J. & Karukka, M. (2003), The standard of user-centered design and the standard definition of usability: Analyzing iso 13407against iso 9241-11, in ‘Proceedings of the Latin American Conference on Human-computer Interaction’, CLIHC ’03, ACM, New York, NY, USA, pp. 53–60.URL: http://doi.acm.org.focus.lib.kth.se/10.1145/944519.944525

Knaster, R. & Leffingwell, D. (2017), SAFe 4.0 Distilled: Applying the Scaled AgileFramework for Lean Software and Systems Engineering, Addison-Wesley Profes-sional.

Lenarduzzi, V. & Taibi, D. (2016), Mvp explained: A systematic mapping study onthe definitions of minimal viable product, in ‘2016 42th Euromicro Conference onSoftware Engineering and Advanced Applications (SEAA)’, IEEE, pp. 112–119.

Measey, P. (2015), Agile Foundations Principles, practices and frameworks, BCSLearning & Development Limited, Swindon.

Miles, M. B. & Huberman, M. (1994), Qualitative data analysis: An expanded source-book, sage.

Moreira, M. E. (2013), Working with Story Points, Velocity, and Burndowns, Apress,Berkeley, CA, pp. 187–194.URL: https://doi.org/10.1007/978-1-4302-5840-7_19

Neuman, L. (2011), Social Research Methods: Qualitative and Quantitative Ap-proaches.

Reinertsen, D. G. (2009), The principles of product development flow : second gen-eration lean product development, Celeritas, Redondo Beach, Calif.

Roulston, K. (2010), ‘Interview ‘problems’ as topics for analysis’, Applied Linguistics32(1), 77–94.

Scaled Agile, I. (2018), ‘What is safe?’.URL: https: // www. scaledagile. com/ enterprise-solutions/what-is-safe/ [Accessed: 2019-01-28]

Scaled Agile, I. (2019), ‘Case studies’.

72

https://www.scaledagile.com/enterprise-solutions/what-is-safe/

https://www.scaledagile.com/enterprise-solutions/what-is-safe/

Bibliography

URL: https: // www. scaledagileframework. com/ case-studies/ [Ac-cessed: 2019-02-06]

Silva, K. & Doss, C. (2007), The growth of an agile coach community at a fortune200 company, in ‘Agile 2007 (AGILE 2007)’, IEEE, pp. 225–228.

Slevitch, L. (2011), ‘Qualitative and quantitative methodologies compared: Ontolog-ical and epistemological perspectives’, Journal of Quality Assurance in Hospitality& Tourism 12, 73–81.

Still, B. & Crane, K. (2017), Fundamentals of user-centered design: A practicalapproach, CRC Press.

Stober, T. & Hansmann, U. (2010), Best Practices for Large Software DevelopmentProjects, Springer.

The LeSS Company, B. (2019), ‘Less case studies’.URL: https: // less. works/ case-studies/ index. html [Accessed: 2019-02-06]

Traylen, H. (1994), Confronting Hidden Agendas: Co-operative Inquiry with HealthVisitors, in P. Reason (ed.) Participation in human inquiry, Sage London.

Vetenskapsrådet (2018), ‘Forskningsetiska principer inom humanistisk-samhällsventenskaplig forskning’.URL: http: // www. codex. vr. se/ texts/ HSFR. pdf [Accessed: 2019-05-10]

Vives, X. (2017), ‘The impact of fintech on banking’, European Economy (2), 97–105.

Wright, A. (2018), ‘How government digital teams can use costof delay to prioritize work’. http://alaniswright.com/blog/how-government-digital-teams-can-use-cost-of-delay-to-prioritize-work/[Accessed: 2019-04-29].

Yin, R. K. (2017), Case study research and applications: Design and methods, Sagepublications.

73

https://www.scaledagileframework.com/case-studies/

https://less.works/case-studies/index.html

http://www.codex.vr.se/texts/HSFR.pdf

http://alaniswright.com/blog/how-government-digital-teams-can-use-cost-of-delay-to-prioritize-work/

http://alaniswright.com/blog/how-government-digital-teams-can-use-cost-of-delay-to-prioritize-work/

AInterview Guide

This appendix presents the interview guide used in the semi-structured interviews.The themes are presented in bold with a selection of its corresponding questions pre-sented underneath.

IntroductionPresentation of the researchers and the study.Inform the interviewee about the anonymity and what the results willbe used for.

Backlog prioritisationWhat differs a project with a high priority from a project with a lowpriority?Do you use any tool for prioritising the backlog?Does the current prioritisation model have an urgency aspect?

Valuation of projectsWhat metrics are used for prioritisation today?How does your prioritisation process look like?Would you like to add any additional metrics to the existing model?Do you feel a need to have a value-driven prioritisation?Do you prefer a monetary or relative valuation of projects? Why?

Challenging valuation casesWhat are the major challenges in valuating projects?Does the current prioritisation model handle these? If yes, how?How do you handle enablers?How large share of the backlog have dependencies?

Backlog prioritisationWhat differentiates a project with a high priority from a project with alow priority?Do you use any tool for prioritising the backlog?Does the current prioritisation model have an urgency aspect?

Monitoring of performance

I

A. Interview Guide

What effects are used for external and internal follow-up?How is performance measured?How can the company obtain a fair comparison between different de-partments and teams?

Summary and wrap upDo you want to add any additional information?Do you have any questions?Can you recommend additional interviewees of interest for this study?Thank the interviewee for their participation.

II

is it safe to use wsjf for prioritisation in financial ...1372030/fulltext01.pdf · trying to scale...

Documents