d8.1.2 – evaluation plan (final) - cordis · 2017. 4. 25. · fp7 - 608991 - streetlife d8.1.2...

FP7-SMARTCITIES-2013

STREETLIFE Steering towards Green and Perceptive Mobility of the Future

WP8 – Impact Assessment & Simulations

D8.1.2 – Evaluation Plan (Final)

Due date: 30.09.2015 Delivery Date: 02.10.2015

Author(s): Philipp Gilka (DLR), René Kelpin (DLR), Davide Frigeri (CAIRE), Alberto Merigo (CAIRE), Mika Vurio (CGI)

Partner(s): CAIRE, CGI, DLR

Editor: Philipp Gilka (DLR) Lead Beneficiary of Deliverable: DLR

Dissemination level: Public Nature of the Deliverable: Report

Internal Reviewers: Astrid Kellermann (SIEMENS), Marco Pistore (FBK)

Ref. Ares(2015)4071248 - 02/10/2015

FP7 - 608991 - STREETLIFE D8.1.2 – Evaluation Plan (Final)

WP8 – Evaluation Plan (final) STREETLIFE Consortium of 96

EXECUTIVE SUMMARY

According to the planned two iterations in STREETLIFE the current ‘Evaluation Plan’ summarizes the evaluation approach for the 2nd iteration of the STREETLIFE project. Based on the valuable results within the 1st iteration the ‘Evaluation Plan’ was revised in order to address the lessons learned. As from the beginning the pilot objectives were beyond the state of the art, two iterations were planned focusing on a stepwise approach in order to realize the set objectives. Therefore the level of technical implementation will differ in both iterations. In addition it was identified as useful to interface directly WP6 field trial planning in order to harmonize even better the activities in this regard. Thus, the 2nd iteration experiments and use cases are described in detail in the present ‘Evaluation Plan’. Thus, a close link between WP6 (pilot execution) and WP8 (impact assessment) is established – bringing together pilot planning, evaluation planning as well as the final evaluation execution and analysis.

In summary, a self-containing document was formed as a useful tool which allows understanding and recalling the 1st iteration’s objectives and corresponding methods. Further the ‘Evaluation Plan’ includes the execution but also results and conclusions. Thus, this report partly repeats and concludes main statements of the initial ‘Evaluation Plan’ (D8.1.1) and relevant results and achieved impacts of the corresponding data analyses (D8.1.2 – Achieved Impacts).

Disclaimer: This project has received funding from the European Union’s Seventh Framework Programme for research; technological development and demonstration under grant agreement no 608991. The information and views set out in this publication are those of the author(s) and do not necessarily reflect the official opinion of the European Communities. Neither the European Union institutions and bodies nor any person acting on their behalf may be held responsible for the use which may be made of the information contained therein. © Copyright in this document remains vested with the STREETLIFE Partners



D8.1.2 – Evaluation Plan

Table of Contents

ABBREVIATIONS ............................................................................................................................................... 5

PARTNER ............................................................................................................................................................. 6

LIST OF FIGURES .............................................................................................................................................. 6

LIST OF TABLES ................................................................................................................................................ 7 1. INTRODUCTION ....................................................................................................................................... 8

1.1. INTENDED AUDIENCE ....................................................................................................................... 8

1.2. OBJECTIVES OF THE IMPACT ASSESSMENT ............................................................................. 8

1.3. DOCUMENT STRUCTURE ................................................................................................................. 9

2. LESSONS LEARNED FORM FIRST ITERATION AND APPLIED FOR SECOND ...................... 10 2.1. MAIN EVALUATION RESULTS FROM FIRST ITERATION ..................................................... 11

2.2. BERLIN PILOT CONSIDERATIONS .............................................................................................. 13

2.3. ROVERETO PILOT CONSIDERATIONS ....................................................................................... 14

2.4. TAMPERE PILOT CONSIDERATION ............................................................................................ 15

3. METHODOLOGY .................................................................................................................................... 16

3.1. IMPACT CATEGORIES..................................................................................................................... 17 3.2. APPROACH FOR ASSESSING THE IMPACTS ON USER BEHAVIOUR ................................. 17

3.2.1. COMMON APPROACHES FOR ALL PILOTS .................................................................................. 17

3.2.2. PILOT SPECIFIC APPROACHES .................................................................................................... 19

3.3. APPROACH FOR ASSESSING THE IMPACTS ON TRANSPORT SYSTEM ........................... 20


3.3.2. PILOT SPECIFIC APPROACHES ...................................................................................................... 20 3.4. APPROACH FOR ASSESSING THE IMPACTS ON ENVIRONMENT ...................................... 21


3.4.2. PILOT SPECIFIC APPROACHES .................................................................................................... 22

4. PILOT SCENARIO DEFINITION .......................................................................................................... 23

4.1. BERLIN ................................................................................................................................................. 23 4.1.1. EVOLUTION FROM 1ST TO 2ND ITERATION ............................................................................... 23

4.1.2. SECOND ITERATION’S SCENARIOS, USE CASES AND EXPERIMENTS ............................. 25

4.1.3. SCENARIOS AND USE CASES ......................................................................................................... 26

4.1.4. EXPERIMENTS ................................................................................................................................... 41

4.1.5. SKIPPED SCENARIOS AND USE CASES ....................................................................................... 47



4.1.6. OBJECTIVES ....................................................................................................................................... 48

4.1.7. RESEARCH QUESTIONS .................................................................................................................. 48 4.1.8. EVALUATION MATRICES – FROM HYPOTHESES TO INDICATORS .................................. 49

4.2. ROVERETO ......................................................................................................................................... 53

4.2.1. EVOLUTION FROM 1ST TO 2ND ITERATION ............................................................................... 53

4.2.2. 2ND ITERATION’S SCENARIOS, USE CASES AND EXPERIMENTS ........................................ 53

4.2.3. SCENARIOS AND USE CASES ......................................................................................................... 54

4.2.4. EXPERIMENTS ................................................................................................................................... 65 4.2.5. SKIPPED SCENARIOS AND USE CASES ....................................................................................... 68

4.2.6. OBJECTIVES ....................................................................................................................................... 69

4.2.7. RESEARCH QUESTIONS .................................................................................................................. 69

4.2.8. EVALUATION MATRICES – FROM HYPOTHESES TO INDICATORS .................................. 71

4.3. TAMPERE ............................................................................................................................................ 79 4.3.1. EVOLUTION FROM 1ST TO 2ND ITERATION ............................................................................... 79

4.3.2. EXPERIMENTS – SECOND ITERATION’S SCENARIOS AND USE CASES ........................... 79

4.3.3. OBJECTIVES ....................................................................................................................................... 85

4.3.4. RESEARCH QUESTIONS .................................................................................................................. 85

4.3.5. EVALUATION MATRICES – FROM HYPOTHESES TO INDICATORS .................................. 86

4.3.6. SCALE OF ENGAGEMENT .............................................................................................................. 91 5. REQUIREMENTS ON THE STREETLIFE SYSTEM ......................................................................... 93

APPENDIX C: LITERATURE ......................................................................................................................... 96



ABBREVIATIONS

BaU Business-as-Usual CO Confidential, only for members of the Consortium (including the Commission

Services)

BER STREETLIFE Berlin-Pilot

CIP City Intelligence Platform D Deliverable

DoW Description of Work

EXP Experiment

FP7 Seventh Framework Programme

FLOSS Free/Libre Open Source Software

FOT Field Operational Test

GPS Global Positioning System

GUI Graphical User Interface

HY Hypothesis IPR Intellectual Property Rights

IRTE Integrated Road Transport Environment ITS Intelligent Transport Systems

MGT Management

MMECP Mobility Management Emission Control Panel MS Milestone

OS Open Source

OSS Open Source Software

O Other

P Prototype

P&R Park and Ride

PI Performance Indicators

PTA Public Transport Authority PTP Pre-trip planning

PU Public

ROV STREETLIFE Rovereto-Pilot

RQ Research Question RTD Research and Development

TAPAS Travel Activity Patterns Simulation

TIC Traffic Information Centre Berlin



TIE Traffic Information Editor TRE STREETLIFE Tampere-Pilot

TSO Transport Service Operator

UC Use Case

UI User Interface

WP Work Package

PARTNER

Fraunhofer Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.

FBK Fondazione Bruno Kessler

SIEMENS Siemens AG

DFKI Deutsches Forschungszentrum für Künstliche Intelligenz GmbH

AALTO Aalto University

DLR Deutsches Zentrum für Luft- und Raumfahrt

CAIRE Cooperativa Architetti e Ingegneri - Urbanistica

Rovereto Comune di Rovereto

TSB Berlin Partner for Business and Technology

Tampere City of Tampere

Logica CGI Suomi Oy

VMZ VMZ Berlin Betreibergesellschaft mbH

LIST OF FIGURES

Figure 1: Evaluation and Impact Assessment Methodology .................................................... 16

Figure 2 Schema of CO2 assessment of ITS ............................................................................ 22

Figure 3: Relation between WP6 (pilots) and WP8 ................................................................. 23



LIST OF TABLES

Table 1: Evaluation results BER (1st iteration) ........................................................................ 11

Table 2 Evaluation results ROV (1st iteration) ......................................................................... 12

Table 3: BER pilot - Second Iteration Scenarios & Use Cases ................................................ 25

Table 4: BER-EXP-5 metrics for PI ......................................................................................... 47

Table 5: BER pilot - First iteration Research Questions .......................................................... 48

Table 6: BER pilot - Second iteration Research Questions ..................................................... 49

Table 7: Evaluation Matrices BER ........................................................................................... 49

Table 8: ROV pilot - Second Iteration Scenarios, Use Cases and experiments ....................... 53

Table 9: Research Question of ROV pilot addressed in Y1 and Y2 ........................................ 69

Table 10: ROVERETO Pilot Evaluation Matrix ...................................................................... 71

Table 11: Research Question of TRE pilot addressed in Y1 and Y2 ....................................... 85



1. INTRODUCTION

1.1. INTENDED AUDIENCE

The present ‘Evaluation Plan’ prepares and structures the evaluation process of the 2nd iteration of the STREETLIFE project by taking into account 1st iterations lessons learned. The idea of a 2nd Evaluation Plan is correlating with the two iterations planed in STREETLIFE. As the pilot objectives were beyond the state of the art, two iterations were planned focusing on a stepwise approach in order to realize the objectives. Therefore the level of implementation differs in both iterations. In addition to that the 1st iteration provided valuable results as lessons learned. In this regard a revised version of the Evaluation Plan was considered as useful. Finally, a direct interface to WP6 field trial planning was identified as meaningful in order to harmonize even better the activities in this regard. Thus, the 2nd iteration experiments and use cases are described in detail in the present Evaluation Plan.

In close cooperation with the pilot evaluation managers the pre-conditions and objectives of the pilot sites were considered elaborated. Based on these, research questions, hypotheses and success criteria have been derived. With the aim of answering these research questions, relevant indicators (performance indicators) have been selected and were defined in detail. In order to gather valuable data, use cases and experiments were designed. The local evaluation manager will be set into the position to have a clear understanding of the evaluation process.

Above this, internal (STREETLIFE) and external evaluation experts will be provided with the intended structure of this final Evaluation Plan. This allows understanding/recalling 1st iteration’s objectives and corresponding methods; its execution during the first field trials, its results and drawn conclusions as well as its application (careful consideration) for the 2nd iteration’s planning. It finally results in only one self-containing document. Thus, this reports partly repeats and concludes main statements of the initial Evaluation Plan (D8.1.1) and relevant results and achieved impacts of the corresponding data analyses (D8.1.2).

1.2. OBJECTIVES OF THE IMPACT ASSESSMENT

The main objective of the present Evaluation Plan is to develop a framework and guidelines to be applied for validation and assessment of the STREETLIFE system at different pilot sites. The STREETLIFE development and implementation work will be assessed with regard to the defined scientific and technological objectives which are according to the DoW the following: System evaluation Obj-8 Customize, deploy, operate and evaluate the proposed Urban Mobility System on three city pilots. Obj-9 Deliver results on impact assessment of the proposed solutions, in terms of traffic, end-users behaviour, and reduction of carbon emissions, and derive useful guidelines on mobility strategies for smart cities of the future.

Above these project level objectives, the pilot sites have a number of objectives which focus specifically on the local needs. These addressing different impact categories and will draw conclusion according to the STREETLIFE effects.



The present Evaluation Plan will provide the basis for evaluation and impact assessment to realize or answer the following:

- Does STREETLIFE have an impact on user behaviour, the traffic system and carbon emissions?

- How can STREETLIFE sustainably change the travel behaviour of citizens into a more eco-friendly behaviour?

- To what extent can STREETLIFE decrease the carbon footprint of individuals and the carbon emissions within a city caused by land-based transport?

In order to realize the envisaged evaluation and impact assessment activities common indicators for performance measurement at each individual level and the whole system will be defined. In addition, methods for data collection and analysis as well as impact assessment will be described.

1.3. DOCUMENT STRUCTURE

The document structure follows the above mentioned task of this report. Chapter 2 describes the lessons learned of the 1st iteration, summarizes its main evaluation results and achievements in terms of the big three impact categories User Behaviour, Transport efficiency and Environment. In addition the pilot specific considerations for the 2nd iteration are specified. In order to consider the revisions and adjustments for the 2nd iteration the adapted Methodology is introduced. The applied experiments, methods, research questions and hypotheses are outlined in Chapter 4. Here, also the 2nd iteration’s experiments and use cases are described and linked with local and global indicators and impact categories. Thus, a close link between WP6 (Pilot Execution) and WP8 (Impact Assessment & Simulation) is established – bringing together pilot planning, evaluation planning as well as the final evaluation execution and analysis. Finally, Chapter 5 discusses the used technical requirements and tools for data acquisition.



2. LESSONS LEARNED FORM FIRST ITERATION AND APPLIED FOR SECOND

A major focus of the 1st pilot iteration is lying on the realisation of the pilot objectives. Each pilot firstly investigated the technology available and how it can be integrated for the need of that specific pilot. Based on the collected information, each pilot then defined a set of scenarios aiming to gather valuable impact results. The impact analysis realized addresses two areas: First, the pilot scenarios drive the technological innovation with respect to the involved city municipality’s objectives and in return, the pilots follow also the main goals of the STREETLIFE project. The intended strategic benefits of the STREETLIFE project focus on sustainable urban mobility which affecting the main three impact categories: User behaviour, Transport Performance and las but not least Environment.

Each of these impact categories is measurable whereas performance indicators can be derived on. However, a lack of measured data leaded to insufficient data availability in some cases which resulted then in slighter reliability.

A redesign of the first Evaluation Plan (D8.1.1) was necessary as evaluation is regarded in all the three pilots as a continuous learning process, leading to incremental understanding and refinement of questions as well as answers. In addition, the remarks from the 1st year review were considered and guided the work of WP6/8. It became apparent that the applied methodology need to be further adjusted and improved for the 2nd iteration field test in order to put more emphasis on the Gamification part while also focusing on the evaluation of the STREETLIFE solutions beyond the three pilot cities. Finally, the following main reasons were identified as crucial:

- Although the user recruitment approach was largely successful the biggest challenge was to keep a significant group of users active for a longer period. An important topic of the 2nd iteration will be therefore user recruitment. A reliable and strategic user recruitment concept needs to be elaborated for all three pilot sites to make sure the pilot can sustain interest and engagement by the participants.

- As part of the user recruitment the gamification topic with its user incentive was elaborated more clearly. In the designed use cases human factors and game-like interactions were specified and addressed. In order to keep a continuous engagement of the user and especially to motivate a change in travel behaviour different game-like interactions were developed to attract user.

- A reliable data acquisition from the field trials is required in order to realize a valuable analysis and assessment. For instance, a link between routing proposals and user tracking data needs to be established in order to verify the selected trip and accordingly the transport mode used.

- In order to gather sufficient data sets to cover the impact assessment adequately a longer field trial time frame at the pilots need to be established which facilitate easy identification of changes in mobility behaviour.

These reasons have indeed guided the work of WP6 and WP8 in the present Evaluation Plan.



2.1. MAIN EVALUATION RESULTS FROM FIRST ITERATION

Occasionally, some of the results reported below may also extend and augment the original Evaluation Plan, whenever data collection and analysis during and after the 1st pilot iteration has made evident features and findings that were not foreseen at the time the Evaluation Plan was set up.

The WP8 project levelled evaluation was focussed on the main global evaluation categories, namely:

i) User Behaviour

ii) Mobility

iii) Environment

The results of the 1st iteration’s WP8 evaluation are reported with the deliverable D8.2.1 Achieved Impacts. The main results in the three STREETLIFE pilots can be summarised as follows:

Berlin

An overall focus of the 1st iteration field trial BER-EXP-2 was set on testing the Berlin App, its applicability for evaluation purposes, corresponding data flows and service interfaces, and the intended Berlin evaluation methodologies as such. The Berlin field trials have been performed as “friendly user tests”. Up to 12 users -familiar with the project STREETLIFE and the Berlin App- used and evaluated the App and the underlying services.

Table 1: Evaluation results BER (1st iteration)

Evaluation Categories 1st iteration’s main results

User Behaviour It can be stated that people provided with an intermodal App are more willing to change mobility behaviour while changing their mode choice, as long as alternative “green” transport means are available. It could be observed independently from the App tested, that also different modes were combined more easily due to the information available.

Mobility In Berlin a simulation-based approach was used to calculate the effects. The results show that share of cycling can be increased by 5% if cycling becomes more attractive. Specifically people who are travelling mid-range trips (<7km) are more willing to enhance their trip radius. At the same time it was observed that the mode share for car decreased by 4%.

CO2- Emissions In a large city like Berlin about 500t could be saved per day for the overall transportation system



Rovereto

The 1st iteration in Rovereto had two experiments: the first one has been conducted on a closed test group of 40 people, and its main goals were to determine whether the ViaggiaRovereto routing App could change user’s behaviour towards multimodal mobility, and analyze the impact of gamification and gaming incentives in this type of environment. The second experiment was an open field testing of the ViaggiaRovereto routing App during a special event that put higher pressure on the mobility system: Rovereto’s Christmas Markets.

Table 2 Evaluation results ROV (1st iteration)


User Behaviour In the 1st iteration a small-scale test of a gamification approach was tested in the City of Rovereto. The results provided show a continuously growing usage of the App during the timeframe of test trials. There was a strong growing of App usage after launching the gamification functionality within the App. We conclude that the results of these tests were consistent with the two hypotheses gamification-related: in fact the Green Game with ViaggiaRovereto reached both its goals, since a) there was an increase in the level of App usage and b) there was also growth in the number of green itineraries chosen.

Mobility The designed STREETLIFE solutions, besides raising the level of awareness and information about sustainable mobility service available in a small city like Rovereto, proved that they could have a real impact on the city traffic system. ViaggiaRovereto and its gamification approach together with the policies that Rovereto municipality chose to promote in the context of the project, which are automated and pushed via the STREETLIFE solution, would have saved almost 25,000 km every day, if all commuters to Rovereto used the App for planning their trips.

Estimating a penetration rate of 75% of smartphone usage in European population in a few years, the distance travelled by car thanks to STREETLIFE will decrease by 4%. That corresponds to a reduction of more than 16,600 km every day.

CO2- Emissions CO2 emissions would have constantly decreased during the trial process, reaching a reduction of 6% in the final week. Finally, if all commuters of a small city would use the STREETLIFE App 4,4t of CO2 could be saved by day. Considering that the total emissions produced by Rovereto car commuters are 71t of CO2 each day, the STREETLIFE solutions would have an impact estimated in a reduction of 6% on transport based emissions.

Similarly to traffic impact, this result does not take into account



how many people can actually use the STREETLIFE App. Assuming that in a few years, 75% of European population own a smartphone, STREETLIFE could save 2.8 t of CO2 every day in a small city. This result reveals that ICT solutions can have a real and measurable environmental impact even in a small city like Rovereto.

Tampere

Table 3 Evaluation results TRE (1st iteration)


User Behaviour The STREETLIFE real-time trip planner provided a multimodal approach to the routing problem, and the suggested routes are more comprehensive and clear about all the possible solutions, thus providing advantages for tourists, people planning new routes and for commuters on long trips within and outside of the city.

There is general consensus on the fact that STREETLIFE has managed to change mobility habits of its users. Environmental awareness of commuters was raised and improved awareness of public transit services was achieved.

Mobility It was estimated with better information 0.5 – 1% modal shift towards public transit can be achieved in a Medium city.

It was possible to state that in the best case scenario in a medium city, 57,000 km could have not been travelled by cars, but by public transport, thus having a real reduction of car traffic congestion.

CO2- Emissions In the environment of the medium city the qualitative evaluation of the STREETLIFE system has given a positive response.

2.2. BERLIN PILOT CONSIDERATIONS

Within the 2nd iteration the Berlin pilot focuses on three main goals:

i) Increase the safety for cyclist

ii) Reducing the carbon footprint of a trip

iii) Improve the transport performance of the city



In addition a crowd-source based assessment of safety aspects of each particular cycling route is envisaged. Therefore a shift in the focus from general modal change observations in the 1st iteration to an assessment of cycling safety perceived and stated by App users is envisaged. The motivation behind this shift is the hypothesis that people feel not comfortable and safe while cycling but safety relevant information might encourage people to cycle. Although the 1st iteration was planned and conducted as friendly user test a sufficient set of data could not be gathered as even in this test procedure we had to deal with drop outs of user. Considering this a strong focus will be laid on user recruitment and to keep them interested for at least the lifetime of the trials. In this regard the pilot partner agreed to sub-contract a specialized user recruitment office. The involvement of external knowhow and experience in terms of user acquisition will support our research activities as the handling of about 50-100 user during the test trial time frame is a major work. Therefore it was decided that the while user support activities will be out sourced.

In order to keep the user interested in using the STREETLIFE App a well-designed and well- performing App was identified as a key precondition. Hence, a number of activities were defined to improve the design and usability before start running with the 2nd iteration field trials.

Finally, the planned and implemented activities will mainly support efficient and effective field trials with the result of realizing a valuable data set. Compared to the 1st iteration were the simulation with TAPAS was conducted mainly on a literature basis, the 2nd iteration will gather a comprehensive set of data. However, this data base is required to calculate valid and representative impacts on CO2 emissions and mobility.

Summing up, the biggest challenge for the impact assessment at the BER pilot will be to evaluate the change of user behaviour, specifically their mode choice and trip length as no baseline trials are foreseen. It is planned to close the gap by using a questionnaire based approach.

2.3. ROVERETO PILOT CONSIDERATIONS

The Evaluation Plan worked in the 1st iteration. The Rovereto pilot cluster managed to collect data from end users’ questionnaires and log files from the app in order to evaluate research questions and hypothesis according to the Evaluation Plan. The approach and the structure of evaluation will be confirmed also for second iteration, consequently research questions and hypothesis involving multimodal mobility are the same as in the 1st Evaluation Plan.

The second iteration will test some new scenarios and use cases, so the Evaluation Plan has been updated and adjusted with a higher and more detailed level regarding research questions and hypothesis that involve two topics:

- gamification - car pooling

With respect to the already existing scenario of Multimodal Mobility, which is a fusion of the older ones “Bike Sharing” and “Park & Ride”, changes that will be implemented in the second iteration are the following:



- in the special events’ experiment, there are margins for improvements in data collection: the open field nature of the experiment generates many data, and in the 1st iteration some of this information got lost.

- the minimum length of walking segments in the multimodal mobility scenario has been changed with the goal of improving reliability of data and obtaining a clearer separation between different modes

- parameters for the definition of “green” routes in the range of solutions proposed from the routing app have been improved according to city mobility policies

- the tracking mode functionality will provide a higher level of reliability with respect to trips that have been really taken following Routing App instructions

2.4. TAMPERE PILOT CONSIDERATION

The Evaluation Plan in the 1st iteration worked relatively well. The Tampere pilot collected data from end users’ questionnaires and used statistics in order to evaluate the pilot according to the Evaluation Plan. The same approach and the structure of evaluation will be also used for second iteration; consequently research questions and hypothesis are the same as in the 1st Evaluation Plan. Following areas of improvement are taken into account for the 2nd pilot iteration evaluation.

Lessons learned in the 1st pilot phase are to improve the participation and make it easier to compare pre/post study data. Actions are already being taken to address them. Relating to participation a discussion has been started with the Tampere University of Technology student's group, who are active on traffic related matters. Plan is to engage them continuously and give them early access in 2nd pilot iteration and to Mixed Reality field trials. Idea is that the improved communication will keep test users interest in the pilot. Participation boost is achieved with continued communication with Tampere Traffic student group and other focus test group members. Additionally rewards for participation and gamers are planned.

In collection of results the following improvements are taken in to consideration. Pre/post questionnaire improvements which allow matching better pre/post questionnaires, tune the questions relating to measure changes in user behaviour and mode changes over time. Weather plays major role in travelling habits, in the second iteration there will be several data collection points in multiple different times on October, February and April to counter how the winters affects the mode share in order to help the comparison of results.



3. METHODOLOGYBased on the methodology described in the initial Evaluation Plan (D8.1.1 Evaluation Plan) the present Evaluation Plan shares common objectives but focusses also on revisions and adjustments due to several reasons which are:

- Pilot specific revised objectives - Lessons learned from the 1st iteration

The STREETLIFE evaluation and impact assessment methodology [Figure 1] is mainly based on the core scientific and technological objectives of the project [STREETLIFE DoW]

Figure 1: Evaluation and Impact Assessment Methodology Figure 1 presents the Evaluation and impact assessment methodology. Based on the overall project objectives one common system architecture [D2.2.1 – STREETLIFE Blueprint Architecture, Security Architecture, and site-specific architectures (initial)] had been developed in order to define the software and service for the STREETLIFE project. This architecture is generic enough to be used for different cities and shall reduce the risk of fragmentation. Each city pilot possesses different needs, ambitions and goals but also different infrastructures and legacy systems. Within the second level of Figure 1 pilot site-specific research objectives were established to focus specifically on the local needs. In order to support the research activities, hypotheses were defined. Based on the objectives, different system functionalities are developed and related uses cases created. The role of the use cases is to describe situations where STREETLIFE would bring additional benefit to the user. With the aim of answering the



defined research questions, performance indicators (PI) were established. Overall, two iterations are planned for testing the system at the different pilots. During the test trials, several measures will be realised to collect relevant data. Within the third level of Figure 1 the gathered data will be analysed. Based on the measures and metrics performance indicators will be derived by mathematical calculation. A statistical evaluation of the derived performance indicator will provide information to the developed functionalities and the relevant impact categories. Finally, in level four the impact assessment will be realized. The defined research questions will be answered with the special focus on user behaviour, impacts on the transport system and environment. Based on these findings future mobility strategies for small, medium and large cities can be discussed as defined in the STREETLIFE DoW.

3.1. IMPACT CATEGORIES

Also for the 2nd iteration, the WP8 evaluation focus is laid on the main impact categories i) User behaviour

ii) Mobility

iii) Environment

STREETLIFE is aiming to showcase in the three pilots that carbon emission can be saved, the performance of the underlying transport systems can be improved, and the behaviour of the users can be significantly changed with the services, applications, scenarios and use cases designed and implemented for a 2nd iteration’s testing.

The WP8 approach is to create a common evaluation methodology. However, with the further deployment of the pilots, its focus and experiments after the results of the 1st iteration have been evaluated, and a re-assessment of the applicability of given evaluation criteria became necessary. For instance, the 2nd iteration the Berlin pilot is strongly focussed on cycling safety and the assessment of respective data sources and statistics.

3.2. APPROACH FOR ASSESSING THE IMPACTS ON USER BEHAVIOUR

3.2.1. COMMON APPROACHES FOR ALL PILOTS

This sub-chapter discusses a common approach for assessing the STREETLIFE system according to the user behaviour. It can be seen as the major aim to assess user acceptance and willingness to use the provided solution. The core question we are going to answer is: How can STREETLIFE sustainably change the mobility behaviour of citizens into a more eco-friendly behaviour?



In order to tackle the envisaged goal different sub-categories of user behaviour needs to be analysed and assessed, such as:

- User Acceptance - Usability - System Usefulness - Functionality of the system - Sustainability of achieved changes - Applied motivators

In this regard several research questions, hypothesis and indicators to measure these sub-categories are identified and addressed in Chapter 4.

With respect to gather information in terms of user behaviour, two data sources are used while focusing mainly on the above-mentioned sub-categories. First, a well-designed questionnaire has been approved as an adequate tool to provide valuable results. Although different services and user interfaces are going to be tested at each single city a common questionnaire will be used. This questionnaire addresses overall relevant items but also additional questions to meet the local requirements and objectives. Similar to the 1st Initial Evaluation Plan (D8.1.1) a baseline and treatment data acquisition is foreseen by using two slightly different questionnaires. A two stage user survey about their daily mobility behaviour and especially their expectation towards the promoted system will bring valuable results. The comparison between similar questions of the two stage questionnaires survey (baseline and post-experiment questionnaire) will provide a clearer picture of the variation in the different sub-categories of user behaviour. The methodical set-up of the questionnaire is based on the Technology Acceptance Model’ (TAM) as already used in the 1st iteration and described in deliverable D8.2.1 Achieved Impacts.

Second, gathering information from the log files of the STREETLIFE system will provide additional benefits in terms of user behaviour evaluation. Valuable data sets could be gathered regarding the users interaction with the system. Moreover positioning data will allow analysing trip details such as transport mode detection.

The test trials are planed with two types of user groups:

- Controlled user group to focus on specific topics in detail and to realise controlled experiments, and

- Open field experiments with the general public which made use of the STREETLIFE systems via the Web or mobile Apps available at the Google Play Store

In addition user group workshops are planned to perform after the pilot execution addressing common research objectives.

Although the described approach allows a better comparison between the pilots it must be considered that the pilots focus on different objectives, implemented different technologies and will probably differ in the number of test trial participants.



3.2.2. PILOT SPECIFIC APPROACHES

BERLIN

The focuses of the BER-pilot are threefold:

i. Increase the safety for cyclists ii. Reducing the carbon footprint of a trip

iii. Improve the transport performance of the city

Changing people’s mobility behaviour is a very challenging task. STREETLIFE addresses this task by incorporating different kinds of data to provide intermodal routing information in order to plan a trip. Additional user preferences allow the user to change easily between different transportation modes and to gain travel comfort.

A conducted requirement analysis shows that people would use bicycles more often, if cycling becomes safer. In this regard, the Berlin pilot partner faces an innovative approach by defining a routing algorithm which takes into account cycling relevant data such as the availability of the different bicycle lanes, road surfaces, traffic signals and incident hotspots.

As a motivating factor to use a bicycle as a means of green transportation, gamification and incentives have been integrated into the Berlin App.

Together with the above mentioned crowdsourcing component these beneficial functionality will be incorporated into the App.

As an additional motivating element to use the bike, gamification and incentives have been integrated into the App. In order to test the developed and implemented components and systems, experiments were planned and elaborated in detail in order to address the related research questions (see Chapter 4.1).

ROVERETO

The approach that will be used to evaluate the impact of STREETLIFE on user behaviour has not changed since the 1st iteration; the aspects that are going to be assessed are the following:

- Change in mode choice; - Ease of use; - Usefulness; - Compliance; - Benefit of gamification.

These categories will be assessed through questionnaires submitted to users and data from log of the app.

In the second iteration, new points of view related to long-term gamification and carpooling will be considered in the evaluation process. These new elements will enrich the assessment of the categories listed. Indeed, thanks to the long-term gamification experiment, we will be



able to evaluate the variation of these aspects and consequently calculate their dynamic trend through time.

Moreover, a qualitative evaluation of social aspects and interaction among STREETLIFE users will be carried out using results from carpooling and long run gamification experiments.

TAMPERE

The transportation flow management is an interesting approach and future possibility for the Tampere transport authority. By smart real-time configuration of the journey planner they can affect mildly travel pattern of people and therefore lead to reduction of congestion. Key test experiment is to see if people behaviour can be changed on a crowded bus stop which gets the whole location congested. In Tampere the winter affects the modal share and can make the comparison of pilot results challenging. Therefore in the 2nd phase data is collected in multiple measuring points on different time points to get better data regarding impacts of user behaviour. Therefore both relate to the Chapter: 3.3. Approach for assessing the impacts on transport system and 3.4 Approach for assessing the impacts on environment

3.3. APPROACH FOR ASSESSING THE IMPACTS ON TRANSPORT SYSTEM


In order to assess the impact of the STREETLIFE system different research approaches will be used. Due to the preconditions at the pilot sites e.g. the TAPAS simulation environment only exists in BER, using one common research approach will not be applied. Similar to the 1st iteration different approaches are required.

As already implemented in the 1st iteration, a quantitative approach based on a microscopic network simulation based on TAPAS is selected for the BER pilot. TAPAS will be feed with mobility data from the field trial. For TRE and ROV the transport simulation network is not available, so an alternative approach had to be selected. Two approaches are foreseen in order to evaluate the implemented system in terms of impacts on the transport system. First, detailed measures of the user mobility behaviour will be acquired. That can be based on existing log files from the STREETLIFE system or external data sources. In addition to that the method of Expert-Interviews would provide added value in order to assess the impacts of STREETLIFE to the transport system.


BERLIN

Similar to the 1st iteration the Travel-Activity Pattern Simulation (TAPAS), an agent-based microscopic simulation model developed for travel demand estimations, will be used (see D8.2.1 Achieved Impacts). TAPAS is a modular simulation, meaning that special variable



sets such as vehicle fleet, possession of public transport cards etc. can be modified without having to change the overall simulation. The simulation is restricted to a geographical area, e.g. a city or a county and allows a detailed illustration of travel behaviour or reactions on transport measures. TAPAS simulates for each person of this population the daily activities, and maps their activity-related trips in space and time. In comparison to the 1st iteration the simulation will be based on data gathered during the field trials. It would then be possible to compare the finding of the 1st iteration which was based on literature and 2nd iteration.

ROVERETO

The approach for the evaluation of STREETLIFE impact on traffic system in Rovereto is based on the calculation of distance travelled by users in each mode of transport before and during the experiments. In fact, baseline data will be collected in the first phase on the trial process or in case of some experiments have already been acquired in the 1st iteration,

The value identified to measure the reduction of traffic achieved thanks to STREETLIFE solutions is the amount of kilometres not travelled by car thanks to the STREETLIFE trial process.

The difference between the baseline and the experiment period allow us to identify the amount of kilometres not travelled by car thanks to STREETLIFE trial process. This value has been identified to measure the reduction of traffic achieved thanks to STREETLIFE solutions. In effect, all the experiments aim to reduce car traffic by cutting down the number of car trip that share the same origin and destination and increasing public transport use and other, non-motorized modes of transport.

TAMPERE

The approach for the TRE evaluation uses the following data as source for evaluation:

- User mobility data collected during the pilot by the multi-modal journey planner; - Questionnaires submitted to user to gather data about mobility habits have been

submitted during period with different weather conditions. Indeed, a pre-study was carried out before there was snow in Tampere, while the post-study was made in the winter period when both private car usage and public transport usage increase and cycling decreases. The weather plays a big role on modal choice and this makes comparison of pre and post-study pilot data evaluation difficult and unreliable;

- Questionnaires submitted to traffic managers in a qualitative ex-post survey

3.4. APPROACH FOR ASSESSING THE IMPACTS ON ENVIRONMENT


In STREETLIFE the schema of CO2 assessment of ITS (already described in D8.2.1 Achieved Impacts) is used to calculate the carbon emission saved due to routing and travel information systems.



Figure 2 Schema of CO2 assessment of ITS (adopted by AMITRAN) However, for the calculation and estimation of traffic impacts different methods have been used. The calculation of local CO2 emission will then be based on the results gathered while analysing the impact on the transport system.


BERLIN

The approach applied is a simulation-based approach. This approach uses the results of the TAPAS microscopic traffic network simulation in order to quantify the kilometre travelled and the transport mode used. In addition, the CO2 values for the different transport modes will be considered for calculation. A distinction of vehicle emission factors is envisaged for the 2nd iteration as this was not possible during the 1st iteration.

ROVERETO The evaluation approach selected to describe the impact of STREETLIFE in terms of CO2 emissions is based on the estimation of the distance not travelled by car thanks to solutions deployed in the trial process.

In order to quantify the CO2 savings, this distance must be multiply by an emission factor calculated through CORINAIR process, a methodology developed by European Environment Agency (EEA) that provides an emission inventory for every category of vehicles. In detail, CO2 estimation is derived from fuel consumption that, like other pollutant agents, is determined with this algorithm which can be found in deliverable D8.2.1 Achieved Impacts.

TAMPERE

The evaluation is based mainly on interviews with the Tampere mobility managers. A detailed calculation method or simulation-based approach will not be implemented as there no sufficient data available.

.



4. PILOT SCENARIO DEFINITION

The purpose of Chapter 4 is depicted with Figure 3. Within the STREETLIFE pilots scenarios are described and composed of Use Cases which are applied by experiments. Within the pilots’ experiments data will be gathered (here Metrics) which finally will be used to derive WP8 performance indicators. This closes the link between pilots and its experiments side and the elements of the WP8 evaluation on the other. Even though a top-down approach has been applied in WP8 in order to quantify answers to relevant questions from a rather global project’s evaluation perspective, also a bottom-up approach has been complementarily used: Pilot specific problems to be solved and evaluated (e.g. cycling safety for the BER pilot) determine necessary local applications and implementations, which define scenarios, use cases, experiments and, consequently, metrics to be collected on a pilot level. Both approaches need to be combined and matched. Thus, WP8 performance indicators and pilots’ metrics have been developed commonly and will be constantly synced.

Figure 3: Relation between WP6 (pilots) and WP8 This is described in the following pilot chapters. It shows what has been changed between the 1st and the 2nd iteration – and why. Pilots’ scenarios, use cases and experiments will be described; and finally, local measurements will be clearly mapped to global (project levelled) evaluation criteria (research questions, hypotheses, and performance indicators).

4.1. BERLIN

4.1.1. EVOLUTION FROM 1ST TO 2ND ITERATION

In accordance with rather general remarks in Chapter 2, here the final goal/orientation of the 2nd pilot iteration has been described as an introduction for the following sub-chapters.



The 2nd iteration’s focus of the Berlin pilot is being laid on a safe cycling routing, gamification and a crowd-source based assessment of safety aspects of each particular cycling route and, in consequence, of the entire Berlin cycling system as a part of the whole transportation system.

An analysis of existing systems and available data within the 1st iteration depicts several available components that could be built upon. The main actor for providing traffic information on behalf of the City of Berlin is the Berlin Traffic Information Centre (TIC Berlin). Presently, the TIC Berlin offers multi-modal routing services for public transportation, car, bicycle and walking. For this purpose the backend system utilises external routing services from the regional public transport association (VBB), TomTom, Google and Bbbike. These routers are integrated in one router developed by VMZ, to generate multimodal trip planning results. If more than one route is an option for a trip from A to B all route options are presented. The VMZ router obtains available real time traffic data which may have an impact on the trip results. Car routes take into account the current traffic situation (LOS1), as well as construction sites and road closures or similar events. Public transportation routes consider the real time departure times, including delays and further related information.

In order to meet the 2nd iterations objectives, a cycling router will be enhanced taking into account safety and comfort aspects. For that a preliminary data handling is necessary. Accident hot spots, provided by the Berlin Police department, subjectively felt points of danger (based on the survey mentioned above) and comfort as well as safety attributes will be added to the routing variables. Furthermore other attributes regarding cycling safety will be taken into account: i.e. reference surface, quality road surface, traffic control at intersections, bicycle traffic at intersections or right-turn green arrow at intersections. All values will be analysed through a client based accident analysis tool ProVista2

Finally, the data exchange and data storage will be realized by the City Intelligence Platform (CIP). Based on the data gathered evaluation results and event based mobility demand forecasting will be provided to the mobility manager and service operator with the MMECP.

Changing the focus of the Berlin pilot has a direct impact on the respective BER research questions, hypotheses, indicators and methods. This mainly applies to the local level where specific metrics are created to quantify specific impacts. But this also applies to the derivation of main (project-levelled) impact categories to be addressed with local measurements and observations.

As for the BER pilot, with the 2nd iteration field test trials the focus is shifted from general modal change observations to an assessments of cycling safety perceived and stated by App users. A more elaborated description of the BER pilot experiments, scenarios and use cases is given with Chapter 4.1.2. The adjustment of evaluation criteria (research questions,

1 Level of service divided into “no delay”, “moderate delay” and “severe delay”

2 Provista Accident Analysis tool - Result of a Research Project 2009-2012



hypotheses, etc.) is given and justified in Chapter 4.1.6 to Evaluation matrices – from Hypotheses to Indicators.

4.1.2. SECOND ITERATION’S SCENARIOS, USE CASES AND EXPERIMENTS

Table 3: BER pilot - Second Iteration Scenarios & Use Cases

ID Name EXP-1

EXP-2

EXP-3

EXP-4

EXP-5

Scenario BER-PTP Pre-trip planning (PTP)

BER-PTP-1 Use Case: trip planning X X X

BER-PTP-2 Use Case: Carbon footprint (CFP) advisory X X X

BER-PTP-3 Use Case: GPS tracking X X X X

BER-PTP-4 Use Case: Crowdsourcing cycling safety assessment X X X X

BER-PTP-5 Use Case: Create user profile with transport preferences X X

Scenario BER-GI Gaming & Incentive

BER-GI-1 Use Case: Game organisation and performance X X X X

BER-GI-2 Use Case: Game participation X X X X

Scenario BER-BUI Bike Usage and Incentives

BER-BUI-1 Use Case: Safe Bike Routing X X X

BER-BUI-2 Use case: Incentive Management X X X X

Scenario BER-MGMT Management

BER-MGMT-1 Use Case: Derivation and Aggregation of main KPI X X

BER-MGMT-2 Use Case: KPI monitoring at STREETLIFE MMECP and website X X X X

BER-MGMT-3 Use Case: event based service adjustment and feedback X X

BER-MGMT-4 Use Case: modal split forecast for an event and transport service adaptation X X

Scenario ID BER-BES Back-end Services

BER-BES-1 Use Case: Scenario simulation X

BER-BES-2 Use Case: Results presentation, feedback and delivery X



Table 3 shows BER scenarios and use cases and its assignment to experiments planned with the 2nd iteration. The following elaborately describes BER scenarios and associated use cases in a structured manner. This structure is taken from a similar WP6 planning document. The description clearly specifies the state of each element – whether it is new or revised. At the end of this chapter those scenarios resp. use cases are listed which have been de-activated/skipped.

Described scenarios and use case take into account three different user perspectives: - End user - Mobility Manager - Event Manager resp. Mobility Service Operator

4.1.3. SCENARIOS AND USE CASES

Scenario ID: BER-PTP Pre-trip planning (PTP) State: revised Narrative: When pre-planning the day, its activities, necessities and appointments at home Silke is using an STREETLIFE Berlin App. In this respect, the STREETLIFE App is to be considered as a multi-modal traveller advisor, combining different mode options within one trip from a given point A to a given point B.

At this particular day Silke has to go to work and has an appointment at the evening in Berlin-Charlottenburg (close to downtown West). Having access to Silke’s STREETLIFE user profile, the App is proposing a list of possible routing options. The App is taking into account actual and forecasted weather, the actual traffic situation public transport schedules, and public transport tickets. Costs and the carbon footprint is calculated and shown for all route options by the STREETLIFE App. For the trip from home to work, Silke decides to take a combination of cycling and public transport (PT). She is participating in a game collecting green credits which can be earned by taking green modes of transportation, such as cycling and PT. Just before starting the morning trip, Silke is choosing in the App the proposed connection (Cycling S-Bahn Cycling). In terms of cycling she chooses safe connections and confirms GPS tracking (the “companion” mode) for the first cycling part of this trip.

When arriving at her destination, Silke is asked to assess the perceived safety of the cycling parts of her trip. When arriving at the train station the “companion” mode is automatically deactivated. Via short vibration and a visual notification the app informs her about the arrival and asks her for assessing the perceived safety of the cycling part of the trip, as she has edited her user profile with respect to a special personal attention to cycling safety. A brief survey screen appears at the App with which she is assessing the safety of this ride as “very high”. Finally, she is quitting the survey by pressing the “send & return” button and turns off the App



Shortly after this, Silke receives feedback from the Bikerider Game.

Stakeholders/actors list: • End user/traveller/commuter

Use Case: Trip planning

Use Case ID: BER-PTP-1 State: revised Use Case Description:

Supports end user planning and evaluating a trip with regard to defined preferences (e.g. safety, comfort, and time) by using a mobile device connected to the internet applied by an individual end user. Primary Actor: end user/traveller Other Actors: none Preconditions:

- Access to multi-modal routing planner available - STREETLIFE user registration with preferences - STREETLIFE App installed - End user location is available

Trigger: end user applies BER STREETLIFE App for trip planning

Basic flow:

1. user starts trip-planning App 2. sets origin and destination coordinates 3. selects/edit one or several user preferences 4. route options will be calculated by the STREETLIFE system and displayed at the App 5. user selects favourite option/route 6. proposed (by system) and viewed (by user) route options as well as selected route will

be stored at CIP

Use Case: Carbon footprint (CFP) advisory


It calculates the carbon footprint for proposed route options for trip request in question. Proposed route options at display contain calculated carbon emission and, thus, help the user to take environmental impact into account for mobility planning.



Primary Actor: end user/traveller Other Actors: none Preconditions:

- STREETLIFE user registration with preferences - STREETLIFE App installed or internet access to STREETLIFE website

Trigger: end user applies BER STREETLIFE App for trip planning (CFP is automatically calculated for integrated modes)

Basic flow:

1. User requests trip itinerary for available mode options (using the trip-planner) 2. CFP returns carbon emission for available route options

Use Case: GPS tracking

Use Case ID: BER-PTP-3 State: revised (former Itinerary tracking and adjustment) Use Case Description:

GPS coordinates of the user’s mobile device is constantly tracked and locally stored. After finishing the trip a tracking data set is sent to and stored at the CIP. There, the km driven by bike are identified and Green Leaves are calculated for the Bikerider Game. The user gets feedback about his status in the game. Primary Actor: end user/traveller Other Actors: none Preconditions:

- a pre-planned trip is designed - a proposed route option is selected and followed - GPS tracking possible - trip/user confirmed to be tracked - GPS tracking periodicity is defined (currently 10 seconds)

Trigger: a pre-planned route option is selected and the companion mode is explicitly selected by the user

Basic flow:

1. a proposed trip option is selected 2. “tracking” is activated and confirmed; the companion mode is activated by the user 3. GPS coordinates are stored locally at the device 4. After finishing the trip a set of trip related GPS coordinates is sent to and stored at the

CIP



Use Case: Crowdsourcing cycling safety assessment

Use Case ID: BER-PTP-4 State: new Use Case Description:

STREETLIFE user assessing service quality and act as “crowdsourcer”. User gives feedback on the cycling route quality. Context awareness is used to ask “the right question at the right time”. In this use case the event “end of trip” is used as a trigger for route safety assessment.

As safety is a focus in Berlin, the traveller can also provide information about unsafe locations by pointing at the locations on the map.

Primary Actor: end user/traveller Other Actors: none Preconditions:

- Selected route: mode/s of transport is/are known and assessable - quantitative (e.g. level of service) and qualitative (e.g. survey methods and

questionnaire items/feedback categories) assessment criteria designed/decided - crowd sourcing / feedback infrastructure - reliability testing procedures defined - uses data from BER-BUI-3 (model used for generating routes)

Trigger: end user finishes the bike ride and receives a feedback screen

Basic flow:

1. User uses BER App 2. User selects a route that includes cycling 3. Feedback screen is presented at the end of the trip referencing to the cycling stretch 4. Data set (GPS track [if given], route assessment, critical points coordinates) is created

and sent to a backend server (CIP) 5. Optional: User uses STREETLIFE App to create new critical points at a Map showing

the performed route after trip OR real-time hotspot annotation by smart watch/phone while riding, comments to be selected from list

Use Case: Create user profile with transport preferences


The App stores user preferences. The idea behind this is to speed up the process of allocating the most suitable route and mode preferences.



In order to participate at the Bikerider game, the user must create a user name and save it. This user name is used to allocate his Green Leaves, trees and to show the high score. Primary Actor: end user/traveller Other Actors: none Preconditions: Connection to STREETLIFE system is available (or App is installed on smartphone) Trigger: end user applies BER STREETLIFE App

Basic flow:

1. end user enters credentials to access the STREETLIFE system 2. end user creates profile based on his/her preferences 3. end user saves profile

Scenario ID: BER-GI Gaming & Incentive State: revised (former: Car Pooling and Incentives) Narrative: After finishing work later this day, Silke is using the Berlin STREETLIFE App again for planning the trip from work to the city centre. Before asking the systems for route proposals Silke is having a look at her online user profile and green leaves account at the BER App. She notices that she is making good progress in the on-going competition and, thus, decides to go for another cycling trip from work to the city centre – collecting a significant amount of green credits. Amongst other multi-modal alternatives, the App is proposing a mono-modal cycling trip. Silke feels very comfortable with this proposal – knowing that the App and the underlying router are paying special attention to Berlin cycling accident hot spots.

When starting the ride she confirms one of the suggestion and by this starts the GPS tracking.

Reaching this trip’s destination Silke finds the safety assessment screen at the App again. As unexpectedly she did not feel safe at the entire trip, she is assessing the overall safety of this trip as “medium” and uses the built-in map functionality of the crowdsourcing component to specify a couple of crossings and road segments of this trip as dangerous and not recommendable. For providing important information for evaluating cycling safety she will be rewarded with further credits. Later she is using the App to checks her account again and sees that she improved her STREETLIFE gaming competition ranking.

As she travelled 6km by bike on her last trip, she earned 60 Green Leaves plus 2 Green Leaves for answering the questions. By this she reached 1024 Green Leaves in total for this game period and is asked to set a virtual tree on the Bikerider Map of Berlin. She observes that already more than 30 trees have been set on the Map and feels that the Bikerider community is growing.

As Silke mostly uses her bike and bike sharing to travel within the city she is one of the top 10 of the Bikerider community and can find her name on the corresponding list of the game. This motivates her to continue and travel by bike even more often.



Stakeholders/actors list: • End user/traveller

Use Case: Game organisation and performance

Use Case ID: BER-GI-1 State: new Use Case Description:

This use cases defines and applies the framework for the BER GAME Bikerider. It specifies the gaming regulations and rules. Credits/rewards can be reasonably calculated, assigned to users’ activities and selections, stored in user profiles and displayed for both the individual user and the competitors in rankings and high-score lists. The game will be separated into two or three campaigns. The personal scores will be shown with the App, whereas anonymized information on the overall ranking could be also aggregated and uploaded to the MMECP and/or the STREETLIFE web site. Primary Actor: Mobility manager Other Actors: end user Preconditions:

- Game rules are set up and communicated via App - Gaming rewards are defined and applied - CIP is able to store user profiles and credit accounts - High-scores and rankings can be calculated, compared and displayed

Trigger: Siemens is starting a gaming campaign

Basic flow: For each individual gaming campaign the following actions need to be applied:

1. Store in an archive last campaign’s participants, winners, scores and rankings 2. Reset all users’ credit accounts to empty 3. Inform user groups on next gaming campaign set-up and timing 4. Officially start the game campaign 5. Individuals scores and a comparative ranking is constantly adjusted and updated with

the App (calculation of green leaves, virtual trees and high score; administration of virtual trees)

Alternative flows: Some derivations of the data (anonymized high-score & rankings) will also be integrated into the MMECP and the STREETLIFE website.

Use Case: Game participation



Use Case ID: BER-GI-2 State: new Use Case Description:

This use cases defines and applies the framework for the BER GAME Bikerider. It specifies the gaming regulations and rules. Credits/rewards can be reasonably calculated, assigned to users’ activities and selections, stored in user profiles and displayed for both the individual user and the competitors in rankings and high-score lists. The game will be separated into two (or maybe three) campaigns. Primary Actor: end user/traveller Other Actors: None Preconditions:

- Game rules are communicated by the project, known and confirmed by user - Gaming rewards are defined and applied - CIP is able to store user profiles and credit accounts - High-scores and rankings can be calculated, compared and displayed

Trigger: user confirms game participation via the App

Basic flow:

1. Participants are using the App for daily mobility planning and execution 2. Specific actions or route selections will be automatically rewarded by green credits

and transferred to users’ accounts

Scenario ID: BER-BUI Bike Usage and Incentives State: revised Narrative: As a modern Berliner, Silke is interested in Berlin city mobility developments and modern mobility trends. Thus, she knows about respective public surveys performed during the last years which investigated accident hot spots and dangerous segments of the Berlin road network. She read a STREETLIFE project announcement some months ago and, thus, decided to become a member of the STREETLIFE user group: In the project leaflet she read that mentioned cycling safety information is taking into account for STREETLIFE mobility planning and App routing.

In her STREETLIFE user profile she laid a special focus on greener modes of transport in combination with the highest possible safety and convenience.

Corresponding data is integrated into the STREETLIFE routing via the underlying routing service of the TIC (Traffic Information Centre Berlin).

Stakeholders/actors list: - End user/traveller



Use Case: Safe Bike Routing

Use Case ID: BER-BUI-1 State: revised Use Case Description:

The bicycle-part of the multi-modal router avoids locations that are dangerous for bicyclists with only small delays accepted. With respect to Silke’s user preferences, proposed route options try to avoid main roads and heavily littered road sections, bigger intersections and accident hotspots for cyclists.

Silke activates the companion mode in the STREETLIFE App, so that the App can give warnings if a dangerous location on the path is coming up. Primary Actor: end user/traveller Other Actors: None Preconditions:

- Bike route chosen - Bike warnings and hotspots given and integrated into routing - Bicycle safety routing activated

Trigger: User selects a route including cycling and has activated respective cycling safety preferences

Basic flow:

1. user activates / configures to use safe routing for cycling 2. user selects routing proposal that contains cycling 3. Positioning is activated 4. STREETLIFE App avoids accident hotspots, unsafe road conditions, unfavourable

lane layouts (no bike lane) etc.

Use Case: Incentive Management

Use Case ID: BER-BUI-2 State: revised Use Case Description:

The management function on the App allows the user to recall gained credits and to take part in the STREETLIFE BER game. The game includes the calculation of Green Leaves, Trees and high score. Green Leaves provide an early feedback at the end of each trip.



Having reached 1,000 Green Leaves, the participant is asked to set a virtual tree on the game map. This element shall enable to observe that a community is established and several travellers are participating at the game. The high score shall encourage to compete with other participants and to use the bike in order to gain more Green Leaves. . Primary Actor: end user/traveller Other Actors: None Preconditions:

- Communication channel and protocols defined and established - Project external functionality can be provided for the project

Trigger: User selects a route including cycling and has activated respective cycling safety preferences

Basic flow:

1. User performed an action to be rewarded by STREETLIFE credits 2. User uses the App to check accounts and gaming ranking

Alternative flows:

1. User could access the overall gaming ranking also via the MMECP user front-end – anonymized high-scores and rankings are derived from user trip data and shown with the MMECP/web site



Scenario ID: BER-MGMT Management State: revised Narrative: Philipp is working at the Berlin Traffic Information Centre as a Traffic Information Editor (TIE) and is responsible for monitoring and evaluating the current traffic situation in Berlin. His main job is to recommend necessary measures and actions on significant changes in transport demand and on severe disturbances of the overall Berlin transport system situation. He is used to work with and to rapidly interpret main KPI, automatically derived from huge amounts of raw data, which help him to “feel the pulse” of the Berlin transport system. The more easy-to-understand information he gets, the better he is able to fine-tune the overall BER transportation system and propose sustainable actions and measures. In his daily business, he deals with a huge amount of information. Thus, he does not need an additional monitor with plain transport figures and data. He needs reasonably aggregated information to be integrated seamlessly and intuitively into his daily operations and routines.

Together with the BER STREETLIFE team, KPIs most relevant for Philipp’s daily operations have been identified; and the MMECP at Philipp’s desk is tailored for this purpose.

Besides that, the MMECP also allows BER STREETLIFE users, such as Silke, to have read access to main KPI as well as to high-scores and rankings of the BER game.

The MMECP also shows the amount of greenhouse gases that could be saved by “playing the STREETLIFE game” in Berlin. The most promising STREETLIFE achievements will also be displayed at the BER section of the STREETLIFE website.

Katrin is an event manager for a very popular events location, the Tempelhofer Field. She knows what kinds of events take place, how big they are, and when they take place. She also wants to make travelling to and from the event as smooth as possible for the citizens and optimize the impact on the environment at the same time. Katrin can also read important KPI on the MMECP. In addition, she can receive from the MMECP analysis and forecast data that give her an estimate on the expected modal split for a given type of event and, knowing the expected overall number of event attendees, the expected number of citizens per transport system. Using this knowledge and her contacts to transportation service providers in Berlin, she can ask prior to the event to have e.g. more shared bikes available in the area or even to have more buses – i.e. a tighter bus schedule – set up around the event area for a given day.

Stakeholders/actors list: - Mobility Manager (Philipp) - Transport (information) service operator (e.g. bike sharing or bus operator) - Event Manager (Katrin)



Use Case: Derivation and Aggregation of main KPI

Use Case ID: BER-MGMT-1 State: revised Use Case Description:

Together with the BER STREETLIFE team, the most relevant KPIs for the operators’ daily operations will be identified. The respective data is identified, analysed, and aggregated in order to be displayed via the MMECP-UI.

Primary Actor: TIE (Philipp), transport service operator Other Actors: None Preconditions:

- Visualization engine for data is available - KPIs are defined

Trigger: output is automatically updated

Basic flow: 1. Actor selects the map view and/or a set of KPIs 2. Aggregation (online or offline) of data takes place 3. Analysis of data regarding the selected KPIs is performed 4. MMECP UI visualizes the requested information

Use Case: KPI monitoring at STREETLIFE MMECP and website


This use case allows transport operators to keep track of the pre-defined KPIs derived from STREETLIFE analyses and relevant for the operation of the BER transportation system. Major project achievements for selected KPIs will also be derived (collected, filtered), for website publication. Primary Actor: TIE, transport service operator Other Actors: end user Preconditions: use case BER-MGMT-1 Trigger: automatically formatted and displayed and reasonably updated



Basic flow: 1. Graphical visualization of public KPIs, including history and target achievement on

web service 2. Update KPI calculation and visualization continuously 3. Use guest book for user recommendations or discussion forum 4. Analyse feedback for system improvement

Alternative flows: Specific update periods for MMECP operator and user views and website feeds

Use Case: Service adjustment and feedback for bike sharing


A Transport Service Operator for bike sharing is supported in adjusting its services according to daily demands or special events and correspondingly growing demand. He will be provided with feedback to evaluate the quality/correctness of any proposed advice/measures that he got from the STREETLIFE system and that told him either to proactively shift bikes among stations or even to resize bike stations in the medium-term.

Philipp, the TIE, is made aware of changing demands and shortages for bike sharing at specific sharing stations. For this purpose, the state of bike sharing stations in pre-defined vicinity is constantly controlled. After reaching pre-defined thresholds, the respective bike sharing service operators will be informed about a shortage/special demand of sharing bikes (as a consequence of slowly increased demand in that location, or possibly due to an event in question).

Primary Actor: TIE (Philipp), transport service operator Other Actors: end user Preconditions:

- Transport service operator for bikes has pre-selected a specific station or an area of interest.

- It exists a given feedback approach including a KPI definition and relevant thresholds. - Definition of event characteristics and categories (if event is planned) exists.

Trigger: A new local event is published to the STREETLIFE-system – OR – the transport service operator triggers the occupation analysis, without an event.

Basic flow:

1. The TSO opens the UI and selects an area of interest that has been specified by the transport service operator

2. The STREETLIFE-system transforms the occupancy information for the querying transport service operator in a customized presentation format.



3. The TSO observes shortages of available bikes at one or more stations. 4. Mobility system sends guidance information including reasoning (“why”) to the

transport service operator. 5. The transport service operator plans on moving some bikes or (in the medium-term) to

resize a station or stations In case of the latter: the transport service operator informs the STREETLIFE system about changed station capacities.

Use Case: modal split forecast for an event and transport service adaptation

Use Case ID: BER-MGMT-4 State: initial Use Case Description:

An event manager together with the Transport Service Operator can use the forecast mechanisms built into the MMECP system to estimate the required transport capacities per means of transport (bus, bike, walking, car …) for a large upcoming public event. Together, they can then decide for which modes of transportation the projected needs are high enough to mandate a higher service schedule (e.g. more buses, bring extra bikes to places near the event …). Ideally, some means to receive feedback after the event allows the evaluation of the quality/correctness of the proposed measures. Primary Actor: TIE, event manager, transport service operator(s) Other Actors: end users (in the role of event attendees) Preconditions:

- Event manager (Katrin) knows about upcoming event, cooperates with TSO - Event characteristics (type, size…) and expected attendees (rough estimates per

category – age, gender…) are known - The Transport service operator (TSO) derives from this a specific set of event

information, suitable as input for forecasting algorithms of the MMECP - Forecasting algorithm has enough suitable previous data on events similar to the

upcoming one and was run (offline) with multiple input data sets

Trigger: A new public event in Berlin is published to the STREETLIFE-system (initiated by event manager Katrin)

Basic flow:

1. The STREETLIFE TIE takes the event data, checks it and transfers it into input suitable for forecasting (where not yet automated)

2. The modal split forecasting is performed by STREETLIFE and expected usage demands are computed (these should indicate confidence regions to allow the fine tuning of any – possibly used – service overprovisioning); one variation is to perform any modal split forecasts offline, i.e. prior to any specific event (to be checked if this



is feasible: runtime, number of input combinations…). In that case, forecasting would be triggered by a scheduler.

3. From the knowledge about bike station occupancy and the computed forecast, MMECP computes an expected demand for the bike stations and an expected final occupancy for “before” and “after” the public event. The MMECP UI must be able to display this predicted occupancy (which could be <0) on the map. Maybe input parameters could be changed interactively, too.

4. Finally, the STREETLIFE system transforms the event-related, information about expected demand for each registered transport service in a customized presentation format.

5. The STREETLIFE system sends notifications/requests to all registered transport service operators, i.e. the transport service operators get notified about the upcoming event and transport needs.

6. The transport service operators respond to the notification; each one may change its service availability in the environment for the given event time.

7. Each transport operator needs to signal back any planned service adaptation; The STREETLIFE system also sends these changes (as a summary) back to the event manager.

Scenario ID: BER-BES Back-end Services State: revised Narrative: Philipp and his colleagues often discuss about “what-if-settings”. What would happen with the Berlin traffic situation, if for instance a city road fee would be implemented in Berlin or if the gasoline prices would increase significantly; what if an increased safety for cyclists or more frequent PT services would convince more car drivers to change to greener modes? What could be achieved in terms of greenhouse gas emissions when radical and innovative transport policies and plans would come into action.

Philipp would really like to know about impacts of possible measures and policies. As this cannot be tested in real-life conditions and such simulation cannot be set up and run in parallel with the operation’s daily businesses, TIE would like to have assistance in simulating such combinations of measures (so called scenarios).

Respective scenarios will be defined and discussed together with the local Transport System Operators. Those scenarios pay special attention to the expectations, needs and wishes of the Transport System Operation as well as to the focus of the STREETLIFE project. This means that a clear added value will be derived from the main goals and objectives of the STREETLIFE project. Jointly defined scenarios will then be simulated, analysed and translated into pre-defined KPI relevant for the Transport Operations. Results (simulation outcomes, KPI, trends and forecasts) will be presented by the Berlin STREETLIFE partners and discussed with the Berlin Transport Operators.



Stakeholders/actors list: - TIEs - Transport service operator (e.g. bike sharing operator)

Use Case: Scenario simulation

Use Case ID: BER-BES-1 State: revised Use Case Description:

Each simulation was achieved by modifying data regarding a specific model/policy decision after inspecting the current mobility situation. Micro simulations are used for simulating specific spots in a “short” time range and macro simulations are used for simulate bigger areas for “long” time. Primary Actor: TIE, transport service operator Other Actors: None Preconditions:

- Berlin KPIs defined - Definition of simulation tools and respective requirements - Input data (e.g. TAPAS) for simulation available

Trigger: In coordination with the Transport System Operation

Basic flow: 1. Select spot or area 2. Select time range 3. Apply respective models/policies 4. Modify data 5. Run simulation

Use Case: Results presentation, feedback and delivery

Use Case ID: BER-BES-2 State: revised Use Case Description:

The results are presented on a map showing how the defined models/policies act on the modified data. Depending on how long the defined time range was, the results were shown in real time or in a fast motion way. Finally a result page with before and after KPIs value added calculation is created. Primary Actor: TIE, transport service operator



Other Actors: None Preconditions:

- Simulation model/policy is applicable - Data is available and modified

Trigger: In coordination with the Transport System Operation

Basic flow:

4.1.4. EXPERIMENTS

In general, the experiment setting of the 2nd iteration is similar to the 1st iteration’s setting. Friendly user feedback on the App as such is collected; but – in addition – also the crowd-sourcing component of the App, the overall BER game scheme and the MMECP visualisation of collected and aggregated data is being tested with a small group of users. The collected feedback will be discussed with the user and the App developers to be finally integrated – as far as possible – into the version of the App which will be applied for the BER field test concentrating at the BER cycling safety assessment in spring 2016.

Experiments descriptions can be found in the following.

Experiment: App Usability and Performance Test

Experiment ID: BER-EXP-1

Goal: After having implemented BER use cases related requirements into the BER App the test BER-EXP-1 is paying particular attention to the App performance, usability, handiness, respective data flows, and back-end communications.

Description:

A group of friendly users collected from BER partner networks installs, uses and feedbacks on this particular part, namely the BER App and its usability.

Scenarios and use cases:

• BER-PTP completely • BER-GI completely • BER-BUI completely

Experiment characteristics

Type beta-testing, focus group Instruments BER App, questionnaires, interviews Stakeholder types: App developer



Size: 15 – 20 users testing the App Duration: 2 – 3 weeks Beginning date: October 2015 End date: October 2015

Experiment engagement plan

Users will be recruited within BER partner companies and respective business and private networks. Every BER-partner proposes and briefs 5-6 users for the tests. Names and email addresses will be safely stored by DLR.

Orientation to WP8 research questions, hypotheses and performance indicators

The main intention of this experiment is to evaluate the BER App from a functioning, usability and performance perspective. Together with experiments BER-EXP-2, BER-EXP-3 and BER-EXP-4, it is to test the interaction between the different components, in particular between the App and CIP in real-life conditions in order to find functioning and usability issues and bugs. Users are requested to report on experienced issues and usability problems constantly in an informal manner. Reported bugs and issues will be collected, analysed and partly discussed in a user group workshop after the testing period. This ensures an improvement of the BER App and its components prior the field test (BER-EXP-5).

As a consequence, those four (beta-testing) experiments will not explicitly address one or several WP8 evaluation performance indicators. Indirectly, as this beta-testing will have an important impact on the App and service quality, it is expected to support the research question “How does trip information change people’s mobility behaviour?” and its HY and PI.

Denominators (metrics) of this experiment will be (amongst others): - Satisfaction with App performance, system response times and perceived quality of

information

- Usability of App with respect to applicability and ease-of-use

- Usability of description of processes and routines, integration of components

- Usage of smartphone resources (processor, battery, sensors)

- Compatibility with crucial phone functions and usage patterns

Experiment: Crowdsourcing Test


Goal: After having implemented use case BER-PTP-4 on crowdsourcing into the BER App the test BER-EXP-2 is paying particular attention to the crowdsourcing component (screen



and dialogue) at the App, the storage of collected data at the CIP and a validation of both raw data and aggregated information.

Description:

A group of friendly users collected from BER partner networks uses and feedbacks on this particular part, namely the BER App crowdsourcing component.


• BER-PTP o BER-PTP-3 o BER-PTP-4

• BER-GI o BER-GI-1 o BER-GI-2

• BER-BUI o BER-BUI-1 o BER-BUI-2

• BER-MGMT o BER-MGMT-2


Type beta-testing, focus group Instruments BER App, questionnaires, interviews Stakeholder types: App and WP3 developer Size: 15 – 20 users testing the App Duration: 2 – 3 weeks Beginning date: October 2015 End date: October 2015


Users will be recruited within BER partner companies and respective business and private networks. Every BER-partner proposes and 5-6 users for the friendly user tests. Names and email addresses will be safely stored by DLR.


Experiment: BER Gaming Test


Goal: After having implemented use case BER-CPI-2 on gaming and incentives into the BER App and back-end services the test BER-EXP-3 is paying particular attention to the gaming



aspect, its calculation methods, technical integration into back-end services and user profiles, and the acceptance by the users.

Description:

A group of friendly users collected from BER partner networks uses and feedbacks on this particular part, namely the BER App gaming component.


• BER-PTP o BER-PTP-1 o BER-PTP-2 o BER-PTP-3 o BER-PTP-4

• BER-GI o BER-GI-1 o BER-GI-2

• BER-BUI o BER-BUI-2

• BER-MGMT o BER-MGMT-2


Type beta-testing, focus group Instruments BER App, CIP, questionnaires, interviews Stakeholder types: App and WP2 developer Size: 15 – 20 users testing the App Duration: 2 – 3 weeks Beginning date: October 2015 End date: October 2015

Experiment Engagement plan





Experiment: MMECP visualization test


Goal: After having implemented the MMECP as a visualizing front-end for both operators and users the test BER-EXP-3 is paying particular attention to the visualization of information aggregated WP4 simulations, from the crowdsourcing App component, and further sources.

Description:

Transport experts from VIZ and DLR and selected test users collected from BER partner networks feedbacks on the MMECP and the integrated information.


• BER-MGMT o BER-MGMT-1 o BER-MGMT-2 o BER-MGMT-3


Type beta-testing, experts interviews Instruments MMECP, CIP, interviews Stakeholder types: WP4 developer Size: 15 – 20 users testing the App Duration: 2 – 3 weeks Beginning date: October 2015 End date: October 2015






Experiment: BER field test on cycling safety


Goal: After having performed tests BER-EXP1 - -EXP4 and taking into account collected and analysed users’ and operators’ feedback for final App and service deployment the overall BER service environment will be tested under real-life conditions with real users.

Description:

After an 8 -10 week field test phase users will be approached with a final acceptance questionnaire and pertly invited to focus group sessions or interviews.


• All scenarios and use cases active


Type open field, focus groups, panel evaluation, etc. Instruments App & services, MMECP, CIP, BER Game, questionnaire, user

traces, system traces, etc. Stakeholder types: end users, transport experts & operators Size: 70 – 100 end users, 5 – 10 transport experts & operators Duration: 3 months Beginning date: March 2016 End date: May 2016


• Main user group will be organised in cooperation with a third party company which are experts in the field of user acquisition and panel selection

• User recruitment activities during the open-door event of the Berlin research institutions at June 13th


The intention of this main BER experiment – bringing together all respective elements and components – is to consider and improve perceived cycling safety. Here, objective (measures) and subjective (observations and surveys) metrics clearly address evaluation performance indicators and research questions – as listed in Table 4.



Table 4: BER-EXP-5 metrics for PI

Exp. Metric Addressed WP8 Evaluation Categories

Impact of provided information on mobility behaviour

User Behaviour

Impact of gamification on mode choices and mobility behaviour

Perceived cycling safety as a driver for non-motorized mode choices

Number of collected green leaves

Trip purposes of planned and performed trips

Event based transport demand

Transport System

Simulated forecasts for events and scenarios

Improved cycling safety statistics

Enhanced cycling routing routines taking into account crowd-sourced accident hotspots

Number of entries (dangerous hotspots) confirmed

Impact of situational variables

Number of new entries added

Modal splits and reported changes Environment - CO2 emission reduction

4.1.5. SKIPPED SCENARIOS AND USE CASES

Due to the adjusted focus of the BER pilot following elements from the 1st iteration have been skipped or replaced by 2nd iteration’s elements.

• Use case BER-PTP-3 “GPS tracking” replaces use case “itinerary tracking and adjustment. It turned out that applying an online itinerary tracking and an adjustment (re-routing) in case of disturbances and delays requires mayor effort and is not in line with the resources available.

• Scenario BER-GI “Gaming & Incentives” replaces scenario “Car-pooling & Incentives”. With the given cycling focus car sharing and pooling scenario and use



cases could not be implemented with the available resources and data. Respective BER-GI use cases have been defined and described as new use case elements. In accordance, the following Car-pooling use cases have been removed:

o Car/Bike sharing booking o Trip communication and Sharing via Social Networks o Trip/Route feedback and tracking

4.1.6. OBJECTIVES

The main objectives of the BER pilot have not been changed - it still addresses the following topics:

• With STREETLIFE the carbon footprint of individuals and the carbon emissions within a city caused by land-based transport shall be decreased.

• With STREETLIFE the travel behaviour of citizens shall be sustainably changed into a more eco-friendly behaviour.

• With STREETLIFE an increasing safety for cyclists shall be realized.

• With STREETLIFE the transport performance of a city will not be decreased.

4.1.7. RESEARCH QUESTIONS

Table 5 shows the 1st iteration research question taken from D8.1.1.

Table 5: BER pilot - 1st iteration Research Questions

Identifier Research questions

RQ-BER-1 Is there a significant change in the mode choice?

RQ-BER-2 Which mode benefits most of the change?

RQ-BER-3 Which type of commuters is most willing to change their mobility habits?

RQ-BER-4 What are the affected trips purposes?

RQ-BER-5 Why people change their mobility behaviour?

RQ-BER-6 How do situational variables (e.g. weather, day time etc.) have an effect regarding the mode change?

RQ-BER-7 How STREETLIFE improves cyclist safety?

With the 2nd iteration’s cycling safety focus the list of BER research questions has been changed as follows in Table 6.



Table 6: BER pilot – 2nd iteration Research Questions

Identifier Research questions

RQ-BER-1 Does STREETLIFE improve cycling safety?

RQ-BER-2 Is there a significant change in the mode choice?

RQ-BER-3 How can gamification approaches support sustainable mobility behaviour?


RQ-BER-5 How does trip information change people’s mobility behaviour?

RQ-BER-6 How situational variables (e.g. weather, etc.) do impact on mode change?

RQ-BER-7 How can public statistics on cycling safety be evaluated and improved?

In correspondence with the updated 2nd iteration BER pilot setup and foreseen experiments, research questions address mainly cycling safety topics. When planning daily trips, users will be provided with information about the cost but also about the environmental impact of available choices. The perceived safety of selected cycling trips and the ratio between user’s attitudes and mode choices are to be evaluated; results (e.g. cycling safety assessment and personnel dangerous hotspots) will be set into relation to given statistics.

4.1.8. EVALUATION MATRICES – FROM HYPOTHESES TO INDICATORS

This chapter compiles relevant information as basis for the impact assessment. For each research question defined in Chapter 4.1.7, the following table list corresponding hypotheses, performance indicators and success criteria.

Table 7: Evaluation Matrices BER

RQ-BER-1 Does STREETLIFE improve cycling safety?

HY-BER-101 STREETLIFE improves cycling safety subjectively perceived by field test users.

PI-101 Perceived cycling safety

Type Subjective, self-reported

Method Questionnaire, interviews, focus groups

Metric Reasonable scale



RQ-RER-2 Is there a significant change in the mode choice?

HY-BER-201 Modal split changes towards “greener” modes of transportation during the test period.

PI-201 Actual shares of available modes

Type Direct, analysis of trip choice data

Method Derivation from trip choice data at CIP

Metric Number of trips and incorporated modes

HY-BER-202 Users more often opt for “greener” modes

PI-202 Percentage of “green” mode choices over time

Type Direct, from CIP data

Method Derivation from trip choice data at CIP

Metric % of total amount of performed trips

RQ-BER-3 How can gamification approaches support sustainable mobility behaviour?

HY-BER-301 Gamification significantly supports sustainable mobility behaviour.

PI-301 Number of collected green leaves

Type Direct, from CIP statistics

Method Derivation from CIP data

Metric # of collected green leaves

PI-302 Impact on gamification on mobility decisions

Type Subjective, from questionnaires and user survey






HY-BER-401 Daily trips (work and school) are mainly affected by changed routines.

PI-401 Trip purposes of planned and performed trips




RQ-BER-5 How does trip information change people’s mobility behaviour?

HY-BER-501 Better information on available mode choices and its impacts significantly change peoples’ mobility behaviour.

PI-501 Impact of provided information on mobility behaviour




RQ-BER-6 How situational variables (e.g. weather, etc.) do impact on mode change?

HY-BER-601 Situational variables do have an impact on mode choices.

PI-601 Impact of situational variables

Type Derived, combining several sources from CIP

Method Derivation and analysis of collected CIP data and external resources (e.g. weather, etc.)

Metric Ratio between performance and external influences



BER-RQ-7 How can public statistics on cycling safety be evaluated and improved?

HY-BER-701 Data and information crowdsourced from field test helps to improve and to evaluate public statistics.

PI-701 PI-702

Number of entries (dangerous hotspots) confirmed Number of new entries added

Type Direct, from crowdsourcing MMECP and CIP

Method Derivation and analysis of collected CIP data

Metric # of respective elements



4.2. ROVERETO


Based on the experience and lesson learned in the 1st iteration (see Chapter 2.3) the second iteration of the ROV pilot was organized in a set of scenarios, use cases and experiments, which are described in the remainder of this Chapter. Table 8 provides an overview of them.

In particular, the differences between the scenarios pursued in the 1st versus the 2nd iteration are the following:

• Park and Ride and Bike Sharing scenarios have upgraded to a unique scenario, called Multimodal Mobility. In this scenario, the STREETLIFE journey planner suggests user’s different trip options that include all green modes of transport. More than P+R and bike sharing trips, users are able to plan their trip taking into account public transport (bus and train), private bike and combination of several modes. The decision to upgrade the scenarios of previous year to the multimodal mobility scenario has been taken examining the results of first year that showed the tendency of citizens to use not a unique mode but several means of transport alternative to car.

• In respect of Y1, a Car Pooling scenario has been introduced. In this scenario people that have in common origin and destination of their daily trip are pushed to share their cars and set up car-pooling groups in order to reduce the number of car travelling in the city. In this scenario local enterprises must be involved so as to promote car-pooling among people working in the same or nearby companies. The time and the effort required to involve local enterprises cause the shift of this scenario to the second iteration.

4.2.2. 2ND ITERATION’S SCENARIOS, USE CASES AND EXPERIMENTS

Table 8: ROV pilot - Second Iteration Scenarios, Use Cases and experiments ID Name EXP-1 EXP-2 EXP-3 EXP-4

Scenario ROV-MM Multimodal Mobility ROV-MM1: Bike sharing X

ROV-MM2: Trip planning X X

ROV-MM3: analysis of current situation of parking system on MMECP

ROV-MM4: MMECP simulation – new parking lot

ROV-MM5: Green Game X

ROV-MM6: crowdsourcing X

ROV-MM7: special events routing X



Scenario ROV-CP Car Pooling ROV-CP1: find a car pooling ride X X

ROV-CP2: offer a car pooling ride X X

ROV-CP3: take a CP ride as passenger X X

ROV-CP4: take a CP ride as driver X X

ROV-CP5: CP personal gamification X X

ROV-CP6: CP collective gamification X

4.2.3. SCENARIOS AND USE CASES

Scenario: Multimodal Mobility

Scenario ID: ROV-MM State: revised As explained in the previous paragraph, this scenario includes the objectives of P&R and BS scenario and involves different stakeholders (commuters, occasional users and the mobility manager) and several use cases. The evaluation of this scenario aims to understand how STREETLIFE impact on user choices and consequently how to better promote and push sustainable trips. The following points are going to be described:

- the aspects of the scenario related to users interested only in the bike sharing system of Rovereto

- the aspects of the scenario related to the use of journey planner by other users not interested in shared bikes;

- the aspects of the scenario related to analysis and operations of mobility manager.

Rovereto municipality wants to improve the accessibility to the centre of the town and has implemented since September 2014 a urban bike sharing system that offers the to its citizens 39 electric bikes located in 13 recharging stations available with a badge with the only condition of leaving them to one of the recharging stations (3rd generation bike sharing called “E-motion"). At the same time, the municipality aims to decrease the number of motorists who drive through the central area and look for parking there.



Narrative Tiziana is a Rovereto citizen who has recognized that the bike sharing service can support her daily need of short-term and short-distance urban transportation. She is also environmentally conscious and intends to participate in the bike-sharing program to improve the quality of life and promote sustainable mobility in her city. Given the varied offering of bike-sharing options, Tiziana would like to interact with the STREETLIFE system with a mobile app to be able to know the availability of bikes in the station where she would like to start and finish her trip. She would also like to be able to get a sense of the environmental achieved with each taken bike trip and to communicate feedback about the service when needed.

Davide is a car driver who uses ViaggiaRovereto to plan in advance his journey to the centre of Rovereto. When he asks ViaggiaRovereto to plan is trip, the app highlights two suggested green travel options that include P&R, bike sharing and public transport, but shows Davide the car trip option as well. Davide may or may not decide to accept the advertised options right away; in both cases, he saves his itinerary on the STREETLIFE system and subscribes to STREETLIFE notifications that are relevant to the chosen route. Later, in the minutes before the course of Davide’s planned trip towards the town centre, he receives timely notifications regarding the projected availability of parking spaces in the town centre (or lack thereof), and expected time to reach his destination including time to look for a parking space; based on these events Davide may be advised and directed to use other modes of transport or the closest P&R to his intended destination.

Nicola is a citizen of Rovereto, who is fitness-conscious, and who during a typical day must run several work and personal errands in the city centre. He is aware of the negative consequences of making frequent and short car trips within the city centre, and is tired of the parking and traffic complications involved. He used to go sometimes on his errands on foot, and enjoyed that, but most of the times that took too long. For all these reasons, he has begun to use the Rovereto bike sharing services more and more frequently.

Cristian works in the mobility management office of the municipality of Rovereto and he is invested in the day-to-day monitoring and management of the parking system. He is interested in consulting an up-to-date dashboard which must report and analyse the most important features about Rovereto parking system. Thanks to these data, he can design policies in order to increase the utilization rate of parking lots located outside, or at the border of, the town centre, and give citizens and visitors who want to reach the historical centre a variety of more sustainable ways to do that, such as public transport, bike, or simply walking.

Moreover, Cristan manages the Green Game in Rovereto, a long-running game for ViaggiaRovereto users like Tiziana, Davide and Nicola which aims at engaging and rewarding citizens for their sustainable mobility behaviours. The game includes a mix of long-term point collection competitions and ranks, shorter-term periodic ranks, badge collection activities, challenges with individual goals, short-term (e.g. daily mini-games) etc. Cristian can launch these various game components as he sees fit; he can also decide upon and arrange in-game incentives as well as material rewards for those components; some of the rewards are offered by the municipality, while others involve local companies that can advertise their products through the App and the game Finally, Cristian evaluates the benefit of gamification on the change of player’s mobility habits, and adjust incentive levels accordingly.



In this scenario, the STREETLIFE system supports both Tiziana’s Davide’s and Nicola’s needs for an efficient way to access Rovereto city centre, as well as Cristian’s needs, both for effective reporting of the state of the parking system and for analysis and decision support.

Use case: Bike Sharing

Use Case ID: ROV-MM1 State: revised Primary Actor: Citizen Use Case Description: the end user through the ViaggiaRovereto routing app manages to find an available bike and uses it

Preconditions: Tiziana has the Viaggia Rovereto app installed on her smart phone Trigger: Tiziana has opened “Available bike” screen in Rovereto Bike Sharing app

Basic flow:

1. Tiziana inspects available bikes on the bike sharing station within a radius centred around its current position 2. Tiziana collects the bike and uses it 3. Tiziana leaves the bike at a new location 4. The STREETLIFE system acknowledges Tiziana for her green mobility choice 5. The STREETLIFE system and the mobile app record and accumulate this information within Tiziana’s profile

5.1 The Gamification system awards N “green leaves” points to Tiziana 5.1.1 Thanks to those points, Tiziana reaches her goal in her personalized weekly challenge in the Green Game 5.1.2 Tiziana is awarded a badge in recognition of her challenge success 5.1.3 Tiziana qualifies for a reward made available by the E-motion service

Use case: Trip Planning

Use Case ID: ROV-MM2 Primary Actor: Car driver State: revised Use Case Description: The end user plans her trip choosing between different options where the first two of the list are the “greenest” ones.

Preconditions: Davide is planning his trip using the STREETLIFE app and multi-modal routing service



Trigger: Davide has opened the "Plan your trip" screen in ViaggiaRovereto app and his destination is in the central area of the town that is pre-defined and recorded in STREETLIFE as traffic-sensitive

Basic flow: 1. Davide asks ViaggiaRovereto multi-modal router to find an itinerary for his trip 2. STREETLIFE system returns the requested car itinerary but also suggest him two greener

option that may include P&R 3. Davide can explore the green option and obtain more information through the

ViaggiaRovereto app 4. Davide decides to take the greenest option and saves the corresponding itinerary 5. The STREETLIFE system acknowledges Davide for his green mobility choice

5.1 The Gamification systems accords to Davide a “Parking Explorer” badge for having used that particular P+R facility for the first time

5.1.1 Davide is notified of the badge and decides to share it 5.1.2 The badge appears on Davide’s player page in the Green Game web UI 5.1.3 Other players are notified of Davide’s achievement

.

Use case: analysis of current situation of parking system on MMECP

Use case ID: ROV-MM3 Primary actor: Mobility Manager State: revised Use Case Description: The city mobility manager monitors the current situation of the parking system through the MMECP

Preconditions: Cristian has to evaluate the situation of parking system in Rovereto Trigger: Cristian has access to MMECP Basic flow:

1. Cristian enters into the MMECP 2. Cristian analyses the occupancy rate for a selected parking lot and for a macro area 3. Cristian analyses the income of parking meters

Use case: MMECP simulation – new parking lot

Use case ID: ROV-MM4 Primary actor: Mobility Manager State: revised Use Case Description: The mobility manager uses the MMECP as a powerful instrument to simulate and analyze the impact of a new parking facility in town



Preconditions: Cristian has to take a decision about the location, the number of parking space and other features of a new parking lot. Trigger: Cristian has access to MMECP Basic flow:

1. Cristian enters into the MMECP 2. Cristian sets some parameters of the new parking lot 3. Cristian estimates the percentage of vehicles that, passing by the location of the new

parking lots, can utilize it 4. Cristian simulates the impact of a new parking lot on occupancy rate of other parking

areas 5. Cristian changes some parameters (capacity and available mobility facilities) of the

new parking lot in order to improve the parking system in the city centre 6. Cristian evaluates the different options 7. Cristian takes a decision about the new parking lot

Use case: Green Game

Use case ID: MM5 Primary Actor: Mobility Manager State: new Use Case Description: End users follow the evolution of the green game through the ViaggiaRovereto App, since their mobility behaviour could be rewarded with prizes and real incentives

Precondition: Green Game has started in Rovereto. Annapaola uses regularly ViaggiaRovereto app and has already accumulated many green leaves. Francesco is a newcomer in the game. Francesco starts accumulating “green leaves” points thanks to his green trips, while Annapaola continues to accumulate green leaves points and climb the general Green Game long-term rank Trigger: Cristian consults a game management console and sees that Francesco, as several other citizens have just entered the game Basic flow: 1. Cristian decides to create a new mini-game for the following day, exclusively geared to

new users registered in Green Game in the last 2 weeks. 2. Cristian decides on incentives and rewards for the mini-game

2.1.The mini-game winner gets a special badge 2.2.The mini-game winner gets a material reward, i.e. a “welcome” package of coupons

offered by the Municipality, mobility operators and local commercial entities. 2.3.The players reaching in the mini-game rank will get a bonus multipliers of their total

“green leaves” points depending on how well they do in the mini-game 3. Cristian launches the mini-game and notifies the new users 4. The mini-game takes its course



5. Cristian assesses the effect of the mini-game on the mobility habits of the participants, in comparison with the in-game incentives they have received 5.1 Cristian adjusts the level of bonus multipliers for next instantiations of mini-games

based on his analysis

Use case: Crowdsourcing

Use case ID: MM6 Primary Actor: Bike sharing user State: new Use Case Description: End Users can be an active actor in the STREETLIFE system by giving feedback about the quality of the service

Preconditions: Nicola has used Rovereto Bike Sharing app to plan his trip by a shared bike Trigger: Nicola has just left a shared bike in a bike sharing station Basic flow:

1. Nicola receives a notification from Rovereto Bike Sharing app in order to evaluate his trip

2. Nicola has the option to send to the system feedback about his trip: feedback includes his satisfaction with the service, the condition of the bike (e.g. it may need repair, or has a flat tire) and will also include information about the actual availability of shared bikes.

3. If Nicola cannot park his bike at the destination for lack of a free or working bike slot, Nicola can send a notification to STREETLIFE system

4. Nicola gets rewarded for its participation in the improvement of the bike sharing service with an in-game incentive within the Green Game

Use case: special events routing

Use case ID: MM7 Primary Actor: Christmas market tourist State: new Use Case Description: During a special event that attracts lots of tourists to Rovereto, the ViaggiaRovereto routing app becomes a useful instrument to release some pressure off the city historical centre. Thanks to the App visitors use the Shuttle Bus to reach the Christmas Markets, and not their cars.

Precondition: Ugo is going to visit Rovereto Christmas markets and he plans his journey in advance Trigger: Ugo has opened the Rovereto Christmas markets web page where he finds the link to the ViaggiaRovereto web app



Basic flow: 1. Ugo opens ViaggiaRovereto webapp on his laptop or mobile phone. 2. Ugo inserts the origin of his journey 3. ViaggiaRovereto journey planner gives him directions to reach Christmas markets and suggests him to park in the P&R area identified for the special event 4. Ugo takes his car and drives towards Rovereto 5. During his trip, Ugo finds some special signposting that guides him to the P&R area 6. Ugo parks in the P&R area 7. Ugo takes the shuttle bus to the city centre

Scenario: Car Pooling

Scenario ID: ROV-CP State: revised Rovereto is the main town of a group of municipalities called Valle Vallagarina. The 80% of commuters who enter in Rovereto every day are coming from Valle Vallagarina. The main roads of entrance into the city and the most common work places of commuters are pretty much recognizable in the urban structure. This situation is an ideal pre-condition to promote ICT solutions that deploy a car pooling service aimed at helping workers-commuters specifically, and which can promotes and incentivize home-to-work-to-home mobility in a sustainable and convenient manner.

Narrative: Marco lives in Valle Vallagarina, about 25 Km outside Rovereto, but has just been hired by company XYZ in Rovereto. He is environmentally-conscious, and would also like to save money on his daily trips to and from work.

Annapaola is also a commuter from Valle Vallagarina, who has been long employed by company XYZ. She is a habitual car commuter (about 20 Km away from Rovereto). However, because of family reasons she uses her car only on Mondays, Wednesdays and Fridays.

Davide is also a habitual car commuter to Rovereto. He is not an employee of XYZ, but works at company ABC in the immediate vicinity (in the same industrial district)

Marco and Annapaola create a community in the STREETLIFE system. The community allow them to ask, offer and negotiate a car ride, among one another and with others who also enter the community. They can specify any personal details and preferences, and the origin and the destination of the trip. A community is organized as a social network: it can include friends and acquaintances, as well as “liked” fellow travellers who have successfully shared a ride in the past. It also can include ant number of co-workers in the same company, in order to facilitate co-workers to find one another when organizing their car pooling trips.



Cristian, who works in the mobility management office of the municipality of Rovereto is tasked with promoting the car-pooling service, with the goal of reducing the number of circulating vehicles going back and forth to the industrial districts of the city during work days. Cristian launches a game that involves both car-pooling app users and enterprises. The game is a combination of individual as well as team long-term competitions, short-term challenges and special events. Teams are the various communities established freely by users of the car-pooling services, as well as entire participating enterprises. The game uses in-game incentives to increment the usage of car-pooling, as well as material rewards offered by the Municipality and commercial entities, including company perks offered by the participating enterprises to their own employees. Cristian can launch these various game components as he sees fit; he can also decide upon and arrange in-game incentives as well as material rewards for those components; finally, Cristian can evaluate the benefit of gamification on the acceptance and usage of the car-pooling service, and adjust incentive levels accordingly.

Use case: Find a carpool ride

Use Case ID: ROV-CP/1 Primary Actor: Commuter / passenger State: revised Use Case Description: The End User manages to arrange a Car Pooling ride as a passenger with the Car Pooling App

Preconditions: Marco has the STREETLIFE mobility app installed on the smart phone and is registered in a car pooling community; Marco and Annapaola belong in the same car pooling community. Trigger: Marco has opened a “Find a ride” screen in STREETLIFE app

Basic flow:

1. Marco specifies his itinerary, including end points and date and time for the requested carpool ride; he can add other details

2. Marco issues the “find a ride” request 3. STREETLIFE car-pooling service finds trip offers that match to Marco’s request 4. Marco chooses Annapaola's trip offer 5. Marco accesses and examines Annapaola’s driver profile maintained by STREETLIFE 6. Marco and Annapaola negotiate and agree upon the terms of the ride 7. Marco accepts the ride and issues a confirmation through the app 8. App sets and send appointment reminder to Marco and Annapaola

Use case: Offer a carpool ride

Use Case ID: ROV-CP/2 State: revised Use Case Description: The End User manages to arrange a Car Pooling ride as a driver with the Car Pooling App



Primary Actor: Commuter (driver) Preconditions: Annapaola has the STREETLIFE mobility app installed on the smart phone and is registered as a car pooler (driver); Marco and Annapaola belong in the same car-pooling community Trigger: Annapaola has opened an “Offer a ride” screen in STREETLIFE app

Basic flow:

1. Annapaola specifies her itinerary, including end points and date and time for the requested carpool ride;

2. Marco chooses Annapaola trip offer. 3. Annapaola and Marco negotiate the terms of the ride 4. Annapaola accepts to offer the ride and issues a confirmation through the app 5. App sets and send appointment reminder to both Annapaola and Marco

User Case: Take a CP ride as passenger

Use Case ID: ROV-CP/3 State: revised Use Case Description: The ride that has been organized with the Car Pooling App takes place, since the designed passenger completes her trip

Primary Actor: Passenger Preconditions: Annapaola and Marco have arranged terms of a ride using the STREETLIFE system. (see Use Cases ROV-CP 1/2) Trigger: Annapaola and Marco meet at arranged place.

Basic flow:

1. Marco goes to the arranged place 2. When they are in the proximity of one another, Marco and his driver Annapaola

complete rendezvous with the support of their STREETLIFE mobile apps. 3. Marco informs the system that the trip ended correctly 4. Marco sends to the STREETLIFE system his feedback about the ride and the driver.

This feedback is recorded by the system 5. The STREELIFE system acknowledges Marco for his usage of the car-pooling service

5.1 Marco accumulates a number of “car-pooling points” calculated based on the trip characteristics

5.2 Marco has reached the goal of 500 “car-pooling points” 5.2.1 Marco is awarded a badge 5.2.2 Marco’s company XYZ is notified of his game achievement



Use case: Take a CP ride as driver

Use Case ID: ROV-CP/4 State: revised Primary Actor: Commuter

Preconditions: Annapaola and Marco have arranged terms of a ride using the STREETLIFE system. (see Use Case ROV – 2/2) Trigger: Annapaola and Marco meet at arranged place.

Basic flow:

1. Annapaola goes to the arranged place 2. When they are in the proximity of one another, Annapaola and her passenger complete

rendezvous with the support of their STREETLIFE mobile apps. Annapaola informs the system that the trip ended correctly.

3. Annapaola sends to the STREETLIFE system her feedback about the ride and the passenger. This feedback is recorded by the system

4. The STREELIFE system acknowledges Annapaola for his usage of the car-pooling service

4.1 Annapaola accumulates a number of “car-pooling points” calculated based on the trip characteristics

4.2 Annapaola’s counter for car-pooling trips offered is incremented by 1 4.3 Annapaola has reached her monthly goal for offered car-pooling trips

4.3.1 Annapaola is awarded a reward by the Municipality for her service

4.3.2 Annapaola’s company XYZ is notified of her game achievement 5 Annapaola and Marco are satisfied with their car pooling experience

5.1 They decide to car pool together regularly every Monday, Wednesday and Friday

5.2 They set their preferences in the car pooling system and their own community accordingly

Use case: Community Gamification

Use Case ID: ROV-CP/5 State: new Use Case Description: End Users are engaged in a regular usage of the App thanks to a gamification system that will award virtual (and real) incentives to the winners of individual game based on the trips taken

Primary Actors: Commuter and Mobility Manager

Preconditions: Cristian manages the car-pooling game from the Mobility Management Office.



Trigger: Cristian’s analysis shows that car-pooling trips with more than 2 people are lagging Basic flow: 1. Cristian launches a new game challenge for communities

1.1.The community that improves the most the average car occupancy for its car-pooling trips in the next month shall win the challenge.

1.2.Cristian contacts companies for setting rewards for company employees who are the challenge winners.

1.3.Company XYZ decides to reward its employees in case they win the challenge with privileged parking spots in the company lot.

2. Annapaola and Marco are made aware of the new car pooling game challenge 3. They decide to actively look for other passengers on their periodic trips

3.1. They know Davide already and they invite him in their community 3.2. (alternative) OR Davide is already part of their community and they negotiate with

him a car pooling ride (see previous use cases) 4. Annapaola, Davide and Marco start car-pooling together regularly on Mondays,

Wednesdays and Fridays 5. At the end of the months, their community wins the challenge

5.1. Annapaola and Marco receive the privileged parking lot pass from XYZ 5.2. Davide receives a reward made available by his own company

Use case: Collective Gamification

Use Case ID: ROV-CP/6 State: new Use Case Description: The gamification system used in ROV-CP/5 to enhance App usage between End Users on a individual basis generates a “virtual battle ground” between companies too

Primary Actor: ABC company

Preconditions: Davide works in ABC company. ABC is taking part to the Car Pooling Green Game for companies; Davide is registered in a car pooling community Trigger: ABC working commuters plan their trips to the workplace with the car-pooling app Basic flow: 1. For every car pooling trip completed by Davide (and any other of its employees),

company ABC receives the corresponding number of “car-pooling points” 2. For every car pooling trip offered by Davide (and any other of its employees) as a driver,

company ABC increases its collective car-pooling trips counter by 1 3. ABC checks its place in the car pooling rank for companies

3.1. ABC decides they want to be more competitive with respect to the car-pooling trips counter

3.2.ABC provide incentives to its staff who drive to work, so that they offer car-pooling trips to others



4. Davide decides to propose within his community he can be a driver on Tuesdays and Thursdays 4.1.Annapaola and Marco accept to take part in these rides 4.2.Davide becomes the driver of car-pooling trips for the 3 of them on those days

5. ABC company sees its car-pooling trips counter and its rank increase with respect to the other enterprises participating in the game

6. The Rovereto municipality acknowledges the green behaviour of ABC and their prominent rank in the Car Pooling Green Game for companies

4.2.4. EXPERIMENTS

Experiment: Car Pooling Beta test

Experiment ID: ROV-EXP-1

Goal: test the car-pooling app and receive feedback and suggestion to improve it.

Description:

A group of friendly users tests the app and sends feedbacks on its usability, weak and strong point and suggests how to improve this instrument.


• ROV-CP1 • ROV-CP2 • ROV-CP3 • ROV-CP4 • ROV-CP5


Type beta-testing, focus group Instruments Car Pooling ROV app, questionnaires, interviews Stakeholder types: App developer, enterprises, Rovereto municipality Size: 20 – 40 end users testing the App Duration: 2 – 3 weeks Beginning date: November 2015 End date: December 2015 Experiment Engagement plan

Users will be recruited at three local companies that agreed to participate in the car pooling pilot. Working commuters will be involved through newsletter and official communication from Human Resources Department and Mobility Manager of the company.



Experiment 2 – Special events mobility (Christmas market)


Goal: test the impact of STREETLIFE on an event that attracts many tourists and notably increase traffic in the city centre.

Description:

Tourists going to Rovereto Christmas markets are suggested to plan their trip using a web app that drive them to a P+R area outside the city centre where they find a shuttle bus service, specifically set up for the event.


• ROV-MM2 • ROV-MM7


Type Christmas markets tourist Instruments ViaggiaRovereto web app Stakeholder types: App developer, Rovereto municipality, Christmas markets organizers Size: Open field experiment Duration: 8-10 days over a period of 4/5 weeks Beginning date: November 2015 End date: December 2015


ViaggiaRovereto web app will be promoted in all the web pages where tourists can find information about Christmas market thanks to the collaboration with Christmas market organizers. Moreover, tourists on their way to Rovereto will find special signposting that guide them to the P&R area.

Experiment 3 – Long Run Gamification


Goal: assess the results of a long-term gamification in terms of users mobility habits and app usage

Description:

Users of the ViaggiaRovereto App will take part in a Green Game, where take part in gamified mobility activities. The game will last for several weeks (up to 12 weeks) and will include a mix of long-term point collection competitions and ranks, shorter-term periodic



ranks, badge collection activities, individual challenges with personal goals, short-term challenges and missions (e.g. daily mini-games) etc. This mix is designed to make the game attractive and playable by newcomers (who are incentivized to compete in the short-term challenges and ranks), as well as to sustain participation of committed players in the long run.

The game awards in-game recognition of sustainable mobility behaviours, for example through “green leaves” points and badges, but also provides material rewards to winners of the various game-like activities and events.


• ROV-MM1 • ROV-MM2 • ROV-MM5 • ROV-MM6


Type Citizens Instruments ViaggiaRovereto web app, Gamification Engine Stakeholder types: App developer, Rovereto municipality Size: Open field experiment Duration: 3 months Beginning date: February 2016 End date: May 2016


ViaggiaRovereto users will be informed about the launch of the Green Game with a notification. Rovereto municipality will promote ViaggiaRovereto and its Green Game among citizens.

Experiment 4 – Car Pooling for commuters


Goal: promote car-pooling among commuters working for Rovereto local companies in order to reduce the number of vehicles circulating in the city. The experiment aims to directly involve also local enterprises and make them set up incentives to promote car-pooling among their working commuters.

Description:

Working commuters will subscribe to Rovereto car-pooling app. They will be part of communities where they can offer and find a ride for their trips. Enterprises will promote car-pooling among their staff. Both end-users and enterprises will take part in a game: individual users will participate in a car pooling game that recognizes with in-game incentives as well as



material rewards their car pooling activities. Enterprises will also take part in the game, and will be recognized for the collective car-pooling activities of their employees.


• ROV-CP1 • ROV-CP2 • ROV-CP3 • ROV-CP4 • ROV-CP5 • ROV-CP6


Type Working commuters Instruments ViaggiaRovereto web app, Rovereto Gamification Engine Stakeholder types: App developer, Rovereto municipality, Enterprises Size: Open field experiment Duration: 3 months Beginning date: February 2016 End date: May 2016


Users will be recruited at Rovereto local companies that have agreed to participate in the car pooling pilot. Working commuters will be involved through newsletter and official communication from Human Resources Department and Mobility Manager of the company. Moreover, Rovereto municipality will promote car-pooling app among its citizens.

4.2.5. SKIPPED SCENARIOS AND USE CASES

The use cases ROV-MM3 and ROV-MM4 that are strongly related to the use of MMECP will be evaluated qualitatively without a specific experiment. The main instrument to assess the usability and the usefulness of the dashboard will be a questionnaire that will be submitted to Rovereto Mobility Manager in order to collect his feedback and suggestions about how to improve the control panel.



4.2.6. OBJECTIVES

The main objectives of the STREETLIFE Rovereto pilot address the following topics:

- CO2 reduction by modal split change towards “greener” modes (project level)

- Improve the city transport quality and efficiency (project level)

- Improve the usage of available parking lots (pilot level)

- Improve Bike Sharing service quality and efficiency (pilot level)

- Boost Car Pooling (pilot level)

- Become a useful instrument in the design of new mobility policies (project level)

- Improve the level of information about mobility to the mobility manager (project level)

- Get people involved using devices that improve their knowledge base on the mobility situation (project level)

- Create an incentive system that attracts users involved (project level)

- Make stable through time benefits of gamification reached by fix and short time experiments (pilot level)

- Make aware local enterprises of their impact on urban mobility and involve them in the promotion of sustainable mobility (pilot level)


Table 9: Research Question of ROV pilot addressed in Y1 and Y2

ID Question Addressed in 1st iteration

Active in 2nd iteration

1st it

erat

ion

RQ

s

RQ –ROV1 Is there a significant change in the mode choice? Y Y

RQ –ROV2 Which mode benefits most of the change? Y Y

RQ –ROV3 Why do people change their mobility behaviour? Y Y

RQ –ROV4 Which type of commuters is most willing to change their mobility habits?

N N

RQ –ROV6 How can STREETLIFE improve the utilisation rate of Bike sharing service on the End User’s side?

Y Y

RQ –ROV7 If there’s a change in the mode choice, what impact does it have on CO2 emissions?

Y Y



RQ -ROV8 Is there a change in the utilisation rate of parking slots available?

Y Y

RQ– ROV13 What is the level of satisfaction from the STREETLIFE users?

Y Y

RQ– ROV14 What is the level of acceptance from the STREETLIFE users?

Y Y

RQ– ROV15 What is the level of compliance to the suggested routes from the STREETLIFE users?

Y Y

RQ– ROV16 Will STREETLIFE reduce time spent in the traffic?

Y Y

RQ– ROV17 If there’s a change in the mode choice, what impact does it have on fuel consumption?

Y Y

RQ – ROV5 Can STREETLIFE increase the level of utilization of Car Pooling as a sustainable alternative to individual mobility by car?

N Y

2nd it

erat

ion

RQ

s

RQ – ROV5 How does an increased amount of information and specification brought by STREETLIFE can improve car-pooling utilization?

N Y

RQ – ROV9 Can the STREETLIFE control panel support with data analysis?

N Y

RQ– ROV10 Can the STREETLIFE control panel support the design of new mobility policies

N Y

RQ– ROV11 Can the STREETLIFE control panel support the management of services?

N Y

RQ– ROV12 Can STREETLIFE use the data contained in users’ profiles efficiently?

N Y

RQ– ROV18 Does gamification have an impact on the app usage?

N Y

ROV-RQ19 If gamification has an impact on the app usage, does it push users towards greener modes of transportation?

N Y

RQ-ROV20 What is the effect of the gamification incentive system on End Users’ mobility behaviour?

N Y

ROV-RQ4 has been excluded from the evaluation process because, after a discussion with Rovereto municipality, we decided to concentrate only on commuter for work. Indeed, this



type of user is the one that mostly impacts on city traffic system, while the majority of students does not have a driving licence and already uses public transport.


Table 10: ROVERETO Pilot Evaluation Matrix

RQ-ROV1ER-2 Is there a significant change in the mode choice?

HY-204 STREETLIFE does increase the use of "green " transport modes

PI-203 PI-204

# of carbon friendly trips km of carbon friendly trips

Type Mobility

Method LOG FILES FROM APP USAGE

QUESTIONNAIRES

Metric

Km travelled for each mode of transport Modal split during experiments

RQ-ROV2R-2

Which mode benefits most of the change?

HY-ROV3 HY-ROV16

The utilization rate of the bike sharing system will rise

The utilization rate of buses will grow

PI-120 PI-112 PI-122 PI-126

Bike sharing distance Turnover of bikes (%) Bike sharing km with P&R Public transport trips with P&R

Type Mobility

Method Log files from app usage Data from bike sharing operator Questionnaires

Metric Km covered, number of trips



RQ-ROV3R-2

Why do people change their mobility behaviour?

HY-503 The user will find the STREETLIFE system useful

PI-305 PI-309 PI-310

Users’ perception Service reliability Awareness level

Type User acceptance

Method Questionnaires

Metric Number of users who gave a positive response to questions

RQ-ROV5R-2

Can STREETLIFE increase the level of utilization of Car Pooling as a sustainable alternative to individual mobility by car?

HY-ROV8 HY-ROV9 HY-ROV10

Car-pooling trips will increase Users increase personal info’s on their profiles in order to have a better car-pooling experience Users utilize the STREETLIFE system to create carpooling groups

PI-113 PI-128 PI-127 PI-129 PI-130

# of car-pooling trips offered car-pooling distance # of warnings # of car-pooling trips travelled # of carpooling groups

Type Mobility, User behaviour


Log files from App usage

Metric # of trips, km, # of groups

RQ-ROV6-2 How can STREETLIFE improve the utilisation rate of Bike sharing

service on the End User’s side?

HY-ROV5 HY-ROV19

The efficiency of the bike sharing system will be higher Users choose trip options that include bike sharing

PI-122 Bike sharing trips with P&R



Type Mobility

Method Log files from app usage Data from bike sharing operator Questionnaires

Metric number of trips, km travelled by bike sharing

RQ-ROV72 If there is a change in the mode choice, what impact it has on CO2 emissions?

HY – 204 HY – 206

STREETLIFE will reduce the CO2 emissions of the daily trip chain STREETLIFE will reduce the CO2 emissions of all traffic, in (part of a) network.

PI-201

Carbon emissions

Type Environment

Method Log files from App usage

Questionnaires

Metric CO2 emissions

RQ-ROV82 Is there a change in the utilization rate of parking slots available?

HY – ROV4 The utilization rate of outer parking spots will grow

PI-118 PI-125

Park & Ride Usage Park & Ride Trips

Type Mobility

Method Log files from App usage

Questionnaires

Metric # of trips, occupancy rates for some parking facilities

RQ-ROV9 Can the STREETLIFE control panel support with data analysis?



HY – ROV11 HY – ROV12

The mobility manager will find the STREETLIFE system useful to analyse actual and previous data; The mobility manager will find the STREETLIFE system useful to have a better integrated data vision;

PI – 308 PI – 309 PI – 304

Service quality; satisfaction; usefulness

Type User behaviour


Metric Number of positive answers to specific MMECP questions

RQ-ROV10 Can the STREETLIFE control panel support the design of new mobility policies?

HY – ROV7

The mobility manager will select better spots for parking lots designed for P+R

PI – 308 PI – 309 PI – 304


Type User behaviour



RQ-ROV11 Can the STREETLIFE control panel support the management of mobility services?

HY – ROV12 HY – ROV13

The mobility manager will find the STREETLIFE system useful to have a better integrated data vision; The mobility manager will find the STREETLIFE system useful to control the services provided;

PI – 308 PI – 309 PI – 304




Type User behaviour



RQ-ROV12 Can STREETLIFE use data contained in users’ profiles in an efficient way?

HY – ROV9

Drivers use the app to inform real time the community about their unavailability to offer a ride and passengers find alternative solutions.

PI – 127

# of warnings

Type User behaviour


Metric Number of warnings, number of feedback given to the system

RQ-ROV13 What is the level of satisfaction from the STREETLIFE users?

HY – 502

The user will find the STREETLIFE system easy to use

PI – 313 PI - 302

Ease of use Users’ perception of comfort

Type User behaviour



RQ-ROV14 What is the level of acceptance from the STREETLIFE users?

HY – 501

The user will use the STREETLIFE system



PI – 311 PI – 318 PI – 317

User acceptance # of users # of active users

Type User behaviour

Method Questionnaires Log files from the app


RQ-ROV15 What is the level of compliance of the suggested routes from the STREETLIFE users?

HY – 602

The user will mostly follow STREETLIFE operational recommendations

PI – 318 PI - 150

# of users Compliance

Type User behaviour

Method Questionnaires Tracking of users

Metric Percentage of trips taken following the App’s instructions

RQ-ROV16 Will STREETLIFE reduce time spent in traffic?

HY – 201

STREETLIFE does not increase total travel time (individual), of a trip (origin-destination)

PI – 307

Journey Time

Type Mobility


Metric Total travel time of a trip

RQ-ROV17 If there is a change in the mode choice, what impact it has on fuel consumption?



HY - 203

STREETLIFE will reduce the fuel consumption of the daily trip chain.

PI - 205

Fuel consumption

Type Environment


Log files from App usage

Metric

kg of fuel

RQ-ROV18 Does gamification have an impact on the app usage?

HY – GE1

Gamification increased app usage

PI-140

# of trips during different phases

Type User behaviour

Method Log files from app usage

Metric

Number of trips taken during different phases of the pilot execution

RQ-ROV19 If gamification has an impact on the app usage, does it push users towards greener modes of transportation?

HY – GE2

Gamification changed user choices on type of trips

PI-203 PI-204


Type User behaviour




Metric

Number of green-friendly trips and its variation during different phases of the pilot execution

RQ-ROV20 What is the effect of the gamification incentive system on End Users’

mobility behaviour?

HY – GE3

The gamification points system doesn’t generate any distortion on End Users’ mobility choices

PI-203 PI-204


Type User behaviour


Metric

Number of green-friendly trips and its variation during different phases of the pilot execution



4.3. TAMPERE


The main focus in the 1st iteration was to take into use new real-time routing service architecture and a mobile STREETLIFE application, utilizing the existing IT infrastructure efficiently. Service reliability and performance, general user acceptance and the modal shift in traffic behaviour were tested/evaluated during this iteration.

Now, by the 2nd iteration, the focus is shifted towards the following topics:

• Utilization of control panel by the Public Transport Authority (PTA) and its effect on public transit passenger flow

• Gamification features and their effect on user acceptance of the STREETLIFE application

• Further development / fine tuning the mobile STREETLIFE application to maximize user acceptance

• New engagement strategies to collect additional user feedback and survey responses

4.3.2. EXPERIMENTS – SECOND ITERATION’S SCENARIOS AND USE CASES

Table: TRE Pilot – Second Iteration Scenarios and Use Cases

ID Name EXP-3 EXP-4 EXP-5

Scenario TRE-03 Transportation flow management

TRE-3/1 Use Case: Mitigate congestions in main bus stations X

Scenario TRE-04 Parking situation control panel

TRE-4/2 Use Case: Increase usage of Park&Ride X

Scenario TRE-02 Multimodal real-time Journey Planner improvements

TRE-02-2 Use Case: Journey Planner gamification X

TRE-02/3 Use Case: Mixed Reality X



SCENARIO: TRE-03 Transportation flow management Narrative: The Public Transport Authority mobility manager (PTA) wants to slightly shift some traffic from the city centre to bus stops nearby. In order to do so, they decreased the algorithmic costs of the specific bus stops a bit. The net effect was that a portion of the traffic started to shift from the hot-spots and hence released some pressure from the centre. Stakeholders: PTA wants to adjust traffic flows with STREETLIFE tools. Added value: Distribute traffic flows to multiple stops in city centre.

Use Case: TRE-3/1 Mitigate congestions in main bus stations

Primary Actor: PTA

Preconditions: Journey Planner management console has mechanisms to alter stop goodness values (or transfer penalty margins for each stop).

Trigger: PTA changes stop goodness value

Basic flow:

1. PTA officer alters stop goodness value in Journey Planner management console and submits changes.

2. The new parameters are used in routing when users are requesting route suggestions

Experiment specifications

Experiment: TRE-EXP-3 Transportation flow management

Goal: Experiment to evaluate the flow management in the city centre by utilizing STREETLIFE real-time journey planner.

Description: There is a bus stop in the city centre where the stop platform length has been reduced and at the same time there is lot of passengers entering the bus. This increases the time bus spends in the stop and overall congestion is caused as stop is used by multiple bus routes. The journey planner should give more cost if change is done on this crowded stop. PTA wants to avoid use of this stop to some extent in order to improve overall flow management of their transport network.

Scenarios and use cases: Goal is to get people changing a few stops earlier with short walking distance. Visible effect should be that when searching to this location journey planner should direct some percentage of users to nearby stops and suggest them walking a short distance. This can be tested by turning the flow management on and off and comparing the resulted route suggestions. Goal is also test various stop goodness values. It is believed too



extreme costs cannot be used so the process of finding suitable cost range is important. After you know the effect of stop costs, they can be applied to key places in the transport network and the flow management reaches the whole network. Stop goodness value (cost) can for example be affected by transport mode, stop amenities, transport policy, congestion, and area of the network (city centre or rural area). Assumption that stop in uncongested area in city centre close to main routes, which have high frequency of public transport and is the first stop of the route should have better stop goodness than a stop in rural area. Side effect should be: user is also happier if he unfortunately misses the transfer, there is more likely a new bus coming soon on a high frequency route.


Type: Open field test Instruments: In-person observations of number of passengers visiting the stop.

Questionnaire and PTA observation interview. Stakeholders: PTA Size: Key locations at city centre, all passengers travelling from these locations. Duration: Continuous Beginning date: Oct 1st 2015 End date: Feb 20th 2016

Experiment Engagement Plan

Engagement with Tampere Traffic Planning has already started and a stop at the city centre has been identified as location where the experiment will be carried out.

Experiment Technical realization

Data source and software components list used in the experiment: Enhancing, configuring and utilizing the real-time journey planner setup in the 1st pilot iteration.

SCENARIO: TRE-04 Parking situation control panel Narrative: The Public Transport Authority mobility manager (PTA) wants to slightly shift some traffic from the crowded parking places in the city centre to Park&Ride locations reducing overall congestion and promoting public transport. In order to do so, they can decrease the algorithmic costs of the specific bus stops a bit as described in TRE-03. But to help this decision making PTA would like to see the parking status in the city centre and Park&Ride locations on top of map. Stakeholders: PTA wants to adjust traffic flows with existing tools and promote Added value: Decision support Use Case: TRE-4/2 Increase usage of Park&Ride

Primary Actor: PTA



Preconditions: Journey Planner management console has mechanisms to alter stop goodness values. Parking situation map is available for decision support. Park&Ride location has free parking spaces.

Trigger: PTA changes stop goodness value after viewing the parking situation

Basic flow:

1. PTA officer views parking situation on map. Map shows availability of free parking and status of Park&Ride locations

2. PTA officer alters Park&Ride locations goodness value in the routing and stop goodness near Park&Ride locations value in Journey Planner management console and submits changes.

3. The new parameters are used in routing when users are requesting route suggestions


Experiment: TRE-EXP-4 Park&Ride usage improvement

Goal: Experiment is to evaluate the flow management by increasing Park&Ride utilization with the help of STREETLIFE real-time journey planner.

Description: PTA wants to increase usage of Park&Ride in order to improve overall flow management of their transport network.

Scenarios and use cases: Goal is to increase usage on Park&Ride. The PTA can modify the Park&Ride locations goodness values and stop goodness values near the Park&Ride locations and the journey planner will more often suggest Park&Ride route to end users. Focus is in Park&Ride but similarly other data can be visualized as decision support.


Type: Focus group and open field test Instruments: Questionnaire and PTA observation interview Stakeholders: PTA Size: Key park and ride locations, all passengers travelling near these. Duration: continuous Beginning date: Oct 1st 2015 End date: Feb 20th 2016


Engagement with Tampere Traffic Planning has already started. The mobility manager will configure the STREETLIFE journey planner with help with Tampere pilot partners.



Experiment technical realization

Data source and software components list used in the experiment:

- Enhancing and utilizing the real-time journey planner setup in the 1st pilot iteration. - Parking place availability API providing data to the CIP

- Siemens CIP for integration to WP4 control panel (MMCEP) - WP4 control panel (MMCEP) to visualize parking situation

SCENARIO: TRE-02 Multimodal real-time Journey Planner improvements Narrative: Lisa has found STREETLIFE app which has gamification. Instead of driving to work and hobbies, she is enthusiastic to collect badges and green leaves in the STREETLIFE app. She even has a small competition with her co-workers who can collect most green leaves in January. In addition Mixed Reality field tests are done, to identify use cases where they would provide additional benefits for people on the move. Stakeholders: Individual citizen: multi-modal real time journey planner helps citizens to make optimal travelling decisions in their daily life. Added value: TRE-02/2: Involve participation to increase usage of

public transportation Traffic flow is improved in the city and additionally parking space issues at business offices are reduced, as people use multiple means of transport compared to only private car. TRE-02/03: Mixed reality help people to navigate in unfamiliar location.

Use Cases: TRE-02/2 Use Case: Journey Planner gamification TRE-02/3 Use Case: Mixed reality

Primary Actor: Citizen Preconditions: TRE-02/2: Citizen has web browser in her phone/tablet.

Journey Planner has gamification. User keeps browser turned on. TRE-02/3: Citizen has tabled with Mixed Reality app with the needed sensors.

Trigger: TRE-02/2: User gets point for certain actions in the journey planner.

Basic flow in TRE-02/2 Journey Planner gamification:

1. Citizen uses STREETLIFE app to search multi-modal route suggestion 2. User gets green leaves when public transport is used. Green leaves / Badges are

given based on areas user visits. 3. Game type is City Explorer where the idea is to encourage using public transport

for all sort of trips. The get users not only to commute to work, but also promote public transport for various leisure trips.



a. Areas are defined by predefined post codes (polygons) to enable the city exploration.

b. GPS data is used to determine the area visit. Additionally there is parameter to take account possible GPS in-accuracy. If a certain threshold value is ok, the location visit can be registered and point given.

c. Points will earn badges and there are multiple different tiers. Badges are given for both absolute amount of locations and relative numbers

d. Weekly / Periodic high score ladder–kind of badges are given too

Basic flow in TRE-02/3 Mixed Reality:

1. User is not familiar with the bus routes proposed by the Journey planner. User uses the Mixed Reality application to identify the exact location of the bus stop, where he has to go.

2. User is getting to the bus, but he is still tens metres away and sees buses coming. However he doesn’t see the bus numbers because of the angle of the bus or buildings preventing visibility. With the Mixed Reality application he can visualise the correct bus on the application screen with estimation if he has time to get to the bus.


Experiment: TRE-EXP-5 Multimodal journey planner improvements and gamification

Goal: Increase user participation and gain more public transport users. Also Mixed Reality field tests are done to evaluate their user acceptance.

Description: PTA wants to increase the usage of public transport with gamification engaging users more. PTA wants learn if Mixed Reality can be used helping passengers’ orientation in unfamiliar location.

Scenarios and use cases: Goal is to get people use public transport with gamification, participation can be seen as a motivation action.


Type: Focus test group Instruments: Questionnaire and PTA observation interview, field tests Stakeholder: Citizen, PTA Duration: Continuous Beginning date: Oct 1st 2015 End date: Feb 20th 2016




STREETLIFE Tampere pilot has already started discussion with the Tampere University of Technology student's group, who are active on traffic related matters. Plan is to engage them continuously and give them early access in 2nd pilot iteration and to Mixed Reality field trials. Hopefully this improved communication will boost the participation and through enthusiast good word spreads.

Experiment Technical realisation

Data source and Software components list used in the experiment:

• Enhancing and utilizing the real-time journey planner setup in the 1st pilot iteration.

• Gamification engine from FBK • Aalto’s Mixed Reality Application for the field tests

4.3.3. OBJECTIVES

The main objectives of the STREETLIFE Tampere pilot address the following topics:

• CO2 reduction by modal split change towards “greener” modes (project-level)

• Improve city transport quality and efficiency (project-level)

• Improve the usage of Park & Ride (pilot-level)

• Get people involved using devices that improve their knowledge base on the mobility situation (project-level)

• Create an easy-to-use app that attracts private car users to public transportation (project-level)


Table 11: Research Question of TRE pilot addressed in Y1 and Y2

ID Question Explored in 1st iteration

Active in 2nd iteration

RQ-TRE-1 Is there a significant change in the mode choice? Y Y

RQ-TRE-2 Why do people change their mobility behaviour? Y Y

RQ-TRE-3 Which type of commuters is most willing to change their mobility habits?

Y Y



RQ-TRE-4 How do situational variables (i.e. weather, day time etc.) have an effect regarding the mode change?

Y Y

RQ-TRE-5 How an increased amount of information and specification brought by STREETLIFE can improve Park & Ride utilization?

N Y

RQ-TRE-6 If there’s a change in the mode choice, what impact does it have on CO2 emissions?

Y Y

RQ-TRE-7 What is the level of satisfaction from the STREETLIFE users?

Y Y

RQ-TRE-8 What is the level of acceptance from the STREETLIFE users?

Y Y

RQ-TRE-9 Will STREETLIFE reduce time spent in the traffic?

Y Y

RQ-TRE-10 Will STREETLIFE reduce time spent searching for a free parking lot?

N Y

RQ-TRE-11 Will STREETLIFE integration to existing IT platforms lead to increased possibilities for further exploitation?

Y Y

RQ-TRE-12 Will STREETLIFE enable Traffic Manager to manage passenger flows by varying stop goodness values?

N Y

RQ-TRE-13 Will STREETLIFE Mixed Reality interfaces improve perception of public transportation?

N Y

RQ-TRE-14 Will STREETLIFE virtual mobility decrease the threshold for using greener means of travel?

N Y


RQ-TRE-1 Is there a significant change in the mode choice?

HY-203 HY-204

STREETLIFE significantly leads to a change in mode choice.

PI-101

PI-102

Mode change

Type Mobility



PI-201

Method Questionnaires, Field study (observations), Mobility manager interview

Metric

Success Criteria

There is 0,5% mode change

RQ-TRE-2 Why do people change their mobility behaviour?

HY-503 The change in modal choice produced by STREETLIFE is more pronounced towards some means of transportation.

PI-101

PI-102

PI-106

User behaviour change

Type Mobility

Method Questionnaires, Field study (observations), Mobility manager interview

Metric

Success Criteria


RQ-TRE-3 Which type of commuters is most willing to change their mobility habits?

HY-TRE01 Private car users will find Park & Ride useful

PI-310

Mode change

Type Mobility

Method Questionnaires, Mobility manager interview

Metric

Success Criteria




RQ-TRE-4 How an increased amount of information and specification brought by STREETLIFE can improve Park & Ride utilization?

HY-TRE02 Park & Ride utilization increases

PI-312

Mode change

Type Mobility


Metric

Success Criteria

Park&Ride usage increaces

RQ-TRE-5 If there’s a change in the mode choice, what impact it has on CO2 emissions?

HY-102 HY-104

STREETLIFE significantly leads to a change in mode choice.

PI-101

PI-102

PI-106

Mode change

Type Mobility


Metric

Success Criteria


RQ-TRE-6 What is the level of satisfaction from the STREETLIFE users?

HY-502

HY-503

HY-104

User approves the solution is helpful and stable

PI-312

Mode change

Type User satisfaction




Metric

Success Criteria


RQ-TRE-7 What is the level of acceptance from the STREETLIFE users?

HY-501

User is happy about the system

PI-312 Mode change



Metric

Success Criteria

User uses the system

RQ-TRE-8 Will STREETLIFE reduce time spent in the traffic?

HY-212

HY-202

User approves the solution is helpful and stable

PI-310

PI-308

Mode change



Metric

Success Criteria

User s travelling is faster, journey time reduced

RQ-TRE-9 Will STREETLIFE reduce time spent searching for a free parking slot?



HY-TRE03 Unnecessary driving in the city centre will be reduced.

PI-101

Mode change

Type Mobility, P&R usage


Metric

Success Criteria

User s travelling is faster, journey time reduced

RQ-TRE-10 Will STREETLIFE reduce time spent searching for a free parking slot?

HY-TRE4 STREETLIFE will stay in production after project has ended

PI-313

Commercial feasibility

Type Mobility, Commercial feasibility

Method Questionnaire, Mobility manager interview

Metric

Success Criteria

Decision makers see the benefit and solution will stay in use after the pilot

RQ-TRE-11 Will STREETLIFE enable Traffic Managers to manage passenger flows by varying stop goodness values?

HY-TRE5 Decreases congestion in central traffic terminal areas

PI-303 Traffic management personnel finds to flow management useful

Type Mobility

Method Questionnaire, Mobility manager interview

Metric

Success Criteria

Traffic management personnel finds to flow management useful



RQ-TRE-12 Will STREETLIFE Mixed Reality interfaces improve perception of public

transportation?

HY-TRE6 Users find bus stops and identify buses easier and more accurately than from traditional interfaces

PI-306

PI-310

Mobile app is useful for the users

Type Mobility

Method Focused field experiments, Questionnaire, Mobility manager interview

Metric

Success Criteria

Field trials results indicate users feel more comfortable

RQ-TRE-13 Will STREETLIFE virtual mobility decrease the threshold for using

greener means of travel?

HY-TRE7 Users can understand a presented route in more comprehensive manner than with traditional interfaces

PI-314 User satisfaction

Type Mobility, Commercial feasibility

Method Focused field experiments, Questionnaire, Mobility manager interview

Metric

Success Criteria

Field trials results indicate users feel more comfortable

4.3.6. SCALE OF ENGAGEMENT

Tampere 2nd phase pilot has focus test group who test the gamification. This will involve invitation of friendly users who will get access to the Journey planner with gamification enhancement. Transportation flow management is public, there mobility manager will change the journey planner parameters and potentially this can affect mildly everyone using public transport in Tampere. Similarly Park and Ride experiment can enhance the overall traffic situation in Tampere.



ID Name Size

Scenario TRE-03 Transportation flow management

TRE-3/1 Use Case: Mitigate congestions in main bus stations Public (thousands users) and Authority

Scenario TRE-04 Parking situation control panel

TRE-4/2 Use Case: Increase usage of Park&Ride Public (thousands users) and Authority

Scenario TRE-02 Multimodal real-time Journey Planner improvements

TRE-02-2 Use Case: Journey Planner gamification 50 user

TRE-02/3 Use Case: Mixed Reality 3-10 user



5. REQUIREMENTS ON THE STREETLIFE SYSTEM

There are different types of measures and metrics that need to be acquired during the field trials in order to derive the required performance indicators for evaluation. Different components of the STREETLIFE system will provide valuable data in order to support the hypotheses validation. Therefore direct measure based on sensor log-files will be gathered, stored and post-processed.

The following components are going to provide evaluation relevant data:

a. Tracking

Data from the mobile device of the user can be collected by means of tracking. The data can be utilized for on device travel assistance services like end-of-route detection, as well as for back-end services like mode detection and Gamification. The log data of an itinerary, including e.g. user preferences, proposed trips, viewed trips and the selected trip, can be further be correlated with the tracking data in order to derive relevant information for impact assessment within STREETLIFE (e.g. for the calculation of modal split). The component is implemented as a part of Android code. The proposed solution covers the following data:

- The activity of the user classified by the Android activity recognition. Possible values are: IN_VEHICLE, ON_BYCICLE, ON_FOOT, RUNNING, STILL, TILTING, UNKNOWN, WALKING. These values can e.g. be used as a first approximation for mode detection or for the validation of bicycle trips.

- The position of the user including: altitude, direction, latitude, longitude, speed.

- Radio mast data including: Cell-ID, location area code, primary scrambling code

- Accelerometer, including: minimum, maximum, and average acceleration as well as standard deviation of the average acceleration at fixed intervals

In the actual WP5 deliverable D5.2.2 further details on tracking (Chapter 2.3.2) and the TrackMe App (Chapter 6.2.2), a standalone implementation of the tracking functionalities, can be looked up.

b. CIP Data base

Within the evaluation context, the CIP mainly interfaces between the Berlin STREETLIFE Mobile App and the VMZ Multimodal router and provides backend services for the Bikerider game and other supporting services.

Furthermore, the CIP stores routing- and tracking and game specific data such as mobile app configuration, user information, user game information and user routing information. For users who have chosen not to participate in the game and use only routing features of the STREETLIFE Mobile App, no tracking information will be stored. Otherwise the users’ tracking information will be stored. The CIP also provides the views and aggregation of the data as per requirements of the STREETLIFE Mobile App through the CIP REST Services. These services are consumed by the STREETLIFE Mobile App to store and retrieve data from the CIP Database.



c. Mode Detection

In order to gather valuable mobility data for evaluation purposes information about the used transport mode and especially a change over time can be gathered while using the user tracking data. Different approaches for different purposes have been developed:

For the Berlin Pilot it is necessary to evaluate the travelled mode in order to answer the defined research questions but also for gamification purposes. Here, the calculation of the Green Leaves, virtual trees and the Highscore for the Bikerider Game need to be correctly.A prototype mode detection module will be used for this purpose, which is available from Siemens CT as a tool developed outside the STREETLIFE project. This mode detection module will be implemented as a Backend Service on the CIP platform. It requires the tracking data from the Berlin STREETLIFE App. The data used for mode detection are: t

- Timestamp

- GPS accuracy

- Longitude

- Latitude

- Acceleration (min, max, average)

- Speed (min, max, and average)

On the basis of this, the travelled modes will be shown by trip, distinguishing three categories, i. e.

- Biking

- Motorized Traffic

- Walking.

For the TRE and ROV local situation the mode detection approach is based on incremental increase in accuracy and functionalities. The approach is also hybrid in the sense that not all computations are one either at the Server or at the Client side, but in both. The initial mode detection algorithm is based on the existing ActivityRecognition API of Google Play services that uses sensor data of Android devices and produces probabilities for the current transport mode. The data logger produces the following information:

- Timestamp

- Actual Activity

- Latitude

- Longitude

- Probability for the user remaining STILL

- Probability for the user being ON FOOT



- Probability for the user being WALKING

- Probability for the user being RUNNING

- Probability for the user being ON BICYCLE

- Probability for the user being IN VEHICLE

- TILTING

Since Google mode detection component fails to distinguish between buses and passenger cars as well as trains and trams from other transport modes, the enhanced version is implemented by post-processing the collected data with the help of external data sets and tracking data. Detailed information of the mode detection process can be consulted in D3.2.2 Mobility Data Integration and Techniques, Chapter 7.

d. MMECP

The Mobility Management and Emission Control Panel (MMECP) is a component to support mobility managers in their daily work of handling dynamic traffic situations. It aggregates and analyses urban mobility data that comes from the Siemens CIP. The MMECP uses a map based view to visualise the results of the analyses. For instance, it can show the current parking lot situation in Tampere and a trend per parking lot regarding its occupancy rate. In certain cases it can also be used as a tool to perform actions on current situations, for instance to give recommendations to an event organiser in handling the approach and departure of the visitors to and from the venue location.

Detailed information of the MMECP can be consulted in D4.1.1 und D4.2.1.

e. Gamification Engine

One major objective of our gamification approach is to support and promote behavioural changes among Smart City citizens that are in line with city-defined sustainability policies and KPIs. The Smart Mobility game must thus promote, supervise and incentivise the use of some of some features across those multiple systems, in a uniform way and must be in line with the aforementioned sustainability policies. More detailed information can be consulted in D5.2.2 – End-user applications techniques and tools.



APPENDIX C: LITERATURE

D2.2.1 - STREETLIFE Blueprint Architecture, Security Architecture, and site-specific architectures (initial)

D3.2.2 - STREETLIFE Mobility Data Integration and Techniques

D4.1.1 - STREETLIFE Requirements analysis, mechanism and technology selection, and control panel specification

D4.2.1 - STREETLIFE Mechanisms and tools for mobility management and emission control

D5.2.2 - STREETLIFE End-user applications techniques and tools

D8.1.1 - STREETLIFE Evaluation Plan (Initial)

D8.1.2 - STREETLIFE Achieved Impacts (Initial)

d8.1.2 – evaluation plan (final) - cordis · 2017. 4. 25. · fp7 - 608991 - streetlife d8.1.2...

Documents