deliverable d7.1 title: training curricula

* Dissemination Level: PU= Public, RE= Restricted to a group specified by the Consortium, PP= Restricted to other

program participants (including the Commission services), CO= Confidential, only for

members of the Consortium (including the Commission services)

** Nature of the Deliverable: P= Prototype, R= Report, S= Specification, T= Tool, O= Other

Deliverable D7.1

Title: Training Curricula

Dissemination Level: PU

Nature of the Deliverable: R

Date: 30/09/2019

Distribution: WP5

Editors: UPV

Reviewers: PSNI, PPM

Contributors: UPV, CBRNE

Abstract: The curricula composed by MAGNETO for training of LEA personnel

Funded by the Horizon 2020 Framework

Programme of the European Union

MAGNETO - Grant Agreement 786629

Ref. Ares(2019)5742851 - 13/09/2019

D0.0 Training Curricula

H2020-SEC-12-FCT-2017-786629 MAGNETO Project of 104

Disclaimer

This document contains material, which is copyright of certain MAGNETO consortium parties and may not

be reproduced or copied without permission. The information contained in this document is the

proprietary confidential information of certain MAGNETO consortium parties and may not be disclosed

except in accordance with the consortium agreement.

The commercial use of any information in this document may require a license from the proprietor of that

information.

Neither the MAGNETO consortium as a whole, nor any certain party of the MAGNETO consortium

warrants that the information contained in this document is capable of use, or that use of the information

is free from risk, and accepts no liability for loss or damage suffered by any person using the information.

The contents of this document are the sole responsibility of the MAGNETO consortium and can in no way

be taken to reflect the views of the European Commission.



Revision History

Date Rev. Description Partner

21/03/2019 1.1 Document template UPV

16/04/2019 1.2 First contribution UPV

30/04/2019 1.3 Second contribution ALL

29/05/2019 1.4 Third contribution UPV

25/06/2019 1.5 Fourth contribution UPV

02/07/2019 1.6 Fifth contribution UPV

20/07/2019 1.7 Draft submission document UPV, CBRNE

20/08/2019 - Peer Review PSNI, PPM

01/09/2019 1.8 Compiled version 1 UPV

10/09/2019 1.9 Compiled version 2 UPV



List of Authors

Partner Author

UPV Francisco Pérez, Victor Garrido

CBRNE Tony Godwin, Nigel hale



Table of Contents Revision History ............................................................................................................................................ 3

List of Authors ............................................................................................................................................... 4

Table of Contents .......................................................................................................................................... 5

Index of figures ............................................................................................................................................. 9

Index of tables ............................................................................................................................................. 10

Glossary ....................................................................................................................................................... 11

Executive Summary ..................................................................................................................................... 12

1. Introduction ........................................................................................................................................ 13

2. Questionnaire ..................................................................................................................................... 14

2.1 Introduction ................................................................................................................................ 14

2.2 Questionnaire analysis ................................................................................................................ 14

2.2.1 General Questionnaire analysis .......................................................................................... 14

2.2.2 Background and Experience of Respondents ...................................................................... 14

2.2.3 Technical and Organisational Capabilities .......................................................................... 15

2.2.4 Previous Training ................................................................................................................. 15

2.2.5 Expectations ........................................................................................................................ 15

2.3 Conclusions: Questionnaire ........................................................................................................ 15

3. Training methodology ......................................................................................................................... 17

3.1 Introduction ................................................................................................................................ 17

3.2 Learning and Teaching Strategies ............................................................................................... 17

3.3 Conclusions: Training Activities .................................................................................................. 19

4. Assisted Distance Learning platform .................................................................................................. 22

4.1 Introduction ................................................................................................................................ 22

4.2 Learning Management System (LMS) ......................................................................................... 22

4.3 Conclusions: (Moodle) ................................................................................................................ 25

5. Training evaluation methods and criteria ........................................................................................... 27

5.1 Introduction ................................................................................................................................ 27

5.2 Evaluation methodologies and strategies ................................................................................... 27

5.3 Conclusions: Evaluation methods, criteria and KPI..................................................................... 29

6. Training curriculum definition ............................................................................................................ 33



7. MAGNETO Training Curriculum .......................................................................................................... 35

7.1 Subject definition ........................................................................................................................ 35

7.2 Subjects objectives ...................................................................................................................... 36

7.2.1 Subject: General knowledge ............................................................................................... 36

7.2.2 Subject: Platform configuration and management ............................................................ 37

8. MAGNETO Training Scheduling ........................................................................................................... 43

9. Summary and conclusions .................................................................................................................. 44

10. References ...................................................................................................................................... 45

Appendix A: MAGNETO Training Questionnaire ......................................................................................... 46

MAGNETO End-Users Training Questionnaire ........................................................................................ 48

Introduction ........................................................................................................................................ 48

Data protection statement and disclaimer ......................................................................................... 48

Instructions ......................................................................................................................................... 49

Personal details ................................................................................................................................... 49

Organization details ............................................................................................................................ 53

Questions on previous trainings received........................................................................................... 56

Expectations on the MAGNETO training ............................................................................................. 67

Appendix B: Summary of Law Enforcement Agency (LEA) Training Questionnaire Responses. ................ 72

Introduction ............................................................................................................................................ 75

A note on the Interpretation of Results .............................................................................................. 75

Personnel Details and Experience / Knowledge (Questions 1 to 8) ....................................................... 77

Respondents Age and Experience (Questions 1 to 3) ......................................................................... 77

Previous Training Courses in Forensic Tools (Question 4) .................................................................. 78

Familiarity with Digital Video Analysis (Question 5) ........................................................................... 79

Familiarity with Digital Audio Analysis (Question 6) ........................................................................... 80

Familiarity with Text and Data Mining Topics (Question 7) ................................................................ 81

Familiarity with Augmented Intelligence (Question 8) ....................................................................... 82

Organization Details ................................................................................................................................ 83

Organization Type (Questions 1 & 2) .................................................................................................. 83

Areas of Interest (for your duties) (Questions 3,4,5) .......................................................................... 83

What are the most important capabilities necessary to counter criminal and terrorist activities?

Please select the importance of each activity (Question 6) ............................................................... 84



Did your organization perform field collaborative experience with other organizations (Question 7)?

............................................................................................................................................................ 85

In the affirmative case were such (collaborating) organizations from the same country or different

countries (Question 8). ....................................................................................................................... 85

What have been the major issues regarding national and/or international co-operation /

communication (Question 9)? ............................................................................................................ 86

Which are your current acquisition methodologies on Video, Audio and/or Text analysis (Question

10) ....................................................................................................................................................... 86

Questions on previous training that has been received ......................................................................... 87

Which types of classes did you attend and how satisfied were you with them? (Question 4.1) ....... 87

Which types of training materials were used and how satisfied were you with them? (Question 4.2)

............................................................................................................................................................ 88

Experience with On-Line Platforms (Question 4.3) ............................................................................ 89

Classroom Information (Question 4.4) ............................................................................................... 90

Legal and Ethical Training ................................................................................................................... 92

Evaluation ........................................................................................................................................... 92

Expectations on MAGNETO Training ...................................................................................................... 94

Type of Class ....................................................................................................................................... 94

Materials ............................................................................................................................................. 95

On-Line Platform ................................................................................................................................. 96

Evaluation ........................................................................................................................................... 97

Personal Expectations ......................................................................................................................... 98

Summary Observations and Conclusions. ............................................................................................... 99

Observations ....................................................................................................................................... 99

Conclusions ....................................................................................................................................... 100

Appendix C: Example of curriculum template oriented to a course of MAGNETO tools. ........................ 101

Definition of curriculum subject and specialists ................................................................................... 101

General knowledge ............................................................................................................................... 101

Description ........................................................................................................................................ 101

General Objective: Basic features of the program ........................................................................... 101

General Objective: Advanced features of the program .................................................................... 102

Platform configuration and management ............................................................................................ 102

Description ........................................................................................................................................ 102



General Objective: Basic features of the program ........................................................................... 102

General Objective: Advanced features of the program .................................................................... 102



Index of figures Figure 1. MAGNETO Training Programme user driven approach. .............................................................. 12

Figure 2. Training user driven approach ..................................................................................................... 17

Figure 3. Neuro-Linguistic Programming (NLP) applied to education ........................................................ 18

Figure 4. MAGNETO proposed activities ..................................................................................................... 20

Figure 5. MAGNETO Training methodologies and materials ...................................................................... 22

Figure 6. LMS functionalities ....................................................................................................................... 23

Figure 6. Biggs and Tang analysis of “Tutor and Student perspectives”..................................................... 28

Figure 8. Evaluation criteria and KPI definition .......................................................................................... 30

Figure 9. Definition of a subject as General and Specific objectives .......................................................... 33



Index of tables Table 1. Example of training activities ........................................................................................................ 19

Table 2. Identified Training Tools ................................................................................................................ 20

Table 3. MAGNETO Training Curriculum ..................................................................................................... 35

Table 4. General Knowledge Objectives Session 1 ..................................................................................... 36

Table 5. General Knowledge Objectives Session 2 ..................................................................................... 36

Table 6. General platform Objectives ......................................................................................................... 37

Table 7. Video Analysis Objectives .............................................................................................................. 38

Table 8. Audio Analysis Objectives ............................................................................................................. 38

Table 9. Text analysis Objectives ................................................................................................................ 40

Table 10. Learn the features of the scenarios Objectives........................................................................... 40

Table 11. Equality and non-discrimination Objectives ............................................................................... 41

Table 12. Privacy and data protection Objectives ...................................................................................... 42

Table 13. Criminal procedure law Objectives ............................................................................................. 42

Table 11. Training scheduling ..................................................................................................................... 43



Glossary

ADL Assisted Distance Learning

DDP Deliverable Development Plan

EAB External Advisory Board

EC European Commission

ETHAB Ethics Advisory Board

GA General Assembly

GDPR General Data Protection Regulation

LEA Law Enforcement Agency

KoM Kick-off Meeting

LMS Learning Management System

MoM Minutes of Meeting

PMC Project Management Committee

PO Project Officer

QPR Quarterly Periodic Report

RTF Result Transferability Framework

SAB Security Advisory Board

STAB Scientific and Technical Advisory Board

TPE Threat Prediction Engine

WP Work Package



Executive Summary MAGNETO is a project founded by the European Commission within the H2020 addressing topic SEC-12-

FCT-2017 “Technologies for prevention, investigation, and mitigation in the context of fight against crime

and terrorism”.

Both video or audio recording, and text mining have become a major resource for legal investigations.

Considering the growing complexity, diversity and overlap of the video-surveillance systems, audio

recordings or huge amount of data coming from external databases or digital documents, the MAGNETO

project has the goal of designing and developing a multimedia analysis platform to simplify these

procedures.

In particular, the main goal of the WP7 “MAGNETO’s Training” is to provide MAGNETO end-users, also

known as LEAs (Law Enforcement Agency), the necessary training to go beyond their current multimedia

analysis knowledge and also cover their needs to successfully handle the MAGNETO tools, i.e the

Multimedia Analysis Platform and the different Multimedia tools to successfully follow the MAGNETO

proposed scenarios.

Figure 1. MAGNETO Training Programme user driven approach.

Deliverable D7.1 “Training Curricula” covers the first two steps of the MAGNETO training development,

as seen in the Figure 1. It aims a first step in defining the training methodology for better practice and

knowledge acquisition of each of the skills proposed by the end-users and the technical partners.

Additionally, methods for measuring up to what extent the training goals have been accomplished have

also been defined and will be adapted to the MAGNETO tools.

The second step within the MAGNETO training development is the definition of a training curriculum. This

curriculum will include several subjects covering all the necessary aspects for the end-users in order to

take full advantage of the MAGNETO platform. It will also cover the subjects scheduling and the necessary

prerequisites to access each subject.



1. Introduction This deliverable is divided in 6 Sections, which include the training questionnaire analysis, the training

methodology, the assisted distance-learning platform, the training evaluation methods and criteria, the

training curriculum definition and finally the training scheduling.

In chapter 2, a study was undertaken to understand current end users training and acquisition

methodologies on multimedia analysis. With the provided questionnaire, the LEAs have been directly

involved in helping identifying and also describing the indicators that allow a relevant assessment in terms

of improving the required skills. Based on the study´s results, an ad hoc training methodology was

definied.

Chapter Σφάλμα! Το αρχείο προέλευσης της αναφοράς δεν βρέθηκε. has been dedicated to defining a

training methodology by analizing/observing different methods and environments. Several learning

methods such us face-to-face workshops, digital manuals, webinars and others, have been studied and

selected to be included in the? MAGNETO Training.

Chapter Σφάλμα! Το αρχείο προέλευσης της αναφοράς δεν βρέθηκε. includes the study of a set of

“Assisted Distance Learning” (ADL) platforms or Learning Management Systems (LMS). As a result, a

platform was selected to host the MAGNETO Training Materials. This platform will provide LEAs members

a training environment according their needs and enable them to use the MAGNETO platform(?) and all

available tools. Moreover, the training will also cover legal, ethical and privacy issues.

Chapter 4 identifies a methodology for measuring the goals of the MAGNETO training and also the learning

progress of the LEAs. A step-wise progressive evaluation process will be incorporate to ensure that end

users acquire the knowledge to improve their skills on multimedia analysis and also on using the

MAGNETO platform and tools.

Chapter 6 contains the definition of the term curriculum in order to better understand the following

sections. Furthermore it includes the description of the key terms “subject”, “objective”, “prerequisite”

and “scheduling”, among others.

Finally, chapter 7 includes the proposal of a first draft for the MAGNETO training schedule divided by

natural days. Each day foresees a six hours of training.



2. Questionnaire

2.1 Introduction This chapter presents a summary of the results of questionnaires returned to the MAGNETO project by LEA project partners. The purpose of issuing these questionnaires was to better understand the end-user requirements and needs related to the scope of the training within the MAGNETO Platform. The target group for the questionnaires were analysts since a previous questionnaire presented in Deliverable 2.2 identified them as the key user group for MAGNETO. Detailed responses from the respondents to the questionnaires are presented in a separate spreadsheet which is available to project partners on an as-needed basis. The questionnaire has assisted the training developers to produce relevant training in formats and ways

that are acceptable to the trainees.

In total there were 31 respondents to the questionnaires from ~10 organizations. Most of them were

national and municipal police agencies.

The Training Questionnaire report is attached to this document and can be found in Appendix B.

2.2 Questionnaire analysis Questions presented to the respondents were largely in the form of binary (yes/no) questions. There were

also a number of questions which required the respondent to select from a 5-point scale ranging from

“very dissatisfied” to “very satisfied” or “unimportant” to “critical”.

In these cases (?), to provide a useful indication of the results obtained across the respondents, answers

were processed into percentages using the simple expediency of calculating the sum of all assigned scores

as a percentage of the theoretical maximum. In the text these are referred to as ‘Compound Scores’.

High Compound scores are interpreted as revealing a consistent and strong affinity to the associated item,

whereas a low percentage score indicates a consistent lack of desire to see the item.

2.2.1 General Questionnaire analysis

Respondents represented a good cross section of age, academic qualification, experience and geographic

location. They had attended a significant number of previous training sessions. The implications of this

being that their(?) feedback is drawn from personal experience and technical knowledge of criminal and

counter-terrorism activities as well as forensics. Since these groups represent the target group of users

of the MAGENTO system, the results are hence important.

2.2.2 Background and Experience of Respondents

Video/Audio Analysis: Although respondents indicated familiarity with topics relating to video and audio

analysis, the examples they quoted were somehow limited.

Data Mining: Respondents indicated good familiarity with these topics and provided a good range of

examples.



Augmented intelligence: Familiarity with this topic was limited and this was reflected in the small number

of examples quoted.

2.2.3 Technical and Organisational Capabilities

The responses indicate that technical capabilities are important to the respondents – perhaps more so for

in-office systems than mobile systems although there was also interest in these. Organisations are

dependent upon a range of resources including the need for co-operation and the use of open intelligence.

In this respect it should be pointed out that collaborations often occur, however not without difficulties

arising due to differences in terms of procedures and data formats.

2.2.4 Previous Training

Respondents have experience with a broad range of class types. There is however a clear preference for

‘guided’ classes including discussion and co-operative working features. These denotes that such sessions

are more valued when they include a strong practical aspect. There was less experience with on-line

platforms than with some other training types but these were also well received on the whole. Again, the

need for feedback and interaction was also noticed.

Training within the LEA’s own offices was seen as an option taking into consideration the added benefit

of reducing travel costs and time. Some also saw the benefit of being away from the office for such

trainings. In all cases, the need for good quality training materials and equipment as well as good

‘charismatic’ trainers and engaging materials was clearly pointed out. The ideal teacher to pupil ratio was

determined at 10 to 15 pupils per teacher.

The use of pc’s / computers was seen as important as well as the need of to keeping the ratio of users per

computer to no more than 2 pupils.

Legal and ethical training was seen as being important (by some) an at the same time participants pointed

out that in the past these topics have not been delivered as they should have.

In terms of evaluation of training, there is a lower incidence of essays and the like being used? by the

respondents and there is an associated lower affinity for these as well.

2.2.5 Expectations

Respondent’s expect guided simulation or similar, with well-presented reference material and guide-

books. There is a strong focus on training being practically focussed on real LEA work – if possible (?) using

examples from their own experiences. The use of video’s and on-line systems is not precluded, however

it is not the respondent’s preference – at least not for main-line training, perhaps more likely for initial

briefing and updates.

2.3 Conclusions: Questionnaire The questionnaire reported here has specifically focussed on the needs of analysts rather than on all

potential user groups identified in D2.2 since this group is the most likely to use the full range of

capabilities delivered by MAGNETO. Analysts are a largely office based user group. Hence the conclusions



presented here are focussed mainly on office usage. Although this excludes some of the potential uses of

MAGNETO, such as on mobile systems for example, it is nevertheless considered to be practicable and

appropriate for a system being developed for TRL 6.

There is clearly a strong interest in a wide and diverse range of training modalities, from technical

documents for self-education to face-to-face training. Participants prefer using Webinars for remote

training rather than videos and also value interaction and feedback features.

Noting again that the system is only intended to be TRL6 and that the training itself can therefore not be

fully developed within the project, a mixed Economy of training methodologies – i.e. one that recognises

the project’s finite budget and the practicalities of getting together groups of analysts from up to ten LEAs

across Europe – is recommended. In all cases there should be a strong focus on practical real-life

examples.



3. Training methodology

3.1 Introduction Magneto training methodology is developed in order to meet the end users training needs on multimedia

analysis. This methodology will establish suitable training environments for increasing the knowledge

acquisition on the use of Magneto tools.

To have a clear idea about the selected methodologies, it is important to comprehend the process of

learning. The first subsection will cover this topic.

3.2 Learning and Teaching Strategies The old-style seminar lessons where the tutor just explains or lectures to their students are no longer

considered to be effective. This was an important aspect to be considered in the evolution process since

the estimated student attention span for the teaching style, “just sit there and listen” is of less than half

an hour.

Recent studies [1], have demonstrated that learning is a circular process. As David A. Kolb [2] pioneered

the idea of experiential learning, which is learning by reflection on doing experiential learning’s focus is

the individual learning process.

Figure 2. Training user driven approach



Figure 2 demonstrated the 4MAT model or the way to label the four types of learners:

• WHY …? Need reason or relevance

• WHAT FOR …? Need additional information and usefulness

• HOW …? Need training to comprehend

• WHAT IF …? Need to investigate it themselves or teach others

As David A. KOLD[2] explains, the best learning (results?) is achieved when performing the displayed

training repeatedly.

Therefore, as a tutor/trainer, one must consider the specific order set out in the diagram while applying

the 4MAT model to follow(?). In summary, in order to keep the attention of the “WHY student”, first the

explanation of “WHY the topic at hand is important” has to first be done. Next, the tutor has to centre on

the “WHAT” and “HOW students”, as they are more patient. The other (who?) between these two comes

from it is also logical to first explain the theory (WHAT) and then the application process (HOW). Finally,

the “WHAT IF students” are rather interested in obtaining more information and therefore need to be

placed at last.

Following on from the topics of communication methods and learning mechanisms, it is necessary to

introduce the Neuro-Linguistics Programming (NLP) models. By using the techniques provided by the NLP

models, it is possible to improve the communication between tutors and students. Therefore, the training

has to implement several communication channels and not just the traditional ones used in seminar

lectures, i.e. auditory word elements but also visuals and practical ones. Considering these aspects, the

classic NLP model proposes the following pyramid for subdividing the learning experience.

Neuro-Linguistics Programming

Students remember

Figure 3. Neuro-Linguistic Programming (NLP) applied to education



Figure 3 demonstrates how the NLP can be applied to educations (the education field/ education?). It is

important to notice that seeing (50%) or seeing and hearing (50%) are fairly effective, but doing or

teaching (90%) however is the best experience for the students to learn and remember a specific topic.

Also, when implementing the learning experience pyramid, the teaching procedure should be arranged

as follows: “First see how something is done, then do it yourself, and, finally, teach it to others”.

3.3 Conclusions: Training Activities Considering the knowledge gained from the previous section, it is recommended to adopt a variety of

training activities to reach students, and to do so in a diverse manner. In summary, it is important that

enough TO DO (TEACH), WRITE (SAY), SEE, HEAR and then READ are included in the learning experience.

As such, the number of methods that can be adopted is quite high. The next table shows some examples.

Table 1. Example of training activities

DO (TEACH) WRITE (SAY) SEE HEAR READ

Small Groups X X X X

Simulations X X X

Case Study X

Practice by doing X

Exercises X X

Sharing Experiences X X X

Group Discussions X X X

Brainstorm X X

Teaching others X

Thinking time X

Demos X X

Lectures X

Further Reading X X

Unfortunately, the scope of the MAGNETO project does not give enough room for implementing all the

above-mentioned activities. There is also the requirement of delivering an efficient and cost effective

training programme that can be delivered (applied?) across multiple Law Enforcement Agencies.

Combining the user-selected preferences (see previous chapter Questionnaire



3.4 Introduction This chapter presents a summary of the results of questionnaires returned to the MAGNETO project by LEA project partners. The purpose of issuing these questionnaires was to better understand the end-user requirements and needs related to the scope of the training within the MAGNETO Platform. The target group for the questionnaires were analysts since a previous questionnaire presented in Deliverable 2.2 identified them as the key user group for MAGNETO. Detailed responses from the respondents to the questionnaires are presented in a separate spreadsheet which is available to project partners on an as-needed basis. The questionnaire has assisted the training developers to produce relevant training in formats and ways


In total there were 31 respondents to the questionnaires from ~10 organizations. Most of them were

national and municipal police agencies.

The Training Questionnaire report is attached to this document and can be found in Appendix B.

3.5 Questionnaire analysis Questions presented to the respondents were largely in the form of binary (yes/no) questions. There were

also a number of questions which required the respondent to select from a 5-point scale ranging from

“very dissatisfied” to “very satisfied” or “unimportant” to “critical”.

In these cases (?), to provide a useful indication of the results obtained across the respondents, answers

were processed into percentages using the simple expediency of calculating the sum of all assigned scores

as a percentage of the theoretical maximum. In the text these are referred to as ‘Compound Scores’.


whereas a low percentage score indicates a consistent lack of desire to see the item.

3.5.1 General Questionnaire analysis


location. They had attended a significant number of previous training sessions. The implications of this

being that their(?) feedback is drawn from personal experience and technical knowledge of criminal and

counter-terrorism activities as well as forensics. Since these groups represent the target group of users

of the MAGENTO system, the results are hence important.

3.5.2 Background and Experience of Respondents


analysis, the examples they quoted were somehow limited.


examples.

Augmented intelligence: Familiarity with this topic was limited and this was reflected in the small number

of examples quoted.



3.5.3 Technical and Organisational Capabilities




In this respect it should be pointed out that collaborations often occur, however not without difficulties

arising due to differences in terms of procedures and data formats.

3.5.4 Previous Training

Respondents have experience with a broad range of class types. There is however a clear preference for

‘guided’ classes including discussion and co-operative working features. These denotes that such sessions

are more valued when they include a strong practical aspect. There was less experience with on-line

platforms than with some other training types but these were also well received on the whole. Again, the

need for feedback and interaction was also noticed.

Training within the LEA’s own offices was seen as an option taking into consideration the added benefit

of reducing travel costs and time. Some also saw the benefit of being away from the office for such

trainings. In all cases, the need for good quality training materials and equipment as well as good

‘charismatic’ trainers and engaging materials was clearly pointed out. The ideal teacher to pupil ratio was

determined at 10 to 15 pupils per teacher.

The use of pc’s / computers was seen as important as well as the need of to keeping the ratio of users per

computer to no more than 2 pupils.

Legal and ethical training was seen as being important (by some) an at the same time participants pointed

out that in the past these topics have not been delivered as they should have.

In terms of evaluation of training, there is a lower incidence of essays and the like being used? by the


3.5.5 Expectations

Respondent’s expect guided simulation or similar, with well-presented reference material and guide-

books. There is a strong focus on training being practically focussed on real LEA work – if possible (?) using

examples from their own experiences. The use of video’s and on-line systems is not precluded, however

it is not the respondent’s preference – at least not for main-line training, perhaps more likely for initial


3.6 Conclusions: Questionnaire The questionnaire reported here has specifically focussed on the needs of analysts rather than on all

potential user groups identified in D2.2 since this group is the most likely to use the full range of

capabilities delivered by MAGNETO. Analysts are a largely office based user group. Hence the conclusions

presented here are focussed mainly on office usage. Although this excludes some of the potential uses of



MAGNETO, such as on mobile systems for example, it is nevertheless considered to be practicable and

appropriate for a system being developed for TRL 6.

There is clearly a strong interest in a wide and diverse range of training modalities, from technical

documents for self-education to face-to-face training. Participants prefer using Webinars for remote

training rather than videos and also value interaction and feedback features.

Noting again that the system is only intended to be TRL6 and that the training itself can therefore not be




examples.

) with the materials and the identified needs regarding the trained tools (see Table 2), it was decided to

implement the following methodologies for the MAGNETO project.

Table 2. Identified Training Tools

Training tool WP.

Ref

Training required Technical

manual

Webinars F2F workshops

External Interfaces WP6 Y X

Background Processing WP3 N

Distinctive Region and Pattern WP3 Y X x

Heterogeneous Data analysis WP3 Y X

Spatio-temporal search of DROP features

WP3 Y X X

Data Storage and Processing WP3 Y X X

Text Translation WP3 Y X X

Semantic information processing interface

WP4 Y X X

Information fusion WP4 Y X X

Evidence discovery WP4 Y X X

Data interaction WP4 Y X X

Situation awareness WP5 Y X X X

Courtproof-awareness WP5 Y X x X

Taking into account the selected methodologies, the training materials could include PDF technical

manuals, on-line presentations through and on-line platform (Webinar) and face-to-face workshops (F2F).

It is important to notice that in parallel to the development of MAGNETO tools, the MAGNETO Training

contents will be simultaneously generated. Hence some of the selected activities could change in the

future.



Figure 4. MAGNETO proposed activities

Regarding MAGNETO F2F workshops, locations or dates have not yet been decided. The purpose of these

workshops is to resolve any issues identified, which are not covered by either technical manuals or

webinars.



4. Assisted Distance Learning platform

4.1 Introduction Due to the international participation in MAGNETO, all the training material will be hosted in an online

learning platform known as a Learning Management System (LMS), with the aim of making easy for the

end-user to access and benefit from it.

Figure 5. MAGNETO Training methodologies and materials

The first fully featured Learning Management Systems (LMS) called EKKO was introduced in 1991. It was

developed and released by Norway’s NKI Distance Education Network. Later on,, New Brunswick’s NB

Learning Network (Rutgers University, NJ, EEUU) presented their own system. In the last 30 years, the

industry has experienced rapid growth, increasing the number of LMS from 15 to more than 700 vendors

on the market nowadays.

4.2 Learning Management System (LMS) LMS platforms are virtual learning spaces created to facilitate the distance training experience, both for

educational institutions and companies. The system chosen allows the creation of “virtual classrooms”,

where the interaction between tutors and students takes place. Users can also undertake evaluations,

exchange files and participate in forums and chats as well as many other additional tools.

Below are listed some of the benefits of LMS platform:

▪ Study at any place and time: this solves the problem of geographic or temporal distances and

offers great freedom in terms of time and pace of learning.

▪ Cheap and flexible way to expand the training.

▪ User friendly application: no great previous knowledge is required in order to use it.

▪ Constant updated learning through the interaction between tutors and students.



Figure 6. LMS functionalities

The basic components or characteristics of any virtual environment, which must also be strongly linked

and interconnected in such a way that they influence each other and provide feedback, can be

summarized in the following:

▪ Centralization and automation of learning management.

▪ Flexibility: the platform can be adapted to the study plans of the institution as well as to the

contents and pedagogical style of the organization. It also allows the trainer to quickly and easily

organize courses.

▪ Interactivity: the user becomes the protagonist of her/his own learning experience through self-

service(?) and self-guided services.

▪ Standardization: this feature allows using courses made by third parties, personalizing the content

and reusing the knowledge.

▪ Scalability: these resources can work with a variable number of users according to the needs of

the organization.

▪ Functionality: features and features(?) that make each platform suitable (functional) according to

the requirements and needs of each individual user.

▪ Usability: ease with which people can use the platform in order to achieve a specific goal.(?)

▪ Ubiquity: ability of a platform to generate peace of mind for the users and make him certain that

everything he needs will be found in that virtual environment. (?)

▪ Integration: the LMS platform must be able to integrate with other business applications used by

human resources and accounting allowing thereby measuring the impact, efficiency, and above

all, the cost of training activities.

Once the basic characteristics of a LMS have been determined, the next step is to (will be?) decide which

LMS to use from all the available ones. One of the main considerations when choosing a LMS platform is

the cost aspect. According to that, LMS platforms are divided into two main types: licensed LMS and Free

GPL platforms open for educational purposes?. However, there are many other important aspects to take



into consideration too when choosing a LMS. Hence deciding which platform best suits the user´s needs

can become a complex task.

Comparing suppliers can be also a demanding task due to the high number of options available. High-end

options do not always guarantee quality and some solutions which offer quick and easy-to-install versions

with basic functionalities, may not meet the organization´s needs in the future.

In summary, although each application is different, there are a number of common factors:

▪ Licensed or free of use

▪ Evaluation of functionality and intention of use

▪ Infrastructure

▪ Contents: activities, configuration of the exercises, interface, interaction, compatibility and

maintenance.

Below, there is a brief description of the LMS platforms that were documented during the research phase.

LMS under license

▪ Blackboard

Blackboard stands out among the proprietary systems. One of the main advantages lies in the great

flexibility it offers due to the many options available for students to learn according to their own style

and rhythm. Blackboard’s work philosophy is very ambitious and its developers team aims at working

together with students and trainers to continuously improve it.

Open LMS

▪ Sakai

Sakai is an international community that collaborates to create technology that improves teaching,

learning and research. The management Committee of the Sakai project is formed by a group of

individuals belonging to various institutions (universities, schools, government, hospitals, etc.) that

provide the necessary leadership for the good direction of the project.

▪ Moodle

Moodle is a project directed and coordinated by an Australian organization of 30 developers, which

is supported financially by a worldwide network of about 60 companies. The learning platform is

designed to provide educators, administrators and students with a unique, robust and secure

integrated system to create personalized learning environments. Moodle can be downloaded on the

web server itself and it offers an assistance service from a Moodle partner. Moodle is the most used

learning platform in the world. It has currently more than 65 million users worldwide.



4.3 Conclusions: (Moodle) As a result of an exhaustive study on the various LMS, it was concluded that in general both open LMS

(Sakai and Moodle) platforms support the needed features for this project. At the end we decide to choose

Moodle due to its large community of active contributors with the aim of continuously improve and

optimize the platform and also due to its large number of modules available.

As previously explained, Moodle is an open source LMS that supports the creation of fully customizable

learning environment. It was designed to cover the needs of educators, students and administrators. It

is a web-based platform developed in PHP technology and MySQL databases.

Moodle provides a large number of features making it the most used LMS in the world:

▪ Solid and confident: Moodle is used for all type of organizations to manage their learning projects

online.

▪ Easy to use: it provides a simple interface to navigate and management the contents with a drag-

and-drop functionality.

▪ Flexible and customizable: as an open source with modular framework it can be customized and

adapted to the individual needs.

▪ Collaborative tools and activities: students and educators can interact and contribute through

tools like chat, forums, surveys, etc.

▪ Always updated by a large community to be aware the requirements of the users (?).

▪ Ubiquitous and multi-platform: the access to the platform is provided by web and its interface is

responsive. Thus it is possible to connect to it by any device or place.

▪ Robust, safe and private: safety controls of the platform controls are updated constantly in order

to protect the privacy and data integrity. For this purpose, Moodle includes system protection

against unauthorized access, data loss or fraudulent use.

The tools available in the platform can be divided into two large groups:

- Resources: used to present information to students

- Activities: used to build to build community and apply the knowledge learned

The most important tools are:

▪ Blackboard Collaborate Classrooms: is a virtual classroom system that allows for complex online

interaction.

▪ Blog: an online journal with content sorted chronologically. Blogs can be groups or an individual

effort(?).

▪ Calendar: all-in-one calendar that can display all the events needs in the course created by

students, administrators or educators.

▪ Chat: real-time discussions for students where they can directly interact with educators.

▪ Database: students and educators collaborate to build a searchable bank of record entries

relating to any topic.

▪ Feedback: students can give their impressions and opinions through surveys, anonymous or not.



▪ Lesson: create programmed learning units that allows tutors to compose the different courses.

▪ Quiz: different types of quizzes to evaluate the knowledge learned.

• Reports: a helpful tool to provide educators extra information on the use of their courses through

logs, live logs, activity report, course participation and statistics .



5. Training evaluation methods and criteria

5.1 Introduction In the final phase of the Magneto project a gradual evaluation process will be established in order to

ensure that the LEAs have acquired the required knowledge for improving their skills on multimedia

analysis.

The first subsection provides an overview of the different evaluation methodologies and strategies in

order to a better understand the selected evaluation criteria.

5.2 Evaluation methodologies and strategies Evaluation is a topic which, in the past, has generated considerable discussion and caused significant

controversy. However, it seems that there is some consensus: evaluation is necessary in order to estimate

or measure the development of a process whose final result is strongly determined by the assessment of

its own stages. Therefore, the evaluation is an essential instrument in the development of the teaching

function.

With regard to the activities, the evaluation is a good indicator, which guides the path of “know-how”,

pointing out alternative options in the search of consequent and operational solutions.

The main characteristics of a good evaluation are as follows:

▪ Explicit

There should be no doubts about what is being evaluated.

▪ Grounded

The necessary reasons must be gathered, so they support the assessment according to the

purposes and objectives of both, the work and the student.

▪ Transparent

Students must know how they will be evaluated as how it will be done.

▪ Additive

The questions should be preferentially independent and contribute separately to the total

performance.

In conclusion, there are no “right” or “wrong” evaluation methods, but there are better and worse ones,

or at least more useful and less useful ones.

Another important aspect to discuss regarding the evaluation methods is the “assessments-objective

alignment”.



Figure 7. Biggs and Tang analysis of “Tutor and Student perspectives”

As Biggs and Tang said in 2009 [7], usually the objectives of the students and the tutors are not the same.

As seen in the Figure 7, the first concern of the tutor is to achieve some objectives about the course.

Subsequently, they perform the teaching activities and at the end an evaluation of the student. On the

other hand, students are concerned about their evaluation and how to achieve a score for their tests

through learning activities. Thus, it is necessary to align the assessment with the objectives of the course.

Once the objectives have been aligned with the evaluation, the tutor has to include a feedback system

that allows him to change/ adapt the training accordingly. A continuous evaluation methodology is

adopted for this purpose. Since the introduction of the Bologna Process in 1999, there are many European

universities that have included this revolutionary system. Through Bologna the student’s evaluation in

which they were evaluated through regular tests of knowledge assimilation, was abolished. By this, tutors

assume the role of counsellor and assess the student’s learning process not merely by sanctioning their

results but by continuously monitoring their work, and hence helping them meeting their objectives.

Furthermore, this system also considers the “student’s competences”, that it is to say, that the students

are capable to demonstrate comtpetency in specific areas.(?)

Thus, although final exams may be utilized, they will not be the unique evaluation method. Therefore, the

procedures designed to facilitate monitoring the student´s learning progress during their academic years

were varied(?).

Several mechanisms are found in the education literature to support continuous evaluation:

▪ Tutor Evaluation

Historically, evaluation has been almost the exclusive preserve of the tutor, who, based on his or

her experience, issues an “authorized opinion”, which allows the student to be placed within an

evaluative framework, linked to the “assignment” or previously established request. This is the

most know evaluation method and is still most of the common ones.

▪ Auto-Evaluation

In the last few years, the concepts of self-evaluation and co-evaluation processes were developed.

The idea behind this is for the students to make judgments of value to their own work, as well as

that of their peers. To guide the students in this task usually questionnaires are especially

designed for this purpose. This situation requires the students, among other things, to



demonstrate objectivity and equanimity, appealing to conscience and maturity. The same

operation can be implemented with groups, where the evaluation was focused on the teammates.

In most common cases, objectivity is not threatened, and sometimes can result in downgrade

evaluation.

▪ Co-Evaluation

By sharing, the affirmation, that the ultimate function of teaching is to achieve the “autonomy”

of the student, one would also agree that what is relevant within the learning system, would also

be to involve the student through their self-criticism, introducing into the trial process their own

self-assessment.

Finally, the possibility of providing different assessment states for the same work or developing exercises

(self-evaluation, cooperative evaluation and tutor evaluation) adds a greater certainty of appreciation of

the progress achieved and allows crossing data and information.

5.3 Conclusions: Evaluation methods, criteria and KPI The key objective is to incorporate in the Magneto training similar evaluation methods applied to other

universities and professional (i.e. CISCO, Palo Alto, etc.) courses performed in the UPV. The preferred

evaluation method will the Test/Quiz/Exam, but as implied previously the “final exam” will be only one

part of the total evaluation.

At the end of each topic (technical manuals and webinars), the user will have the chance to do a no-graded

small quiz. The answers to the test will be given at the end of it. This quiz can be done several times and

serves as an auto-evaluation mechanism for the student to measure their understanding and learn the

most important content of a subject.

Then, at the end of each chapter there will be a graded test. that will cover only the knowledge of this

chapter. In this case the test can be done twice since the answers will not be provided.

At the end of the training, and once the students have passed the chapter tests, they will have to pass a

final test that will gather questions from all the previous ones. The students will only receive the grade at

the end. This test can be undertaken only once.

The summative evaluation will take into account the results of the chapter tests and final test. Each part

will cover a percentage of the final grade, 40% to 60%, respectively. In addition, each individual test will

need to be passed with a score of at least 50% to successfully complete the course.



Figure 8. Evaluation criteria and KPI definition

Following the scheme depicted in Figure 8 and considering the available information on the features and

design of the Magneto tools, the KPI can be categorized and defined as follows:

General Objective

Improve the current knowledge on heterogeneous data mining techniques of the MAGNETO LEA’s

▪ Specific Objective

o Gain knowledge to properly use the MAGNETO Distinctive Region and Pattern tool

▪ Configure and manage the basic features of the software

KPI: 80% students pass with a grade greater than 6/10

▪ Configure and manage the extended features of the software



o Improve the background knowledge on Heterogeneous Data analysis tool

▪ Understand basic heterogeneous data analysis concepts applied to the software



o Improve data storage knowledge

▪ Learn how to store, modify, and delete files properly on the platform



o Gain knowledge to properly use the MAGNETO Text Translation tool







General Objective

Improve the current knowledge advanced (?) semantic reasoning of the MAGNETO LEA’s


o Gain knowledge to properly use the MAGNETO Semantic Information Processing tool






o Gain knowledge to properly use the MAGNETO Information Fusion tool






o Improve the background knowledge on Evidence Discovery techniques

▪ Understand evidence discovery techniques applied to the software



o Improve the background knowledge on Data Interaction

▪ Learn to correctly interpret pre-processed data


General Objective

Improve the current knowledge on augmented intelligence of the MAGNETO LEA’s


o Gain knowledge on Situation Awareness

▪ Learn to read an interpret geographic information


▪ Learn to manage, represent and visualize data from the different sources



o Gain knowledge on managing Court-proof Evidences

▪ Learn how to understand and manage control files



o Gain knowledge on integration and interoperability

▪ Learn how to integrate and interoperate the MAGNETO platform with other

systems and data sources





o Improve the knowledge on the legal procedures, privacy and ethical principles to ensure

it will be accomplished during the MAGNETO project

▪ Understand evidence discovery techniques applied to the software



o The training achieves an overall good satisfaction level from the students

▪ Survey passed to the students

KPI: 80% of the students are satisfied with the used training methodology

KPI: 80% of the students consider the materials adequate to the training

KPI: 80% of the students would recommend this training experience

Prior KPIs have been defined at an early stage considering a general approach. However, these KPIs

will be re-evaluated by the time the tools will be implemented and available for analysis.



6. Training curriculum definition A Curriculum is the subjects comprising a course of study in a training process. The curriculum depicts the

sequence of steps, which will guide the coaches and trainees to the education target (learning goals?).

The curriculum describes the subjects, objectives and the necessary evaluating processes required. In

addition, the interaction between coaches and trainees would (will?) be incorporated, as well as the

materials, content and resources required.

The following items should be included in the curriculum:

• Number of subjects

• Order of the subjects

• Timing of all subjects

• Mentor of each subject

Therefore, the entire curriculum of training will cover several subjects, taking into account the

requirements and the increasing difficulty. In addition, it will be developed according to the necessary

time for each subject to examine all the material, exercises and prepare the evaluation. This is necessary

in order to create a schedule that is fit for purpose. Finally, a subject specialist (Mentor) will be assigned

to each subject in order to be a point of contact for concepts that the main coach could not handle

(during…?). The tutor will be responsible for recording issues to be resolved through weekly dialogue with

the specialist.

Figure 9. Definition of a subject as General and Specific objectives

The subject’s composition is showed in Figure 9. Each subject is divided into several general objectives.

Each of these general objectives could do refer to general topics. There can be as many general objectives

as required. For example, in a Math subject a general objective would be “Learn basic statistics”.



At the same time, each general objective could (can?) be sub divided into specific objectives. Specific

objectives are created to specify only one single task. Following the same example, in this case, a specific

objective could be “Learn to divide two numbers”.

Specific objectives could have a set of prerequisites. Sometimes it is necessary to gain additional

knowledge in order to understand the topic. For example, in order to resolve the specific objective “Learn

to divide two numbers”, first the student would need to “Learn to add two numbers”.

Finally, each specific objective would have a determinate method to be evaluated. For example, in the

last examples, the best way could be to pass a test.



7. MAGNETO Training Curriculum

7.1 Subject definition This section will cover the MAGNETO training curriculum. It is important to remember that this curriculum

may change in the following months while the Multimedia Analysis Platform and its key features are

further developed.

Table 3. MAGNETO Training Curriculum

No. Subject Name Prerequisite Mentor (Subject

Specialist)

Mentor

(Company)

Duration

1 General knowledge Not required Victor Garrido UPV 12 h

2 Platform configuration

and management

Subject 1 Francisco Pérez UPV 6 h

3 Video Analysis Subject 1 – 2 Francisco Pérez UPV 6 h

4 Audio Analysis Subject 1 – 2 Francisco Pérez UPV 6 h

5 Text Analysis Subject 1 – 2 Francisco Pérez UPV 6 h

6 Scenarios training Subject 1 – 2 Francisco Pérez UPV 18 h

7 Legal, ethical and

social

Not required Francisco Pérez UPV 12 h

Table 3 shows the seven subjects that have been identified for the MAGNETO training. They can divided

into three parts. The first one will incorporate the necessary background knowledge in order to continue

with the next part of the training (Subject 1). The second part encompasses all the subjects that cover the

Multimedia Analysis Platform and its tools (Subject 2 – 6). Finally, the last part, will provide the necessary

legal, ethical and social considerations (Subject 7).



7.2 Subjects objectives The subjects have already been defined, and now we have (we will?) to determinate their objectives, both

general and specific. The consideration (?) will contain objective, prerequisites and evaluation method.

7.2.1 Subject: General knowledge

This subject will include all the minimum necessary knowledge that is required in order to create the first

steps in the platform. For example, it will include the technical vocabulary, the use of encoders and data

formats as well as the most common (most commonly used) audio, video and text processing.

General Objective 1.1. – Learn the basic principles in multimedia tools.

This general objective encompasses the main principles in multimedia technology. For example, technical

vocabulary and different kinds of formats and encoders.

Table 4. General Knowledge Objectives Session 1

No. Specific Objectives Prerequisites Evaluation method

1.1.0 Multimedia Vocabulary Not required Quiz

1.1.1 Standard multimedia formats Not required Quiz

1.1.2 Basic video Not required Exercise

1.1.3 Basic audio Not required Exercise

1.1.4 Basic Text Not required Exercise

1.1.5 Upload files Not required Exercise

1.1.6 Basic graphs Not required Exercise

General Objective 1.2. – Learn the advanced principles in multimedia tools.

After learning the basics, this general objective covers more advanced principles in multimedia

technology. In this part, the student will interact with the multimedia data, common good practices and

format conversions.

Table 5. General Knowledge Objectives Session 2


1.2.0 Specificity of multimedia recordings General Objective 1.1 Quiz

1.2.1 Map orientation General Objective 1.1 Exercise



1.2.2 Basic scenarios General Objective 1.1 Exercise

1.2.3 Advanced functions General Objective 1.1 Exercise/Quiz

1.2.5 Basic architecture interactions 1 General Objective 1.1 Exercise/Quiz

7.2.2 Subject: Platform configuration and management

The MAGNETO Multimedia Analysis Platform is the center of MAGNETO training curricula. This tool covers

the main functionalities and will provide all the analysis tools developed in the project.

General Objective 2.1. – Learn the general functions of the platform.

This general target focuses on the core functionalities of the Multimedia Analysis Platform such as basic

programming skills, architecture design, BBDD management.

Table 6. General platform Objectives


2.1.1 Basic architecture interactions 2 General knowledge subject Exercise

2.1.2 BBDD NoSQL management General knowledge subject Exercise/Quiz

2.1.3 Basic Docker composition General knowledge subject Quiz

2.1.4 Pub/Sub platform approach General knowledge subject Exercise/Quiz

2.1.5 Advanced functions General knowledge subject Exercise

7.2.3. Subject: Video Analysis

This subject explains one of the analysis tools included in the Multimedia Analysis Platform. The video

analysis tool is a set of task related modules that teach the users how to analyze the video content

according to different properties such as patterns detection, face recognition and behavior identification

among others.

General Objective 3.1. – Learn the features of the video analysis tool.

The general goal (of?) is intended to teach the students how to use the MAGNETO video-based and video

synchronization.



Table 7. Video Analysis Objectives


3.1.1 Input of media data into

MAGNETO System

Exercise

3.1.2 Initiate video analysis modules to

extract relevant information

Spec. Objective 3.1.1 Exercise

3.1.3 Check results in User-Interface Spec. Objective 3.1.2 Exercise/Quiz

7.2.4. Subject: Audio Analysis

This subject explains one of the analysis tools included in the Multimedia Analysis Platform. The audio

analysis tool is a set of task related modules to analyze the audio content according different properties.

The main module detects the text that contains the audio, and transcribes it.

General Objective 4.1. – Learn the features of the audio analysis tool.

The general goal (of?) is intended to teach the students how to use the MAGNETO audio-based and audio

transcription.

Table 8. Audio Analysis Objectives


4.1.1 Input audio files into MAGNETO

System

Exercise

4.1.2 Initiate audio analysis modules to




7.2.5. Subject: Text Analysis

This subject explains one of the analysis tools included in the Multimedia Analysis Platform. The text

analysis tool is a set of task related modules that teach the users how to analyze the text content according

to different properties. One module detects keywords, another module could detect the sentimental

analysis.

General Objective 5.1. – Learn the features of the video analysis tool.

The general goal (of?) is intended to teach the students how to use the MAGNETO text-based and text

analysis.



Table 9. Text analysis Objectives


5.1.1 Input text data into MAGNETO

System

Exercise

5.1.2 Initiate text analysis modules to




7.2.6. Subject: Scenarios training

This subject explains the different existing scenarios and how the MAGNETO platform and tools can

improve the daily tasks.

General Objective 6.1. – Learn the features of the scenarios.

The general goal is to follow the real proposed scenarios using the MAGNETO platform and tools and

check how the proposed solution can improve the daily tasks of the LEAs.

Table 10. Learn the features of the scenarios Objectives


6.1.1 Scenario 1: UC1 & UC3 Riots and

terrorist attacks

Subjects 1-5 Exercise

6.1.2 Scenario 2 UC2 & UC4 Ilegal

economic transactions

Subjects 1-5 Exercise

6.1.3 Scenario 3 UC5 Identity crimes Subjects 1-5 Exercise

Subject: Legal, ethical and social

The objective of the ethical and legal considerations module is to provide training and orientation to the

end-users of the MAGNETO Multimedia Analysis Platform and analytics tools regarding to the ethical and

legal limits of their deployment. The classes will teach how the tools(?) (container) can be deployed

respecting ethical and societal values, and their development into the law (?). However, the functionality

and contributions of the tools to law enforcement needs can still be retained (?). This part (of the training

(?) ) will focus on three main areas: non-discrimination, privacy and data protection as well as criminal

procedure law.



General Objective 7.1. – Equality and Non-Discrimination.

The course on Equality and Non-Discrimination teaches Magneto users on equality and non-discrimination

matters when using the Magneto Multimedia Analysis Platform and analytics tools. Users begin the course

with an overview of the technical notion of equality as a principle of justice and the fundamental rights to

non-discrimination and equality, followed by EU equality principles pertinent to law enforcement

activities. The course also teaches from an equality perspective, how the deployment of analytics modules

can lead to unfair forms of differentiation and decision-making. On the other hand, it also explains how

discrimination can result out of the different stages of the analytics process.

Finally, the course will approach how to avoid ethnic profiling. According to international human rights

law, ethnic profiling is forbidden. Therefore, it is important for LEAs, and law enforcement data analysts

in particular, to gain a strong knowledge base on this topics too. The course will teach how specific

characteristics could be searched within multimedia data bases.

Table 11. Equality and non-discrimination Objectives


7.1.1 The Fundamental Right to Equality and

Non-Discrimination

Not required Quiz

7.1.2 Law Enforcement and EU Non-

Discrimination Law

Spec. Objective 7.1.1 Quiz

7.1.3 How Algorithms Differentiate Fair and

Unfair Forms of Differentiation


7.1.4 Ethnic and Discrimination Profiling: How to

Avoid Ethnic Profiling


General Objective 7.2. – Privacy and Data Protection.

The objectives of the privacy and data protection course are to inform and teach LEAs about the

fundamental rights to privacy and data protection, the General Data Protection Regulation and the Law

Enforcement Directive. This (?) is another key requirement as the collection and processing of personal

data are an integral component in the use of the MAGNETO Multimedia Platform and analytics tools.

The course will also address how these rights have shaped current EU data protection legislation, for

example such as the GDPR.



The course will primarily focus on the EU legal framework and consider the ethics and illegality of mass-

surveillance.

Table 12. Privacy and data protection Objectives


7.2.1 The Fundamental Rights to Privacy and Data

Protection and EU Data Protection Law

Not required Quiz

7.2.2 EU data protection law: key definitions Spec. Objective 7.2.1 Quiz

7.2.3 EU Data Protection Law: Data Quality Principles Spec. Objective 7.2.2 Quiz

7.2.4 Specific Data Protection Requirements for LEAs Spec. Objective 7.2.3 Quiz

7.2.5 Biometrics, Automated Decision-Making

(Profiling) and EU Data Protection Legislation


7.2.6 The ethics and illegality of Mass-Surveillance Spec. Objective 7.2.5 Quiz

General Objective 7.3. – Criminal Procedure Law.

The training module aims to teach the users of MAGNETO modules about the basic principles of criminal

procedure law. Although criminal procedure law is regulated primarily on the national level, these

courses will be provided on a multi-national level. For this reason, the first part will mainly focus on the

basic international principles of criminal procedure law and the right to fair trial in particular. However,

throughout the course and where relevant (?), national examples of criminal procedure law will be

provided. Users are advised to first follow the privacy and data protection course.

Table 13. Criminal procedure law Objectives


7.3.1 International Principles of Criminal Procedure

Law (The Right to Fair Trial)

Not required Quiz

7.3.2 Data Collection from multimedia sources Spec. Objective 7.3.1 Quiz

7.3.3 The legality of multimedia data as evidence in

court






8. MAGNETO Training Scheduling

The MAGNETO training has been scheduled to be completed in less than three working weeks (this is only a first proposal). That means 13 days of 6 work hours

a day. the multimedia analysis Users can skip the first lessons if they have already basic knowledge in multimedia analysis.

Table 14. Training scheduling

D1 D2 D3 D4 D5 D6 D7 D8 D9 D10 D11 D12 D13 D14 D15 D16 D17 D18 D19 D20 D21

General

Knowledge x x WE WE WE WE WE WE

Platform

Configuration and

management

x WE WE WE WE WE WE

Video Analysis x WE WE WE WE WE WE

Audio Analysis x WE WE WE WE WE WE

Text Analysis WE WE x WE WE WE WE

Scenarios training WE WE x x x WE WE WE WE

Legal, ethical and

social WE WE x WE WE x WE WE

Final Test and

general issues WE WE WE WE x x WE WE





9. Summary and conclusions This deliverable has presented the MAGNETO training methodology, the evaluation methods and criteria,

the training curriculum and the preliminary scheduling needed to complete each subject.

A questionnaire has been designed and requested to be filled by the end-users to have a better

understanding about their current knowledge and background in multimedia analytics. The objective of

this questionnaire was also to measure the end-users satisfaction with previous trainings and gather their

suggestions regarding the development of MAGNETO training in order to ensure their needs are met.

The result of this survey has been the design of a training methodology focused on MAGNETO end-users

needs. The state of the art research on teaching methodologies combined with the results of MAGNETOs

end-users questionnaire helped us selecting a set of training activities and material, including technical

manuals, webinars and face-to-face workshops. It must be noted that face-to-face workshops will be

scheduled first after the training has begun since the purpose of these workshops is to resolve issues and

questions that might arise during the on-line training.

It is important to mention that, the generation of MAGNETO training contents will be in parallel with the

tools development (?), so some of the defined activities cold be changed according new identified

features. In summary, each training methodology will be checked when the tools will be available for

testing and training content generation.

Taking into account that MAGNETO is an international project and each partner is located in a different

country, the training material will be hosted in an on-line learning platform to make it accessible for the

end-users and take maximum benefits (?). After the survey performed on different LMS (?), it was decided

to use Moodle as the MAGNETO e-learning platform.

Following the evaluation methods implemented in the UPV since the Bologna Process was introduced, a

set of evaluation methods and criteria have been defined. Continuous evaluation is preferred by end-

users and trainers and the evaluation methods will be Test/Quiz and practical exercises.

Additionally, the terms of curriculum (?), “subject”, “objective”, “prerequisite” and “scheduling” have

been explained, and an example of MAGNETO Curriculum has been provided in Appendix A, for a better

understanding of the concepts.

Following the explanation, a first version of MAGNETO training curriculum was defined with a total of

seven identified subjects, organized in three main blocks: background knowledge, Multimedia analytic

tools, and finally the legal, ethical and social guidance. Also, a set of general and specific objectives has

been defined as well as their preferred evaluation method.

Finally, a first draft of the MAGNETO Training schedule was proposed with a duration of 78 hours.



10. References

[1] https://en.wikipedia.org/wiki/Experiential_learning

[2] https://learningfromexperience.com/

[3] https://comparisons.financesonline.com/sakai-vs-moodle

[4] https://sourcecontribute.com/2015/04/16/freeopen-source-elearning-management-systems-

moodle-vs-sakai/

[5] https://www.insidehighered.com/blogs/digital-tweed/looking-beyond-sakai

[6] http://presentations.cita.illinois.edu/2011-03-csun-

lms/?utm_campaign=elearningindustry.com&utm_source=%2Flms-accessibility-comparison-

blackboard-vs-desire2learn-vs-moodle-vs-sakai&utm_medium=link

[7] https://pdfs.semanticscholar.org/1f96/f4a66767293cf2902652ac9eadfa8ef51f13.pdf

https://en.wikipedia.org/wiki/Experiential_learning

https://learningfromexperience.com/

https://comparisons.financesonline.com/sakai-vs-moodle

https://sourcecontribute.com/2015/04/16/freeopen-source-elearning-management-systems-moodle-vs-sakai/

https://sourcecontribute.com/2015/04/16/freeopen-source-elearning-management-systems-moodle-vs-sakai/

https://www.insidehighered.com/blogs/digital-tweed/looking-beyond-sakai

http://presentations.cita.illinois.edu/2011-03-csun-lms/?utm_campaign=elearningindustry.com&utm_source=%2Flms-accessibility-comparison-blackboard-vs-desire2learn-vs-moodle-vs-sakai&utm_medium=link



https://pdfs.semanticscholar.org/1f96/f4a66767293cf2902652ac9eadfa8ef51f13.pdf



Appendix A: MAGNETO Training Questionnaire

MAGNETO END USER

TRAINING QUESTIONNAIRE

Dissemination Level: RE

Nature of the Deliverable: O

Date: 24/10/2018

Distribution: WP7

Editors: UPV

Reviewers: UPV

Contributors: ALL

Abstract: This questionnaire is a part of the requirements collection phase of Multimedia Analysis and

correlation engine for orgaNised crime (MAGNETO)






Table of Contents

MAGNETO End-Users Training Questionnaire ............................................................................................ 48

Introduction ........................................................................................................................................ 48

Data protection statement and disclaimer ......................................................................................... 48

Instructions ......................................................................................................................................... 49

Personal details ................................................................................................................................... 49

Organization details ............................................................................................................................ 53

Questions on previous trainings received........................................................................................... 56

Expectations on the MAGNETO training ............................................................................................. 67



MAGNETO End-Users Training Questionnaire

Introduction

This questionnaire is a part of the requirements collection phase of Multimedia Analysis and correlation

engine for orgaNised crime (MAGNETO) which is a project within the H2020 Programme funded by the

European Commission, addressing topic SEC-12-FCT-2017 Technologies for prevention, investigation, and

mitigation in the context of fight against crime and terrorism. More specifically, MAGNETO addresses the

technical research, innovation work and newly developed technologies that meet the real operational

needs of the Law Enforcement Agencies (LEA) unifying the different evidence sources (video, audio

text/documents, social media and Web data, telecom data, surveillance systems data, police databases

etc.) Moreover, the LEAS, from now on “end-users”, know very well what they require, especially based

on their recent experiences of investigating huge amount of data from to crime and terrorist attacks.

The purpose of this questionnaire is to better understand the end-user requirements and needs related

to the scope of the training in the MAGNETO Platform.

Although the MAGNETO Platform will provide a user interface as user-friendly and intuitive as possible

and also different interoperability tools, a suitable training will be prepared to ensure optimal use of a

sophisticated analytic platform. Based on this, the MAGNETO training group will develop a comprehensive

training programme targeting the end-users’. Besides MAGNETO technical aspects, training will also cover

related legal, ethical and privacy issues.

More precisely, these training activities will cover two main targets:

• The end-users will be trained during the project to be able to optimally use MAGNETO prototype

during assessment and demonstration in field tests. The potential target audience is estimated (?)

several LEA data analytic experts from the LEAs involved as project partners, the MAGNETO

advisory board, and in the MAGNETO expert community.

• Training material and tools for training of end-users (train-the-trainer?) will be elaborated to

spread the knowledge to additional LEA data analytic experts not involved during the project. As

such, the potential audience can increase up to hundreds of LEA officers in Europe.

Data protection statement and disclaimer

The MAGNETO consortium – particularly all those organizations involved in gathering, processing and

analyzing end-user needs – fully understand the sensitive nature of the subject and will not include any

information arising during an interview that is not suitable for the public domain.

At the same time, all interviewees’ personal details will remain anonymous and ‘firewalled’.



Instructions

Please complete the form selecting the satisfaction/importance of the following requirements:

Satisfaction level in previous trainings

(0 is the lowest importance and 4 is the highest importance)

0 Very dissatisfied The training will be improved if removed.

1 Dissatisfied Nice to have, but the training will be fully useful even without it.

2 Match Expectations If removed, the training will be only partially useful.

3 Satisfied This improved the training.

4 Very satisfied This was a key aspect of the training.

Importance level for the future MAGNETO training

(0 is the lowest importance and 4 is the highest importance)

0 Unimportant With or without this, the training is exactly the same.

1 Wish Nice to have, but the training will be fully useful even without it.

2 Important Without this, the training will be only partially useful.

3 Serious Without this, the training will be usable but not useful.

4 Critical Without this, the training will be not useful at all.

Personal details

Please complete the following form with your personal information. 1. Age

18-20 years

26-30 years

31-40 years

41-50 years

50+ years

2. Grade of studies



Technical School

Graduate/Degree

Master

Phd

3. Years of experience in the field of criminal and terrorist counter activities.

No experience

1-5 years

6-10 years

10+ years

4. Number of previous training courses received in the field of forensic tools.

0 courses

1-2 courses

3-5 courses

5+ courses

Please write down the courses:



5. Please check bellow the Digital video analysis topics you are familiar with:

Digital video recording/storage process

Frame-rate/Resolution/Color types (RGB/YMK)

Metadata included in the video

Video Coding

Please write down some examples you know:

Video Encapsulation


Image and video processing (2D/3D Fourier Transformations)

Motion detection

Video tracking

Categorization of objects in videos using a samples from a data base

Human face recognition techniques

Human behavior recognition techniques

6. Please check bellow the Digital audio analysis topics you are familiar with:

Digital audio recording/storage process

Sample-rate/Bit depth/Signal types (Mono/Stereo)

Metadata included in the audio file

Audio Coding




Audio Encapsulation


Audio processing (Filter, transformations)

Audio-text conversion

Speaker recognition

Audio pattern recognition

7. Please check bellow the Text and Data Mining topics you are familiar with:

Distributed storage platforms (SharePoint, HDFS, …)


Metadata included in the text file

Files preprocessing

Text gathering techniques (Web Scraping, documents parsing, sensors data gathering …)




Automatic language translation

Semantic information processing

8. Please check bellow Augmented Intelligence topics you are familiar with:

Visual Analytics

Immersive visualization

Haptic Interaction

Operational and Situational Awareness

Achieving Solid and court-proof Forensic Evidence


Organization details

Please complete the following form with the information of your organization.

1. General Information

Name

Organization

Department

2. Organization type:



National police

Municipal police

Private security forces

Military forces

Public bodies

Other (please describe below)

3. Do CRIMINAL activities belong to the area of your specific interest/duties?

Yes

No

4. Do TERRORIST attacks belong to the area of your specific interest/duties?

Yes

No

5. Do you perform any particular tasks in the field of counter criminal and terrorism activities?

Yes

No

If “Yes”, could you briefly describe the task you perform?

If “No”, do you think that MAGNETO research could be beneficial to your organization?

Yes

No



6. What are the most important capabilities necessary to counter criminal and terrorist activities? Please

select the importance of each activity.

Technical capabilities

Sensors 0 1 2 3 4

Mobile devices 0 1 2 3 4

Mobile platforms 0 1 2 3 4

Technical resources 0 1 2 3 4

Information systems 0 1 2 3 4

Analysis tools 0 1 2 3 4

Organizational and individual capabilities

Human resources 0 1 2 3 4

Procedures/Policing 0 1 2 3 4

Cooperation 0 1 2 3 4

Open Intelligence 0 1 2 3 4

Knowledge 0 1 2 3 4

Awareness 0 1 2 3 4

Proactivity 0 1 2 3 4

Mind-set 0 1 2 3 4

Other

Luck 0 1 2 3 4

7. Did your organization perform field collaborative experience with other organizations?

Yes

No

8. In affirmative case, were such organizations from the same country or different countries?

Same

Different

9. What have been the major issues regarding national and/or international

cooperation/communication aspects?

Language

Procedures

Communication channel

Data format



Others(Please describe below)

10. Which are your current acquisition methodologies on Video, Audio and/or Text analysis?

Questions on previous trainings received

Type of class

Relative to the previous trainings, could you answer the following question regarding which type of classes

did you received.

Type of class Seminar (In person)

YES

NO

Satisfaction 0 1 2 3 4



Briefly describe

your opinion

Best

Worst

Type of class Video recorded lectures (On-Line lectures)

YES

NO


Briefly describe

your opinion

Best

Worst



Type of class Webinars (On-Line Lectures)

YES

NO


Briefly describe

your opinion

Best

Worst

Type of class Guided simulations (Practical lectures with exercises)

YES

NO


Briefly describe

your opinion

Best

Worst



Training materials

Relative to the previous trainings, could you answer the following question regarding which type of

materials were provided.

Type of material Manuals / PDFs / PowerPoints

YES

NO


Briefly describe

your opinion

Best

Worst

Type of material Video explanations

YES

NO


Briefly describe

your opinion

Best

Worst



Type of material Exercises

YES

NO


Briefly describe

your opinion

Best

Worst

Online platform

Relative to the previous trainings, could you answer the following question regarding online platforms.

Online platform

information

Did the course provide an online platform?

YES

NO


Briefly describe

your opinion

Best

Worst



Online platform

information

Where you able to access the online platform outside from work?

YES

NO


Briefly describe

your opinion

Best

Worst

Online platform

information

Did the online platform provide a forum for teachers and students to

interact?

YES

NO


Briefly describe

your opinion

Best

Worst



Classroom information

Relative to the previous trainings, could you answer the following question regarding the classroom where

the training took place.

Classroom information Did you perform the trainings in the LEAs offices?

YES

NO


Briefly describe

your opinion

Best

Worst

Classroom information How many students where in your class?

Less than 10

10 – 20 students

20 – 30 students

More than 30 students

Briefly describe

your opinion

Opinion



Classroom information Number of students per computer in the class room

No computer

1 student

2 students

3 students

More than 3 students

Briefly describe

your opinion

Opinion

Classroom information Technical capabilities of the classroom computers?

Good overall computer

Internet connection

Possibility to run Virtual Machines

Good graphic card with capabilities to run 3D environments

Briefly describe

your opinion

Opinion



Legal and ethical training

Relative to the previous trainings, could you answer the following question regarding the training received

on legal procedures and ethical principles.

Legal and Ethics Did you received training on legal procedures and ethical principles?

YES

NO


Briefly describe

your opinion

Best

Worst

Legal and Ethics Did the class perform discussion on legal and ethical topics?

YES

NO


Briefly describe

your opinion

Best

Worst



Evaluation

Relative to the previous trainings, could you answer the following question regarding the evaluation.

Evaluation In past courses, at the end, did the students have to pass an evaluation?

YES

NO


Briefly describe

your opinion

Best

Worst

Evaluation Quiz

YES

NO


Briefly describe

your opinion

Best

Worst



Evaluation Exercises / Problems / Written Test

YES

NO


Briefly describe

your opinion

Best

Worst

Evaluation Projects / Essays / Manuscripts

YES

NO


Briefly describe

your opinion

Best

Worst



Expectations on the MAGNETO training

Type of class

Type of class Video recorded lectures (On-Line lectures)

Importance 0 1 2 3 4

Type of class Webinars (On-Line lectures)


Type of class Guided simulations (Practical Lectures with exercises)


Type of class Other types

Description



Materials

Type of material Manuals / PDFs / PowerPoints


Type of material Video explanations


Type of material Exercises


Type of material Other types

Description



Online platform

Online platform

information

Manuals / PDFs / PowerPoints


Online platform

information

Video explanations


Online platform

information

Exercises


Online platform Other features

Description



Evaluation

Evaluation Quiz


Evaluation Exercises / Problems / Written Test


Evaluation Projects / Essays / Manuscripts


Evaluation Other types

Description



Personal Expectations

Personal Expectations Which skills do you consider that the training should focus on?

Description



Appendix B: Summary of Law Enforcement Agency (LEA)

Training Questionnaire Responses.

Title: Summary of LEA Training Questionnaire

Responses

Dissemination Level: CO

Nature of the Deliverable: Internal Report

Date: March 2019

Distribution:

Editors:

Reviewers:

Contributors:

Abstract: This document contains the details of …training questionnaire returns






Table of Contents

Appendix B: Summary of Law Enforcement Agency (LEA) Training Questionnaire Responses. ................ 72

Introduction ............................................................................................................................................ 75

A note on the Interpretation of Results .............................................................................................. 75

Personnel Details and Experience / Knowledge (Questions 1 to 8) ....................................................... 77

Respondents Age and Experience (Questions 1 to 3) ......................................................................... 77

Previous Training Courses in Forensic Tools (Question 4) .................................................................. 78

Familiarity with Digital Video Analysis (Question 5) ........................................................................... 79

Familiarity with Digital Audio Analysis (Question 6) ........................................................................... 80

Familiarity with Text and Data Mining Topics (Question 7) ................................................................ 81

Familiarity with Augmented Intelligence (Question 8) ....................................................................... 82

Organization Details ................................................................................................................................ 83

Organization Type (Questions 1 & 2) .................................................................................................. 83

Areas of Interest (for your duties) (Questions 3,4,5) .......................................................................... 83

What are the most important capabilities necessary to counter criminal and terrorist activities?

Please select the importance of each activity (Question 6) ............................................................... 84

Did your organization perform field collaborative experience with other organizations (Question 7)?

............................................................................................................................................................ 85

In the affirmative case were such (collaborating) organizations from the same country or different

countries (Question 8). ....................................................................................................................... 85

What have been the major issues regarding national and/or international co-operation /

communication (Question 9)? ............................................................................................................ 86

Which are your current acquisition methodologies on Video, Audio and/or Text analysis (Question

10) ....................................................................................................................................................... 86

Questions on previous training that has been received ......................................................................... 87

Which types of classes did you attend and how satisfied were you with them? (Question 4.1) ....... 87

Which types of training materials were used and how satisfied were you with them? (Question 4.2)

............................................................................................................................................................ 88

Experience with On-Line Platforms (Question 4.3) ............................................................................ 89

Classroom Information (Question 4.4) ............................................................................................... 90

Legal and Ethical Training ................................................................................................................... 92

Evaluation ........................................................................................................................................... 92



Expectations on MAGNETO Training ...................................................................................................... 94

Type of Class ....................................................................................................................................... 94

Materials ............................................................................................................................................. 95

On-Line Platform ................................................................................................................................. 96

Evaluation ........................................................................................................................................... 97

Personal Expectations ......................................................................................................................... 98

Summary Observations and Conclusions. ............................................................................................... 99

Observations ....................................................................................................................................... 99

Conclusions ....................................................................................................................................... 100



Introduction This document presents a summary of the results of questionnaires returned to the MAGNETO project by LEA project partners. The purpose of issuing the questionnaires was to better understand the end-user requirements and needs related to the scope of the training in the MAGNETO Platform. The questionnaires were focussed on Analysts as a previous questionnaire presented in Deliverable 2.2 identified that these were perhaps the key user group for MAGNETO. This is an internal report to assist with the development of the Deliverable 7.1 (Training Methodology) and Deliverable 7.2 (Training Tools). Detailed responses from the respondents to the questionnaires are presented in a separate spreadsheet which is available to project partners on an as-needed basis. This summary report will help the training developers to produce relevant training in formats and ways


In total there were 31 respondents to the questionnaires from ~10 organisations, mostly in the form of

national and municipal police.

The raw results of the questionnaires are summarised below along with brief summaries of the

implications of those results, where such comment is warranted. The request, question or prompt that

was used to solicit a response from the respondents is given at the start of each sub-section. Questions

presented to the respondents were largely in the form of binary (yes/no) or questions but there were also

a number which required the respondents to select from a 5 point scale. These were processed as

described in Section 1.1.

Where text answers were also provided by the respondents, to expand on their multiple choice answers,

key points from these are also presented, but readers are encouraged to read the full responses contained

in the spreadsheet if a fuller understanding is needed.

Occasionally, direct quotes from the respondents are re-produced below. These are shown in italics in

quotation marks e.g. “Thus”.

A note on the Interpretation of Results

Questions presented to the respondents were largely in the form of binary (yes/no) or questions but there

were also a number which required the respondents to select from a 5 point scale;

In respect of aspects of previous training that the respondents had received, the options were

Option

/ Score Satisfaction Explanation

0 Very dissatisfied The training will be improved if removed.

1 Dissatisfied Nice to have, but the training will be fully useful even without it.



2 Matched Expectations If removed, the training will be only partially useful.

3 Satisfied This improved the training.

4 Very satisfied This was a key aspect of the training.

In respect of aspects of possible future training the options were

Option

/ Score Expectation Explanation

0 Unimportant With or without this, the training is exactly the same.

1 Wish Nice to have, but the training will be fully useful even without it.

2 Important Without this, the training will be only partially useful.

3 Serious Without this, the training will be usable but not useful.

4 Critical Without this, the training will be not useful at all.

In these cases, to provide a useful indication of the results obtained across the respondents, answers have

been processed into percentages using the simple expediency of calculating the sum of all assigned scores

as a percentage of the theoretical maximum. So, for example, if all 31 respondents select option 4 then

this would be converted to 100% (31*4 out of a theoretical maximum of 31*4) and one rated as ‘4’ by 5

of the users, ‘3’ by 15 of the users and ‘2’ by 6 users would be converted to 74%

[(5*4+15*3+6*2)/((5+15+6)*4)]. These are referred to in the text as ‘Compound Scores’


whereas a low percentage score indicates a consistent lack of desire to see the item. A mid-range

percentage score represents a mixed response. These scores can be used to set priorities for the design.

Note that not all respondents, responded to all questions, so that the sample size varies with question.

Sufficient data are presented so that these variances are clear.



Personnel Details and Experience / Knowledge (Questions 1 to 8)

Respondents Age and Experience (Questions 1 to 3)

Request: Please complete the following form with your personal information.

Respondents were asked to indicate their age, relevant levels of academic qualifications, years of

experience and familiarity with the types of technologies likely to be relevant to the MAGNETO system.

These are summarised in the graphics below.



Previous Training Courses in Forensic Tools (Question 4)

Request: Please indicate the number of previous training courses you have received in the

field of forensic tools.

21 respondents (68%) indicated a variety of previous training courses covering topics such as mobile

phone surveys, forensic computing, pc forensics/ data recovery as well as more general training in criminal

investigation and the use of industry standards such as i2 iBase and i2 Analyst from IBM and mapping

systems such as ArcGIS from esri. It is notable that 10 respondents indicated that they had not received

any previous training in forensic tools, but this does not mean that they have not received training in other

types of tools.



Familiarity with Digital Video Analysis (Question 5)

Request: Please check bellow the Digital video analysis topics you are familiar with.

Request: Respondents were asked to provide text examples of video coding and video

encapsulation.

10 respondents (32%) provided examples of video coding which included items such as analogue to digital

conversion, metadata analysis, differences in video resolution codecs, frame rates and associated storage

requirements and use of standard software such as NACSPORT. 4 respondents (13%) provided examples

of video encapsulation referring to standards such as .avi, .mov and MPEG.



Familiarity with Digital Audio Analysis (Question 6)

Request: Please check bellow the Digital audio analysis topics you are familiar with.

Request: Respondents were asked to provide examples of Audio Coding and Audio

Encapsulation.

9 respondents (29%) provided examples of audio coding which included items such as examples of

metadata, references to standards such as MP3, FLAC, M4A and AAC. A number of the respondents

indicated that audio coding was not relevant to their work area. 4 respondents (13%) provided additional

details of audio encapsulation but only ‘Linear pulse Code Modulation’ was mentioned by name.



Familiarity with Text and Data Mining Topics (Question 7)

Request: Respondents were asked to indicate the Text and Data Mining topics they were

familiar with

Request: Respondents were asked to provide examples of Distributed Storage Platforms and

Text Gathering Techniques.

9 respondents (29%) provided examples of Distributed Storage Platforms which included items such

Microsoft Sharepoint (Microsoft), HedVig and SWARM, Dataminer (Oracle?), I2 (IBM) and Azure

(Microsoft). 9 respondents (29% but not exactly the same group of respondents) provided examples of

Text Gathering Techniques such as Python Web Scraping, ARMITAGE, CeWL and others.



Familiarity with Augmented Intelligence (Question 8)

Request: Please check below the Augmented Intelligence topics you are familiar with.

Request: Respondents were asked to provide examples of the topics they were familiar with.

6 respondents (19%) provided additional responses to the request for examples but only one response

was directly relevant – mentioning haptic feedback – and 3 of the responses noted that the respondent

had no knowledge about Augmented Intelligence.



Organization Details

Organization Type (Questions 1 & 2)

Areas of Interest (for your duties) (Questions 3,4,5)

Questions (3 and 4)

Percentage of respondents

answering “Yes”

Question 3: Do CRIMINAL activities belong to the area of your specific interest/duties?

84%

Question 4: Do TERRORIST attacks belong to the area of your specific interest/duties?

68%

Question 5: Do you perform any particular tasks in the field of counter criminal and terrorism activities?

61%

Respondents answering ‘Yes’ to question 5 indicated that they were involved with i) teaching police

officers (including teaching in the field of terrorism and counter crime) (2 respondents), ii) analysis of data

for presentation to other officers (6 respondents) iii) forensic analysis (2 respondents), iv) Threat and Risk

Analysis (2 respondents) and v) searching the internet for criminal activity (1 respondent).

Note that some of the respondents who answered ‘No’ to Question 5 and some of the respondents who

answered ‘Yes’ to Question 5, confirmed that they thought that the MAGNETO research would be

beneficial to their organisation and 1 respondent indicated that they thought that MAGENTO research

would not benefit their organisation.



What are the most important capabilities necessary to counter criminal and terrorist

activities? Please select the importance of each activity (Question 6)

Request: Respondents were asked to select the importance of a range of technical and

organizational capabilities.

Technical Capabilities

All capabilities scored highly for importance, the calculated compound scores (see section 1.1) are

Technical Capability

Compound Scores (%)

Analysis Tools 63

Information Systems 66

Technical resources 62

Mobile Platforms 52

Mobile devices 52

Sensors 47

Organizational and Individual Capabilities



All capabilities, apart from ‘Luck’ have compound scores at around 65 (see below), indicating significant

importance to the respondents.

Organizational and Individual Capabilities

Compound Scores (%)

Luck 42

Mind-set 63

Pro-activity 63

Awareness 62

Knowledge 63

Open Intelligence 63

Co-Operation 66

Procedures / policing 60

Human resources 59

Did your organization perform field collaborative experience with other organizations

(Question 7)?

27 of 30 respondents indicated that they had performed collaborative exercises and 3 said that they had

not.

In the affirmative case were such (collaborating) organizations from the same country

or different countries (Question 8).

20 of 31 respondents indicated that they had collaborated within the same country and 16 of 31 indicated

that they had collaborated outside of their own country (N.B. some respondents had collaborated both

within and out of their own country).



What have been the major issues regarding national and/or international co-

operation / communication (Question 9)?

Request: Respondents were asked to provide further text details where they had selected

‘others’.

One respondent indicated the need to use define an agreed set of specific terms before entering such

joint ventures, one respondent referred to different data protection acts in the single federal states, one

noted issues with knowledge sharing and platforms but a number also noted that many of the difficulties

can be easily overcome.

Which are your current acquisition methodologies on Video, Audio and/or Text

analysis (Question 10)

Respondents provided a range of responses covering bespoke forensic software (2 respondents), open-

source/freeware/’standard’ software (3 respondents) and manual analysis (2 respondents).



Questions on previous training that has been received

Which types of classes did you attend and how satisfied were you with them? (Question

4.1)

Respondents were asked to provide example of the worst and best aspects of the types of classes ranked

above.

Seminar (In person) – ‘Best’ and ‘Worst’ Respondent Comments

The best aspects of seminars were noted as being related to having face-to-face training (direct

interaction) and the possibility for attendees to ask questions. But it was also noted that this required

competent and charismatic teachers, a clear practical approach and the ability of the seminar leader to

‘dynamically modulate’ (adapt to attendees questions). Others noted that the ability to exchange ideas

with other attendees was also valuable.

The need to ensure that attendees had the requisite prior knowledge was noted as was the need for a

flexible schedule with sufficient time.

Video recorded lectures (On-Line lectures) – ‘Best’ and ‘Worst’ Respondent Comments

A key benefit of video recorded lectures was noted as being the ability of trainees to watch the teaching

content again (e.g. if missed first time or not fully understood first time). The lower financial and logistical

burden of such training (compared with other types) was also noted as a key benefit. On the negative

side many respondents noted that the lack of the possibility to ask questions and to seek clarification was

an issue. The need for technically good videos (clear language and video quality) was also noted.

Webinars (On-Line Lectures) – ‘Best’ and ‘Worst’ Respondent Comments



Webinars were noted as being good for getting an initial introduction to a subject or for upgrading existing

knowledge but there was perhaps a suggestion that inflexibility and the ‘inability to ask questions’ was a

problem for ‘new’ learning. The possibility of ‘bringing together a vast and heterogeneous audience’ was

noted as a benefit as were the low financial and logistic burdens but the potential problems with differing

time zones among attendees was also noted. The need for attendees to be susceptible to distraction and

hence ‘The student must be very motivated to be constant’ was also noted. The potential for interaction

was noted as a benefit.

Guided simulations (Practical lectures with exercises) – ‘Best’ and ‘Worst’ Respondent

Comments

Respondents noted that the combination of theory and plenty of practice was the key benefit for this type

of training; especially the ability to try things out with a trainer present to provide guidance. The

importance for these sessions to be focused on ‘how it’s actually done’ was also noted. The need to have

an appropriate student to teacher ratio ‘in order that students get the attention needed e.g if struggling

to understand a particular matter’ was noted as well as the fact that ‘Different learners’ technical

background cause slow down’. The potential relatively high costs of these types of sessions was also noted

as a concern.

Which types of training materials were used and how satisfied were you with them?

(Question 4.2)

Respondents were asked to provide example of the worst and best aspects of the types of materials

ranked above;

Manuals / PDFs / Powerpoints - ‘Best’ and ‘Worst’ Respondent Comments

Respondents indicated that the key value of Manuals and PDFs was that they could be referred to during

and after the training for a deeper understanding or to refresh the memory/ act as a reference guide. The



need for sufficient detail in manuals was also noted. On the other hand although ‘A good Powerpoint or

image can sometimes provide a clearer message than a document with 1000 words’ was noted so was the

fact that they can ‘Sometimes be boring and disengaging’.

Video Explanations - ‘Best’ and ‘Worst’ Respondent Comments

Comments similar to those for Video recorded lectures in 0 were noted.

Exercises - ‘Best’ and ‘Worst’ Respondent Comments

Hands-on learning was noted as being a crucial part of training which reinforced studies and help

attendees to ‘settle knowledge’. It was noted that ‘The best way to learn is by being hands on and trying

it yourself.’ It was also, however, noted that ‘It can be daunting in the wrong environment. May need to

be closely guided depending on confidence of the student’. It was also noted that thus can be overlooked

or not done properly if something else does not go to plan.

Experience with On-Line Platforms (Question 4.3)

Respondents were asked to provide example of the worst and best aspects of the platform approaches

ranked above.

Provision of On-Line Platforms - ‘Best’ and ‘Worst’ Respondent Comments

One respondent noted that ‘An online learning platform that offers not only videos, exercises, learning

materials but also the ability to communicate with other trainees has, in my point of view, a substantial

benefit’ , it was also noted that they had the benefit of being available without travel (costs) and that they

could be fitted around a schedule. The need for platforms to be kept up-to-date and to ensure that there

was a mechanism for users to clarify concepts was also noted.



Ability to Access On-Line Platforms from Outside Work - ‘Best’ and ‘Worst’ Respondent

Comments

The need for such platforms to be compatible with private pc’s / mobile devices was noted. The difficulty

of using such approaches and the need for arrangements for ‘secure’ access – an perhaps the need to

access them at work offices - were also noted.

Provision of a Forum for teachers and students to interact - ‘Best’ and ‘Worst’ Respondent

Comments

The possibility for Forums to work well and be beneficial was noted but the need for a quick response

time (to questions) was especially noted.

Classroom Information (Question 4.4)

In-Office Training - ‘Best’ and ‘Worst’ Respondent Comments

In-office training was seen as a useful training method but the need for appropriate quality training rooms

and equipment was noted. The need to verify the available equipment’s functionality in advance was

noted. The absolute need to perform training in-house for some ‘secure’ systems was noted. On the

down-side it was noted that ‘You are in the usual workplace and available for any job consultation’ , which

implies the need for in-house training to be treated as unavailable time in the same way as out of office

training.

Class Sizes and Equipment – Previous Training Experiences



Class Sizes and Equipment - Opinions

There was general consensus among those specifically commenting on class sizes that somewhere

between 8 to 15 students per teacher is most effective.

It was noted that ‘Teacher to student ratio is vitally important to allow for unforeseen circumstances such

as technical issues and diversity in abilities of students’.

Number of Computers – Opinions

With respect to number of computers it was generally noted that 2 people per computer was the correct

ratio and that having 2 had the benefit that the students could help each other.

Computer Capabilities – Opinions

There was a general opinion that ‘Secure and fast internet connection is important, ideally multiple IP lines

for classroom’. Also, some noted that ’reasonably specced computers with capability to run virtual

machines etc’ was desirable but that this was dependent on the subject being taught.



Legal and Ethical Training

Legal and Ethical Training and Discussions – ‘Best’ and ‘Worst’ Responder Comments

The importance of this type of training was noted but it was also noted that it can be ‘A difficult subject

to make interesting’ and that it is ‘Generally done by information overload’. The need to make the training

role specific was noted, especially as it ‘Can be less relevant to certain activities and difficult to make

interesting for some students’.

Evaluation

Need to pass an evaluation – ‘Best’ and ‘Worst’ Responder Comments

Respondents interpreted evaluations as being ‘evaluations of the course’ or ‘of the attendees’. The use of

evaluations as a method of ensuring that attendees had acquired the required knowledge and

understanding was widely recognized ‘It effectively measures the learners’ level of understanding of the

subjects’ but it was also noted that ‘This is vital to ensure student maintains focus throughout the training

and allows you to confirm their capability at the end of training’ and that ‘By taking part in an evaluation,

participants in the seminar not only have the opportunity to give feedback to the head, but can actively

contribute to adapting and improving the seminar’. It was also recognized that any such evaluations or



questionnaires need to be customized to the actual course (rather than being generic), not be too

extensive and that they must actually be acted upon by the trainers. The potential for evaluations of

the attendees to introduce stress and be counter-productive was also recognized.

Post Training Quiz – ‘Best’ and ‘Worst’ Responder Comments

The value of using quizzes to ‘keep students focused’ and to check the level of understanding was

recognized but the need to focus questions carefully on the subject matter and to allow discussion of

answers was also noted.

Exercises / problems / written tests – ‘Best’ and ‘Worst’ Responder Comments

The value of exercises and written tests and that ‘test helps the students to realize what they really

learned and they can prove with a test or certificate that they gained the knowledge of that course’ was

recognized. The need for there to be practical aspects in exercises was also recognized – ‘Enables students

to demonstrate practical techniques learnt rather than regurgitating theory’. The difficulty of ‘Sometimes

the teacher's interpretation could be subjective’ was noted by some.

Projects / essays / manuscripts – ‘Best’ and ‘Worst’ Responder Comments

The suitability of these types of evaluations for academic training was recognized, and that they are ‘Not

helpful or relevant for practical operational training’. But it was also recognized that it ‘Allows more

flexibility than exams and a higher personal input’ and that it ‘Forces you to deepen the subject’

understanding. The additional time, effort and dedication required (from attendees) was also recognized.



Expectations on MAGNETO Training

Type of Class

The associated Compound Scores are:

Type of Class

Compound Scores (%)

Guided Simulations 63

Webinars 46

Video recorded lectures 44

Notes from ‘Other Types’

Other types of training that were noted included workshops, personal seminars, on-line seminars,

practical classes and online systems like ‘Moodle’.



Materials


Type of Material

Compound Scores (%)

Exercises 56

Video Explanations 49

Manuals / PDFs / PPT 60


Other types of material that were noted included help in the same program, reference material, tutoring,

individual support and ‘Practical LEA's case stories solved with Magneto help’.



On-Line Platform


On-Line Platform Information Type

Compound Scores (%)

Exercises 54

Video Explanations 52

Manuals / PDFs / PPT 57


Other types of Information that were noted included

• ‘- the possibility of communication with other participants and lecturers

• access to the online platform not only web-based but app-based, too

• in the learning platform, a web-based training in advance of the training, in order to already

provide basic knowledge, so that all course participants rank at the same level at the beginning of

the course. ‘

• On-line tutors

• Practical tests

• ‘If an online platform will be available there should be some areas open to questions and answers

from users (like forum or chatrooms) as so as a wiki where tutorial can be shared among

practitioners.’



Evaluation


Evaluation Type

Compound Scores (%)

Projects / Essays / Manuscripts

28

Exercises / Problems / Written tests

51

Quiz 45


Other types evaluation that were noted included

• Continuous evaluation

• Case studies/resolution

• Multiple choice

• Certification



Personal Expectations

Personal expectations expressed by the respondents generally stated a need for training to focus on the

MAGNETO program – as seen in comments such as ‘training should empower the students to

independently use the Magneto-Software/Framework and to realize how they can improve their daily

analysis work with this framework’, and ‘I would expect a deep explanation for each of the MAGNETO

platform’s features’.

Some also noted that training ‘in the fundamental aspects of analysis’ should also be included. Others

noted ‘I would prefer to receive lessons related to the underlying technologies in order to better catch all

possible applications’. However, some noted that ‘The training should focus on what is new and what skills

are required to operate, process and analyse any new software, tools and output; It should assume a pre-

existing knowledge of law enforcement forensic processes and IT infrastructure within a law enforcement

and open-source environment.’

The need for focused example case studies and practical training was also recognized.

The value of training sessions which present the possibility of networking with other trainees as also noted

– as another resource for further discussion on MAGNETO and related experiences.



Summary Observations and Conclusions.

Observations

Overall


location. They had also attended a significant number of previous training sessions. The implications of

this being that their feedback is drawn from personal experience and technical knowledge of criminal and

counter-terrorism activities as well as forensics. As these represent a key group of anticipated users for

the MAGENTO system, the results are important.

Background:


analysis the examples they quoted were somewhat limited.


examples.

Augmented intelligence: Familiarity with AI topics is limited and this is reflected by the small number of

examples quoted.

Technical and Organisational Capabilities




In this respect it is noted that collaborations often occur, although these are not without some difficulties

arising from things such as differences in procedures and data formats.

Previous Training

Respondents have experience of a broad range of class types but there is a clear preference for these to

be ‘guided’ and to include the opportunity for discussion and co-operative working. There is also good

indication that such sessions are more valued when they include a strong practical aspect. Experience

with on-line platforms was less than with some other training types but these were also well received on

the whole. Again, the need for feedback and interaction was also noted.

Training within the LEA’s own offices was seen as possible and having the added benefit of reducing travel

costs and time, but others also saw the benefit of being away from the office for such training. In all cases,

the need for good quality training materials and equipment was noted as was the need for good

‘charismatic’ trainers and engaging materials. Teacher to pupil ratios were noted as ideally being 10 to

15 pupils per teacher.



The use of pc’s / computers was seen as important and the need to keep the ratio of users per computer

to no more than 2 was noted.

Legal and ethical training was seen as being important (by some) but there is good indication that this has

not been well delivered in the past.

In terms of evaluation of training, there is a lower incidence of essays and the like being used by the


Expectations

Respondent’s expectations are for guided simulation or similar, with well-presented reference materials

and guide-books. There is a strong focus on training being practically focussed on real LEA work – possibly

using examples from their own experiences. The use of video’s and on-line systems is not precluded but

this is not the respondent’s preference – at least not for main-line training, perhaps more for initial


Conclusions

Since the questionnaire reported here has focussed on the needs of Analysts rather than the full potential

user group identified in D2.2 – these being the user group most likely to use the full range of capabilities

delivered by MAGNETO – and this is a largely office based user group, the conclusions presented here are

in turn focussed on office usage. Although this excludes some of the potential uses of MAGNETO, such as

on mobile systems, it is nevertheless considered to be practicable and appropriate for a system being

developed for TRL 6.

There is clearly interest for a diverse range of training modalities ranging from technical documents for

self-education through to face to face training. Remote training using Webinars, rather than videos, is

also indicated, especially where there is the facility to have interaction and feedback.

Noting again that the system is only intended to be TRL6 and that the training itself cannot therefore be




examples.



Appendix C: Example of curriculum template oriented to a

course of MAGNETO tools.

Definition of curriculum subject and specialists

Subject Name Mentor Duration

General knowledge Victor Javier Garrido X theory hours + X practice hours

Platform configuration and

management

Francisco José Pérez X theory hours + X practice hours

General knowledge

Description

General knowledge is a main requirement, that every person needs to know in order to understand the

basic tools.

General Objective: Basic features of the program

Specific Objectives Prerequisites Evaluation method

Basic video Not required Exercise

Basic audio Not required Exercise

Basic Text Not required Exercise

Upload files Not required Exercise

Basic graphs Not required Exercise



General Objective: Advanced features of the program


Map orientation Not required Exercise

Basic scenarios Not required Exercise

Advanced functions Basic course Exercise/Quiz

Basic programming 1 Not required Exercise/Quiz

Basic architecture 1 Not required Exercise/Quiz

Platform configuration and management

Description

Platform configuration and management is general understanding about the architecture and Data Base

infrastructure.

General Objective: Basic features of the program


Basic programming 2 General knowledge course Exercise

Basic architecture 2 General knowledge course Exercise

Basic BBDD management General knowledge course Exercise/Quiz

Basic Docker General knowledge course Quiz

General Objective: Advanced features of the program


Rabbit Basic course Exercise/Quiz

MongoDB management Basic course Exercise/Quiz

Docker composition Basic course Exercise/Quiz

Sentimental analysis Basic course Exercise

Advanced modules Basic course Exercise

deliverable d7.1 title: training curricula

Documents