subject matter model cognitive processes...

Subject Matter Model

Cognitive Processes Model

Integration Model & Interoperability Guidelines

Interaction & Presentation Model

2

Motivation

3

“The works of its artists, architects, musicians, writers and scientists, the […] tangibleand intangible works through which the creativity of that people finds expression:languages, rites, beliefs, historic places and monuments, literature, works of art,archives and libraries”. UNESCO World Conference on Cultural Policies, Mexico, 1982

4

relevance as it envelopes not only what we know about our past, but also our currentand future vision of what we are.

: employment, social cohesion, potential to revitalizeareas and promote education and sustainable tourism.- €3.2 billion was invested in heritage (European Regional Development Fund)- €1.2 billion on rural heritage (Agricultural Fund for Rural Development)- around €100 million worth of heritage research was funded from the 7th Framework

Programme.In 2014-2020 these numbers are rising.(European Commission, Creative Funding Europe Programme Presentation)

Motivation

Problem

6

7

Problem

Research Goals

8

Research Methodology

9

Design Science research methodology for information systems research. Peffers et al. 2007

Hypothetico-deductive approach

Tested in assistance IS

Oriented towards solutions

Flexible

Validation possibilities: case study,action research, empirical studies,statistical approaches…

‐

Research Methodology

10

Initial Hypothesis

11

Research Questions

12

Exploring the Problem: state of art

13

14

Exploring the Problem: Conceptual Approaches

15

Exploring the Problem: Process ApproachesCOGNITIVE PROCESSES CHARACTERIZATION VISUAL SOFTWARE-ASSISTANCE

How can we extract what cognitive processes are the user/professional performing with data?

Approaches:

Shadowing, interviews and surveysDiscourse analysis

Test-bed method User centred Feedback

• Vagueness and subjectivity• Bias

Analytical method IA support Big amount of data

• Check with experts• Static

16

COGNITIVE PROCESSES CHARACTERIZATION

Literature Review

Tested characterization IA support Big amount of data

• Doman-dependent• Different levels of

abstraction & goals

17

Source Characterisation Abstraction Discipline Formalization

COGNITIVE PROCESSES CHARACTERIZATION

18

Information Retrieval Approach Ad Hoc Modelling Approach

• Automatic processing focused

Semantic approach

• Domain focused

Extraction of information from textualsources in a quantitative level (countingoccurrences, frequency indicators…).

Unable to provide information about thecomplex semantic relations betweenelements in the text.

Not as powerful as an approach based onheavy linguistic techniques- (causalrelations…)

Not possible to attain a high degree of standardization.

Working inside the context of a well-defined domain

Accurate and rich semantic management

Extraction of information related toequivalence relations or hierarchicalrelations (thesauri, topic maps…).

A new model creation for each application

Heuristic & Probabilistic approach

Rolland, C., 2013. Conceptual Modeling and Natural LanguageAnalysis, in Seminal Contributions to Information SystemsEngineering. Springer Berlin Heidelberg. 57-61.

COGNITIVE PROCESSES CHARACTERIZATION: EXTRACTING COGNITIVE PROCESSES FROM TEXTUAL SOURCES

19

Discourse Analysis

Suitable and Agile basis to structure and to extract cognitive processes & semantic relations in free-style textual sources.

Hobbs, J.R., 1985. On the coherence and structure of discourse.

Iterative text breaking: until reaching the

level of single clauses

1Hobbs’s

coherence relations labelling

between clauses

2Coherence

relation characterization

3Validation of

the knowledge underlined.

4

From Linguistics, several techniques.

Emerged from the necessity to discover meaning in terms of the narrative elements presented in a discourse.

Used to study verbal communications and textual documents: organization of the language, above or under sentence or paragraph level.

Successfully applied to different fields, with different degrees of automation. e.g. Legal texts, biomedicine…

20

Cognitive Processes in CH: Characterization proposed

21

Based on experience:

◦ Fujitsu, Microsoft, SQI, JPL, plus other 20, Input from 25+ countries, Australian AS 4651 (2004)

◦ Field consulting from UTS, Neco, Top-notch academic research

◦ Led from academia, Submission February 20th, 2012

ISO/IEC 24744 is an international standard to model methodologies in IS

We extended it to apply discourse analysis.

ISO/IEC 24744

Language

Name

ModelKind

/Name

/Description

Model

/CreationTime

/LastChangeTime

/Status

ModelUnitModelUnitKind

/Name

Definition

1..*1..*Uses

10..*

L1: Language

Name = “Discourse Language”

DiscourseModel DiscourseModelUnit

ISO/IEC 24744

Derived language

MUK1: ModelUnitKind

Name = “Discourse Model Unit”

Definition = ...

MK1: ModelKind

Name = “Discourse Model”

Description = …

Our Modelling Language

3 different relevant areas:

• Narrative or discursive elements in the source text themselves (sentences, clauses…)

• Domain constructs, entities of the reality. Representing following an object-oriented approach)

• Hobbs’ coherence relations. 22

DiscourseModelUnit

CoherenceRelationDiscourseElement

Content

Feature

Name

Type

Value

Content

Entity

Id

1 1

Is AnInstance Of

Is AnInstance Of

10..*

0..* 1

0..* 0..*

DomainElement

LinguisticRelationFormalRelation

EvaluationRelation ExplanationRelation

ExemplificationRelation

GeneralizationRelation

OccassionRelation

BackgroundRelation

ElaborationRelation

ParallelismRelation

ViolatedExpectationRelation

ContrastRelation

DiscourseFragment Discourse

Clause Sentence

0..*

1…*

1..*

Reference

0..* 10..* 0..*

Refers To Makes

0..1

1

23


24

Explanation Relation Model


“Six of these millstones (…) had been thrown away as aresult of becoming overly worn or havingbroken.”[Gianotti et al. 2011].

Contrast Relation Model

“Another stone ring was also identified in the southeastquadrant although, unlike the former, it is made of smalland medium-sized blocks laid over the underlyingrock.”[Gianotti et al. 2011].

Our Methodological Elements

25

ISO/IEC 24744 PROCESS DIAGRAM ISO/IEC 24744 ACTION DIAGRAM

How can we express visual software-assistance?

Modelling approachesAbstract specifications: code libraries & languages Formal representation of

presentation & interaction mechanisms Existing tools Domain & implementation

independent

• Modelling expertise needs• MDE Oriented

Formal representation ofpresentation & interaction mechanisms Existing tools

• Technical expertise needs• Conceptual connection with

implementation

26

InfoVis techniques

Tested assistance Existing tools

• Domain-dependent• Different levels of

abstraction & goals• Not enough formalized

VISUAL SOFTWARE-ASSISTANCE

Exploring the Problem: Interaction Approaches

27

Exploring the Problem: Interaction Approaches

InfoVis Techniques+

RIA (Rich Internet Applications) patterns.

VISUAL SOFTWARE-ASSISTANCE

Exploring the problem: prior empirical studies

28

1. Stacked Bar Chart 2. Line-based Chart 3. Simple Bar Chart 4. Bubble-based Chart 5. Customized Venn Diagram 6. Treemap7. Geographical map 8. Scatter Chart

E. Study 3

The Study of Information Visualisation Techniques in

Cultural Heritage

E. Study 1

An Analysis of Textual Sources

Using our Discourse Analysis

Methodology

E. Study 2

Empirical Validation of the Cognitive Processes

Characterisation in Cultural Heritage Via

Thinking Aloud Protocols

InfoVisTechniques

Wohlin, C., P. Runeson, et al., 2012. Experimentation in software engineering. Springer Science & Business Media.

E. Study 4

Empirical Validation of Visualisation Techniques

Via Thinking Aloud Protocols

How can we assist these cognitive processes through software?• Visualization and adapted interaction patterns• Knowledge extraction techniques: Pattern recognition suggestion

Knowledge Generation Process

Assisted Knowledge Generation Process

31

Problem: in summary…

32

Problem: in summary…

Solution Proposal

34

CH Subject Matter Model: CHARM as reference model, anextension for each application.We define ConML packaging and clustering mechanisms forknowledge generation purposes.

Cognitive Processes Model: Cognitive processes are an intrinsicelement which must be modelled if we wish to assist to knowledgegeneration.

Interaction and Presentation Model, based on an hierarchy ofpatterns, to define the software-assistance offered.

Integration Pivot Model for integration and interoperabilitybetween: Subject Matter, Cognitive Processes and Interaction &Presentation Mechanisms.

Subject Matter Model

35

Standardization vs. Customization: Adopt + Extend approach

Archaeological Valorization Hillfort

Research Valorization

Discipline: 1 Text

Construction Constructive Element

Constructed Entity (A)

Material: 1..* enum Material

0..*

1..*

Wall

0..*1..*

Area

Is Located In

0..*

1

Valorizes 1..*0..*

Object

Evaluable Entity (A)Valorization (A)

Content: 1 Text

Valorizes 1..*0..*

Object

Is Located In

0..*

1

Archaeological Site

Event (A)

Moment: 1 Time

Event of Discovery

Derived EntityIs Result Of

1

0..*

Occurs To 1..* 0..*

Not only for museums

or spatial approaches

Support for Cultural Heritage

Domain particularities:

Subjectivity, Temporality, Vagueness

CH Object Oriented ontology

ConML Packages and Clusters

36

To delimit sub-areas of the subjectwithin the particular model being workedwith in each case of software assistance.

To indicate semantically-cohesive groups and differentlevels of importance in concepts.

To establish the basis for the interpretation mechanismsused during knowledge generation processes.

37

Cognitive Processes ModelBased on Hobbs coherence relations & ourcognitive processes characterization

Based on RIA patterns modelling & ourvisualization techniques results with CulturalHeritage specialists

Interaction& Presentation Model

38

LEVEL 2: Timeline IU

?Problem: analysis needs in temporalaspects of the data: temporal attributes,classes with values in attributes whichchange over time.

?Solution: To organise the informationin order to select, visualise and interactwith values of variable attributes overtime interface.

Interaction & Presentation Model: a pattern example

Integration Pivot Model

39

2 Types of Interoperability guidelines defined:

DOMAIN INDEPENDENT, e.g.Building Cognitive Processes-> STRUCTURE IU

CULTURAL HERITAGE DEPENDENT, e.g.If the main class of the package concerns

stratigraphic information and the cognitive process to assist is Clustering -> SEQUENTIAL IU

Do the software framework models allow us…

… regarding the three aspects necessary in order to provide knowledge generation assistance in a ?

…and the offered in this case?

Validation

41

42

Analytical Validation: A Romea

• Morphological structure of the mound• Oldest chronological adscription• Later phases that can be attributed to the barrow (phases

of occupation, abandonment, change of use, etc.)• Do the material findings correspond to the stratigraphic

sequence of the site?• Are there any nearby areas of activity generated by the

presence of groups of humans linked to the barrow duringits construction or even before?

RESEARCH CHALLENGES:

FRAMEWORK APPLICATION STEPS:

1. A ROMEA SUBJECT MODEL: TYPES & INSTANCES, EXTENSION2. A ROMEA SUBJECT MODEL: PACKAGES & CLUSTERS3. A ROMEA COGNITIVE PROCESSES FOR ASSISTANCE4. A ROMEA INTERACTION PATTERNS OFFERED5. A ROMEA INTEGRATION PIVOT MODEL DEFINED

43

A Romea Subject Model

44

A Romea Subject Model

CHARM Extension:

183 classes, 9 packages, 11 clusters

Implemented as relational database: types & instances

3481 objects, 4050 links, 11601 values

45

A Romea Cognitive Processes Interaction & Presentation Model

Cognitive Processes Instantiation:

4 Inference types supported

Interaction & Presentation Instantiation:

6 Interaction Units

46

A Romea Integration Model

IntegrationPivot Model Instantiation

Interoperability GuidelinesImplementation

Analytical Validation achievement

47

48

Empirical Validation

Implementation of the full framework using iOS technology, version 8.4 SDK (Software Development Kit) for development in Apple iPad devices.

Relational database implementation.CoreData library for short memory data.

49

Empirical Validation: Prototype

COGNITIVE PROCESS & PACKAGE SELECTIONCASE STUDY SELECTION

50


STRUCTURE IU VALUE COMBINATION IU

51


SEQUENTIAL IU TIMELINE IU

Empirical Validation

52

Spreadsheet vs Assistance Prototype

Response Variables/Metrics:

Statistical Paired design modelblocked by experimental objects

Wohlin Methodology, 2 sessions

16 heterogeneous participants,sample statistically relevant

1. Accuracy (Task Precision)2. Efficiency (Time)3. Productivity (Task Precision/ Time)4. Satisfaction (PEOU, PU, ITU)5. Quality of the Knowledge Generated (Mistakes and Report Satisfaction)

A ROMEA

53

Empirical Validation: Tasks

H: Partner textual abstract evaluation

FORNO

DOS MOUROS

A & D (Combining): How many?

B (Clustering): Grouping by averagefragmentation

C & F (Situating): Chronologicalattributes and temporal intervals

E (Building): Name of attributes in theinformation structure of stratigraphy

G: Textual valoration (Abstractproduction): What heritageconclusions do you draw from thecase study in question?

Wohlin methodology, based on Cook & Campbell

4 threats types: Conclusion, Construct, Internal and External Validity

24 threats analyzed, e.g.:

54

Empirical Validation: Threats of validity study

Conclusion Validity

Construct Validity

55

Empirical Validation: Results on Accuracy

56

Empirical Validation: Results on Satisfaction

Empirical Validation: Results on Efficiency

57

Empirical Validation: Conclusions

Hypothetico-deductive conclusions

58

7 sub-problems identified, mainly absence of cognitive processes formalization and

traceability treatment.

Hobb’s relations but organized in terms of user’s goal. Own characterization defined.

Bubble charts & Specific timelines and structure mechanisms.

Hierarchical pattern structure defined.

Using the framework proposed, we improved accuracy, productivity, user

satisfaction and knowledge generation in reports (only InfoVis assistance).

Critical Analysis

59

Empirical Validation Restrictions:

1. The efficiency issue: …produced by different levels of expertise between methods?…our prototype learnability is not enough?

2. Sample validation only in method’s comparison

Other framework issues:

1. Assistance defined only in terms of visualization techniques, lack of treatment of knowledge extraction mechanisms (proposed in the initial hypothesis).

2. Different Cultural Heritage visions, different sub-disciplines.

3. There is no an unique software-assistance form.

Larger empirical study with different

expertise levels

Knowledge extraction assistance

incorporation

Model formalization Abstraction levels Flexibility (adopt + extend approach) Domain Independence of cognitive &

interaction models

Contributions of the Ph.D. research

61

1.

2.

3.

4.

5.

62

Ph.D. Communication

4 Conferences CORE B Ph.D. CAISE’13 CORE A JOCCH ACM Journal8 Books Chapters

Archaeology & Antiquity Sciences MSc. (USC, Spain).

Software Engineering MSc. (Andrés Bello U., Chile).

CSIC Digital Humanities Laboratory (UNED)National CNCR Center, ChileBasque Country University (UPV/EHU)

8 Workshops and Seminars

Organizing committee member CAiSE 2013. Valencia, Spain.Organizing committee member JIA2012. S. de Compostela, Spain.

Hands-On Archaeological Conceptual Modelling WS, CAA 2014.Co-chair of CAA sessions from 2011 to 2016.

Chair in RCIS 2014 & RCIS 2016 Sessions.CAA Program Committee Member since 2011.RCIS Program Committee Member since 2012.

SLE 2016 Artefact Evaluation Committee Member.

MARIOL: Abstract Reference Models for Information on Cultural Heritage. MINECO. 2014-2017.ARIADNE: Advanced Research Infrastructure for Archaeological Dataset Networking in Europe. FP7 INFRASTRUCTURES. 2013-2017.RCINDIM: Tecnologías e Información para la gestión Integrada de Desastres Naturales. CONICYT FONDAP, Chile. 2013-2017.Textura: Consolider INGENIO 2010 Tecnologías para la conservación y valorización del Patrimonio Cultural. 2007-2013.MIRFOL: Metodoloxía Integral Para A Representación Formal Do Patrimonio Cultural. INCITE, Xunta de Galicia. 2009-2012.

Future roadmap

Detecting other cognitive processes & software interaction patterns in Cultural Heritage

which should be incorporated into the proposed conceptual framework.

63

Testing the framework in other complex heritage areas: Anthropology & Art?

Research ConnectionsOther integration proposals of cognitiveprocesses in IS: Intention Mining.

How analysts use this conceptualframework in order to define softwareassistance?

New metrics to evaluate the level ofassistance obtained in knowledgegeneration in any domain.

Incorporate MDE paradigms into theconceptual framework presented

Application of discourse analysistechniques in SE, especially in therequirements phase.

64

Emerging research linesTAP (Thinking Aloud) for thecharacterisation, definition and extractionof cognitive processes in Humanities areas.

ISO/IEC 24744 Discourse AnalysisMethodology for traceability of theknowledge generated, specially relevant inhumanities, but also explored in RE.

Semi-automatization of the software-assistance, expanding the types ofassistance offered: via knowledge extractiontechniques & TDM applications.

“In literature and in life (and in research!)

we ultimately pursue, not conclusions, but beginnings ”

- Sam Tanenhaus

subject matter model cognitive processes...

Documents