1 challenge the future considering cognitive aspects in designing cyber-physical systems: an...
TRANSCRIPT
1Challenge the future
Considering cognitive aspects in designing cyber-physical systems:an emerging need for transdisciplinarity
Wilfred van der Vegte and Regine VroomDelft University of TechnologyFaculty of Industrial Design EngineeringDepartment of Design Engineering
2Challenge the future
Faculty of Industrial Design Engineering
3Challenge the future
Contents
• Cyber-physical systems
• CPSs design – involved disciplines
• Disciplinary approaches: mono, multi, inter, intra, and trans
• Flavours of transdisciplinarity
• Two directions of research:
• Simulating cognitive loads and processing times
• Informing systems and mental models
• Concluding remarks
4Challenge the future
Cyber-Physical systems (CPSs)
CPSs are integrations of computation with physical processes, in which embedded computers and networks monitor and control the physical processes, usually with feedback loops where physical processes affect computations and vice versa.
Example of a CPS:Swarming Micro Air VehicleNetwork (SMAVNET) @ EPFL, CH• Rapidly creates communication
networks for rescuers in disasterareas
• Sensor networking technologies• Swarm intelligence
5Challenge the future
CPSs design @ IDE– involved disciplines
• Industrial Design Engineering (IDE), • Cognitive Psychology, • Psychophysiology, • Information and Communication Technology (ICT) • Disciplines commonly involved in an interdisciplinary faculty
of IDE such as:• Materials technology• Manufacturing technology• Human factors• Electronics• Mechanical engineering• Marketing• etc.
6Challenge the future
Addressing cognitive aspects
• In predecessors of CPSs (mechatronic/smart systems, etc.) ICT and physics were already heavily involved.CPSs will increasingly incorporate (distributed) artificial cognitionin interaction with human cognitionHandling cognitive psychology issues will be a key challenge in the near future of CPS development
• Cognition-related issues:
• Allocation of cognitive tasks between human and CPS
• Cognitive matching of inputs/outputs between human and CPS
• Preventing information overload of human users
• Enabling CPSs as safety-critical systems
• Objective: cognitive symbiosis between human and CPS
7Challenge the future
Transdisciplinary vs. intra-, inter- and multidisciplinary
Flavours of transdisciplinarity
engineeringdesign
other area of development
(e.g., healthcare)
end users/consumers(e.g., product users)
end users/consumers(e.g., patients)
engineeringscience
other area ofscience
(e.g., medical)mono
mono mono
intra
intra
intra
intra
inter
multi
multi
inter
trans
8Challenge the future
engineeringdesign
other area of development
(e.g., healthcare)
end users/consumers(e.g., product users)
end users/consumers(e.g., patients)
engineeringscience
other area ofscience
(e.g., medical)
transdisciplinary design=
transdisciplinary research
Flavours of transdisciplinarity
9Challenge the future
Two directions of research
1. Simulating cognitive loads and processing times
2. Informing systems and mental models
10Challenge the future
1.0 Simulating cognitive loads and processing times
• Key application area: deployment of CPSs as safety-critical systems
• Revision of decision-making responsibilitiesCPS human
• Simulation of human mental processes together with models of products and systems (in particular, CPS)
• Goal: evaluate CPS during development identify bottlenecks to be addressed in the CPS’s designincluding service design, task design/allocation
11Challenge the future
How to simulate human thinking and human reasoning?
1.1 Human-cyber-physical systems – how can we simulate?
• Interactive simulation vs. fully virtual simulation:
• Safety-critical systems identification of incidents happening once in ~1,000 years.
• Interactive human-in-the-loop simulation must be real-time,but we cannot run a simulation for 1,000 years!
• we need faster-than-real-time simulation fully virtual, even humans
Use simulation tools common in embedded systems engineering
(procedural logic, state machines)
Avoid time-consuming physics simulations based on geometric discretisation (e.g., FEM): use simplified models instead.
Take shortcuts: disregard perception, motor skills, etc.
humanCPS; environment
information processing
physics
12Challenge the future
1.2 Simulating human thinking and human reasoning
• Two aspects:logic of decision making and processing time of decision making
• Logic of decision making:
• What action is taken under what condition?e.g. “IF cup is full THEN retrieve cup from machine”:straightforward execution of ‘normal’ use,assuming a particular history of preceding events.
• But can a simulation predict a user acting according tothe production rule “IF cup is full THEN stick finger in it”? →unlikely!
• Yet we can try to generate typical aberrations from ‘regular use’:so-called error phenotypes (Hollnagel):actions accidentally in wrong order, accidental repetition, etc.,by applying systematic variations
13Challenge the future
1.3 Simulating human thinking and human reasoning: processing time
• Processing time:How long does it take to accomplish a given action, taking into account aspects such as memory retrieval, memory capacity, learning, multitasking, distraction, etc.
• These aspects can be simulated using cognitive architectures such as ACT-R
• A cognitive architecture is
• a blueprint of the human mind
• based on findings from brain science
• filled with psychologically validated task modelsexpressed as production rules
14Challenge the future
1.4 ACT-R cognitive architecture
simulationof CPS &environment
ACT-Rsimulation(human)
declarative module ( temporal cortex / hippocampus)
intentional module( not identified)
external world
retrieval buffer( ventrolateral prefrontal cortex)
visual module( occipital cortex)
visual buffer( parietal cortex)
motor buffer( motor cortex)
goal buffer ( dorsolateral prefrontal cortex)
motor module (motor cortex / cerebellum)
central production system
( basal ganglia)
• ACT-R models are task specific, programmed in LISP by skilled, dedicated cognitive scientists
• Most tasks require scientists to create new customized models, that have to be validated in laboratory experiments with human subjects
• Intensive collaboration between cognitive scientists and designers of CPSs seems inevitable
15Challenge the future
1.5 Example CPS for simulating cognitive loads & processing times:
Advanced support of emergency response
16Challenge the future
2.0 Informing systems & mental models
Informing CPSs (e.g. informing public traffic systems)• aims to find novel means to inform users and to find new
symbiotic relations between human and cyber-physical systems;
• based on which designers can be supported in the early stages of CPS development;
• the objective is to avoid situations where users are mentally or perceptually overloaded and to precisely give the information that will help to take a right decision to react.
17Challenge the future
2.1 Project purpose
The purpose of this project is to gain a better understanding in the manner in which MMs influence our interaction with the informing part of CPSs, and to provide guidelines for designers based on these insights
Cyber-PhysicalSystem
Human “system”
Processing
CPS output adapted to the cognitive capabilities of
individual user(s) in a specific situation
CPS input
Sensors /detectors
Human input
Senses
Brain (cognition, including knowledge, experiences,
reasoning)
Human output
Human output detected by a CPS
CPS informs (or offers other functionality) to human
Current situatio
n
Current detection e.g. through motion detection, smart
phone connection, id tag, …
18Challenge the future
2.2 Informing systems & mental models
Goal: Include cognitive insights to influence the adaptability of CPSs.Approach: Study the behavior of mental models
Future situatio
nMental model
Cyber-PhysicalSystem
Human “system”
Processing
CPS output adapted to the cognitive capabilities of
individual user(s) in a specific situation
CPS input
Sensors /detectors
Human input
Senses
Brain (cognition, including knowledge, experiences,
reasoning)
Human output
Mental model: internal representation that people hold of
an external reality that allows them to explain, interact, and
predict that reality (from cognitive psychology)
Mental model
19Challenge the future
Mental model
Cyber-PhysicalSystem
Human “system”
Processing
CPS output adapted to the cognitive capabilities of
individual user(s) in a specific situation
CPS input
Sensors /detectors
Human input
Senses
Brain (cognition, including knowledge, experiences,
reasoning)
Human output
2.3 Future situation
CognitiveScience
Design Engineering
precisely give the information that will help to take a right decision to
react
Designerly cognitiveinsights
Current situatio
n
20Challenge the future
2.4 Designerly cognitive insights
a. Study behaviour of mental models: • Is there inertia when switching from one mental model towards
another?• E.g. if an unexpected situation occur, will there be a different reaction
on the same situation if the person was reading an exciting book than
when he was playing football? Difficulty: perception influences can’t be
reset (“undo” or “delete”)
• How to identify inaccuracies and gaps in a mental model (i.e. in a person’s knowledge and experience)?• Mental models are inaccurate and incomplete. Insights in how to
determine the gaps and the faults incorporate clues to better inform
people.
21Challenge the future
2.4 Designerly cognitive insights cont’d
b. Study relationships between mental states and cognition at one side and physical human data (facial expressions, gestures, heart beat etc., i.e. psychophysiology) on the other side.• In addition to search for direct determination methods, indirect
measurements might be useful: some facial expression may indicate
that a person doesn’t understand a message for example.
c. How to effectively address the major gaps and faults in a mental model? • Effect of senses to address, effect of amplitude of the message
(audio volume, pressure level in haptic information, etc.)
The bridge towards guidelines for designers of informing CPSs: insight in the mental model states and behaviour will enable designers to design CPSs with adaptive capabilities on the user’s cognition.
22Challenge the future
Concluding remarks – cognitive modelling
• Two research directions
• one to allow virtual testing of CPSs by designers, taking into account speed and capacity of human cognition in the interaction with CPSs
• one to provide knowledge to designers, so that CPSs can adapt themselves to mental models maintained by their users
• Both entail transdisciplinary collaboration with cognitive scientists –from disjunct research communities with different scientific approaches
• Cognitive architectures are based on empirical laboratory experiments
• Mental models are captured based on interviews
• Common goal: achieve symbiotic relationship between human and CPS
23Challenge the future
Concluding remarks – Transdisciplinary design (or: research)
Defining what is transdisciplinary and what is not, is probably not as simple as we have suggested:• Knowledge value chain often more complex than
research development/design application• What is one discipline? How to deal with hierarchies of
disciplines?(e.g. engineering electrical, mechanical, civil, ...)
• If we have learned enough from working with expert scientists from another discipline, can we eventually do the trick ourselves?Does it mean that a new discipline has formed and the activity is no longer ‘transdisciplinary’? If so, is that bad?