vocabularies for description of accessibility issues in mmui Željko obrenović, raphaël troncy,...
TRANSCRIPT
![Page 1: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/1.jpg)
Vocabularies for Descriptionof Accessibility Issues in MMUI
Željko Obrenović, Raphaël Troncy, Lynda HardmanSemantic Media Interfaces, CWI, Amsterdam
[email protected] http://www.cwi.nl/~obrenovi/
![Page 2: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/2.jpg)
Introduction
• What (output) modalities aremost suitable in which situation?
• How should different (output) modalitiesbe combined?
• Using rich accessibility descriptionof MM UIs to answer these two questions
![Page 3: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/3.jpg)
Motivation
• Expressing human functionality andanatomical structures required by modalities– Capturing existing knowledge– Reasoning over modalities
• Limited scope, small non-standard vocabularies – Card, Mackinlay, Robertson:
• Morphological analysis of input device;
– Modality theory, Niels Ole Bernsen:• Based on taxonomy of output modalities;
• Linguistic, analog, arbitrary, static, media…
![Page 4: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/4.jpg)
Describing Accessibility Issuesin Multimodal User Interfaces
• MM user interfaces - systems that communicate a message, an effect,– Stimulating a particular human
functionality or anatomical structures
• Interaction constraints– Influence of various factors on human
anatomical structures and functionalities.
![Page 5: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/5.jpg)
Describing Accessibility Issuesin Multimodal User Interfaces
Modality
Modality
Modality
User constraints
Social constraints
Environment constraints
Device constraints
Accessibility issues(interaction constraints)
Mu
ltim
od
al is
su
es
Computer Human
Sensing
Perception
Cognition
Motor skills
Linguistic skills
![Page 6: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/6.jpg)
Interaction Modalities
Interaction Constraints
Interaction Context
described in terms of
ReasoningFramework
define
query result
Applications
described in terms ofVocabularies
![Page 7: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/7.jpg)
Vocabularies
WHO Resources
Bioinformatics
Additional concepts
ReasoningFramework
query result
Applications
Interaction Modalities
Interaction Constraints
Interaction Context
described in terms of
define
described in terms of
![Page 8: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/8.jpg)
Vocabularies: ICF1/2
• WHO International Classification ofFunctioning, Disability and Health (ICF):– 2001, 9 years revision, widely used in health
community
• Describing “the person in his/her world“– Applicable to all people, whatever their health
condition– Carefully designed to be relevant across cultures
as well as age groups and genders– Uses neutral terms (function vs. disease)– Allows for an assessment of the degree of disability
![Page 9: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/9.jpg)
Vocabularies: ICF2/2
• Around 1500 concepts:– Body functions and structure– Activities (tasks and actions by individual)
and participation (involvement in a life situation)– Environmental factors
• Problems with formalization– Not defined by knowledge engineering experts
• Formalized ICF Checklist: 180 core concepts
• Possibility to reuse millions of profilesand statistical data expressed by ICF!
![Page 10: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/10.jpg)
Vocabularies: Anatomy
• Foundational Model of Anatomy (FMA)– Detailed description of human anatomy– Open source and available for general use– Used in bioinformatic community– ~ 100 000 concepts– Available in OWL format– Problems: size
![Page 11: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/11.jpg)
Vocabularies: Interaction Effects• Some examples:
– Gestalt visual grouping• By similarity, motion, texture, symmetry, proximity,
parallelism, closure, good continuation
– Gestalt visual highlighting• By color, polarity, brightness, orientation, size, motion,
flicker, depth, shape
– 3D cues• Visual: stereo vision, motion parallax, linear
perspective, rel. size, shadow, familiar size, interposition, horizon
• Audio: inter-aural time/intensity difference, HRTF, head movement, echo, attenuation of high frequencies
![Page 12: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/12.jpg)
Vocabularies
WHO Resources
Bioinformatics
Additional concepts
ReasoningFramework
query result
Applicationsdefine
described in terms of
Interaction Context
described in terms of
Interaction Modalities
Interaction Constraints
![Page 13: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/13.jpg)
Example Modality: Speech
![Page 14: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/14.jpg)
Example Constraint: Noise
![Page 15: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/15.jpg)
Interaction Constraints
Interaction Context
•Device profile•User profile•Environment profile
Userinterface
described in terms of
described in terms of
Vocabularies
WHO Resources
Bioinformatics
Additional concepts
ReasoningFramework
query result
Interaction Modalities
define
described in terms of
Applications
![Page 16: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/16.jpg)
Reasoning: Using Descriptions
• Possible with rich and explicit descriptionsof (implicit) accessibility requirements
• What modalities are suitable in which situation?– Combining descriptions of constraints
and modalities (speech in noisy environment?)
• How should different modalities be combined? – Combining modality descriptions to identify conflicting
requirements (speech & short term memory);
![Page 17: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/17.jpg)
Implementation
• In early stage• Using Semantic Web technologies:
– OWL, RDF standards– Tools and database to support reasoning
• Sesame, Jena…;
– Reusing some of our and existing toolsfor data visualization and exploration• Facet browsing
![Page 18: Vocabularies for Description of Accessibility Issues in MMUI Željko Obrenović, Raphaël Troncy, Lynda Hardman Semantic Media Interfaces, CWI, Amsterdam](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649ee15503460f94bf1e4d/html5/thumbnails/18.jpg)
Conclusions
• Expressing human functionality andanatomical structures required by modalities
• Relations with other projects:– W3C Multimodal Interaction Framework - content– W3C Accessibility Initiative (WAI) - guidelines
• Future work:– Full implementation of reasoning framework– Solving problems with vocabularies:
• No relations among concepts, overlapping – New vocabularies are coming:
• The Human Brain Project (with FMA)• DARPA Digital Human – unifying medical ontologies