meta-adaptivity and self-regulation: towards the next generation of adaptive systems alexandros...
TRANSCRIPT
![Page 1: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/1.jpg)
Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems
Alexandros Paramythis
Johannes Kepler University
Linz, Austria
![Page 2: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/2.jpg)
Paramythis, Meta-adaptivity and Self-regulation 2
Outline Introduction
» Problem space
Meta-adaptivity and self-regulation
» Definitions
» Examples
» Operational requirements
An example
» The system
» Evolution of adaptive behaviour
Adaptive system development revisited
» New possibilities
» New requirements
» Applicability
» Overhead
» Constraints
Discussion
![Page 3: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/3.jpg)
Introduction
![Page 4: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/4.jpg)
Paramythis, Meta-adaptivity and Self-regulation 4
Glossary! SeR Self-Regulation AS Adaptive System(s) SeRAS Self-Regulating Adaptive System(s)
![Page 5: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/5.jpg)
Paramythis, Meta-adaptivity and Self-regulation 5
Problem space (1/2) Design / authoring of adaptive systems still hard,
despite recent progress in
» availability of frameworks and tools
» accumulation of validated design knowledge
Two major difficulties (among others):
» sometimes the design corpus is unavoidably poor, incomplete, or based on intuition
» “evolution” of an adaptive system is an exclusively “manual” task
![Page 6: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/6.jpg)
Paramythis, Meta-adaptivity and Self-regulation 6
Problem space (2/2) A symmetric situation in the evaluation of adaptive
systems
Because of the complexity involved,
» people are often not willing to interfere with a working system; thus,
» deployed adaptive systems are even less likely to be evolved than their non-adaptive counterparts
![Page 7: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/7.jpg)
Paramythis, Meta-adaptivity and Self-regulation 7
Meta-adaptivity to the rescue Let the system “participate”, by
» “trying out” alternative adaptive behaviours / strategies (e.g., when designers are unsure about their applicability)
» “building up” new pieces of adaptation design knowledge through the “findings” of such trials
» aggregating results across a large number of users, to derive knowledge even in the absence of trials
To achieve the above, use
» Meta-adaptivity, and, specifically, self-regulation
![Page 8: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/8.jpg)
Meta-adaptivity and Self-regulation
![Page 9: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/9.jpg)
Paramythis, Meta-adaptivity and Self-regulation 9
Meta-adaptivity and the second-level cycle
Environment Adaptor mechanism
Users
input
output
ADAPTIVEADAPTIVETHEORYTHEORY
low level theory
variant
Lower Level Adaptor
HigherLevel
Adaptor
User Interface Variants(flexibility)
Interaction Cues(evidence of user needs)
User / TaskModels(the needs)
Logical diagram for a two-level adaptation architecture for user interfaces;adapted from (Totterdel and Rautenbach, 1990)
![Page 10: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/10.jpg)
Paramythis, Meta-adaptivity and Self-regulation 10
Self-regulation: Main points Self-regulating adaptive systems (SeRAS)
» are entry level “meta-adaptive” systems
» have “second-level” adaptation cycle
» “learn” dynamically how to modify their behaviour to accommodate different users, context of user, etc.
» typically “know” a priori the alternative adaptive behaviours / strategies
![Page 11: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/11.jpg)
Paramythis, Meta-adaptivity and Self-regulation 11
Examples of SeRAS (1/2) Recommender system
» Recommends items to users
» Capable of multiple recommendation strategies• Item characteristics, User characteristics, Collaborative filtering,
combinations thereof
» Capable of switching between strategies
» The criteria for success may vary• E.g., user never shows interest in recommended items
![Page 12: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/12.jpg)
Paramythis, Meta-adaptivity and Self-regulation 12
Examples of SeRAS (2/2) Adaptive Collaboration Support System
» Adaptively supports the establishment of groups that collaborate on a topic (e.g., learning task)
» Support is in the form of “neighbourhood” visualisations
» Capable of calculating / visualising neighbourhoods in multiple ways
» Criteria for success of a given neighbourhood algorithm / visualisation
• E.g., actual group establishment and longevity
![Page 13: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/13.jpg)
Paramythis, Meta-adaptivity and Self-regulation 13
Operational Requirements for SeRAS Observing interaction
Observing adaptive behaviour
Self-evaluation
Modifying adaptive behaviour
![Page 14: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/14.jpg)
Paramythis, Meta-adaptivity and Self-regulation 14
SeRAS Requirements: Observing interaction Already part of the first-level adaptation cycle for all
adaptive systems
» Prerequisite for adaptive behaviour in the first place!
No additional implications.
![Page 15: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/15.jpg)
Paramythis, Meta-adaptivity and Self-regulation 15
SeRAS Requirements: Observing adaptive behaviour
Adaptive behaviour must be “broken down” to distinct constituents
» Granularity may vary
This requirement is possible to relax somewhat, but
It restricts the range of adaptive systems on which self-regulation can be applied
![Page 16: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/16.jpg)
Paramythis, Meta-adaptivity and Self-regulation 16
SeRAS Requirements: Observing adaptive behaviour
Implications
» Notification when adaptations occur
» Identification of adaptations • Semantically interpretable, or, • Uniquely identifiable, or, at least,• Of a uniquely identifiable type
![Page 17: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/17.jpg)
Paramythis, Meta-adaptivity and Self-regulation 17
SeRAS Requirements: Self-evaluation Most demanding of the requirements
Involves assessment of (degree of) success of adaptive behaviours
An overwhelming range of possibilities!
Proposal: principled approach based on “expectations”
![Page 18: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/18.jpg)
Paramythis, Meta-adaptivity and Self-regulation 18
SeRAS Requirements: Self-evaluation Proposed approach
» Define expectations (adaptation “theory”)
» Operationalise expectations with respect to• User interactive behaviour• Changes in the modelled interaction state
» Provide SeRAS with expectations expressed in computable form
» Computation models may vary as needed
![Page 19: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/19.jpg)
Paramythis, Meta-adaptivity and Self-regulation 19
SeRAS Requirements: Self-evaluation Implications
» Quantifying changes in interaction state• Input: direct user input, current values from the static and dynamic
models of the system, “historical” values from the same models, as well as interim results from previous calculations
• Output: depends on computational approach• Computation: depends on system, but generality possible given
sufficient similarities in AS• Association with behaviours being evaluated: Understanding vs.
simple identification
![Page 20: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/20.jpg)
Paramythis, Meta-adaptivity and Self-regulation 20
SeRAS Requirements: Modifying adaptive behaviour
Implies either (or both) of
» Changing first-level strategy / theory• Most straightforward of the alternatives
» Overriding adaptation outcomes
Plausibility and feasibility depend, mainly, on the decision making approach of the adaptive system
![Page 21: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/21.jpg)
Paramythis, Meta-adaptivity and Self-regulation 21
SeRAS Requirements: Modifying adaptive behaviour
Implications of changing first-level strategy / theory
» The most readily attainable level of “intervention”
» Requires that alternative strategies are represented in a way that allows for:
• identifying them individually,• “knowing” whether they can be combined and in what ways, • (de-) activating them on demand
» Ideally, strategies would be conveyed to the system in a declarative manner
• alternatively, any approach which would result in a run-time model of adaptation would also suffice from a technical perspective
![Page 22: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/22.jpg)
Paramythis, Meta-adaptivity and Self-regulation 22
SeRAS Requirements: Modifying adaptive behaviour
Implications of overriding adaptation outcomes
» To override the outcome of adaptations, the system must be able to understand and predict that outcome
» This, in turn, implies the need for a “model” of adaptive behaviour (again, granularity may vary)
• nevertheless, more fine-grained than the strategy level
» It also implies that systems will be able to have models of reasonable interventions, in response to prescribed adaptive behaviours
» All in all, a very powerful approach, but quite some progress required before it is more readily attainable
![Page 23: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/23.jpg)
Paramythis, Meta-adaptivity and Self-regulation 23
The SeRAS spaceM
od
elli
ng
“black-box”
“wh
ite-
bo
x”
Decision Making
“bla
ck-b
ox”
“white-box”
•insufficient second-level inputs •self-evaluation possible only through global metrics and direct user feedback•interventions only at global scope and only in the form of disabling
•full-scale self-evaluation possible, but •difficult or impossible to associate self- evaluation context with adaptations•interventions only at global scope and only in the form of disabling
•self-evaluation possible only through global metrics and direct user feedback•fine-grained interventions possible, but•only external adaptation overriding feasible, and•impossible to associate interventions with adaptation logic
•full-scale self-evaluation possible •fine-grained interventions possible
self-evaluation capabilities
interventioncapabilities
![Page 24: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/24.jpg)
The example
![Page 25: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/25.jpg)
Paramythis, Meta-adaptivity and Self-regulation 25
The system Along the lines of NetCoach and AHA!
Main characteristics
» Domain model: small, course-specific, module- and concept- oriented ontology as the
» User model: overlay model over the domain
» Updates in the user model: through direct observation and interpretation of user actions
» Adaptation logic: rule-based
» Adaptive function: generation of recommendations / predictions about the suitability of modules in relation to the user’s current knowledge
![Page 26: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/26.jpg)
Paramythis, Meta-adaptivity and Self-regulation 26
The design question What is the best way to convey system
recommendations / predictions to users?
» “Competing” adaptation strategies:
![Page 27: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/27.jpg)
Paramythis, Meta-adaptivity and Self-regulation 27
Evolution of adaptive behaviour Assumptions
» Design goal 1
• Provide navigation assistance so that users do not encounter concepts they are not “ready” for
» Design goal 2
• Apply as few restrictions as possible on navigation
» However, no evidence as to what strategy to use when / for whom
Keep in mind
» Iterative approach
• Example goes through 3 iterations, but this is rather arbitrary
» Can use design “input”
• Again, example assumes none, but no reason why one cannot have a corpus to start with
![Page 28: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/28.jpg)
Paramythis, Meta-adaptivity and Self-regulation 28
Iteration 1: “Tabula rasa”
A. No annotationA. No annotation
B. Colored linksB. Colored links
C. Colored bulletsC. Colored bullets
D. Custom iconsD. Custom icons
E. Link hidingE. Link hiding
“LA
_Str
ateg
ies”
Step 1: Define strategies
Strategies
A.
No
anno
tatio
n
B.
Col
ored
link
s
C. C
olor
ed b
ulle
ts
D. C
usto
m ic
ons
E.
Link
hid
ing
A. No annotation X X X X B. Colored links C. Colored bullets X D. Custom icons E. Link hiding
Step 2: Specify whether / how they can be combined
Use
r m
odel
M1: maximise (UC1 over UC2 ) using “LA_Strategies”
Step 3: Specify self-regulation metrics
UC1 UC2
# of links followed (total)# of links followed (total)UC2UC2
# of “ready” links followed# of “ready” links followedUC1UC1
Ready to start testing Ready to start testing
![Page 29: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/29.jpg)
Paramythis, Meta-adaptivity and Self-regulation 29
Iteration 2: Selection, categories, priorities (1/3)
A. No annotationA. No annotation
B. Colored linksB. Colored links
C. Colored bulletsC. Colored bullets
D. Custom iconsD. Custom icons
E. Link hidingE. Link hiding
“LA
_Str
ateg
ies”
Step 1: (system) Eliminate “unnecessary” strategies
D. + E. D. + E.
C. + E. C. + E.
B. + C. B. + C. B. + D. B. + D. B. + E. B. + E.
Step 2: (system) Provide preliminary categorisation and rankingStep 3: (designer) Add semantics
Cat. I
Cat. II
Cat. III
absolute freedom, no support
absolute freedom, explicit support
restricted navigation, partially enforced path
Continue testing Continue testing
![Page 30: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/30.jpg)
Paramythis, Meta-adaptivity and Self-regulation 30
Iteration 2: Selection, categories, priorities (2/3) Evidence from first round of testing
» Suggests that some strategies can be eliminated • e.g., because they did not satisfy the metric(s) for any user• in our case this will be B –link colour only– and all combinations
» Provides support for a tentative categorisation and “ranking” • on the basis of, e.g., how well strategies “performed”; more general
similarities in their effects; similarities in the user population on which they are most effective; etc.
» Semantics of findings “added” by the designers
![Page 31: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/31.jpg)
Paramythis, Meta-adaptivity and Self-regulation 31
Iteration 2: Selection, categories, priorities (3/3) Pending issues
» In which “direction” is the ranking to be applied / tested?
• “liberal” to “restrictive” (i.e., from I to III) for this example
» What are the “default” and / or “fallback” strategies?
• “default” is category I, and “fallback” is category III for this example
![Page 32: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/32.jpg)
Paramythis, Meta-adaptivity and Self-regulation 32
Iteration 3: Binding to concrete adaptation logic Evidence from second round of testing might result in
the following concrete adaptation logic
» Novices and students unfamiliar with the knowledge domain / material category III
» Within category III, apply new ranking: (a) Strategy E
(b) Combined D and E
(c) Combined C and E
» Reserve category II for users sufficiently familiar with the system and the recommendation mechanism
» Only use category I for experienced users that are also familiar with the knowledge domain
» …
![Page 33: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/33.jpg)
Adaptive system development revisited
![Page 34: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/34.jpg)
Paramythis, Meta-adaptivity and Self-regulation 34
New possibilities and requirements The good – self-evolution
» Categorisation or “clustering” of strategies and “ranking”
» Derivation of concrete adaptation knowledge / logic
The bad – non-trivial
» Several technical requirements• Quite a few, but most importantly, the system must be capable of
self-evaluation• A bonus requirement for the user modelling community: the type of
analysis that takes place as part of self-evaluation requires that the system has access to “historical” states of a user’s model
» Potentially more involved design process
![Page 35: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/35.jpg)
Paramythis, Meta-adaptivity and Self-regulation 35
New possibilities (1/2) Derivation of new adaptation knowledge / logic
» Analysis of similarities between the user models of users for whom adaptation strategies have resulted in comparable ouput from the self-regulation metrics
» Identification of “discriminating” user model attributes / values
» Human-assisted integration of new knowledge (enrichment with semantics also desirable in the process)
![Page 36: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/36.jpg)
Paramythis, Meta-adaptivity and Self-regulation 36
New possibilities (2/2) Categorisation or “clustering” of strategies and
“ranking”
» Identification of strategies that have similar effects (with respect to metrics) given sufficiently similar user models
• provisional “clustering”, as well as preliminary “cause and effect” patterns
» Identification of differentiating subsets of models that render certain strategies more effective than others in a given context
• combined with semantic meta-data (e.g., level of navigation freedom a strategy affords) this can be turned into a ranking
![Page 37: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/37.jpg)
Paramythis, Meta-adaptivity and Self-regulation 37
New requirements (1/2) Emerging technical requirements
» Adaptation strategies must be represented independently from the “driving” adaptation logic
• strong requirement
» Adaptation strategies (expressed potentially as sets of actions) must be applicable in combination
• weak requirement; can be simulated, albeit with extra work
» There must exist a representation of one or more adaptation “goals” that drive self-evaluation and selection / application of strategies
• again, a strong requirement, which, additionally precludes trivial approaches to the first two
![Page 38: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/38.jpg)
Paramythis, Meta-adaptivity and Self-regulation 38
New requirements (2/2) Emerging technical requirements (cont.)
» The system must be capable of maintaining and employing a ranking amongst (combinations of) strategies
• strong requirement if ranking is desirable; manually attempting that typically prohibitive because of overhead
» Most importantly, the system must be capable of self-evaluation itself, which requires
• that the system “knows” about alternatives, and• that the system has a way of assessing said alternatives with
respect to the degree they satisfy design requirements
» And a bonus requirement for the user modelling community• the type of analysis that takes place as part of self-evaluation
requires that the system has access to “historical” states of a user’s model
![Page 39: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/39.jpg)
Paramythis, Meta-adaptivity and Self-regulation 39
The role of meta-adaptivity Why do we need meta-adaptivity at all? What does it
bring to the design table?
» Capacity to test with end users large numbers of alternative behaviours
» (Semi-) automatic derivation of adaptation knowledge within the system
• although not discussed today, this approach can also be used to validate existing adaptation logic
» In short: it helps us design in ways that would be too expensive to apply manually
![Page 40: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/40.jpg)
Paramythis, Meta-adaptivity and Self-regulation 40
The role of self-regulation Why self-regulation as a specific meta-form?
» Although not an exceptionally sophisticated form of meta-adaptivity, self-regulation suffices for the scenaria discussed today
» It is easier to implement than other forms, because it does not presuppose the generation of new strategies by the system
» In cases where the first-level adaptation cycle uses a declarative form of specification of adaptive behaviours, self-regulation can be implemented orthogonally to the primary adaptation mechanism
![Page 41: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/41.jpg)
Paramythis, Meta-adaptivity and Self-regulation 41
Discussion (1/2) Applicability of the proposed approach?
» Of course, not universal
» Requires a “running system”
» Better suited to cases with a limited design corpus to boot, and / or with several competing design alternatives
Design / authoring overhead?
» On the one had, several additional tasks• formulation of strategies; formulation of metrics; review and validate
system results and propositions; incorporate improvements
» On the other hand though, major overlap with tasks that would need to be performed anyway in iterative design
• even though strategies and metrics may not need to be expressed in formal / computable forms
![Page 42: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/42.jpg)
Paramythis, Meta-adaptivity and Self-regulation 42
Discussion (2/2) Additional constraints:
» Not a replacement for user studies!• but possibly a tool to facilitate aspects of such studies
» A tool to be used with care • by nature, self-regulation can pose an even greater threat to
usability qualities if used carelessly, either at design time, or in deployed systems
And, of course, new roles for humans!
![Page 43: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/43.jpg)
Alexandros Paramythis
Johannes Kepler University
Linz, Austria
Thank you! . Questions?
![Page 44: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/44.jpg)
F.A.Q.s – Extra slides
![Page 45: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/45.jpg)
Paramythis, Meta-adaptivity and Self-regulation 45
F.A.Q.s Can self-regulation be implemented generically?
» Open-source framework with scheduled release in October is under active development
• Applicable to any XML “pipeline”; support for HTML as well• Pluggable user modelling components, and pluggable adaptation
logic (reasoning) components• Support for adaptation actions and strategies; independent from
either of the above through configurable “bindings”• Self-regulation features
– Integrated second-level cycle– Built-in expression language for formulating metrics – Analysis uses variations of existing data mining algorithms (mainly
different forms of multivariate analysis, reverse derivation of associations from clustering, etc.)
– Accumulated observations can be output as reports, or using the same expression language as metrics
– Too many more details to list here; ask for more info.
![Page 46: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/46.jpg)
Paramythis, Meta-adaptivity and Self-regulation 46
F.A.Q.s What are some examples of a self-regulation metrics?
» One approach, using absolute thresholds:(count of followed links when in state “ready” / count of followed links when in state “not-ready”) > 0.5
» Alternative approach with relative ordering: maximise (count of followed links when in state “ready” / count of followed links when in state “not-ready”)
![Page 47: Meta-adaptivity and self-regulation: Towards the next generation of adaptive systems Alexandros Paramythis Johannes Kepler University Linz, Austria](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649f1c5503460f94c32d97/html5/thumbnails/47.jpg)
Paramythis, Meta-adaptivity and Self-regulation 47
F.A.Q.s Isn’t link annotation too simple an example?
» What happens when we address more complex issues?
» Or, what happens when we want to test several design aspects in parallel?
In fact, this is exactly why this approach is being proposed.
» It can “scale” well to more complex design issues• for instance, it can very easily be applied recursively; it can be extended to
accommodate for potentially competing design goals
» When applied properly, the discriminatory capacity of the approach is only limited by the make-up of the user sample participating in tests
• the more potentially interacting design variables one works with, the more care one must take in deciding the number and characteristics of participating users – just like in any user study