designing and validating a structuring method for a multipurpose knowledge repository and evaluating...
TRANSCRIPT
Designing and validating a structuring method for a multipurpose knowledge
repository
And evaluating the method in a software prototype
Lars van der MeerUniversiteit Utrecht
Master Business Informatics28-08-08
1/53
Agenda
• Background• Awareness of problem• Research question• Research method• Results• Conclusions• Discussion and future work
2/53
Background
3/53
KnowledgeKnowledge
Knowledge managementKnowledge
management
Knowledge repository
Knowledge repository
Multipurpose knowledge repository
Multipurpose knowledge repository
General
Specific
Knowledge
Data: ‘Known facts or things used as a basis of inference or reckoning’
Information: ‘Systematically organized data that tells us something when we interpret it’
Knowledge: ‘Actionable information’
(Jashapara, 2004)
Knowledge
Data: ‘Known facts or things used as a basis of inference or reckoning’
Information: ‘Systematically organized data that tells us something when we interpret it’
Knowledge: ‘Actionable information’
(Jashapara, 2004)
Knowledge management
‘..all methods, instruments and tools that in a holistic approach contribute to the promotion of
core knowledge processes’ (Mertins et al., 2000)
Knowledge management
‘..all methods, instruments and tools that in a holistic approach contribute to the promotion of
core knowledge processes’ (Mertins et al., 2000)
Knowledge repository
‘A knowledge repository is a tool that contributes to knowledge management by supporting capturing,
sharing and (re)usage of knowledge’
Knowledge repository
‘A knowledge repository is a tool that contributes to knowledge management by supporting capturing,
sharing and (re)usage of knowledge’
Multipurpose knowledge repository
‘A knowledge repository that enables different organizations to structure knowledge in a way that fits their knowledge domain and purpose of use’
Multipurpose knowledge repository
‘A knowledge repository that enables different organizations to structure knowledge in a way that fits their knowledge domain and purpose of use’
Background
‘It has become all too easy to accumulate knowledge in digitized form … after a point, search and recontextualization costs outweigh the potential benefits from reusing the knowledge’ (Raghu, 2005).
Challenge is making knowledge efficiently retrievable• Search engine• Browsing
4/53
Agenda
• Background• Awareness of problem• Research question• Research method• Results• Conclusions• Discussion and future work
5/53
Awareness of problem
• Domain modelling schemeso Taxonomy
6/53
Taxonomy
• Windows explorer• Knowledge system (TOPdesk)
7/53
Awareness of problem
• Domain modelling schemeso Taxonomyo Thesaurus
8/53
Thesaurus
Category FAQ NT
FAQCategory BTPoints of interest RT
9/53
Awareness of problem
• Domain modelling schemeso Taxonomyo Thesauruso Ontology
10/53
Ontology
• A vocabulary with definitions• All relations between the concepts
o Hierarchical relationso Associative relations
• Difference compared to thesaurus, is the level of detail
11/53
Awareness of problem
• Domain modelling schemeso Taxonomyo Thesauruso Ontologie
• Structuring methodso Taxonomy
12/53
Taxonomy
13/53
TOPdesk 4
FAQ
Incident-beheer
Configuratie-beheer
Aandachts-punten
Incident-beheer
Configuratie-beheer
TOPdesk 4
FAQ
Aandachts-punten
Incident-beheer
Configuratie-beheer
Incident-beheer
Configuratie-beheer
Taxonomy disadvantages
• Knowledge item can not be used on multiple places in the structure
14/53
Taxonomy
15/53
TOPdesk 4
FAQ
Incident-beheer
Configuratie-beheer
Aandachts-punten
Incident-beheer
Configuratie-beheer
Kennis ItemKennis Item
Taxonomy disadvantages
• Knowledge item can not be used on multiple places in the structure
• Browsing can only be done mono-dimensional
16/53
Taxonomy
17/53
TOPdesk 4
FAQ
Incident-beheer
Configuratie-beheer
Aandachts-punten
Incident-beheer
Configuratie-beheer
Product: TOPdesk 4Category: FAQProductmodule: ?
Taxonomy
18/53
TOPdesk 4
FAQ
Incident-beheer
Configuratie-beheer
Aandachts-punten
Incident-beheer
Configuratie-beheer
Product: TOPdesk 4Category: ?Productmodule: Incidentbeheer
Taxonomy
19/53
TOPdesk 4
FAQ
Incident-beheer
Configuratie-beheer
Aandachts-punten
Incident-beheer
Configuratie-beheer
Product: TOPdesk 4Category: ?Productmodule: Incidentbeheer
Taxonomy disadvantages
• Knowledge item can not be used on multiple places in the structure
• Browsing can only be done mono-dimensional
• Knowledge items are a fundamental part of the structure
20/53
Taxonomy
21/53
TOPdesk 4
FAQ
Incident-beheer
Configuratie-beheer
Aandachts-punten
Incident-beheer
Configuratie-beheer
ArchiveFAQ
Awareness of problem
• Domain modelling schemeso Taxonomyo Thesauruso Ontologie
• Structuring methodso Taxonomyo Tagging
22/53
Tagging
• Advantage: flexibility• Disadvantage: will become a mess of
definitions
23/53
ItemItem
FAQ TOPdesk 4
Awareness of problem
• Domain modelling schemeso Taxonomyo Thesauruso Ontologie
• Structuring methodso Taxonomyo Tagging o Wiki
24/53
Wiki
• Advantage: flexibility• Disadvantage: gets a mess of
relations and definitions
25/53
ItemItem
ItemItem ItemItem
Awareness of problem
• Domain modelling schemeso Taxonomyo Thesauruso Ontologie
• Structuring methodso Taxonomyo Tagging o Wikio Topic maps (Pepper, 2000)
26/53
Topic map
27/53
TOPdesk 4
FAQ
Incident-beheer
Configuratie-beheer
Aandachts-punten
Kennis Item
Product
Categorie Categorie
Productonderdeel
Productonderdeel
Product: TOPdesk 4Category: ?Productmodule: Incidentbeheer
Kennis Item
Awareness of problem
• Domain modelling schemeso Taxonomyo Thesauruso Ontologie
• Structuring methodso Taxonomyo Tagging o Wikio Topic maps (Pepper, 2000)o Knowledge entry maps (Helms;
Brinkkemper; v Oosterum; Nijs; 2005)
28/53
Knowledge entry map
29/53
Agenda
• Background• Awareness of problem• Research question• Research method• Results• Conclusions• Discussion and future work
30/53
Research question
‘Is knowledge retrieval in a multipurpose knowledge repository using a specialized structuring method for a multipurpose knowledge repository, more efficient than knowledge retrieval in a multipurpose knowledge repository using a taxonomy structure?’
31/53
Agenda
• Background• Awareness of problem• Research question• Research method• Results• Conclusions• Discussion and future work
32/53
Research method• Design research
33/53
Takeda et al. (1990)
Awareness of problemAwareness of problem
SuggestionSuggestion
Suggestion
• Existing structuring methods• TOPdesk
34/53
TOPdesk• TOPdesk company
o Software developer from Delfto 250+ employeeso Offices in Delft, Kaiserslautern, London
• TOPdesk producto Service managemento Standard producto Important modules
• Incident management• Configuration management• Knowledge base
35/53
Suggestion• Existing structuring methods• TOPdesk• Requirements engineering
o 10 semi-structured interviews with users, developers, sales people, consultants
o 2 Joint requirements sessionso Analysed incidents registered by the helpdesko Analysis of the current systemo Recorded in ‘natural language’o Built an Acces applicationo Development of scenarios
36/53
Research method• Design research
37/53
Takeda et al. (1990)
Awareness of problemAwareness of problem
SuggestionSuggestion
Develop-ment
Develop-ment
EvaluationEvaluation
Evaluation
• Scenarios structured with structuring method in prototype
• Validation through test sessions o Stratified sample group of 20 persons
• 3 categories of web experience
o 4 knowledge retrieval assignmentso 2 systems
• Current TOPdesk knowledge system Taxonomy• Prototype with new structuring method
o Exactly the same test conditionso Measurement
• Amount of Time• Amount of Clicks
38/53
Research method• Design research
39/53
Takeda et al. (1990)
Awareness of problemAwareness of problem
SuggestionSuggestion
Develop-ment
Develop-ment
EvaluationEvaluation
ConclusionConclusion
Agenda
• Background• Awareness of problem• Research question• Research method• Results• Conclusions• Discussion and future work
40/53
Results: Requirements
41/53
Results: Scenarios• 2 scenarios• “Small and simple knowledge domain”
o 1 system administratoro Limited knowledge for setting up a structureo A few persons adding knowledgeo Need for a flexible structure
• “Large complex knowledge domain”o Multiple system administratorso A lot of knowledge for setting up a structureo Hundreds of persons adding knowledgeo Need for a compelling structure
42/53
Results: Development
Solution: a “topic map” that can be created from within an application, which also allows a flexible structure, and relationships between knowledge items
43/53
Characteristic Topic maps KEM
Wiki Tagging
Structure without set-up X X
Knowledge item to knowledge item relationship
X
Topdown browsing X
Compelling structure X
Results: Development
Multipurpose Knowledge Repository Structuring Method (MKRSM) model
44/53
Results: Development
45/53
Results: Development
Multipurpose Knowledge Repository Structuring Method (MKRSM) meta-model
46/53
Results: Prototype
47/53
Results: Validation
48/53
Amount of clicks
Amount of time
Scenario 1: Small and simple knowledge domain
Results: Validation and Evaluation
MAC
Assignments1Taxono-my
1MKRS-M3 2
3,6 2,9 3,7 3AVG
22,1 19,4 21,5 20,3Average time between 2 clicks taxonomy structure: 6,0 sAverage time between 2 clicks MKRSM structure: 6,7 s
Results: Validation and Evaluation
49/53
Amount of time
Scenario 2: Large and complex knowledge domain
Amount of clicks
9,15 4,55 10,2 4,4
66,7 43,7 70,7 45Average time between 2 clicks taxonomy structure: 6,9 sAverage time between 2 clicks MKRSM structure: 8,8 s
Agenda
• Background• Awareness of problem• Research question• Research method• Results• Conclusions• Discussion and future work
50/53
Conclusions
• The developed structuring method improves knowledge retrieval efficiency in a multipurpose knowledge repository compared to a taxonomy structure.
51/53
Discussion and futurework
• Premise that sample group was a realistic representation of the population
• Testing maintainability• Search engine remains important• Automated suggestions for
associations• Technical design• Standardized structures
52/53
Questions?
»More information:»www.larsvandermeer.nl/thesis
53/53