Semantic Memory Semantic Memory Architecture for Architecture for

Knowledge Acquisition Knowledge Acquisition and Managementand Management

Włodzisław DuchWłodzisław Duch

Julian SzymańskiJulian Szymański

Semantic MemorySemantic Memory

Endel Endel TulvingTulving „ „Episodic and Semantic MemoryEpisodic and Semantic Memory” ” 19721972

Semantic memory refers to the memory of meanings and Semantic memory refers to the memory of meanings and understandings. It stores koncept-based,understandings. It stores koncept-based,

generic, context-free knowledge. generic, context-free knowledge.

One of types of long-term memory. Together with episodic memory One of types of long-term memory. Together with episodic memory make up the category of declarative memory. (the others are make up the category of declarative memory. (the others are

episodic and procedural) episodic and procedural)

Semantic memory includes generalized knowledge that does not Semantic memory includes generalized knowledge that does not involve memory of a specific event. involve memory of a specific event.

PPerernamnament container for general knowledgeent container for general knowledge (facts, ideas, words, (facts, ideas, words, problem solving)problem solving)

Hierarchical Model Hierarchical Model Collins & Quillian, 1969Collins & Quillian, 1969

Semantic network Semantic network Collins & Loftus, 1975Collins & Loftus, 1975

Knowledge representationKnowledge representationwCRKwCRK

Interactive semantic Interactive semantic spacespace

Concept Description VectorsConcept Description VectorsCobraCobra

is_ais_a animalanimalis_ais_a beastbeastis_ais_a beingbeingis_ais_a brutebruteis_ais_a creaturecreatureis_ais_a entityentityis_ais_a faunafaunais_ais_a objectobjectis_ais_a organismorganismis_ais_a reptilereptileis_ais_a serpentserpentis_ais_a snakesnakeis_ais_a vertebratevertebratehashas bellybellyhashas body partbody parthashas cellcellhashas chestchesthashas costacostahashas digitdigithashas facefacehashas headheadhashas ribribhashas tailtailhashas thoraxthorax

Semantic Space explorationSemantic Space exploration

Binary dictionary searchBinary dictionary search222020 = 1048576 = 1048576

Binary search – not acceptable in complex Binary search – not acceptable in complex semantical applicationssemantical applications

Semantic space can be search using Semantic space can be search using context – based algorithm. Similar to word context – based algorithm. Similar to word game.game.

CConcept space oncept space narrowed narrowed by subsequent by subsequent user answersuser answers

20 questions game 20 questions game algorithmalgorithm

, where p(keyword=vi) is fraction of concepts for which the keyword has value vi

Subspace of candidate concepts O(A) are selected according to:

O(A) = {i; d=|CDVi-ANSW| is minimal}

,where CDVi is a vector for i-concept and ANSW is a partial vector of retrieved answers

● we can deal with user mistakes choosing d > minimal

0

( ) ( ) log ( )K

i i

i

I keyword p keyword v p keyword v

1

01 ( )

0( , ) , : ( , )

( ) 2

N

n nn

if y NULLdist CDV ANSW

yd CDV ANSW where dist x y if x NULL

len ANSWx y

Data aquisitionData aquisition How to obtain semantic data?How to obtain semantic data?

Wordnet Wordnet Relations for Semantic category: animalRelations for Semantic category: animal

7543 7543 objects and objects and 1696 1696 featuresfeatures Truncated using word popularity rank:Truncated using word popularity rank:

IC – IC – information contentinformation content is an amount of appearances of the is an amount of appearances of the particular word in WordNet descriptionsparticular word in WordNet descriptions

GR - GR - GoogleRankGoogleRank is an amount of web pages returned by is an amount of web pages returned by Google search engine for a given word Google search engine for a given word

BNC are the words statistics taken from BNC are the words statistics taken from British National British National NorpusNorpus..

- Semantic Space reduced to - Semantic Space reduced to 889 889 objects andobjects and 420 420 featuresfeatures

( )max( )

IC GR BNCRank word

Rank

Active learningActive learning Data from wordnet:Data from wordnet:

Not complete Not complete Not common senceNot common sence Sometimes specialised conceptsSometimes specialised concepts

Basic dialogs for obtaining new relationsBasic dialogs for obtaining new relations I give up. Tell me what did you think of?I give up. Tell me what did you think of? Tell me what is characteristic for <concept>Tell me what is characteristic for <concept> ??

Knowledge correction :Knowledge correction :

, where:, where:WW00 – initial weight, initial knowledge – initial weight, initial knowledgeANS – answer given by userANS – answer given by userN – amount of answersN – amount of answersββ - parametr for indicating importance initial knowledge - parametr for indicating importance initial knowledge

0 *N

w ANSw

N

The gameThe game Giraffe: Giraffe:

[is vertebrate] [is vertebrate] YY,, [is mammal] [is mammal] YY, , [has hoof] [has hoof] YY, , [is equine] [is equine] NN, , [is bovine] [is bovine] NN, , [is deer] [is deer] NN, , [is swine] [is swine] NN, , [has horn] [has horn] NN, , [has horn] [has horn] NN,, [is sheep] [is sheep] NN,, [is antelope] [is antelope] NN,, [is bison] [is bison] NN. . System correctly guess concept System correctly guess concept giraffegiraffe..

- Yuppi i’ve won! Let’s talk about Yuppi i’ve won! Let’s talk about giraffegiraffe. . Tell me what is characteristic forTell me what is characteristic for giraffegiraffe??

After entering keyword. Semantic memory is reorganised, and ready to play new After entering keyword. Semantic memory is reorganised, and ready to play new games.games.

LionLion: :

[is vertebrate] [is vertebrate] YY,, [is mammal] [is mammal] YY,, [has hoof] [has hoof] NN, , [has paw] [has paw] YY,,[is canine] [is canine] NN,, [is cat] [is cat] YY,, [is wildcat] [is wildcat] YY

The different way for organizing concept lion in WordNet taxonomy, causes the game The different way for organizing concept lion in WordNet taxonomy, causes the game goes in wrong way and system fails guess this concept:goes in wrong way and system fails guess this concept:

[is leopard] [is leopard] NN,, [is painter] [is painter] NN,, [is puma] [is puma] NN,, [is lynx] [is lynx] NN,, [is lynx] [is lynx] NN. . I give up. What it was? Lion …I give up. What it was? Lion …

After giving right answer system reorganizes its knowledge and next game for After giving right answer system reorganizes its knowledge and next game for searching concept lion is finished with success:searching concept lion is finished with success:

[is vertebrate] [is vertebrate] YY, , [is mammal] [is mammal] YY, , [has hoof] [has hoof] NN, , [has paw] [has paw] YY,, [is canine] [is canine] NN, , [is cat] [is cat] YY , , [is wildcat] [is wildcat] YY, , [is leopard] [is leopard] NN, , [has mane] [has mane] YY, , I guess it is lion.I guess it is lion.

AvatarAvatar

Experimental resultsExperimental results How many games do we need do How many games do we need do

clarify semantic space?clarify semantic space? proportion failed games proportion failed games NNff performed to achieve first performed to achieve first

success.success.

The semantic memory errorThe semantic memory error: : where where NNss is amount of the games finished with success and N is amount of the games finished with success and N

is total games amount, for searching first 10 concepts were is total games amount, for searching first 10 concepts were

0.220.22 How it changes during learning How it changes during learning

process?process? Avg Density features / objectAvg Density features / object

2.39fNQN

N

23

24

25

26

27

28

29

30

0 1 2 3 4 5 6 7 8 9 10

lp

CDVdens

1 SNE

N

Thank youThank you