www.sharon-it.com1 refining – finding words/expanding taly sharon [email protected]...

34
www.sharon-it.com 1 Refining – Finding Words/expanding Taly Sharon taly @ sharon -it.com sharont @alum. mit . edu

Upload: alberta-stevens

Post on 30-Dec-2015

221 views

Category:

Documents


1 download

TRANSCRIPT

Page 2: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com2

Contents• Expanding/Learning terms• Categorization/Clustering engines• Google Suggest• SurfWax FocusWords• When you don’t know where to

start

Page 3: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com3

Make Longer Queries

•Yahoo

•Harvest Digital

Average Search Terms per Query

Overall Experienced

Page 4: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com4

Adding Words• Holocaust

23,200,000• holocaust memorial

836,000• holocaust memorial budapest

42,300• holocaust memorial budapest danube

4,910• holocaust memorial budapest danube promenade

692

Page 5: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com5

Classification/Categorization

• Classification: the process of deciding the appropriate category for a given document.

• Examples:– deciding what newsgroup an article belongs

to. – what folder an email message

should be directed to.– what is the general topic of an essay.

Page 6: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com6

Clustering

• The process of automatically grouping documents.

Page 7: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com7

Search Categorization/Clustering

• The result documents are ordered according to categories.

• The searcher can select the relevant category to display the related documents.

• Examples:– Vivisimo/Clusty– Excite– Teoma– Exalead

Page 8: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com8

Clusty

Page 9: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com9

Excite

Page 10: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com10

Ask

Page 11: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com11

Exalead

Page 12: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com12

Google Suggest

• As you type – you get query suggestions and number of results per query.

Page 13: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com13

Google Suggest (2)

Page 14: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com14

Google Suggest

Page 15: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com15

Page 16: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com16

Yahoo Search Assist

Page 17: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com17

SurfWax FocusWords• SurfWax has an option “Focus”• This option invokes the FocusWords

mechanism• You get suggestions to make your

query:– Broader– Similar– Narrower

• http://www.surfwax.com

Page 18: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com18

SurfWax FocusWords

Page 19: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com19

When you don’t know how to start

• Reverse Dictionary• Glossaries and

Dictionaries• Taxonomy/

Folksonomy• Pearl Culturing• Analyzing pages• Finding similar pages

– Google’s related:– Alexa www.alexa.com

Page 20: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com20

Reverse Dictionary• OneLook reverse dictionary:

http://www.onelook.com/reverse-dictionary.shtml

• Example: “bird of prey” => raptor• Example: economical measure of a

nation’s wealth => Gross Domestic Product

Page 21: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com21

Page 22: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com22

Glossaries and Dictionaries• Google search:

– <topic>– glossary OR thesaurus OR dictionary OR taxonomy

• Example 1: agriculture glossary– http://www.cnie.org/nle/AgGlossary/AgGlossary.htm– http://www.cahe.nmsu.edu/news/aggloss.html– http://agriculture.house.gov/info/glossary.html

• Example 2: agriculture thesaurus– http://agclass.nal.usda.gov/agt/agt.shtml– http://www.fao.org/aims/ag_intro.htm (multilingual)

• http://www.glossarist.com– http://www.glossarist.com/glossaries/business/primary-in

dustry/agriculture.asp

Page 23: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com23

Page 25: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com25

del.icio.us

Page 26: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com26

del.icio.us

Page 27: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com27

Pearl Culturing• What to do when you don’t have the

category nor the right keywords?• Find one good relevant website• Look it up in directories• You will find:

– the category/main keywords– authoritative websites

• Useful search engine: Exalead

Page 28: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com28

Analyze Pages• Distilling: what is problematic in a bad page?

– what is wrong? Is there an interfering keyword/term appearing.

– Remove interfering terms (using “-”).• Identifying clues and patterns in a good page.

– Read the document, what are the clues? – Look for new keywords, word combinations and

other things differentiating between it to non-authoritative documents.

– Use frequency counter:• http://www.wordcounter.com/• http://www.georgetown.edu/faculty/ballc/webtools/web_

freqs.html

Page 29: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com29

Frequency Counter

Page 30: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com30

Page 31: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com31

Wordcounter

Page 32: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com32

Page 34: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu

www.sharon-it.com34

Exercises1. How are bad user interfaces called (hint: try Google suggest)2. Reverse dictionary

a. Find relevant keywords for chemistryb. What is the terminology for when menstruation stops?c. How was the separation between the west and the soviet union

called?3. What are the related terms to Competitive Intelligence?4. Check suggestions from Google Suggest for a query starting

with biofuel.5. Using SurfWax, learn options to focus or broaden the query:

biodiesel.6. Identify the most relevant terms in the website: www.uspto.gov 7. Identify the most relevant terms in the Biofuel Wikipedia entry

http://en.wikipedia.org/wiki/Biofuel.8. Search in Onelook reverse dictionary and in other glossaries

terms: fuel, natural energy, geothermal, and other terms. Look at the results.