A.Frank1

Internet Resources Discovery (IRD)

Whither Search Engine (SE)?! Some Practical

Recommendations

A.Frank2

Contents• When not to use SEs?

• Use patterns of SEs

• Rules for choosing SEs

• Practical recommendations

• General/Specialty subject SEs

A.Frank3

Googlism

A.Frank4

When not to use SEs?• You know it all.• You prefer asking friends (or paid experts ).• You know the Web site for it (and didn’t forget the

exact URL or have auto-completion or bookmark or can access through another known site).

• You already found a specific/relevant digital library or database (maybe in Invisible Web).

• Tired of paid inclusions, SE spamming, and sponsored commercial results.

• Tired of chasing down useless URLs.

A.Frank5

Use patterns of SEs (2004)

iProspect SE User Attitudes Survey Results, March 2004 (www.iprospect.com/) –

57% of Web users use the same SE when they are looking for information.

Most searchers (92%, up from 71% in June 2002) are loyal to their favorite SE, and stick with it (by modifying their query) even if they don't initially find what they're looking for (in the first 3 pages).

Just 30% of Web users have a few specific SEs they use regularly.

Only 13% use a different SE depending on what they are looking for at that time.

Google has a "loyalty rate" of 66%, Yahoo! is next at 55%, followed by MSN at 54% and AOL at 49%.

A.Frank6

Web

IndexIndex DDirectoryirectory

WEB

Which kind to use? All Which kind to use? All

SSearch earch EEnginengine

GeneralGeneral SpecialtySpecialty GeneralGeneral SpecialtySpecialty

Meta-SMeta-Search earch EEnginengine

A.Frank7

When to use an Index?• Need to search for a narrow piece of

information.

• Have a specific objective/site in mind.

• Want to find/rank many related Web sites.

• Want to factor quantity in (index has crawler based results).

• Need to check/fix spelling (based on Web statistics).

A.Frank8

When to use a Directory?• Clear about the exact topic of your query. • Need general information on a rather broad

topic/category.• Want to amass knowledge on a fairly wide subject.• Would like to browse (and then search) a certain area.• Want to factor quality in (directory has human-

powered results), not quantity. • Need information that is usually carefully evaluated

and even annotated.

A.Frank9

When to use a Meta-SE?• When single Basic-SE fails to provide good results. • One-stop shopping - prefer to search multiple

SEs/sites at once to get blended ranked results (so as to save effort/time).

• When the query is simple (complex fields/options don't usually work).

• Searching for multi-faceted topics. • Want to get clustered results to focus search on the

relevant keywords. • Looking for current events/news.

A.Frank10

When to use a Specialty-SE?• When general-SE fails to provide good results.

• When your target is very topic/technology specific.

• Want to find more than just Web pages/sites.

• Need more results from the Invisible Web.

• Want your search terms to more likely have the meanings you intended them to have.

A.Frank11

So how to choose your SE(s)?

A.Frank12

General rules for choosing SEs

• Use "major" SEs that are both well-known and well-used (and that hopefully won’t be downgraded or disappear soon ).

• Prefer SEs that employ both a huge index and a comprehensive directory (gives better results; can also switch between).

• Stick to SEs of established companies that treat search as their main business/expertise.

A.Frank13

Technical rules for choosing SEs• Insist on a thin (user-friendly) interface but also

support an advanced search page. • Fully index a broad range of file types. • Provide multimedia (images, audio/MP3, video)

search tags (also news, products). • Suggest spelling correction based on Web statistics.• Have a featured toolbar (easier to invoke from the

desktop).• Enable “Cursor Search” of word(s) on a Web page by

right-clicking the mouse.

A.Frank14

Theory vs. Practice!?

A.Frank15

Popularity of Google vs. Yahoo!

Search for “google” in – Google: 48,700,000 results.

– Yahoo!: 57,500,000 results.

Search for “yahoo!” in – Google: 152,000,000 results.

– Yahoo!: 110,000,000 results.

Maybe “Know thy enemy”?!

A.Frank16

Common sense rules for choosing SEs

"לעולם אל תשים את כל הביצים בסל אחד":• !בחר ביותר מאחד

"טובים השניים מהאחד":• !בחר בשניים עיקריים

"... עד שיבוא (הכתוב) השלישי ויכריע ביניהם":•!כדאי אבל שיהיה גם שלישי )מסוג שונה( כגיבוי

"בתי הקברות מלאים בכאלו שחשבו שאין להם •תחליף":

!אז כדאי שיהיו גם תחליפים לכל אחד ואחד

A.Frank17

Top 5 Search Engines/Sites (Jan 2004)

A.Frank18

Share of Searches Sites (Dec 2004)

A.Frank19

Share of Search Providers (Dec 2004)

A.Frank20

Practical recommendations

• Two major SEs (usually use both):1. Google (GG)

– Alternative: Personalized Google (labs.google.com/personalized)

2. Yahoo! search (YH)– Alternative : Teoma/AskJeeves (AJ)

• One Meta-SE (as a backup):3. DogPile

– Alternative: Vivisimo\Clusty

Note: Choices are not Hebrew oriented.

A.Frank21

המלצות פרקטיות (לעברית)

שני מנועי חיפוש עיקריים (לרוב שימוש בשניהם):•גוגל ישראל 1.

חלופה: מורפיקס (לצורך המורפולוגיה)–

וואלה! חיפוש 2.חלופה: נענע/תפוז (אבל כולם פורטלים בעיקר)–

מנוע חיפוש-על אחד (כגיבוי):•סטארט3.

) (מנוע חיפוש משלנו 2Findחלופה:–

הערה: התמיכה שלהם בעברית עדיין לאמושלמת.

A.Frank22

So what was the message ?

A.Frank23

So how does it look?

A.Frank24

Why use Google? (1)

• Biggest, most comprehensive coverage: ~8 billion Web pages (but ~1 billion of it isn’t full-text

searchable!) ~10 billion documents, if you count images and newsgroup

postings.

• Fastest around.

• Most relevant results (voted 3 times most outstanding SE by Search Engine Watch readers).

• Provides good directory results (PageRanks results of DMOZ Open Directory).

A.Frank25

Why use Google? (2)

• Has thinnest interface around. • But provides rich set of advanced search

features/tools(/hacks).• Finds similar/related pages. • Supports Web pages translation.• Cached (HTML) copy of pages (great for quick

view of DOCs/PDFs and for 404s ).• Google alert – use of push technology.

A.Frank26

Why use Yahoo! search? (1)

• Has brand new Yahoo! search – gives highly relevant Web results (at Google level ).

• Still supports an expert’s humanly-compiled directory (dir.yahoo.com).

• Has (also) a thin interface (search.yahoo.com) while providing a rich set of advanced search features/shortcuts.

A.Frank27

Why use Yahoo! search? (2)

• For legacy reasons (oldest of all directories). • Puts particular emphasis on personalization

and customization (my.yahoo.com).

• Had enough of Googlism (www.googlism.com ). • It devoured/uses (know-how from) Overture

(Inktomi, AltaVista and AllTheWeb, etc)

A.Frank28

Yahoo acquisitions?!

Yahoo

google

A.Frank29

4 Things Yahoo can and Google can’t!

• Find websites linking to a page– linkdomain:

• Mix syntax– Link:amdocs.com site:gov

• Long queries (>32 terms)– Especially important when using OR

• Search for XML/RSS

A.Frank30

New alternative - Why use MSN?

• Fresh from the Oven -- Launched Nov 11, 2004.

• Vast index of information: ~5 billion documents.

• The most up-to-date information – MSNbot is active all the time.

• Direct answers -- from Microsoft Encarta®, encyclopedia.

• Direct actions -- to MSN channels.

A.Frank31

Why use Teoma?

• It’s an expert/guide (term in Gallic ).

• Provides “subject specific” ranking of pages.

• One search – three responses:

1. Results: Lists relevant Web pages

2. Refine: Suggestions to narrow your search.

3. Resources: Recommends link collections from experts and enthusiasts.

A.Frank32

Why use AskJeeves (that uses Teoma)?

• Provides a Natural Language interface (uses NLP).

• Suggests related searches.

• Hides technical details (of Teoma).

• Just purchased Interactive Search Holdings (MyWay, MySearch, My Web Search, iWon, and Excite).

A.Frank33

Why use DogPile?

• Customizable list of indices, directories and SSEs.• Indices searched include Google, Yahoo, and

AskJeeves/Teoma. Directories searched include About, LookSmart and Open Directory.

• View results by their relevance or by search engine. • Designed to identify the (non-)commercial intent of a

user's search - proposes to refine your results.• Winner of “Best Meta Search Engine” award from

Search Engine Watch for 2003.

A.Frank34

Why use Vivisimo\Clusty?

• Indices searched include MSN, GigaBlast, Lycos, and WiseNut. Directories searched include LookSmart and Open Directory.

• Provides automatic clustering in hierarchical folders.• The results are grouped into successively narrower

subcategories, allowing to drill down through a topic without additional searching.

• Won second place for “Best Meta Search Engine” in the 2003 Search Engine Watch awards and winner in 2002.

A.Frank35

Choice of SEs is a delicate balance

A.Frank36

General Subject SEs

lii.org• Librarians’ carefully selected index to the internet

www.ipl.org/• Evaluated and annotated subject directory

infomine.ucr.edu• Selected scholarly internet resource collections

www.about.com/• Academic collection of "sites" on many subjects

www.finderseeker.com/• Search Engine for Specialty Search Engines

A.Frank37

Specialty Subject SEs

www.search.com/sitemap• Directory for Meta Specialty Searches

www.leidenuniv.nl/ub/biv/specials.htm• A collection of special SEs

www.academicinfo.net/• Educational subject directory

www.searchability.com/ • Multi-subject guides to specialized SEs

lib.nmsu.edu/instruction/specialtysearch.htm• Lists of Specialty Search Engines

A.Frank38

Ready Reference Deskswww.refdesk.com/

• Comprehensive reference deskwww.ipl.org/

• Ready reference collectionlii.org/search/file/reference

• Ready reference and quick factsacademicinfo.net/reffind.html

• Educational reference desk

www.faganfinder.com/• Help people find what they are looking for

A.Frank39

If more time ... we could SEEk more

A.Frank40

Bibliography/Credits http://searchenginewatch.com/reports/article.php/2156431

searchengineshowdown.com/ www.noodletools.com/debbie/literacies/information/

5locate/adviceengine.html infopeople.org/search www.lib.berkeley.edu/TeachingLib/Guides/Internet/ www.monash.com/spidap.html www.searchlore.org www.li-net.net/il/search (Hebrew)

A.Frank41

Bottom line

Seek/Search the best way for you!