how google works: are search engines really dumb and why educators

How Google Works:Are Search Engines Really

Dumb and Why VEMAns Should Care

Paul Barron

Director of Library and Archives

George C. Marshall Foundation

[email protected]

All Right Reserved. This presentation may be copied and distributed for

nonprofit educational purposes only. Revised October 2010

mailto:[email protected]

The School Librarian: The Keystone of Learning 2222222

Search Engines – The Web’s Gatekeepers

“Search engines are

the most important

gatekeepers in today‟s

digitally network environment.”

Regulating Search Engines: Taking Stock and Looking Ahead

Urs Gasser

Yale Journal of Law and Technology

Spring 2006

http://tinyurl.com/332rna


And we know our students …

For them, “to Google” is a lifestyle, a habit

pattern. Do you agree?

JEFF STAHLER: (c) Columbus Dispatch Dist.

by Newspaper Enterprise Association, Inc

“Whereas libraries once seemed

like the best answer to the

question, Where do I find…? the

search engine now rules.”

“No Brief Candle: Preconceiving Research

Libraries for the 21st Century;” Part II

Council of Library and Information Resources

http://www.clir.org/pubs/reports/pub142/pub142.pdf


If educators hope …

To change students’ excessive use of Google,

educators must embrace

Google and learn how

the search engine works, in order …

To influence students to integrate Google use

with other reliable sources of information.


Also, to improve students’ media literacy …

They need to understand how the search engine

determines and ranks results.

Why?

Students “trust” search engines and perceive credible

sites because a site was returned at the top of the

results by the search engine.

“Trust Online: Young Adults’ Evaluation of Web Content”

Journal of Communications: 4 (2010), 468-494

ijoc.org/ojs/index.php/ijoc/article/download/636/423


Presentation Objective

Increase our understanding of how search engines

and Google work by dispelling search engine myths

Propose a plan to increase the use of library research

databases

Not by excluding Google use

Integrate Google use with use of library databases

Goal - Enable us to help our students become better

researchers

http://infotrac.galegroup.com/itweb/tel_p_jackmsl

http://infoweb.newsbank.com/


Presentaion Objective: Dispel …

Search engine myths:

understand a searcher’s query,

treat all sites and domains the

same when determining results, and

determine the results based on the popularity of the

site with searchers.

I‟m

.edu.

I‟m

.net.

But we‟re

not equal.


Why should we learn how Google works?

“Users are not familiar with how search engines “find”

what they are looking for. [U]sers

might benefit from having more

information [how] Google “crawls”

the Web and determines how a website

is ranked.”

In Google We Trust: Users’ Decisions on Rank, Position, and Relevance

Laura Granka and others

Journal of Computer-Mediated Communication

User Experience Researcher

Google, Inc


The Center of the Search Universe

2010

2000

http://www.bruceclay.com/serc_histogram/histogram.htm2000


# 1 Search Engine in the

http://gs.statcounter.com

10

Google

80%


Google’s Influence

“They [Google] are going to have more influence and

more control over the structure of the world information

industry than any other.”

David B. Yoffie

Senior Associate Dean, Chair, Executive Education; Harvard Business School

Quoted in: “Google Ends Microsoft’s Yahoo Search;” New York Times, May 6, 2008

http://www.nytimes.com/2008/05/06/technology/06google.html


Google’s Power: From Duke U Law

“Google has become the index of choice for online

information; [it] steers our thoughts and

learning online. Google‟s control …

constitutes an awesome ability to

set the course of human knowledge.”

Google‟s Law

Greg Lastowka

Duke University School of Law

http://works.bepress.com/cgi/viewcontent.cgi?article=1003&context=lastowka

12


Without access to Google …

“Chinese scientists say their research will be

dramatically compromised if Google shuts down its search

engine in China.”

“One scientist [said], „If I lose Google,

it will [be] just like a man without his eyes.

Another says it „would be like life without electricity.‟ ”

“A land without Google?”

Nature 463, 1012-1013 (2010)

http://www.nature.com/news/2010/100224/full/4631012a.html


And from another law review …

“Whoever controls search has enormous

influence on us. Search engines shape what we

read, who we listen to, and who gets heard.

No search engine comes closer to controlling

search than Google.”

James Grimmelmann; "The Google Dilemma"

New York Law School Law Review ; Jan. 2009: 939

http: works.bepress.com/james_grimmelmann/19

http://www.nyls.edu/


Student’s # 1 Online Information Source

15

Google Google was the go-to

resource for almost all of the

students in the sample.

Nearly all of the students in

the sample reported always

using Google, both for

course-related research and

everyday life research.

“How College Students Seek

Information in the Digital Age”

http://tinyurl.com/yfp7ol5


Should educators be concerned?

“There are consequences to

our students and our educational

system if we [allow] a search

engine to define the parameters

of effective research.”

The University of Google: Education in the (Post) Information Age

Tara Brabazon


Expectations of Search Engine Users

“Searchers are now expecting

search engines to not just index the

Internet; they are starting to expect that

the search engine will understand the

intent of the query better.”

Interpreting Hitwise Statistics on Longer Queries

http://blog.ask.com/2009/03/interpreting-hitwise-statistics-on-longer-queries.html

And

neither

does

Google!


Expectations of Search Engine Users

“We expect a lot search engines. We ask them vague

questions about topics that we are unfamiliar and anticipate

a concise organized response.”

“You would have better success if

you laid your head on the keyboard

and coaxed the computer to read your mind.”

Understanding Search Engines: Mathematical Modeling and Text Retrieval

Michael W. Berry and Murray Browne


Why? Because …

“I‟ve been at this for two decades

now; search isn‟t out of its infancy yet.

The science is at the point where we

are crawling. Soon we‟ll walk. I hope in

my lifetime; I‟ll see search enter its

adolescence.”

Amit Singhal; Google Fellow

“This is tough stuff”

25 February 2010

http://googlepolicyeurope.blogspot.com/2010/02/this-stuff-is-tough.html


More significantly …

“(S)earch engines have no

understanding of words or

language. (They) don't

recognize user intent, can't

distinguish goal-oriented search

from browsing search.”

A ResourceShelf Interview: 20 Questions with Dr. Gary Flake, Ph.D.

Head of Yahoo! Research Labs

http://searchenginewatch.com/showPage.html?page=3372051

Thursday, June 3, 2004

http://www.flickr.com/photos/x180/296282975/


And …

“We can write a computer program to beat the very

best human chess players, but

we can't write a program to

understand a sentence anywhere near the precision

of a child.”

“Helping Computers Understand Language”

Steven Baker, Google Software Engineer

Official Google Blog

January 19, 2010

21


If Google doesn’t understand my query …

… how does Google

determine how to select and rank the

results in response to my query?


How does Google determine a result’s ranking?

http://www.google.com/corporate/tech.html


How Google Determines Relevance

Top 5 Ranking Factors Rank

Keyword Focused Anchor Text from External Links(The anchor text is the text description of a link.)

1

External Link Popularity

(Quantity/quality of external links)

2

Diversity of Link Sources

(Links from many unique root domains)3

Keyword Use Anywhere in the Title Tag 4

Trustworthiness of the Domain Based on Link Distance from

Trusted Domains (Trust diminishes as links become further removed from the trusted site.)

5

http://www.seomoz.org/article/search-ranking-factors#overview



Ranking Factors Rank

Diversity of Link Sources

(Links from many unique root domains)6

Page-Specific TrustRank (Whether the individual page has earned links from trusted

sources)

7

Global Link Popularity of the Domain

(Based on an Iterative Link Algorithm) 8




Ranking Factors Rank

Link Diversity of the Domain

(Based on number/variety of unique root domains linking to pages on

this domain)

9

Links from Hubs/Authorities in a Given Topic-

Specific Neighborhood 10

Iterative Algorithm-Based, Global Link Popularity (PageRank)

11


See also: “Exclusive: How Google’s Algorithm Rules the Web”

http://www.wired.com/magazine/2010/02/ff_google_algorithm/all/1


In summary:

“Google uses many factors in ranking.

PageRank … the best known … evaluates:

how many links there are to a web page from other pages, and

the quality of the linking sites.”

“With PageRank, five or six high-quality links from

websites would be valued much more highly than twice as

many links from less reputable or established sites.”

Librarian Central

How does Google collect and rank results?

http://www.google.com/librariancenter/articles/0512_01.html


What factor was missing in that table?

The fact that

the site is

popular with

us, the

searchers who

view the sites!


Searcher’s Usage Data – Low Importance


Consider Google’s challenge when …

Google processes

hundreds of million queries every

day and “20% of those queries

have never been seen before.”

Amit Singhal; Google Fellow

“Half of the search terms

[searched for in Google] occur

less than once a month.”Peter Norvig

Google Inc.Director of Search Quality and Research

“Masters of Information”

Forbes; September 5, 2005

www.google.com/trends


Google’s Solution

"We believe the approach which relies heavily on an individual's

tastes and preferences [to rank results] just doesn't produce the

quality and relevant ranking that our algorithms do." Amit Singhal

Google Fellow

“This is tough stuff”

25 February 2010

http://googlepolicyeurope.blogspot.com/2010/02/this-stuff-is-tough.html


Continuous Process Improvement

Google’s Matt Cutts answered

the question, “How many search

algorithm changes were made

in 2009?”

Matt said that Google makes a

change per day to the search

algorithm. He said, “in 2009,

Google probably made between 350 to 400 changes to

the search algorithm.”“Google Makes One Change Per Day To Search Algorithm”

April 22, 2010

http://www.youtube.com/watch?v=1_jm_isupFY&feature=player_embedded


Google and Usage Data

“Peter Norvig confirmed that

Google does collect usage data.

However when Google tries

new ranking models, Google does not use real

usage data to tune their search ranking algorithm.”

“How Google Measures Search Quality”

Datawocky

http://tinyurl.com/6mpt4u


Why!?!

First: “We have all been trained to trust

Google and click on the first result.”

Ibid

College students trust Google; they

click on the number one abstract most

of the time, even when the abstracts are

less relevant.”In Google We Trust: Users’ Decisions on Rank, Position, and Relevance

Laura Granka

Journal of Computer-Mediated Communication


College Students (& some Faculty) Trust Google

“[S]tudents and faculty appear to be satisfied, especially

with Google. [Librarians] stressed that there is blind trust

and an increasing reliance on search results, especially on

whatever appears on the first couple of screens.”

Search engine use behavior of students and faculty: User perceptions and implications for future research

Oya Y. Rieger. First Monday

http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/viewArticle/2716/2385


Trusting Google too Much?

“Second: For informational queries …

if a result on page 4, provides better

information than the results on the first three

pages, users will not know this result

exists!

Therefore, usage behavior does not

provide the best feedback on the rankings.”

“How Google Measures Search Quality”

Datawocky

http://tinyurl.com/6mpt4u

But we are

the best

results!


Looking at Only How Many Results!

“[F]ixations are around the top 5 results; the

majority of clicks are on the top 3 results.”Has Google Gotten Better

http://thinkeyetracking.com/Blog/?p=4

Searchers

reviewed the first

10 results in 2005.


And in ’09 …

“… most users

found what they were

looking for among the

first two results

and they never

needed to go further

down the page.”

“Eye-tracking studies: more than meets the eye”

http://googleblog.blogspot.com/2009/02/eye-tracking-studies-more-than-meets.html


Consider this …

“The computer screen is … literally

a small thing [that] may display just

over 300 words. If this world becomes

our reality, we actually are relying on

less information, not the more that is available.”

“The Google-ization of Knowledge”

Natasja Larson, Laura Servage, and Jim Parsons

Faculty of Education; University of Alberta

http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/28/03/99.pdf


From Google’s CEO

"I worry that instantaneous technologies might be

bad for deep reading of longer things like books and

magazines. We don't know what that means for

cognition." “Davos Day 3: Technology Is Like Magic”

http://www.technologyreview.com/blog/pontin/tags/eric+schmidt/

40


Question! Just librarians?

Can librarians help us rediscover the value of deep reading?

“The danger for librarians is that they will

think of themselves as Googlers. They need to

resist the pressure to rely on search engines.”

“Librarians can help us broaden out beyond the results that

search engines provide. [They can] and take us off the beaten track

to cover more perspectives and to offer more intellectually rigorous

takes on a particular subject.”

“An Interview with Nicholas Carr;” Information Outlook; http://www.sla.org/io/2010/04/863.cfm

41


Intellectual Vigorous Content

42

http://www.procon.org


Google doesn’t need to consider …

… the popularity of a website with

searchers because their algorithm is

so up-to-date that Google always

returns the best results.

Right?

RIGHT!

RIGHT!


Relevance in Google = Only an Opinion

Google’s … “assessments of the "value"

of a web page are subjectively-determined [by]

formulae to come up with a ranking. PageRanks

are opinions. They're professional opinions, but

they remain opinions.”

“Google Replies to SearchKing Lawsuit”

http://research.yale.edu/lawmeme/

Thursday, January 9, 2003

Google v.


Google's rankings are protected opinion.

"The court simply finds there is

no conceivable way to prove that the

relative significance assigned to a given

Web site is false. Accordingly, the court concludes

Google's PageRanks are entitled to full constitutional

protection.”

“Judge Dismisses Suit Against Google”

http://news.cnet.com/2100-1032_3-1011740.html

May 30, 2003


Evaluating Google’s Opinion

Google returns all sites with the phrase,

martin luther king.


Google’s 3rd Result


Martin Luther King.org Homepage


Martin Luther King.org is hosted by …


The student wants to know …

Why was that site returned

as the 3rd result among the

millions of results!?!

I thought Google and

other search engines always

returned the best results.


144,000 Links from .edu Sites

For what purpose are the

academic libraries linking

to martinlutherking.org?

51


Links from .edu & .gov Sites = Trust

“Google places a heavy bias on informational

resources; .edu and .gov sites tend to rank

higher than others.”

“Google is the best at

determining true link quality and places a lot of

weight on domain trust levels.”

“Can You Please Them All?”

http://www.bruceclay.com/blog/archives/2006/08/can_you_please.html

Reputation

and well

established


High Quality Links

“Contextual links on the pages of certain:

Industry research websites

Major news websites

High Google Page Rank websites

University websites

Major industry organizations

High traffic websites in your industry

Major corporate websites”Pandia Search World

“Link building for high quality links”

http://www.pandia.com/sew/2595-quality-links.html

http://www.stanford.edu/

http://research.microsoft.com/c/1040


Google‘s opinion about webpages is important.

What can I do to influence the results

returned by Google?


Question!

Search Engine Components

Spider/Web Crawler/Robot

Index

Search Engine

The only feature that you can

control is the query entered

into the search engine.


With the Fewest Words as Possible

“There's a real imbalance in

Web search. Users give us three

words at a time. People type the

query "map," and then they get

upset if it's not the map they were

thinking of.”“The Future of Search:

The head of Google Research talks about his group's projects.”

MIT Technology Review

http://tinyurl.com/2pmfsu

81% of search engine queries are 4 words or

less.


Remember the Google Trends?

How many of the queries were more than three

(3) words?

82%

http://blog.alessiosignorini.com/2010/02/average-query-length-february-2010/


Queries by Middle School Students

“A predominate difficulty students experience while performing

Web-based research is constructing effective search strings.”

“[M]iddle school students demonstrate

unsophisticated skills when constructing

search strings, using mainly broad terms

and phrases.”

“Internet Searching by K-12 Students: A Research-based Process Model”

http://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/1b/a8/26.pdf


Queries by High School Students

“[H]igh school students

struggle with conceptualizing

the topic for their query, sometimes omitting

required concepts.”

“Internet Searching by K-12 Students: A Research-based Process Model”

http://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/1b/a8/26.pdf


Queries by College Students

“[S]earch engines generally performed poorly, a lack of

computer skills and an inability

to construct appropriate search

statements limited students'

success.”

Nowicki, Stacy.

Student vs. Search Engine: Undergraduates Rank Results for Relevance

portal: Libraries and the Academy - Volume 3, Number 3, July 2003

60


Can students craft a four-word query on …

… the effects on climate change on global

temperatures and sea levels worldwide?

61


Why are WIKIPEDIA results ranked so highly?

Is there a preference for WIKIPEDIA or other

top level domains?

Why Search? Just Take Me to WikipediA.

62


Yes!

“Google‟s authority-based algorithm is

domain-centric. Google has focused on

domain-trusting by pushing to

the top of the results massive sites

like Wikipedia that couldn‟t have been

created by spammers.”The Google Cache

Google’s New Algorithm: if($domain==’wikipedia.org’){$rank=1;}

http://tinyurl.com/yv3xo6

63


WIKIPEDIA – One of the Most Important Domains

http://www.seomoz.org/top500

64


WIKIPEDIA Use by Students

“Wikipedia is a unique and indispensible resource for

students; [they] begin their research process

with Wikipedia; [it is] their „first go-to place."

“Wikipedia entries provide „a simple

narrative that gives me a grasp‟ and [that]

„helps when I have no idea what to do for

a research paper.‟ ”

“Finding Context: What Today’s College Students Say about Conducting Research in the Digital Age”

Project Information Literacy Progress Report: February 2009

http://www.projectinfolit.org/pdfs/PIL_ProgressReport_2_2009.pdf

65


From the Founder of WIKIPEDIA

“For God sake, you‟re in college; don‟t

cite the encyclopedia.”Jimmy Wales

Wikipedia Founder Discourages Academic Use of His Creation

The Chronicle of Higher Education

http://tinyurl.com/lxhxo

66


So tell me again, what can an educator do?


We probably won’t see this from …

You are not lucky. Connect to the nearest reference librarian.


But this is not the answer!


Why?!?

“Schooled on Google and WIKIPEDIA, [students] want to

inquire, not rely on the professor for a detailed roadmap. They want

an animated conversation, not a lecture.

They want an interactive education. Rather

than waiting for a trusted professor to tell them what's going on, they

find out on their own from Google to WIKIPEDIA.”

The Impending Demise of the University

Don Tapscott

http://www.edge.org/3rd_culture/tapscott09/tapscott09_index.html


Teach the Use of Advanced Search Syntaxes

“Advanced syntax users demonstrate search expertise

that the majority of user population does not. They are:

more adept at combining query operators to

formulate powerful query statements and

return more relevant results

Not only were they more successful in their searching,

they were consistently more successful.”

Investigating the Querying and Browsing Behavior of Advanced Search Engine Users research.microsoft.com/~ryenw/papers/WhiteSIGIR2007b.pdf


Searching: An Aid to Complex Reasoning

“Internet searching engages complicated brain activity which may

help improve brain function. [The] Web-savvy group registered activity in the

areas of the brain which control decision-making and complex reasoning.”

“UCLA study finds that searching the Internet increases brain function”

November 15, 2008

http://newsroom.ucla.edu/portal/ucla/ucla-study-finds-that-searching-64348.aspx

Brain Activity from Internet Search

The image on the left displays brain activity while reading a book;

the image on the right displays activity while engaging in an Internet search.


Now that we hooked ’em with Google …

By teaching options …


From Google to the Proprietary Databases

Demonstrate that Google syntaxes and queries,

with minor modification work in the proprietary

databases and may provide more relevant sources.

http://www.galeschools.com/


From Google to Proprietary Database

Students can limit the

search to the structure of

the document.


Academic Journals


It works - minds can be changed!

“I modeled my presentation, „Internet Privacy Laws are

Necessary,‟ for juniors (AP English 11) after the VEMA presentation.

“They searched their usual way. Then I

demonstrated a search using advanced search

techniques. Then they searched using advanced

search syntaxes.”

“They narrowed their search results from one

million to 55 and it was amazing how many hits were on target.”

Nancy Keenan; Library Media Specialist/Computer Coordinator

Glenvar High School

Salem, Virginia


Minds can be changed!

“I revised my lesson plan for teaching students how to search the

Web and library databases. Students were

frustrated using the Web; when we got to

Gale and ABC-CLIO their amazement in the difference of the quality of

information was priceless. One student researching working women of

the 1930s said, „I found much more in Student Resource Center.‟ ”

Lori Donovan, NBCT, Teacher-Librarian

Thomas Dale High School

Chester, Virginia


Don’t underestimate the importance of friends!

http://www.oclc.org/reports/2005perceptions.htm

Gang, the

librarian is better

than Google!


Are you thoroughly googled?


Student Research in the 21st Century

“[N]o matter where students are enrolled, [the]

information resources they have, and how much time they

have, [even with] the proliferation of digital information

resources research seems to be far more difficult to

conduct in the digital age.”

Finding Context: What Today’s College Students Say about Conducting Research in the Digital Age

Project Information Literacy Progress Report: February 2009

http://www.projectinfolit.org/pdfs/PIL_ProgressReport_2_2009.pdf


Learn How Search Engines Work

“Given their popularity, knowing more about

search engines is vital to understanding

information access in a digital age.”

The social, political, economic, and cultural dimensions of search engines: An introduction.

Hargittai, E., (2007).

Journal of Computer-Mediated Communication, 12(3), article 1.

http://jcmc.indiana.edu/vol12/issue3/hargittai.html

I understand Google; I‟ll search for

title pages with domain-limited

Boolean queries.


And the winner is …


Web Searching Tutorials

Title URL

A Short and Easy Search

Engine Tutorialhttp://www.pandia.com/goalgetter

Boolify: Interactive

Boolean Search Tool

Generator

http://www.boolify.org

Finding Information on

the Internet: A Tutorial

http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/FindInfo.html

http://tinyurl.com/leqf

Web Research,

Evaluation, & Citation

http://www.arlington.k12.ma.us/ahs/instructionaltech/docs/webre_eval_cit.pdf

http://tinyurl.com/6bejgh

Web Search Strategies

in Plain English (Video) http://www.commoncraft.com/search

http://www.pandia.com/goalgetter

http://www.boolify.org/

http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/FindInfo.html

http://tinyurl.com/leqf

http://www.arlington.k12.ma.us/ahs/instructionaltech/docs/webre_eval_cit.pdf

http://tinyurl.com/6bejgh

http://www.commoncraft.com/search


Other Search Engines to Consider

Search Engine Address Remarks

AltaVista www.altavista.com Useful for Link Checking

Bing www.bing.com Bing provides results

to YAHOO!

Duck Duck Go www.duckduckgo.com Creator wants

recommendations from

educators

E-mail Gabriel Weinberg - [email protected] or go to

http://duckduckgo.com/feedback.html

Exalead www.exalead.com/search Useful for Link Checking

Picsearch www.picsearch.com Find animated images in

advanced search

mailto:[email protected]


Questions, Comments, or Suggestions?

how google works: are search engines really dumb and why educators

Documents