tests of sources assessing source credibility. is the source known? unknown or incomplete sources...

Tests of SourcesTests of Sources

assessing source credibilityassessing source credibility

Is the source known?• unknown or incomplete

sources (“they say…” “I read somewhere…” “experts say…”)

• pseudo-sources, “fronts” for special interests

• Anyone can create a Website• beware of anonymous information

on the Web• beware of Websites with no clear

affiliation• beware of personal Websites or

blogs• Primary versus secondary,

tertiary sources

Is the source qualified?

• Expertise: does the source know his/her stuff?

• Even experts can be wrong• consider education, degrees,

credentials, licenses• consider honors, awards, peer

recognition• consider background and expertise

relevant to the topic or issue• expertise in the blogosphere

Source qualifications on the Web

• Stanford University study found that the average consumer paid more attention to superficial aspects of a site, such as visual cues, rather than content

• 46.1% based their evaluation on the overall design look.

• Project for Excellence in Journalism found that 12 percent of those surveyed said they believe what they read in blogs all or most of the time.

• 56 percent said they could believe newspapers and television news.

False expertise

• “genuine” versus phony psychics?• As seen on TV• Infomercial spokespersons• The ichthys symbol as a sign of

trustworthiness• “based on a true story” or “inspired by a

true story”• The Amityville Horror:

http://www.snopes.com/horrors/ghosts/amityville.asp

Published print media

• Scholarly refereed journal articles are the gold standard

• Peer review• Blind submission• Rejection rate

• Newspapers, Wire services (AP, Reuters, etc.)• For books, is the publisher well-known, reputable?• Beware of “vanity” publications

• Controlled clinical trials are the gold standard in medicine

• double-blind studies• Placebo effects

On-line sources• Is the Website part of an

official organization, academic institution, or news group?

• Does the article include a bibliography?

• References can be used to verify the information reported

• for Websites: does the site give the source’s position, institutional affiliation and contact information?

• Is the material well-written and well-referenced?

• Are there numerous typos, grammatical mistakes, and spelling errors?

• Obama’s fake Kenyan birth certificate

On-line and published sources• What does the domain

name tell you? (.edu, .gov, .org, .net

• Organizational or institutional affiliations?

• Can the information be verified for accuracy?

• How recent is the information?• Is there a copyright date?• When was the material created

or posted?• Has the material been updated

or revised?

• Who is the sponsor?• Are there links to information

about the author or sponsor?• Are there citations and

references?• Are there many broken links?

Wickipedia: trust but verify• Based on hit counts, Wikipedia

is among the top 40 websites• However, popularity is not

synonymous with accuracy• Important errors have been

found in Wikipedia• False information on Robert

Kennedy’s assassination• Kazakhstan’s entry was

altered to include information from Borat

• David Beckham was a Chinese goalkeeper in the 18th Century

• Entries can be submitted anonymously

• Biased and prejudicial information

• Encyclo-vandalism• How reliable is “collective

intelligence”?• Does fact-checking work?• Openness is the problem

and the solution

Reliability

• Reliability: can you trust the source?• results of a drug or alcohol test• predictions of economic forecasters• can you trust what you read on the Web?

• Health information sites• Conspiracy sites

Author James Frey Author James Frey admitted on Oprah admitted on Oprah Winfrey’s show that many Winfrey’s show that many of the details in his of the details in his memoir, memoir, A Million Little A Million Little PiecesPieces, were fabricated. , were fabricated. The book was ranked #1 The book was ranked #1 on the New York Times on the New York Times best seller list for 15 best seller list for 15 weeks.weeks.

Reliability of claims by the White House

• Donald Rumsfeld on ABC, March 2003: “We know where they (weapons of mass destruction) are. They're in the area around Tikrit and Baghdad and east, west, south and north somewhat."

• George W. Bush, U.N. address, September 2003: “The regime of Saddam Hussein cultivated ties to terror while it built weapons of mass destruction.”

• Dick Cheney on Larry King Live, May 2005: "The level of activity that we see today from a military standpoint, I think, will clearly decline. I think they're in the last throes, if you will, of the insurgency.“

• Dick Cheney, January 2004: “There's overwhelming evidence there was a connection between al Qaeda and the Iraqi government.”

Reliability continued

• The CIA whiffs on “curveball”• The CIA ignored warnings that an informant

code-named "Curveball" — the chief source of prewar intelligence about Iraqi germ weapons — was unreliable. (L.A. Times, April 2, 2005)

• Claims that Iraq was producing biological weapons were based almost entirely on accounts from a defector who was described as "crazy" by his intelligence handlers and a "congenital liar" by his friends. (L.A. Times, April 1, 2005)

• German officials said that they had warned American colleagues well before the Iraq war that Curveball's information was not credible - but the warning was ignored (The Guardian, April 4, 2004)

Objectivity

• Objectivity: is the source neutral, impartial, unbiased?

• Rathergate … Dan Rather was blamed for erroneous claims in a 60 Minutes story that Bush had dodged National Guard duty.

• The story relied on faked documents that CBS was warned about

• Mary Mapes, the producer, and three other CBS employees were fired over the scandal.

Source bias• Is Fox “fair and balanced”?• Does the media have a liberal bias?

• The general public is convinced the media has a liberal bias

• surveys show that journalists tend to take more liberal stands on issues and vote more for Democrats (Journal of Broadcasting & Electronic Media/March 2005)

• But most major media companies are owned by conservatives and publishers are motivated by profits (Alterman, 2003)

• “a link between reporters’ political beliefs and news coverage has never been convincingly established” (Lee, 2005)

Hostile Media Effect

• Partisans tend to believe the media is biased against their side of an issue.

• Ross, Lepper, & Vallone (1985) showed 144 observers six television news segments about Israel's 1982 war with Lebanon.

• Pro-Arab viewers “heard” 42 positive references to Israel and 26 references that portrayed Israel negatively.

• Pro-Israeli viewers “heard” 16 positive references to Israel and 57 negative references.

• Both groups saw identical videotaped segments

Source bias continued• Corporate bias

• Medical research sponsored by drug manufacturers

• Magazines and advertising profit interests• Political bias

• special interest groups: NRA, PETA, ACLU

• Regulatory agencies that are “cozy” with the industries they regulate

Bias in pharmaceutical trials• American Journal of Psychiatry, April,

2006: many clinical trials show bias toward the sponsoring company’s drug.

• In clinical trials that were partially or fully funded by the drug’s manufacturer, 90% of the findings favored the sponsored drug

• The study showed many of the biases were minor or correctable

Reluctant testimony• Reluctant testimony is when a source

makes an admission that is not in his/her self interest

• In 2008, even Karl Rove, the architect of negative campaigning, acknowledged that some of McCain’s ads had “gone one step too far”

• Feb 4, 2005 Colin Powell told the Washington Post that he regretted recommending military action against Iraq, given that there were no WMDs.

• Norma McCorvey, a.k.a.“Jane Roe” in the Rove V. Wade Supreme Court Case, was once a pro-choice advocate. She has since switched sides and is now a vocal anti-abortion activist.

• How reliable is http://www.drkoop.com and other medical information websites?

• How reliable are “birther” claims that Barack Obama is not a U.S> citizen? http://www.birthers.org/ http://www.orlytaitzesq.com/

• http://www.youtube.com/watch?v=keiFFoTyALk

• How credible is this Bigfoot Website?• http://www.bfro.net/

• How credible is the Flat Earth Society?• http://www.lhup.edu/~dsimanek/fe-scidi.htm• http://www.alaska.net/~clund/e_djublonskopf/FlatHome.htm

• Are Holocaust denial groups credible?• http://www.ihr.org/• http://www.codoh.com/• http://www.fpp.co.uk/

Applying Tests of Sources