1 West Street New York NY 10004 | 646-545-3900 | info@networkedinsights.com | networkedinsights.com

White Paper

Keyword Search vs. Discovery

TEENWOLF

1 West Street New York NY 10004 | 646-545-3900 | info@networkedinsights.com | networkedinsights.com

2

Networked InsightsWhite Paper

Many tools are available to marketers for monitoring social media conversations. And it isn’t hard to find them. Just do a keyword search.

Identifying these tools by using a search engine is particularly fitting, because most of those tools employ some form of keyword search as the cornerstone of their technology. And for many information retrieval tasks, keyword search works well.

But search isn’t always the answer for marketing professionals — especially those who view themselves as marketing technologists — who are growing ever more sophisticated in their approach to social media analysis. Sometimes, search-based approaches are by nature flawed, pro-ducing information that is misleading or inadequate. Even if the informa-tion is on target, assembling and analyzing it can take a painfully long time.

Search’s limitations are especially evident when the goal is to uncover themes emerging from people’s social media conversations without injecting into the analysis any preconceptions of what they will be talking about. In other words, when you truly want to find out what you don’t know, search may be as likely to send you down rabbit trails as it is to reveal the truth.

Thematic discovery is an alternative approach to search that can provide a more accurate, authentic view of the topics that people are engaged in. Along with identifying key themes within a social media data set, thematic discovery can help in identifying relationships between themes that may not have an obvious connection, how the themes correlate, and where people are likely to be talking about them simultaneously. Such informa-tion can help marketers develop media plans, choose celebrities to en-dorse products, and tap into the diverse interests of their target markets.

This white paper discusses the uses and limitations of keyword search-based tools and the key characteristics of a thematic discovery tool that can provide insights that search cannot.

Keyword Search vs. DiscoveryThematic discovery

can help uncover— • What topics are being discussed in

connection with the user’s and competitors’ brands

• How the volume of the user’s conversations compares to that of competitors

• What unexpected topics are being discussed by a given group of people

• How the user’s and competitors’ PR and marketing efforts changed the way people talk about brands

Movies

MusicTV

Brands

Games

3

Keyword Search vs. Discovery

White Paper Networked Insights

When search works wellKeyword search is an indispensable tool for virtually anyone who uses the Internet. It is most effective when:

You know exactly what you’re looking for. Search does an excellent job of finding specific items of information.

You have a narrow range of acceptable, predetermined results. Search also works well when you have in mind what the results should look like and say. You only require a few answers to your question. Top search results can be on target, but the farther down the list you go, the less likely the results are relevant to what you are looking for.

Very specific matches are the desired output. Search emphasizes specific matches of terminology because it relies on the language that you pro-gram into the search box.

Context is not important or even valuable. Search works well if all you want to do is find something, but you don’t care where it came from or what it’s about.

Why search stumblesWhile the characteristics of search make it work well for finding specific information items, it has several limitations when conducting thematic analysis:

Keyword search introduces considerable distorting biases to results. Performing a keyword search requires you to create a list of words and phrases, which by definition, reflect your expectations of the themes that will be present in a given data set. That expectation introduces a bias in the results because the search will not find results outside of those predefined keywords. If, for example, you try to learn from conversations about smartphones, you will need to develop a search query that lists all the topics and brands that you think will be present. But there is no way for that query to find topics or brands that are not included in the keyword string. In other words, you can’t search for something unless you have an idea of what you’re searching for.

Another factor potentially influencing search results is “prosecutor’s fal-lacy.” This is the statistical phenomenon that, in a large data set, no matter what you’re looking for, you will find it, regardless of whether it is statisti-cally important or valid.

Search emphasizes specific matches of terminology because it relies on the language that you program into the search box

traditional search

4

Keyword Search vs. Discovery

White Paper Networked Insights

Search can miss relevant data sets. While keyword search can yield a lot of data, several issues can diminish its usefulness and efficiency when applied to thematic analysis:

Lack of fluency in the vocabulary being used. In keyword search, a user can misunderstand the subtle differences in how people talk about the same concept. Thematic discovery provides the ability to identify similar con-cepts even if the language used to describe those topics varies.

Search results may not produce enough relevant responses. Using search to find themes may not produce a statistically valid sample. Again using the smartphone example, a search for street-map applications may deliver a few highly targeted results, but it will likely not yield enough data points to constitute a statistically valid sample of the entire data set. Many lower ranking results from a keyword search will not even be about the intended subject matter at all.

Lack of context to bridge result sets. Keyword search delivers an ordered list of individual results ranked by relevance to the search query. But there is no additional information provided to help the user understand how the results relate to each other. By contrast, thematic discovery not only iden-tifies the themes that are present in a given data set, but also describes whether those themes connect to one another, and if so how. Oftentimes, the context of the results is as valuable as the results themselves.

Multiple word meanings make searching for themes more difficult. Say the word “apple,” and someone may think about a food, a company or a com-puter. Differentiating these meanings with a keyword search tool requires complex queries containing many exclusions. The longer the query, the longer it takes to build and process that query, and the greater the chance of human errors as terms are added.

Search can be time consuming and costly.Understanding multiple themes requires developing multiple keyword search queries. Every theme within a data set will require its own tailored search. The broader the analysis, the more time intensive it becomes.

Highly manual processes are required to measure, compare and trend search results. The ranked results from a keyword search are typically not a usable format for further types of analysis. Specifically, performing the-matic discovery from those results requires considerable manual effort to convert the data into consumable information.

Traits of a good thematic discovery toolThe goal of thematic discovery is to understand the prominent topics within a given data set and how they relate to one another. The resulting information can help answer critical business questions.

Search cannot contemplate the context of how words and phrases are used in relationship to one another; it simply can identify whether or not that word or phrase is present.

traditional search

5

Keyword Search vs. Discovery

White Paper

If you are trying to find out what people really think about your brand, markets, products and competitors, thematic discovery will enable you to approach the task without biases, capture all key conversations and gather information quickly.

Questions about this report? Want

to learn how thematic discovery can

improve your marketing decisions?

Let us know by visiting

www.networkedinsights.com/contact

Networked Insights

An effective thematic discovery tool can help users:Eliminate as many user biases as possible. A discovery tool needs to pro-duce organic results uninfluenced by the user. In any discovery initiative it’s imperative to avoid injecting what you know into the process.

Identify all the themes present in a data set. You can use search to identify themes you know about, but how do you look for something that you didn’t know was there? A discovery tool should uncover every theme present.

Explain the relative size of themes and whether they are gaining or losing momentum. Comparing themes can show which are capturing the largest conversations and how the themes are trending against one another over time.

Provide the flexibility to change the granularity of the analysis. Ideally, a discovery tool should allow users to define the size of a theme that is important to them. Are you looking for the three or four most important themes, or for outliers? In some cases, outliers may be valuable to mar-keters because they help identify previously unidentified niche markets. A good tool will enable the user to look at various levels of granularity in themes.

Separate noise from the signal. In any given discovery process there will be some data points that have no value on their own and do not relate mean-ingfully to any other part of the data set. The tool should eliminate these points from the data set to prevent clouding the results.

Understand the relationship between those themes. What is the likelihood that someone might be talking about any two or more themes at a time? Statistics can provide those answers and a good discovery tool will not just find the themes but also identify the number and strength of correlations that exist between them.

Work quickly and efficiently. The process of defining the data set for thematic analysis should be quick and painless. The results should be easily understood through a well-crafted visualization. And the entire process should move quickly enough to provide same-day results from the time a business question is asked to the time it is answered.

Ending the search for (search) alternativesKeyword search will always have a role in finding the names of a compet-ing company’s officers, the address of tonight’s restaurant reservation and the score of last night’s game. But if you are trying to find out what people really think about your brand, markets, products and competitors, themat-ic discovery will enable you to approach the task without biases, capture all key conversations and gather information as quickly as possible—to truly know what you don’t know.

646-545-3900info@networkedinsights.comnetworkedinsights.com© 2012 Networked Insights, Inc. All rights reserved.