anna tomkowicz listening - 2011

Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011

A presentation from the NewMR ‘Listening is the New Asking’ Text Analytics Event - March 8 2011

The sponsor of the ‘Listening is the New Asking’ event is Zinc Research For more information about NewMR events visit newmr.org

The copyright for this material is jointly owned by The Future Place and the presenter.

‘Online Text Analytics – Top Ten Tips: Welcome to the oldest new game in town’ Anna Tomkowicz Freshminds Research, UK

Online Text Analytics – Top Ten Tips Welcome to the oldest new game in town 8th March 2011



There are a lot of conversations going on out there...

>  In 2010, Twitter users sent 25 billion Tweets and more than

100 million new accounts were registered.

>  More than 3.5 billion pieces of content are shared each week on Facebook.

>  Purpose built Facebook pages have created more than 5.3 billion fans.

>  In 2009, there were 126 million blogs on the internet.

Source: e-consultancy, Twitter


Powered by

... and lots of automated tools to analyse them



It’s all really confusing


I will take you through Top Ten Tips for using automated tools for text analytics


0

1000

2000

3000

4000

5000

6000

7000

Twitter

Forum

Blog

Different tools lead to different results


Check which areas of the internet your tool is actually monitoring 1

•  You are probably missing out on data on closed communities. Most tools are unable to trawl private areas on Facebook.

•  You can use youropenbook.org to source

quotes from open profiles, but there is no tool enabling robust analysis.

•  There are serious ethical implications in

sourcing quotes from closed communities.


Check how your tool recognises location or nationality 2

•  Most tools use the domain name to determine location (com vs. co.uk), which may lead to bias.

•  The language used on a forum isn’t

necessarily indicative of the user nationality.


Check how recent your data is 3

•  Some tools provide you with fresh data every day, with mentions from the most popular websites coming in first.

•  Other tools may lag up to a week.

•  Sometimes, speedy delivery of the data happens at the expense of accuracy, so a quick data set may not be a very good one.

•  Make a decision – do you need insight from great quality data, or the ability to quickly react at the first indication of a crisis?


Automated sentiment analysis is difficult 4

20%

70%

10%

•  Tools use complicated algorithms to detect sentiment.

•  There are certain limitations, e.g. challenges

around irony.

•  Most tools are between 60%-85% accurate, but if you look at the usual sentiment metrics observed in consumer goods, you will see that if you mark all mentions as ‘neutral’ you will be on average 70% accurate.

•  Accuracy goes down once you compare automated coding with manual coding on a one by one basis as opposed to on aggregate.


Emerging topics are sensitive to the search terms used

Topic Alterian Biz360 Brandwatch Nielsen Radian6 Scoutlabs Sysomos Baristas Yes

Burger King Yes

Cannabis Yes

Caribou Yes

Celebrity Yes Yes

Costa Yes

Credit Cards Yes

Fair Trade Yes

Falling in love Yes

Flat White Yes Yes

Freebies Yes

Gold Card Yes

Goo Yes

Guns Yes Yes Yes Yes Yes Yes Yes

Keep going Yes

Proximity Yes

Rewards Yes

Service Yes

Sundays Yes

Survey Yes

Wifi Yes

5

•  Imposing a search term can have an impact on all your results and terms you are detecting


Which leads us to our over-arching tips

6 •  Make sure you have clear business objectives before you commit to online text analytics.

7 •  Don’t try to distill the analysis to a single metric.

8 •  Use multiple data sources and robust sampling techniques.

9 •  Tools change every week, so stay abreast of the developments and changes in methodology.

10 •  Challenge the findings!


Q & A

Anna Tomkowicz FreshMinds

Ray Poynter The Future Place


FreshMinds Research

FreshMinds Research is an award-winning research-led consultancy specialising in qualitative & quantitative market research and business analysis. We bring together the brightest minds* and the freshest thinking to create effective, commercial solutions to every client challenge. Clients include HSBC, The Cabinet Office, innocent and eBay; from tech start-ups to global FMCG companies. And no two projects are the same. Our approach is always fresh and energetic, our focus always commercial. We aim to inspire, offer new perspectives and help you move your business forward. * (ours, yours, consumers’ and experts’)

For more information, please contact:

Anna Tomkowicz Lead Research Manager [email protected]

020 7692 4554

Phil Burgess Head of Consumer Research

[email protected] 020 7692 4549

FreshMinds 229-231 High Holborn

London WC1V 7DA www.freshminds.co.uk

T: 020 7692 4300 F: 0870 460 1596

E:[email protected]

anna tomkowicz listening - 2011

Marketing

uk newmr listening

market research

online text analytics

new asking event

new accounts

newmr events

oldest new game

automated tools