new directions in social media tom reamy chief knowledge architect kaps group
TRANSCRIPT
New Directions in
Social Media
Tom ReamyChief Knowledge Architect
KAPS Group
http://www.kapsgroup.com
2
Agenda
Introduction – Social Media and Text Analytics Deeper than Positive-Negative
Building a Foundation for Social Media– Adding intelligence to BI, CI, and Sentiment
New Dimensions for Social Media– New taxonomies– Cognitive Science of Emotion
Applications– Expertise Analysis, Behavior Prediction, Crowd Sourcing– Integration with Predictive Analytics, Social Analytics
Conclusions
3
KAPS Group: General
Knowledge Architecture Professional Services – Network of Consultants Partners – SAS, SAP, IBM, FAST, Smart Logic, Concept Searching
– Attensity, Clarabridge, Lexalytics, Strategy – IM & KM - Text Analytics, Social Media, Integration Services:
– Taxonomy/Text Analytics development, consulting, customization– Text Analytics Fast Start – Audit, Evaluation, Pilot– Social Media: Text based applications – design & development
Clients: – Genentech, Novartis, Northwestern Mutual Life, Financial Times,
Hyatt, Home Depot, Harvard Business Library, British Parliament, Battelle, Amdocs, FDA, GAO, etc.
Applied Theory – Faceted taxonomies, complexity theory, natural categories, emotion taxonomies
Presentations, Articles, White Papers – http://www.kapsgroup.com
4
Introduction – Social Media & Text AnalyticsBeyond Simple Sentiment Beyond Good and Evil (positive and negative)
– Social Media is approaching next stage (growing up)– Where is the value? How get better results?
Importance of Context – around positive and negative words– Rhetorical reversals – “I was expecting to love it”– Issues of sarcasm, (“Really Great Product”), slanguage
Granularity of Application– Early Categorization – Politics or Sports
Limited value of Positive and Negative– Degrees of intensity, complexity of emotions and documents
Addition of focus on behaviors – why someone calls a support center – and likely outcomes
5
Introduction – Social Media & Text AnalyticsBeyond Simple Sentiment Two basic approaches:
– Statistical Signature of Bag of Words – Dictionary of positive & negative words
Beware automatic solutions – Accuracy, Depth Essential – need full categorization and concept extraction to get
full value from social media Categorization - Adds intelligence to all other components –
extraction, sentiment, and beyond Categorization/extraction rules – not just topical or sentiment
Combination with advanced social media analysis Opens up whole new worlds of applications
6
Text Analytics Foundation for Social MediaText Analytics Features Noun Phrase Extraction
– Catalogs with variants, rule based dynamic Sentiment Analysis
– Objects and phrases – statistics & rules – Positive and Negative Auto-categorization
– Training sets, Terms, Semantic Networks– Rules: Boolean - AND, OR, NOT– Advanced – DIST(#), ORDDIST#, PARAGRAPH, SENTENCE
Text Analytics as Foundation– Disambiguation - Identification of objects, events, context– Build rules based, not simply Bag of Individual Words
New Dimensions for Social Media
New Taxonomies – Appraisal – Appraisal Groups – Adjective and modifiers – “not very good”– Four types – Attitude, Orientation, Graduation, Polarity– Supports more subtle distinctions than positive or negative
Emotion taxonomies – Joy, Sadness, Fear, Anger, Surprise, Disgust– New Complex – pride, shame, embarrassment, love, awe– New situational/transient – confusion, concentration, skepticism
Beyond Keywords– Analysis of phrases, multiple contexts – conditionals, oblique – Analysis of conversations – dynamic of exchange, private language
10
New Applications in Social Media
Expertise Analysis – Experts think & write differently – process, chunks– Categorization rules for documents, authors, communities
Applications:– Business & Customer intelligence, Voice of the Customer– Deeper understanding of communities, customers – better models– Security, threat detection – behavior prediction, Are they experts?– Expertise location- Generate automatic expertise characterization
Behavior Prediction–TA and Predictive Analytics, Social Analytics Crowd Sourcing – technical support to Wiki’s Political – conservative and liberal minds/texts
– Disgust, shame, cooperation, openness
11
12
New Applications in Social Media
Analysis of Conversations– Techniques: self-revelation, humor, sharing of secrets, establishment
of informal agreements, private language– Detect relationships among speakers and changes over time– Strength of social ties, informal hierarchies
Combination with other techniques– Expertise Analysis – plus Influencers– Quality of communication (strength of social ties, extent of private
language, amount and nature of epistemic emotions – confusion+) Experiments - Pronoun Analysis – personality types Essay Evaluation Software - Apply to expertise characterization
– Model levels of chunking, procedure words over content
13
New Applications in Social MediaBehavior Prediction – Telecom Customer Service Problem – distinguish customers likely to cancel from mere threats Analyze customer support notes General issues – creative spelling, second hand reports Develop categorization rules
– First – distinguish cancellation calls – not simple– Second - distinguish cancel what – one line or all– Third – distinguish real threats
14
New Applications in Social MediaBehavior Prediction – Telecom Customer Service
Basic Rule– (START_20, (AND, – (DIST_7,"[cancel]", "[cancel-what-cust]"),– (NOT,(DIST_10, "[cancel]", (OR, "[one-line]", "[restore]", “[if]”)))))
Examples:– customer called to say he will cancell his account if the does not stop receiving
a call from the ad agency. – cci and is upset that he has the asl charge and wants it off or her is going to
cancel his act– ask about the contract expiration date as she wanted to cxl teh acct
Combine sophisticated rules with sentiment statistical training and Predictive Analytics and behavior monitoring
15
New Applications: Wisdom of CrowdsCrowd Sourcing Technical Support Example – Android User Forum Develop a taxonomy of products, features, problem areas Develop Categorization Rules:
– “I use the SDK method and it isn't to bad a all. I'll get some pics up later, I am still trying to get the time to update from fresh 1.0 to 1.1.”
– Find product & feature – forum structure– Find problem areas in response, nearby text for solution
Automatic – simply expose lists of “solutions”– Search Based application
Human mediated – experts scan and clean up solutions
16
New Directions in Social MediaText Analytics, Text Mining, and Predictive Analytics Two Systems of the Brain
– Fast, System 1, Immediate patterns (TM)– Slow, System 2, Conceptual, reasoning (TA)
Text Analytics – pre-processing for TM– Discover additional structure in unstructured text– Behavior Prediction – adding depth in individual documents – New variables for Predictive Analytics, Social Media Analytics– New dimensions – 90% of information
Text Mining for TA– Semi-automated taxonomy development – Bottom Up- terms in documents – frequency, date, clustering– Improve speed and quality – semi-automatic
17
New Directions in Social MediaConclusions Social Media Analysis requires a hybrid approach
– Software, text analytics, human judgment – Contexts are essential
Text Analytics needs new techniques and structures – Smaller, more dynamic taxonomies– Focus – verbs, adjectives, broader contexts, activity
Value from Social Media Analysis requires new structures– Appraisal taxonomies, emotion taxonomies Plus– Better models of documents and authors – multi-dimensional
Result:– Enhanced sentiment analysis / social media applications– Develop whole range of new applications
Questions?
KAPS Group
http://www.kapsgroup.com
Upcoming: Text Analytics Summit – Boston - June
Text Analytics World – Boston – October – Speakers?