Text Mining: Finding Nuggets in Mountains of Textual Data

Jochen Dijrre, Peter Gerstl, Roland Seiffert

Presented by Ebrahim Kobeissi

Text in this color are my additions/modifications

1

Outline

Motivation Methodology Feature Extraction Clustering and Categorizing Some Applications Comparison with Data Mining Conclusion & Exam Questions

2

Motivation

A large portion of a company’s data is unstructured or semi-structured

Letters Emails Phone transcripts Contracts

Technical documents Patents Web pages Articles

3

Definition

Text Mining: the discovery by computer of new, previously

unknown information, by automatically extracting information from different written resources

4

Typical Applications

Summarizing documents Discovering/monitoring relations among people,

places, organizations, etc Customer profile analysis Trend analysis Documents summarization Spam Identification Public health early warning Event tracks

5

Outline

Motivation Methodology Comparison with Data Mining Feature Extraction Clustering and Categorizing Some Applications Conclusion & Exam Questions

6

Methodology: Challenges

Information is in unstructured textual form Natural language interpretation is difficult &

complex task! (not fully possible) Watson is a step closer

Text mining deals with huge collections of documents Impossible for human examination

7

Methodology: Two Aspects

Knowledge Discovery Extraction of codified information

Feature Extraction Mining proper; determining some structure

Information Distillation Analysis of feature distribution

8

Two Text Mining Approaches

Extraction Extraction of codified information from single

document Analysis

Analysis of the features to detect patterns, trends, etc, over whole collections of documents

9

Outline

Motivation Methodology Feature Extraction Clustering and Categorizing Some Applications Comparison with Data Mining Conclusion & Exam Questions

10

IBM Intelligent Miner for Text

IBM introduced Intelligent Miner for Text in 1998 SDK with: Feature extraction, clustering,

categorization, and more Not a drop in application

Traditional components (search engine, etc) The rest of the paper describes text mining

methodology of Intelligent Miner.

11

Feature Extraction

Recognize and classify “significant” vocabulary items from the text

Categories of vocabulary Proper names – Mrs. Albright or Dheli[sic], India Multiword terms – Joint venture, online document Abbreviations – CPU, CEO Relations – Jack Smith-age-42 Other useful things: numerical forms of

numbers, percentages, money, etc12

Canonical Form Examples

Normalize numbers, money Four = 4, five-hundred dollar = $500

Conversion of date to normal form Morphological variants

Drive, drove, driven = drive Proper names and other forms

Mr. Johnson, Bob Johnson, The author = Bob Johnson

13

Feature Extraction Approach

Linguistically motivated heuristics Pattern matching Limited lexical information (part-of-speech) Avoid analyzing with too much depth

Does not use too much lexical information No in-depth syntactic or semantic analysis

14

Advantages to IBM’s approach

Processing is very fast (helps when dealing with huge amounts of data)

Heuristics work reasonably well Generally applicable to any domain

15

Outline

Motivation Methodology Comparison with Data Mining Feature Extraction Clustering and Categorizing Some Applications Conclusion & Exam Questions

16

Clustering

Fully automatic process Documents are grouped according to

similarity of their feature vectors Each cluster is labeled by a listing of the

common terms/keywords Good for getting an overview of a document

collection

17

Two Clustering Engines

Hierarchical clustering Orders the clusters into a tree reflecting various

levels of similarity Binary relational clustering

Flat clustering Relationships of different strengths between

clusters, reflecting similarity

18

Clustering Model

19

Categorization

Assigns documents to preexisting categories Classes of documents are defined by providing a set

of sample documents. Training phase produces “categorization schema” Documents can be assigned to more than one

category If confidence is low, document is set aside for

human intervention

20

Categorization Model

21

Outline

Motivation Methodology Feature Extraction Clustering and Categorizing Some Applications Comparison with Data Mining Conclusion & Exam Questions

22

Applications

Customer Relationship Management application provided by IBM Intelligent Miner for Text called “Customer Relationship Intelligence” “Help companies better understand what their

customers want and what they think about the company itself”

23

Customer Intelligence Process

Take as input body of communications with customer

Cluster the documents to identify issues Characterize the clusters to identify the

conditions for problems Assign new messages to appropriate clusters

24

Customer Intelligence Usage

Knowledge Discovery Clustering used to create a structure that can be

interpreted Information Distillation

Refinement and extension of clustering results Interpreting the resultsTuning of the clustering processSelecting meaningful clusters

25

Outline

Motivation Methodology Feature Extraction Clustering and Categorizing Some Applications Comparison with Data Mining Conclusion & Exam Questions

26

Comparison with Data Mining

Data mining Discover hidden

models. tries to generalize all

of the data into a single model.

marketing, medicine, health care

Text mining Discover hidden

facts. tries to understand

the details, cross reference between individual instances

biosciences, customer profile analysis 27

Conclusion

This paper introduced text mining and how it differs from data mining proper.

Focused on the tasks of feature extraction and clustering/categorization

Presented an overview of the tools/methods of IBM’s Intelligent Miner for Text

28

Exam Question #1

What are the two aspects of Text Mining? Knowledge Discovery: Discovering a common

customer complaint in a large collection of documents containing customer feedback.

Information Distillation: Filtering future comments into pre-defined categories

29

Exam Question #2

How does the procedure for text mining differ from the procedure for data mining? Adds feature extraction phase Infeasible for humans to select features manually The feature vectors are, in general, highly

dimensional and sparse

30

Exam Question #3

In the Nominator program of IBM’s Intelligent Miner for Text, an objective of the design is to enable rapid extraction of names from large amounts of text. How does this decision affect the ability of the program to interpret the semantics of text? Does not perform in-depth syntactic or semantic analysis

of the text; the results are fast but only heuristic with regards to actual semantics of the text.

31

Questions?

32