31

CHAPTER 3

MINING UNSTRUCTURED DATA USING ARTIFICIAL NEURAL NETWORK

AND FUZZY INFERENCE SYSTEMS MODEL FOR CUSTOMER RELATIONSHIP

MANAGEMENT

3.1 INTRODUCTION

When managing thousands of customers, business will have difficulty sustaining the

rising costs created by interactions among people. However, if all customer data is inserted

into a database, the resulting records will provide a detailed profile of these customers and

their interactions with one another, and will be an important resource for businesses that wish

to probe customer data, customer needs, and customer satisfaction levels. Data mining uses

transaction data to gain a better understanding of customers and effectively discover hidden

knowledge through the insertion of business intelligence into the process of customer

relationship management. Wong et al. (2004) argued that data mining is an approach to assist

companies in developing more effective strategies to meet the requests of selective

customers. Data warehousing is useful and accurate for assembling a business’ dispersed

heterogeneous data and providing unified convenient information access technique. Data

mining technology can be used to transform hidden knowledge into manifest knowledge. An

integrated CRM system is extremely flexible. It can adjust customer needs throughout a

product’s life cycle, and to analyze and actively monitor customer preferences. Therefore,

one of the best competitive strategies is the successful utilization of Information Technology

(IT) to swiftly and effectively integrate business knowledge and provide the business with

timely quality decision support.

The most common forms of customer interaction are as follows: (1) Face-to-face

interaction with retail personnel; (2) Calls to customer service centers and conversations with

customer service representatives; (3) Comments on company websites; and (4) Feedback

expressed through e-mail. Customer data harvested through these methods is usually

unstructured; however, most data mining technologies can only handle structured data.

Therefore, during customary data warehousing processes, unstructured data is not taken into

account and much valuable customer information is lost.

Structured systems are those where the data and the computing activity is

predetermined and well-defined. Structured systems are designed by, built by, and operated

by the IT department, Automatic Teller Machine (ATM) transactions, airline reservations,

32

manufacturing inventory control systems, and Point of Sale (POS) systems are all forms of

structured systems. Unstructured systems are those that have no predetermined form or

structure and are usually full of textual data. Typical unstructured systems include email,

reports, contracts, transcribed telephone conversations, and other communications. The

person doing the communication can structure the message in whatever form is desired, using

language in any form desired. The following figure 3.1 shows the unstructured and structured

systems.

Unstructured Systems Structured Systems

Fig. 3.1: Unstructured and Structured Systems

This research uses content analysis to transform unstructured textual content into

structured data. Unstructured data has no identifiable structure. It includes bitmap

images/objects, text and other data types that are not part of a database. It is a generic label

for describing any corporate information that is not in a database. It can be textual or non-

textual. Textual unstructured data is generated in media like email messages, PowerPoint

presentations, Word documents, collaboration software and instant messages, Narayani et al.

(2010). Some current technologies used for content searches on unstructured data require

tagging entities such as names or applying keywords and meta tags. Therefore, human

intervention is required to help make the unstructured data machine readable. These construct

a more complete customer data platform for data mining analysis and the extraction of hidden

individualized knowledge for optimizing marketing strategies.

33

3.2 DESIGN METHODOLOGY

The information flow from data collection to usable knowledge is shown in Figure

3.2. According to Figure 3.2, the first steps are to apply pre-established selection rules in the

integration of primitive data, to decide whether to keep or discard data, and to decide the

subset to which the data belongs.

Fig.3.2: Knowledge Discovery Process

The next steps are to clean up and reorganize the data by discarding unnecessary or

redundant information, to establish record-keeping formats and contents, and to ensure the

integrity and consistency of the data in order to construct a data platform. Thereafter, the

organized data is grouped into related subjects through data transformation and data mining

processing methods are used to determine data models and to further define relationships

among various data for reference in storage and query computation. After analysis and

interpretation, the resulting models can become useful knowledge and tools for decision

support.

Integration Database Preprocessing

DataMining

Post processing

Knowledge

34

3.2.1 DATA COLLECTION

Data can be collected from the following sources:

(i) The Electronic News System

(ii) The customer service hotline

(iii) Word documents

The following is a description of these types of data:

i. Electronic News System – Internet marketing content which includes featured

reports and industry analysis.

ii. Customer Service Hotline (Customer Service Data) – The customer service hotline

is established to provide appropriate and timely responses to customer demands and

complaints.

iii. Word Documents – Textual Information.

3.2.2 EVALUATION OF CONTENT ANALYSIS

The customer data can be imported from e-mail, the contents of which are diversified

not manifestly structured. Main goals are: (1) Goal seeking or discovering core customers;

and (2) evaluation of service strategies and service efficacy. A detailed evaluation proceeded

from the following three fronts:

i. Formulation of Customer Portfolios – Customer attributes and contents of inquiries

are classified.

ii. Using Data mining technologies and methods to analyze consumer behavior and to

provide the company with an ideal service model that is designed for consumers

and oriented by customers.

iii. Providing the company with more effective marketing strategies that allow for

customer retention and identification of potential customers and new customers.

3.2.3 DATA MOVEMENT AND DATA TRANSFORMATION

The following steps outline the process of data transformation:

Step 1: Environment settings

Environment settings refer to the data mining system’s architecture

Step 2: Formulation of data models

After ascertaining requirements, the method of data storage should be determined.

There are three levels of data modeling: (1) Conceptual: Expresses the relationships among

the entities included in the data; (2) Logical: Completed without regard to which type of

35

database is to be used. This step is only completed after the user confirms the accuracy of the

model; and (3) Physical: The logical data model is actually built in database.

Step 3: Data Movement and Data Transformation

The company can extract data from customer service center systems and convert the

data into Excel format. Content analysis can be used to analyze the data content of customer

inquiries. After the analysis is completed, the data combined with other data in the database,

and the data transformations will be done. Unstructured mail feedback details from each

customer must be collected. All these feedback details were stored in text files. Each line was

tokenized and the occurrences of words were stored. Figure 3.3 shows the occurrences of

each word.

Fig. 3.3: Occurrences of Each Word

The tokenizing process is shown in figure 3.4. The feedback details about each

product for each customer were rated and stored as “Bad”, “Satisfactory”, “Good”, “Very

Good”, and “Excellent” and recorded as “.csv” format in Excel. Table 3.1 shows the sample

feedback about products. Specific occurrences of feedback rating are shown in figure 3.5.

36

Figure 3.6 shows the specific item feedback. Figure 3.7 shows the chart for specific customer

feedback.

Fig.3.4: Tokenizing the Process

37

Table 3.1: Feedback about Products

cust_id item1 item2 item3 item4 item5 item6 item7 item8 item9 item10 item11 item121 bad good bad excellent verygood bad bad bad satisfactory good bad2 verygood bad bad satisfactory good bad excellent verygood bad bad bad satisfactory3 satisfactory good bad excellent verygood satisfactory good bad excellent verygood satisfactory good4 bad bad satisfactory good bad excellent verygood bad bad bad satisfactory good5 verygood bad bad satisfactory good bad excellent verygood bad bad bad satisfactory6 satisfactory good bad excellent verygood satisfactory good bad excellent verygood satisfactory good7 bad bad satisfactory good bad excellent verygood bad bad bad satisfactory good8 verygood bad bad satisfactory good bad excellent verygood bad bad bad satisfactory9 satisfactory good bad excellent verygood satisfactory good bad excellent verygood satisfactory good

10 bad bad satisfactory good bad excellent verygood bad bad bad satisfactory good11 verygood bad bad satisfactory good bad excellent verygood bad bad bad satisfactory12 satisfactory good bad excellent verygood satisfactory good bad excellent verygood satisfactory good13 bad bad satisfactory good bad excellent verygood bad bad bad satisfactory good14 verygood bad bad satisfactory good bad excellent verygood bad bad bad satisfactory15 satisfactory good bad excellent verygood satisfactory good bad excellent verygood satisfactory good16 bad bad satisfactory good bad excellent verygood bad bad bad satisfactory good17 verygood bad bad satisfactory good bad excellent verygood bad bad bad satisfactory18 satisfactory good bad excellent verygood satisfactory good bad excellent verygood satisfactory good19 bad bad satisfactory good bad excellent verygood bad bad bad satisfactory good20 satisfactory good bad excellent verygood satisfactory good bad excellent verygood satisfactory good21 bad bad satisfactory good bad excellent verygood bad bad bad satisfactory good22 verygood bad bad satisfactory good bad excellent verygood bad bad bad satisfactory23 satisfactory good bad excellent verygood satisfactory good bad excellent verygood satisfactory good24 bad bad satisfactory good bad excellent verygood bad bad bad satisfactory good25 verygood bad bad satisfactory good bad excellent verygood bad bad bad satisfactory26 satisfactory good bad excellent verygood satisfactory good bad excellent verygood satisfactory good

38

Fig.3.5: Specific Occurrences of Feedback Rating

Fig.3.6: Specific Item Feedback

39

Fig.3.7: Customer Feedback

3.3 ARTIFICIAL NEURAL NETWORK (ANN) AND FUZZY INFERENCE

SYSTEMS (FIS) MODEL

It is observed that the use of Neural Network (NN), fuzzy logic (FL) and Genetic

Algorithm (GA) has been increased during the last decade in CRM-related applications.

However, the focus often has been on a single technology heuristically adapted to a problem.

For example, while NNs have the positive attributes of adaptation and learning, they have the

negative attribute of a “black box” syndrome. By the same token, Fuzzy Logic has the

advantage of approximate reasoning but the disadvantage that it lacks an effective learning

capability. Merging these technologies provides an opportunity to capitalize on their strengths

and compensate their shortcomings.

Fusion of Artificial Neural Network (ANN) and Fuzzy Inference Systems (FIS) have

attracted the growing interest of researchers in various scientific and engineering areas due to

the growing need of adaptive intelligent systems to solve the issues related to e-commerce.

ANN learns from scratch by adjusting the interconnections between layers. FIS is a popular

computing framework based on the concept of fuzzy set theory, fuzzy if-then rules, and fuzzy

reasoning. A key challenge for business firms is to manage customer relationships as an asset.

Compared to traditional mathematical modeling, fuzzy modeling possesses some distinctive

advantages, such as the mechanism of reasoning in human understandable terms, the capacity

of taking linguistic information from human experts and combining it with numerical data

and the ability of approximating complex non-linear functions with simple models. To create

40

an effective toolkit for the analysis of customer relationships, a combination of relational

databases and Adaptive Neuro-Fuzzy logic is proposed.

3.3.1 ADAPTIVE NEURO FUZZY INFERENCE SYSTEM (ANFIS)

The ANFIS approach learns the rules and membership functions from data. ANFIS is

an adaptive network. An adaptive network is network of nodes and directional links.

Associated with the network is a learning rule - for example back propagation. It is called

adaptive because some, or all, of the nodes have parameters, which affect the output of the

node. These networks are learning a relationship between inputs and outputs.

Rules. The if-then rules have to be determined somehow. This is usually done by

‘knowledge acquisition’ from an expert. It is a time consuming process that is fraught

with problems.

Membership functions. A fuzzy set is fully determined by its membership function.

ANFIS architecture is shown below in figure 3.8. The circular nodes represent nodes

that are fixed whereas the square nodes are nodes that have parameters to be learnt.

Layer 1 Layer 2 Layer 3 Layer 4 Layer 5

1w 1w 11fw X

F

Y 2w 2w 22fw

A1

A2

B1

B2

Fig.3.8: ANFIS Architecture for Two Rule Sugeno System

A Two Rule Sugeno ANFIS has rules of the form:

111111 ryqxpfTHENBisyandAisxIf

222222 ryqxpfTHENBisyandAisxIf

41

For the training of the network, there is a forward pass and a backward pass. The

forward pass propagates the input vector through the network layer by layer. In the backward

pass, the error is sent back through the network in a similar manner to back propagation.

Layer 1

The output of each node is:2,1)(,1 iforxO

iAi

4,3)(2,1 iforyO

iBi

)(,1 xO i is essentially the membership grade for x and y .

The membership functions could be anything but for illustration purposes we will use the bell

shaped function given by:

ib

i

i

A

acx

x 2

1

1)(

where iii cba ,, are parameters to be learnt. These are the premise parameters.

Layer 2

Every node in this layer is fixed. This is where the t-norm is used to ‘AND’ the membership

grades - for example the product:

2,1),()(,2 iyxwOii BAii

Layer 3

Layer 3 contains fixed nodes, which calculate the ratio of the firing strengths of the rules:

21,3 ww

wwO iii

Layer 4

The nodes in this layer are adaptive and perform the consequent of the rules:

)(,4 iiiiiii ryqxpwfwO

The parameters in this layer ( iii rqp ,, ) are to be determined and are referred to as the

consequent parameters.

Layer 5

There is a single node here that computes the overall output:

42

i i

i iii

iii w

fwfwO ,5

It can be shown that for the network described if the premise parameters are fixed the

output is linear in the consequent parameters.

S = set of total parameters

1S = set of premise (nonlinear) parameters

2S = set of consequent (linear) parameters

So, ANFIS uses a two pass learning algorithm:

Forward Pass. Here 1S is unmodified and 2S is computed using a Least Squared Error

(LSE) algorithm.

Backward Pass. Here 2S is unmodified and 1S is computed using a gradient descent

algorithm such as back propagation.

The summary of the process is given below:

The Forward Pass

Present the input vector

Calculate the node outputs layer by layer

Repeat for all data A and y formed

Identify parameters in 2S using Least Squares

Compute the error measure for each training pair

Backward Pass

Use steepest descent algorithm to update parameters in 1S (back propagation)

For given fixed values of 1S the parameters in 2S found by this approach are guaranteed to

be the global optimum point.

The neuro-adaptive learning method works similarly to that of neural networks.

Neuro-adaptive learning techniques provide a method for the fuzzy modeling procedure to

learn information about a data set. Fuzzy Logic Toolbox software computes the membership

function parameters that best allow the associated fuzzy inference system to track the given

input/output data. The Fuzzy Logic Toolbox function that accomplishes this membership

function parameter adjustment is called anfis. The anfis function can be accessed either from

the command line or through the ANFIS Editor Graphical User Interface (GUI). Using a

43

given input/output data set, the toolbox function anfis constructs a Fuzzy Inference System

(FIS) whose membership function parameters are tuned (adjusted) using either a back

propagation algorithm alone or in combination with a least squares type of method. This

adjustment allows your fuzzy systems to learn from the data they are modeling.

A network-type structure similar to that of a neural network, maps inputs through

input membership functions and associated parameters. Then through output membership

functions and associated parameters to outputs, it can be used to interpret the input/output

map. The parameters associated with the membership functions changes through the learning

process. The computation of these parameters (or their adjustment) is facilitated by a gradient

vector. This gradient vector provides a measure of how well the fuzzy inference system is

modeling the input/output data for a given set of parameters. When the gradient vector is

obtained, any of several optimization routines can be applied in order to adjust the parameters

to reduce some error measure. This error measure is usually defined by the sum of the

squared difference between actual and desired outputs. An anfis uses either back propagation

or a combination of least squares estimation and back propagation for membership function

parameter estimation.

3.3.2 ANFIS MODEL FOR CUSTOMER RELATIONSHIP MANAGEMENT

The modeling approach used by anfis is similar to many system identification

techniques. First, we need to hypothesize a parameterized model structure (relating inputs to

membership functions to rules to outputs to membership functions, and so on). Next, we have

to collect input/output data in a form that will be usable by anfis for training and then use

anfis to train the FIS model to emulate the training data presented to it by modifying the

membership function parameters according to a chosen error criterion. ANFIS Editor

Graphical User Interface (GUI) is used to create, train, and test Sugeno-type fuzzy systems.

Figure 3.9 shows an ANFIS Editor GUI. Figure 3.10 presents ANFIS model structure. After

the generation of FIS, the model structure has been viewed by clicking the structure button in

the middle of the right side of the GUI.

44

Fig.3.9: ANFIS Editor GUI

Fig.3.10: ANFIS Model Structure

The branches in this graph are color coded. Color coding of branches characterizes the

rules and indicate whether or not and, not, and or, are used in the rules. The input is

represented by the left-most node and the output by the right-most node. The node represents

a normalization factor for the rules. Figure 3.11 shows the FIS Editor. Figure 3.12 shows

membership function editor. Figure 3.13 represents Rule Editor. Figure 3.14 shows rule

viewer. Figure 3.15 shows surface viewer. Figure 3.16 shows the ANFIS model structure. 30

sample rules are mentioned here.

45

Fig.3.11: FIS Editor

Fig.3.12: Membership Function Editor

46

Fig.3.13: Rule Editor

47

Fig.3.14: Rule Viewer

Fig.3.15: Surface Viewer

48

Fig.3.16: ANFIS Model Structure

3.4 SUMMARY

Content analysis has been used to analyze text data. It is used to transform

unstructured customer service information into structured customer service data. This is used

to find ways to analyze text data in order to discover more latent knowledge. ANFIS model

has been used to discover the customer knowledge.