tools to find satisfaction level

8/9/2019 Tools to Find Satisfaction Level

http://slidepdf.com/reader/full/tools-to-find-satisfaction-level 1/9

Tools for Measuring Customer Satisfaction

Douglas Andrea, Sezai Dogdu, Carolynne O'GradyApril 20, 2000

ARTICLE TOOLS

Email Print Reprints Share

Useful insights for anyone who is interested in measuring and improving overall customer

satisfaction.

Customer satisfaction can be defined as thedegree to which a customer's expectations of

a product or service are in line with what the product provides. 1 As customer satisfaction is

a function of the customer's attitude, it is avery hard concept to measure. One way of

measuring customer satisfaction is to getcustomer input from various channels, suchas surveys, focus groups, or customer

complaints and combine this information inan attempt to get an accurate measure of

customer satisfaction.

This already hard task of measuring customer

satisfaction can become even more complex

when it is not obvious who the customer is. As an example, the company described in thefollowing case study has direct customers as well as resellers and OEM customers, creating a

large variety within the group and a wide variety of customer expectations.

The following discusses the multiple ways of measuring customer satisfaction, how this

information can be combined, and the use of this information to increase customer

satisfaction.

Methodology

Table 1. Conversion table used in study.

Advertisement



Throughout the research, two methods have been utilized to organize customer feedback gathered from different sources. One of them is the structured method, where the customers

are requested to choose one of the given alternatives. For instance, rating the satisfactionlevel of a product on a scale. Surveys are an example of this type of feedback. When

organizing structured information, a rating scale from 1-5 has been used. A 5 indicates very

high customer satisfaction, while a 1 indicates very high customer dissatisfaction.

In the second method, the customers are given the freedom of providing any feedback they

would like to give. Complaints to a call center or discussions in a focus group are some

examples of this type of feedback, where there might be many different answers. This kind of

feedback is referred to as "unstructured" feedback. For instance, a customer might say, "I like

the product," while another one might say "It is uncomfortable." As can be seen, this

information is very "vague" in nature and hard to compare and combine. The main reason is

that the unstructured information is in the form of natural language, which is hard to classify.

To solve this problem, a methodology similar to the concept of "fuzzy logic" is used. Fuzzy

logic is a concept based on the "degrees of truth" rather than the "true" (1) or "false" (0)

approach in the classical logic.2

Fuzzy logic uses 0 and 1 values as extreme cases of truth, but

also includes the various states in between. A classical logic example can be seen whenmeasuring height; when someone over six feet tall is defined as tall, a person who is 5'11"

will be considered short. In fuzzy logic, however, he can have a 0.9 membership value to thetall set. In fuzzy logic, membership functions are generally defined in analytical or graphical

form. As unstructured customer feedback is based on natural language, the ability of fuzzylogic to describe the in-between values makes it a valuable tool for this study.

In this study, a group of possible words and phrases that can be used by a customer to

indicate their satisfaction level are assigned a number between 1 and 5, similar to the concept

of membership function. Words such as "excellent" and "great" demonstrate high customer

satisfaction, while "poor" or "does not work" demonstrate poor customer satisfaction. Table 1

provides a sample. When a customer uses a word that is not included in the category listdemonstrated on the table, this new word/impression is evaluated, assigned a number and

included in the table; thereafter, the same rating system is used for that word to maintainconsistency.

Table 1 is used as a conversion matrix between natural language and numerical expression of

customer satisfaction level, so unstructured information can be combined with the structuredinformation. Two main differences between this study and fuzzy logic are that, fuzzy logic

uses values between 0 and 1, while this study used values between 1 and 5 to equate to

structured information. Also, in fuzzy logic, membership functions can be defined on a

continuous scale such as "0.45 tallness," while this study has only five strict customer

satisfaction levels .

It is obvious that during the establishment stage of Table 1, where words and impressions areassigned to the numerical values, there is a bias coming from the researcher. As long as

consistency is maintained throughout different types of feedback, this bias should not besignificant.

After the conversion of the natural language to numbers, the structured and unstructured

information sets can be combined. In the case study, after different types of customer



satisfaction information was gathered, it was all summarized to a number, Ci, which is between 1 and 5. Naturally, the higher the Ci is, the higher the customer satisfaction level.

Customer Satisfaction Surveys (Direct Users)

Figure 1. Customer response on the technical quality of

surveyed products.

One of the most popular methods of measuring customer satisfaction is to use surveys where

customers are requested to answer predetermined questions. It is important to get a largeenough sample size to ensure that the sample is representative of the customer population.

Ignoring the size and nature of the samples may cause misleading results. Surveys can bedone via mail, phone conversations, e-mail or through the Internet. Direct mail has

traditionally been the most popular way of conducting customer surveys, although the low

participation from customers has been a problem.3

Further, some customers are not willing tofill out information and return the surveys, even if pre-stamped return envelopes have been

provided. It is often considered a time consuming task with no tangible benefit to the

customer. In addition, phone interviews might not be effective for direct users when the

sample size is large. With the fast progress of information technology, it is expected thatusing the Internet and e-mail for conducting customer surveys will increase. Some of the

advantages of using the Internet for these types of surveys include ease of use and ease of data collection. Internet surveys are easily completed with just click of a mouse, thus giving a

higher response rate. Since data is electronically stored, the time-consuming data entry step is

eliminated.

In this case study, the Internet was used as an alternative to traditional customer survey

methods. A survey was created based on input from different internal company departmentssuch as sales, design, quality and marketing. The target of the survey was to find out the

general customer perception about the quality of the products and services.

In the survey, a total of twelve questions were asked. A couple of general questions aimed at

understanding the general customer demographics; these were followed by questions

regarding different aspects of the product's design, performance and cost. In these questions,

participants were requested to rate on a scale of 1 to 5 (1 being highly unsatisfactory, 5

indicating highly satisfactory) their response to questions such as, "How satisfied are you



with the quality of the company's products?" Also, a "comments" section was created whereusers could give any input they felt that was not covered in the survey.

Survey questions were converted to HTML format and the survey was uploaded onto the

Internet. The customers who visit the company's Web site were requested to participate in the

survey. Sometimes a higher participation can be obtained by offering an incentive (i.e., a

gift).3 In this case study, special discounts-including free shipment with a product purchase-was provided as an incentive, and 245 customers participated in the survey. As the responses

were electronically stored, the results were easily analyzed. Figure 1 presents the customer

response on the technical quality of the products.

Participants were asked to use a rating system that features numbers, since this is easy

information to collect and organize. Only the information gained from the "comments"

section is converted from natural language to a number using the methodology described

above. Overall, the customer satisfaction index, Ci, is calculated as 4.12 for customer surveys.

Measuring Customer Satisfaction for OEM Customers

OEM customers differ from direct customers in that they buy larger quantities and are not the

direct users of the product. When dealing with OEM customers, two issues should be kept in

mind: the OEM's overall satisfaction as a customer and the satisfaction level of the end-users,

where the OEM is a representative of their opinions. OEM customer opinions should be

evaluated as the voice of a much larger population. An OEM representative who bought

100,000 units should be evaluated as a representative of 100,000 users. Also, the "quality" of

the information received from an OEM representative is dependent upon that representative's

level of contact with both the end-users and the customer service department. Written or

verbal feedback is the general method in this category.

In this case study, phone interviews were conducted with OEM representatives where they

were asked to evaluate different aspects of the products such as quality, reliability, deliveryand product-to-market cycle time (when customizing products). Respondents were asked to

choose among rating categories, ranging between 1 and 5. As the number of OEM customersis a lot less than the number of end-users, all OEM customers were used as a sample in this

study.

Just like the customer survey performed via the Internet, this survey contains more structuredinformation, which is easy to organize and evaluate. The customer satisfaction index (Ci) for

OEM customers is calculated as 4.60.

Focus Groups

Focus groups, which have become very popular today, refer to a group of users who evaluate

the product at the development stage or when an improvement is planned for an existing product. Group members provide feedback. Participation is more open with this method

versus surveying direct or OEM customers because participants do not necessarily need to be



existing customers. Feedback of non-customers is still very valuable to understanding theexpectations of potential customers.

In this case study, 75 users were asked to provide feedback on three different products, where

they compared or provided feedback about the products. This is the best application of

unstructured information, due to the fact that the participants are given total freedom in

evaluating the products without having any specific categories guiding their answers. Themain reason for doing this is to allow users to be creative and provide feedback in the way

they are comfortable. Then the results gathered from this stage are converted using the

methodology described above. The customer satisfaction index (Ci) is calculated as 4.02 for

the focus group.

Customer Complaints

Another important indicator of customer satisfaction levels are customer complaints.

Customers who are not happy with the products or services they purchased generally

communicate their dissatisfaction to the company. This information is very valuable becauseit is initiated by the customer. A drawback is that customer complaints generally appear in a

random fashion, with no predictable time or nature, which makes them hard to collect andorganize. Also, it is important to note that, unlike dissatisfied customers, satisfied ones do not

generally provide any feedback. So the information gained from the customer complaintshave a negative bias, which should be taken into consideration during the analysis.

During this case study, in order to utilize the information gained from the customer

complaints, special forms were designed and distributed to the company's call center,

customer support representatives and sales personnel. These forms facilitated data collection,

such as the specific product for which the customer had complaints, details of the complaints,

and so forth. This is another example of unstructured information. This information is thenconverted to a number between 1 and 5, to correlate with other sources of information. In this

case study, there were five customer complaints surveyed, and they had a customer satisfaction index (Ci) value of 2.25.

As discussed previously, there are two different types of information that can be gained from

customers: structured and unstructured. Converting the natural language using a similar concept to "fuzzy logic" enables data collectors to use and combine these different

information types. In this study, results gained from different information sources were

converted to a number between 1 and 5. Those who require more accuracy can use a more

detailed scaling system, such as a scale from 1 to 100, which would provide more details .

However, it would be harder to classify this information in great detail.

The second step is to combine the information provided in Table 2. The following method isused in this study to combine this information:

In Equation 1, CTI represents the total (overall) customer satisfaction index; Ci represents

individual customer satisfaction indices from different sources of feedback; Wi represents therelative importance weighting of the customer satisfaction method; and n is the total number

of different customer feedback methods. Table 2 summarizes the different types of customer



feedback types, the customer satisfaction index value for each of these indicators, and theweighting values for each type.

For this particular case study, the response from OEM customers was more significant than

focus groups, customer surveys and customer complaints due to the surveying company's

sales portfolio. These weighted numbers will change from company to company or will vary

at different times in the same company based on where sales are derived-retail, value-addedreseller or OEM. In addition to that, the sample size used for each type of customer feedback

is an important factor that has to be taken into account. It is the responsibility of the

researchers to appropriately assign weights to the feedback. For this case study, the overall

customer satisfaction index (CTI) is calculated as 4.25, using the values provided in Table 2

and using Equation 1. This value suggests that, overall, the customers are satisfied-to-very

satisfied with the products and services of the surveying company.

It also should be noted that each time something is summarized, some information is lost.

The goal of this study is to provide an index that can combine different sources of customer

satisfaction information and provide a single measure to represent the satisfaction level. The

information gained from all these different feedback types should be used to improve the

quality of the products and services.

Combining and Comparing Structured and Unstructured Information

As discussed previously, there are two different types of information that can be gained from

customers: structured and unstructured. Converting the natural language using a similar concept to "fuzzy logic" enables data collectors to use and combine these different

information types. In this study, results gained from different information sources wereconverted to a number between 1 and 5. Those who require more accuracy can use a more

detailed scaling system, such as a scale from 1 to 100, which would provide more details .

However, it would be harder to classify this information in great detail.

The second step is to combine the information provided in Table 2. The method at left is used

in this study to combine this information.

In Equation 1, CTI represents the total (overall) customer satisfaction index; Ci represents

individual customer satisfaction indices from different sources of feedback; Wi represents therelative importance weighting of the customer satisfaction method; and n is the total number

of different customer feedback methods. Table 2 summarizes the different types of customer feedback types, the customer satisfaction index value for each of these indicators, and the

weighting values for each type.



Table 2. Customer feedback types, customer

satisfaction indices and weight values.

For this particular case study, the response from OEM customers was more significant than

focus groups, customer surveys, and customer complaints due to the surveying company¿ssales portfolio. These weighted numbers will change from company to company or will vary

at different times in the same company based on where sales are derived¿retail, value-added

reseller or OEM. In addition to that, the sample size used for each type of customer feedback

is an important factor that has to be taken into account. It is the responsibility of theresearchers to appropriately assign weights to the feedback.

For this case study, the overall customer satisfaction index (CTI) is calculated as 4.25, using

the values provided in Table 2 and using Equation 1. This value suggests that, overall, the

customers are satisfied-to-very satisfied with the products and services of the surveying

company.

It also should be noted that each time something is summarized, some information is lost.

The goal of this study is to provide an index that can combine different sources of customer satisfaction information and provide a single measure to represent the satisfaction level. The

information gained from all these different feedback types should be used to improve the

quality of the products and services.

Using Information to Increase Customer Satisfaction

Figure 2. Product development cycle.

When applied correctly, customer feedback is a very important tool to the success of a

company. Once the customer feedback is retrieved, the input provided by the customers

should be carefully evaluated and used to improve the overall satisfaction level of all



customers. If this step of the customer satisfaction process is not executed carefully, theabove procedures are really a waste of time.

Figure 2 demonstrates the overall product development cycle; as the figure shows, it all starts

with the customer and ends with the customer. Throughout this study, all the customer

feedback was shared with the appropriate departments to take improvement actions. Overall,

a total of 48 improvement ideas were collected from customer surveys, including new product ideas, improvements on existing products and ideas for better customer service. All

these ideas were applied to increase the overall customer satisfaction level.

Similar to any quality improvement activity, improving customer satisfaction is a continuous

process. After customer feedback is received and an action is taken, customer satisfaction

should be measured again.

A Continuous Process

This study demonstrates an effective use of innovative methods for measuring customer satisfaction. The suggested index can be used to summarize the customer satisfaction

information from different sources down to a single number.

Currently, this study provides a snapshot of the customer satisfaction level, but does not

provide any information about the trends. Replicating the study at a later date will provide

information on the effectiveness of the improvement actions, as well as providing a tool to

analyze the trends in the customer satisfaction level over time. As customers' attitudes

towards the products and services change very slowly, a six-month to one-year replication

period is suitable for measuring customer satisfaction. Although this time period is suitable

for surveying purposes, the process of improving customer satisfaction should be a

continuous one. Also, the measures discussed herein were the most suitable ones for the purposes of our company. Other companies might want to integrate other measures, such as

customer loyalty, customer returns and the cost of dissatisfied customers.

Measuring Customer Satisfaction is Useful

Understanding customer satisfaction and the ability to meet customer expectations has adirect effect on a company's revenue performance. The ability to react to shortcomings will

lead to higher customer retention and follow-on sales, as well as increased customer loyalty.

Measuring customer satisfaction can be very useful for companies to evaluate where theystand with respect to the competition and end-user, and to find out which areas requireimprovement. Customer feedback from direct or OEM customers, focus groups and customer

complaints are representative of some of the available tools for measuring customer satisfaction levels. Due to their nature, some of this information is structured and some of it is

unstructured. However, converting natural language to numbers enables the combination andcomparison of these different types of information. An index based on weighted averages can

be used to get a single output from all these different sources of information. Tracing this



index over a period of time can provide useful insights about the trends in customer satisfaction.

Acknowledgements

The aut hors would like t o t hank t he individual cust omers and OEM partners who have

participated in t he company's surveys. In addition, we would like t o t hank all t he empl o yees

of Andrea Electr onics C orporation for t heir continuous support and suggestions for

measuring and impr oving cust omer satis f action. Especially, we would like t o t hank Melissa

Marcic for her e fforts during t he preparation and analysis of t he cust omer surveys, Lauren

Smit h and Jean Mulry for t heir e fforts wit h t he OEM cust omer interviews and Bradley

Borders for his focus gr oup work and insig hts.

tools to find satisfaction level

Documents