how to conduct a survey - by nbri inc - america - ebook

8/10/2019 How to Conduct a Survey - By NBRI Inc - America - eBook

1/24NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBR

A PRIMER ONSURVEY RESEARCH

BY DR. JAN G. WEST, PH.D.

How to Conducta Survey
http://www.youtube.com/embed/arHFglXtG70?autoplay=1


2/24

2HOW TO CONDUCT A SURVEY: A Primer on Survey Research

NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBR

Youve been tasked with conducting a survey of your customers and/or employees, so the rst thing youre doingis searching the internet for how to information. Or, like most people, youve always thought that building a surveymeant pulling questions out of the air, slapping them on a piece of paper, and calling it a survey! But now thatyoure trying to build one yourself, youve run into your rst stumbling block: the survey questions themselves.

What do I ask? How do I ask them? Youve come to the right place! Not only will you learn how to conduct surveyresearch here, but you will also learn why. When the why rings true with good, common sense, you will havegreat con dence in how you run your study, and you will be able to defend your actions and recommendations toothers. Most importantly, you will be obtaining fact and not ction from your research, so that stakeholderswho use your information can be con dent when basing important decisions and actions upon it.

The goal of great research is to drive business outcomes: improve Customer Satisfaction, Customer Intent toReturn or Willingness to Recommend, and Employee Engagement in order to improve Financial Performance.Survey Research without Return on Investment may provide interesting information, but is truly a waste of timeand money. For over 3 decades, the National Business Research Institute (NBRI) has helped business leaders

understand the psychological constructs of attitude, opinion, and belief that determine employee and customerbehavior, and drive pro tability. In this eBook, we share our knowledge of the following aspects of survey research:

1. SURVEY CONTENT IS YOUR SURVEY VALID? ........................................................................................Page 4 a. Topics

b. Questions

c. The Scale

2. DEPLOYMENT METHODOLOGIES ..................................................................................................................Page 8 a. Paper

b. Online c. Telephone

d. In Person

3. REPRESENTATIVE SAMPLE IS YOUR DATA VALID? .............................................................................Page 10 a. Con dence Levels and Sampling Error

b. Sample Size Chart

4. REPORTING & THE STATISTICS OF SURVEY RESEARCH .......................................................................Page 13 a. Percent Favorable and Top Box Scores

b. Mean Scores

c. Benchmarking Scores

5. TARGETED MARCHING ORDERS .................................................................................................................Page 18 a. Descriptive Statistics

b. Inferential Statistics

6. TURNING DATA INTO ACTION .....................................................................................................................Page 22


3/24



TEST YOUR KNOWLEDGE OF SURVEY RESEARCH!Thinking about a recent survey project or a survey currently in deployment, answer True or False for each of thefollowing statements. Each statement is worth 5 points.

1. My survey is valid represents the issues facing the target audience. T / F

2. My survey questions have been standardized through testing on millions of individuals. T / F

3. My data is valid represents the thinking of the target population. T / F

4. I have achieved Representative Sample at a minimum of 95% Con dence and 5% Error at the T / F Total Population Level and every segregation thereof.

5. The response scale I use does not bias my data arti cially. T / F

6. Percent Favorable Scores are context, and are not used to interpret results or drive decision-making. T / F 7. Mean Scores are context, and are not used to interpret results or drive decision-making. T / F

8. I only base decisions on Benchmarked Scores. T / F

9. Qualitative Data is only used to drive survey content. T / F

10. My survey questions are pure, clean, and actionable. T / F

11. One of the primary root causes of Customer Intent to Return is Employee Engagement. T / F

12. One of the primary root causes of Financial Performance is Customer Intent to Return. T / F

13. My data tells me the root causes of my Customers Intent to Return. T / F

14. My data tells me the root causes of my Employees Engagement. T / F

15. My Survey Vendor takes me from reporting my survey data to implementing solutions in 3 weeks. T / F

16. My Survey Vendor has the ability to benchmark my data against my industrys data. T / F

17. I receive targeted action items following the survey. T / F

18. My Survey Vendor assists with communications to my target population. T / F

19. I have a project team to guide me through the survey process. T / F

20. My Survey Vendor provides Synergistic Research for my organization. T / F

BONUS: The integrity of my data is absolute, so I have con dence tying improvement goals T / F to management bonus plans.

In Best in Class Research Studies, such as those provided by NBRI, the answer to every question above is True!A = 90-100 points B = 80-85 points C = 70-75 points D = 60-65 points F = 55 or fewer points


4/24



1 SURVEY CONTENT

The rst step, of course, is deciding on survey content. Do you decidethis alone? Do you ask co-workers or other management personnel

for their input? Do you ask the audience what they think you shouldbe asking them? Never has it been more true than in survey researchthat garbage in = garbage out. This can be a frightening anddaunting task, as you realize the critical nature of customer andemployee opinion as driving forces on your organizations nancialperformance. What if customers do not intend to return to yourbusiness? What if employees are apathetic, and are actually runningoff customers? This makes it imperative that each study providemanagement with pure, valid data fact and not ction. While itis easy to conduct survey research improperly, it is just as easy toconduct it properly by adhering to a few, simple principles.

There are really only three sources of survey content: you, us, and them. You, most likely an Executive of a MajorCorporation; Us, Survey Research Consultants; and, Them, Your Target Audience. From the horses mouth comes to mind about now because who better to tell you what you should ask on your survey than those who willbe responding to it? Your employees and customers know very well what you should include in your survey, as theylive the issues the survey will address every day. Clearly, your employees and customers are the best source ofinput on what the survey should cover.

Great, you think. Now I have to do research before I can even begin the survey. And how does one go about

conducting Preliminary Research?

TOPICS

Beginning in 1982 and for over 3 decades now, NBRI has been conducting one-on-one telephone interviews ofsmall, strati ed, random samples of individuals in employee and customer populations within virtually everyindustry sector. As we have done so, we have discovered the issues that are important to all such populations.Thirty years ago, we called these organizational dynamics because thats exactly what they were, and there wereno other names for them. Today, we call them topics. Topics are simply broad categories of survey questiontypes. Some of the most common Customer and Employee Survey Topics are shown on the following page.


5/24



CUSTOMER SURVEY TOPICS

Brand Engagement Ethics Sales AssistanceCommunications Financing Sales Process

Company Behavior Friendliness & Helpfulness Service & SupportCompany Image Invoicing & Statements Services

Competitive Position Job Knowledge TeamworkControl Systems Pricing Technical Expertise

Cost & Value Product Delivery Technical SupportCreativity Products Values

Customer Loyalty Professional Conduct Wait TimeCustomer Service Project Management WarrantiesDecision Making Quality Website

Employee Behavior Safety

EMPLOYEE SURVEY TOPICS

Autonomy Handling Ambiguity Professional ConductBene ts Health Pro t Improvement

Career Development Human Resources Project ManagementChange Management Integrity Quality

Climate Interpersonal Skills RecognitionCommunications Job Satisfaction RespectCompany Image Job Training Results DrivenCompensation Leadership Safety

Competitive Position Leading Change Sexual HarassmentControl Systems Learning Agility Short & Long Term Goals

Corporate Performance Life Balance Social ActivitiesCreativity Management Style Social ResponsibilityCulture Managing Performance Strategic Agility

Customer Service Mergers & Acquisitions SupervisionDeveloping People Morale Teamwork

Diversity Organization Technology

Employee Commitment Organizational Change ValuesEmployee Values Organizational Structure VisionEngagement Performance Evaluations Work Life

Ethics Planning Working RelationshipsFlexibility Policies

Fostering Relationships Productivity


6/24



QUESTIONS

Also beginning in 1982 and for over 3 decades now, NBRIhas been writing the questions to assess those topics;questions that have become the very foundation of theSurvey Research Industry. So, as you see, you do not needto interview your customers and employees rst! You donot need to discover the issues of importance to them.And you do not need to write the survey questions. Thathas all been done for you by NBRI.

It is important to reiterate that you are conducting psychological research of your customers and employees. What? I just wanted to survey them! Right! And assessing the psychological constructs of attitude, opinion, and belief, thosehuman perceptions that drive behavior, is by de nition, psychological research! As such, how each survey question is

phrased is very important. The wording of each question has the potential to bias responses arti cially, either to thepositive or negative.

Consider, for example:

I can always talk to my supervisor when I need to versus

My supervisor is available when I need him/her

People will typically zero in on the word always, and thats not really the point of the question, is it? The wordalways causes the respondent to drift toward a negative rating because no one is always available. Since the

intent is to determine if supervisors are available when needed, the latter question will gather clean, pure data,whereas the former question will generate data that is arti cially skewed to the negative. The result: managementmay believe there is a problem when there isnt one. There will be no red flags warning about the corruptinginfluence of the wording on the data.

Consider another example:

My Account Executive is friendly and knowledgeable

This is what is commonly called a double-barrel. It is seeking to assess two issues in one question. When not if you receive less than 100% positive feedback, the next question Management will ask you is: Which issue were

they rating friendliness or knowledge? This is best broken out into two, separate questions.

And there are many more examples of poor survey question wording, such as vague wording that provides nomeaningful, actionable data, and even harmful wording that disseminates a negative connotation about theCompany to the audience.

Today, it is NBRI questions that are used by thousands of small and mid-sized organizations, as well asone-fourth of the largest corporations in the United States, and numerous organizations around the world.Each question you may select for inclusion in your study is a standardized, scienti c, psychological research


7/24



instrument that has been tested on millions of individuals. Or would you rather reinvent the wheel and riskconducting a survey that delivers misinformation?

The order in which questions are presented may also bias responses. It is important to randomize the order ofquestions on employee surveys, but not customer surveys. Customers should be taken through the survey in athoughtful, logical order, with questions grouped by topic. But employees may have biases against certain topics,so the order of survey questions to employees should be randomized to eliminate biasing the data.

THE SCALE

Even the scale may cause your data to be more positive or negative than data collected with an unbiased scale.Consider, for example, a scale of Excellent, Extremely Good, Very Good, Good, Fair. This type of scale will producedata that is arti cially biased to the positive, rendering the results worse than meaningless the data will actuallybe misleading.

The scale typically used in psychological research is a 6-point, balanced scale with 3 points of positive and 3points of negative. This way, the respondent has high, medium, and low options of both the positive and negative.And, there is no neutral point. After all, the intent of the survey is to gather opinions. If your survey is valid, de nedas representative of the issues facing your audience, then as psychologists, we know that an opinion exists foreach and every survey question. Yes, it is a psychological fact that if you have knowledge of an issue, you willnecessarily have an opinion about it. Even if it is as simple as vanilla ice cream. If you have experienced it, youhave an opinion. The goal is to gather opinions.

THE NORMATIVE SCALE

StronglyAgree

ModeratelyAgree

SlightlyAgree

SlightlyDisagree

ModeratelyDisagree

StronglyDisagree

6-point scale

Balanced

Unbiased to the positive or negative

100 years of psychological research

No neutral point: force the opinion

When NBRI has imported data for analysis from organizations that previously conducted their own research andused a neutral point, we have found that respondents mark the neutral point about 20% of the time. This datamust be discarded. It is meaningless. This means that 20% of the survey dollars were just spent on no data at all!When a respondent wants to skip a question, they can and they will. Why would anyone want to offer a no opinionoption on every single survey question?


8/24



2 DEPLOYMENT METHODOLOGIES

PAPER

Paper surveys, while far less popular since the inception of the internet, representa valuable methodology for reaching the large numbers of employees andcustomers without computer access, email addresses or telephone numbers.When considering deploying a paper survey, outbound and inbound postage mustbe considered, as well as ink, paper, printing, folding, stuf ng, sealing, etc., and thelabor cost to do so. Furthermore, one can expect only about a 10% response ratefrom customers asked to respond to a paper survey. If this is your only option, besure to check out Section 3 of this eBook to learn about the validity of data anddetermine whether or not you will be able to have con dence in your results.

ONLINE

Online surveys have become the most popular means of deploying a survey becauseof their time- and cost-effectiveness. In addition, surveyors can quickly respond topoor scores and negative feedback from the respondent submitting the survey. This iscritical, indeed, as customers who experience issues are much more likely to patronizean organization again if issues are dealt with promptly and satisfactorily, while thosewhose issues are ignored or dealt with poorly tend to spread the word and cost theorganization not only their own business but that of many others, as well.

Because of the relatively low cost of online surveys, and the higher customerresponse rate of 20% as compared to paper surveys, many Do-It-Yourself optionshave sprung up on the internet in the last decade. Companies like Zoomerang and Survey Monkey make millionsof dollars per year by offering basic survey programming for the uneducated who would create their own surveys.Completely ignored by these tools are considerations of biases in the wording of the questions, the validity of thesurvey and the validity of the data gathered, reaching representative sample, employing appropriate statisticalanalyses and data interpretation, benchmarking, and assistance with responding to survey results. This placesevery user in the precarious situation of basing important business decisions on misinformation. Users of thesetools generally have no idea that their data is worse than worthless it is actually misleading.

There are no warning signs in research. You either know how to conduct research properly, or you dont. Most college-educated business people do not learn how to conduct psychological research unless they have participated indoctoral-level courses that emphasize research methods. Companies like Zoomerang and Survey Monkey perpetuatethe myth that a survey can be created by pulling questions out of the air and sending it to a large number of people.Research that drives your business when done properly will provide a return on investment that is huge comparedto the cost of a study done well. While cost is an important consideration, the choice between spending a few hundreddollars for junk data versus a few thousand dollars for hard, objective data should clearly be no choice at all.


9/24



TELEPHONE

The highest labor cost but also the highest response rate of all for customer surveysis through telephone deployments. An online survey is still programmed, but insteadof deploying it online, telephone interviewers read the survey to participants over thephone and enter the data online for them.

Some survey research organizations offer IVR (Interactive Voice Response) as aninexpensive alternative to live telephone interviewers, but beware of the insult thismay cause to your valued customers! NBRIs research shows that ill-will is oftencaused by the use of recorded telephone interviewers, and highly recommendsagainst it. On the other hand, customer loyalty actually increases when you put yourcustomer feedback in the hands of a highly skilled researcher. Deeper exploration of opinions and beliefs is onlypossible between two humans, not between a human and a machine, and customers are keenly aware of whether

or not you feel they are worth it.Due to the fact that interviewers are able to call a phone number numerous times if no one is reached the rstseveral times, and can do so without upsetting anyone, response rates of customer surveys deployed via telephoneare typically around 30%! Caller ID and leaving a call-back number also increase participation.

IN PERSON

The rarest of all survey deployments is in person. Due to the high labor cost,usually out of the of ce and often out of town, thus requiring transportation andaccommodations, it is unusual to nd a situation that is not better handled viapaper, online, or telephone. Indeed, only when extremely high response rates arerequired due to the small size of the population should In Person Deployment evenbe considered. With proper planning and scheduling, however, customer responserates of 100% are possible through this methodology!

The response rates for employee feedback do not differ by deployment methodology: one can expect an 85% to95% response rate when Best Practices are followed.

The response rates for customer surveys differ greatly by deployment methodology as discussed above.


10/24



3 REPRESENTATIVE SAMPLE

Just as a valid survey truly represents the issues facing the target population, valid data truly representsthe thinking of the target population. The validity of psychological data is easily determined by measuring the

Con dence Level and Sampling Error.Important! All surveys have Con dence Levels and Sampling Error, including one in which 100% of the populationparticipates. It is critical to measure and report these statistics in any form of survey research in order to prove thevalidity of the data.

CONFIDENCE LEVELS AND SAMPLING ERROR

Lets de ne Con dence and Error. They are both probabilities. Con dence is the probability that the data does,in fact, reflect the thinking of the target population, as if every person had participated. This is an important

consideration since gathering data from 100% of any population, even an employee population, is typicallyimpossible. Someone is on vacation; someone refuses to participate; someone is out sick; etc. The issue is evenmore complex with customers, particularly if you have hundreds of thousands of them and dont want to spend themoney surveying each and every one.

CONFIDENCE LEVEL

The probability that the samplerepresents the whole.

SAMPLING ERROR

The probability that there are differencesin the sample from the whole.

So, if you have a population of 100 employees or customers, and only 10 complete the survey, does your datarepresent how all 100 people think? What if you have 10,000 employees or customers, and 1,000 complete thesurvey? Will a 10% response rate from any size population provide us with the con dence to claim that our data isvalid and truly represents the thinking of the entire population?

Sampling Error is basically the opposite of Con dence Level. Sampling Error is the probability that we have apreponderance of people in our data that feel differently than the majority, such as extremely pleased or displeasedpeople. Data with large amounts of Error in it misrepresents the thinking of the population as a whole, and is notvalid. Error, then, must be kept to a minimum.

Reminder! All surveys have Sampling Error, including one in which 100% of the population participates, and sinceit is critical to report this statistic in research, you need to know how to measure it. Do we pull numbers out of the


11/24



air? Do we hope, think, wish our way to 50% being o.k. 75% being o.k. 90% being o.k.? Certainly not! This ishard, objective, scienti c research upon which our Clients often base multi-million dollar decisions. We have toknow, and so do you!

Based on 100 years of psychological research, NBRI depends on a proven Sample Size Chart!

SAMPLE SIZE CHART

Across the top of the chart shown below are Con dence Levels, from 99% to 90%. Under the Con dence Levelsare Sampling Error Levels from 5% to 3%. Con dence can be as high as 99.99% or very low. Generally, Con dencebelow 80% is considered junk data. Sampling Errors can be as low as less than 1% or very high. Generally,Sampling Error above 7% is considered junk data. The left column shows various Population Sizes.

SCIENTIFIC PSYCHOLOGICAL RESEARCH

SAMPLE SIZE CHART

Con dence Level 99% 95% ResponseRate

99% 95% 90%

Sampling Error 5% 5% 3% 3% 3%

Population Size50 46.5 44.2 88.4% 48.6 47.7 46.8

100 86.9 79.3 79.3% 94.8 91.4 88.2500 285.1 217.2 43.4% 393.3 340.4 300.2

1,000 381.9 269.2 26.9% 604.7 488.2 409.52,000 498.2 322.2 16.1% 959.2 695.8 546.310,000 622.2 369.9 3.7% 1556.3 964.2 699.150,000 654.8 381.2 0.7% 1777.7 1044.8 740.5

100,000 659.2 382.6 0.4% 1809.9 1055.8 746.0

The box around 95% Con dence Level and 5% Sampling Error highlights the fact that these levels are perfectlyacceptable in business and science. Note that with a Population size of 1,000, 269.2 completed surveys arerequired to reach a 95% Con dence Level and 5% Sampling Error, but 604.7 completed surveys are required to

reach a 99% Con dence Level and 3% Sampling Error. It is not necessary to spend money collecting and analyzingmore data than is needed without an imperative to do so. Many of NBRIs studies are submitted to governmentalagencies, such as the Federal Drug Administration (FDA), and a 99.9% CL with less than 1% SE is required. Ingeneral, however, NBRI recommends the 95%/5% to Clients because of the high validity of the data and the cost-savings to do so.

So, does 10% from populations of 100 or 1,000 obtain valid data for us? As the chart shows, a 79.3% ResponseRate, or 79.3 completed surveys is required to obtain valid data at a 95% Con dence Level and 5% Sampling Error


12/24



from a total population of 100 individuals, so 10% would be junk data. For a total population of 1,000 individuals,a 26.9% Response Rate would be required for a 95% & 5% result, so 10% would be insuf cient in this case, as well.However, 10% would be more than enough to reach 95% & 5% for 10,000 people and 100,000 people, as only 3.7%and 0.4% Response Rates are needed for these populations, respectively.

It is important to note that valid data must be obtained at the total population level and at each level of segregationthereof. For example, if your total population in your Market Research Study is 1,000 people, and you want to learnhow the thinking of 500 males and 500 females differs, you will need a 43.4% Response Rate, or 217.2 completedsurveys from each subgroup. Likewise, valid data based on the Con dence Level and Sampling Error you selectmust be obtained for each division, business unit, or department of a Company in an Employee Survey if you wantto compare the thinking of such groups.

Clearly, conducting Scienti c Psychological Research that results in pure, clean, hard data that truly representsthe thinking of the population and its subgroups is best done by those with extensive education, training, andexperience in the eld. Sources of error abound and are typically overlooked by the untrained eye. Defending thequality and purity of ones research is a basic requirement for Doctoral Degrees, and represents a key value thatPh.D.s at NBRI bring to every Clients Survey Research Study.


13/24



4 REPORTING & THE STATISTICS OF SURVEY RESEARCH

There are basically only 3 statistics used in Survey Research: percent favorable (or top box), mean scores, andbenchmarking scores. These are known as Descriptive Statistics because they describe the body of data in terms

of its strengths and weaknesses. But these statistics cannot tell you the most important issues to work on. Thiswill be addressed later employing Inferential Statistics.

When reporting results, it is best to report all 3 types of descriptive statistics so that the report user has the fullcontext of the information from which all scores are derived. Consider the report format shown below.

In the upper left corner is the Topic: Employee Behavior. Then, there are 3 Survey Questions assessing this Topic:

Company personnel deliver what is promised.

Company personnel exhibit professionalism.

Company personnel are well trained.

Beneath each Survey Question are demographic segregations of the data:

Total Customer Base 2013.

Total Customer Base 2012 for historical comparison, also known as internal benchmarking.

Customer Type 1, 2, 3, and 4.


14/24



The 2nd column shows the N, or Number Responding. The 3rd column shows the Mean Scores. The 4th columnshows the Benchmark Scores. Benchmark Scores are graphed and color-coded in the 5th column. Finally, theScale and entire Distribution of Responses (DOR) are shown in the last 6 columns.

In this way, the report user has all of the information in one place. This shows exactly how many individualsanswer in exactly which way, and rolls up all of the information to Total Customer Base benchmarked against theIndustry level via the Benchmark Score.

Now lets look at the 3 demographics outlined in red: Customer Types 2, 3, and 4, and lets consider eachstatistic individually.

PERCENT FAVORABLE AND TOP BOX SCORES

As scienti c researchers who always choose the objective, absolute over the subjective, debatable, NBRI warnsClients about Percent Favorable and Top Box Scores as we believe them to be misleading.

First, one must de ne what constitutes Percent Favorable and Top Box Scores. Bob may feel that it should onlybe those who Strongly Agree. Sally may want to de ne it as those who Strongly and Moderately Agree. And Tedmay feel it should include all respondents who agree whether Strongly, Moderately, or Slightly. This has allowedsubjectivity to enter into what was otherwise an objective research study, which greatly weakens it. How onede nes Percent Favorable or Top Box is clearly open to debate.

Next, lets say they decide that Percent Favorable should include all respondents who marked Agree, includingStrongly, Moderately, and Slightly. In the example above, Customer Types 2, 3, and 4 outlined in red all equal 100%Percent Favorable! If they report the single statistic of 100% for each group, their readers will believe there were no

differences between these groups in this instance. Clearly, this would be false information.

Finally, how does one decide if a Percent Favorable or Top Box Score is high or low, good or bad? The only option isto compare the score to a scale of 1 to 100. Unfortunately, the scale does not represent reality, and results in falseinformation! The same is true with Mean Scores, so we discuss this in more detail below.

MEAN SCORES

In our example above, it is apparent that there are differences between the Customer Types 2, 3, and 4, with MeanScores of 4.77, 5.45, and 5.67, respectively. The Mean Score is the average of all scores, and is calculated by

weighting the six response choices. It is clear from the Mean Scores that true differences exist between thesegroups, but what do the Mean Scores mean? Is Customer Type 4 signi cantly higher than Customer Type 3 and 2?And are these good, average, or poor scores? Does the Vice President of Customer Type 4 qualify for a biggerbonus than the Vice President of Customer Type 2?

Many people compare the Mean Score to the Scale itself to determine whether it is high or low, good or bad. Hint:the mean of a 6-point scale is 3.5 (not 3, see below). Comparing the Mean Score to the scale (whether it is aboveor below the mean of the scale, for instance) results in false information because the scale does not represent


15/24



reality! That is, the average of all responses to all questions from all groups of all people is not 3.5! Consider thetwo Customer Survey questions in the picture below, both of which receive a Mean Score of 3.99, for example.

BENCHMARKING SCORES

What if you knew how people normally answer any given question? NBRI does! This is called normative orbenchmarking data.


StronglyAgree

ModeratelyAgree

SlightlyAgree

SlightlyDisagree

ModeratelyDisagree

StronglyDisagree

6 5 4 3 2 1

3.99

Is this a high score or a low score?

I would recommend doing business with the Company to others.Sales Representatives are responsive to my needs.

NBRI compares the data from your respondents with data from people in your industry who have answered

the same survey questions about their places of work (employees) or about your competitors (customers)!Benchmarking Data is a mapping of mean scores from thousands of individuals to a Scale of the 1st to 100thpercentiles. The Average of all mean scores is at the 50th percentile; the 75th percentile represents StretchPerformance; and the 90th to 100th percentiles represent Best in Class Performance. These are the Companieswith the highest Employee Engagement, Customer Loyalty, and Financial Performance!

When we look at Benchmarking Data, we may nd that a Mean Score of 3.99 is a poor score for one surveyquestion, but a great score for a different survey question! The charts below demonstrate this principle.

For the survey question, I would recommend doing business with the Company to others, a mean score of 3.99

is a poor score at the 38th percentile of this particular benchmarking database, with the Industry Average at 4.56(50th), Stretch Performance at 5.18 (75th), and Best in Class Performance at 5.64 (90th).


16/24




StronglyAgree

ModeratelyAgree

SlightlyAgree

SlightlyDisagree

ModeratelyDisagree

StronglyDisagree

6 5 4 3 2 1

I would recommend doing business with the Company to others.

For another survey question, Sales representatives are responsive to my needs, a mean score of 3.99 is a Best in

Class score at the 96th percentile of the benchmarking database!


StronglyAgree

ModeratelyAgree

SlightlyAgree

SlightlyDisagree

ModeratelyDisagree

StronglyDisagree

6 5 4 3 2 1

Sales representatives are responsive to my needs.

In short, it is impossible to understand the meaning of any score percent favorable, top box, mean score, or anyother without benchmarking data.

Another powerful bene t of benchmarking data is the ability to group results by quartile into a SWOT Analysis:Strengths, Opportunities, Weaknesses and Threats. When the Total Company receives a benchmark score, each

Business Unit, Division, and Department receives benchmark scores, each Customer Type or other CustomerDemographic Group receives a benchmark score, and every Topic and Question on the survey receives abenchmark score, it is immediately evident how each score ranks against the Industry (external benchmarking)and against each other (internal benchmarking). In addition, because 5 or more benchmarking percentilesis statistically signi cant (de ned as true, factual, not due to chance), you know immediately if differencesbetween scores in the present study are signi cant or not, and you know immediately if differences betweenscores in the present versus past studies are signi cant or not. The goal of great research is to drive businessoutcomes: improve Customer Satisfaction, Customer Intent to Return or Willingness to Recommend, and Employee

3.99MEAN96TH

2.2650TH2.7875TH3.6490TH

3.99MEAN38TH

4.5650TH

5.1875TH

5.6490TH


17/24



Engagement in order to improve Financial Performance. Survey Research consistently provides interestinginformation about an organization but wastes time and money if no action is taken on the results to effect a returnon investment. When actions are taken, it is important to measure their effects. Benchmarking data provides animmediate understanding of the differences between scores.

SWOT ANALYSIS

Category Benchmarking RangeStrength 75th to 100th Percentile

Opportunity 50th to 74th PercentileWeakness 25th to 49th Percentile

Threat 1st to 24th Percentile

Immediate understanding of results

5 or more percentiles = statistical signifcance


18/24



5 TARGETED MARCHING ORDERS

DESCRIPTIVE STATISTICS

In the Chapter above, we discussed the 3 basic statistics of Survey Research: percent favorable (or top box),mean scores, and benchmarking scores. These are known as Descriptive Statistics because they describe thebody of data in terms of its strengths and weaknesses. In particular, employing benchmarking scores puts us inthe strong position of seeing exactly where the organization is performing well and where it needs interventionby region, or department, or customer type, or any other demographic, as well as by topic and question, and as anorganization as a whole.

The chart below shows typical results for improvements in survey question scores from an Employee Survey thathas been deployed annually for 3 years.

ITEM SWOT ANALYSIS

In the First Annual Assessment in 2011, there were a preponderance of Threats and Weaknesses, shown in green. Inthe Second Assessment in 2012, the distribution of Item Scores was more evenly spread from Threats to Strengths,shown in light blue. And in the Third Assessment in 2013, there is a preponderance of Strengths and Opportunities,shown in dark blue. Moving the scores at the item or survey question level ensures Total Company improvement, asin the example on the following page.

Strength100th - 75th

16

5

0

8 755

15

1

6

1012

2013 2012 2011

0

10

20

30

N u m

b e r o

f I t e m s

Opportunity75th - 50th

Weakness50th - 25th

Threat25th - 1st


19/24



Ice Cream Sales Swimming Accidents

TOTAL COMPANY SCORE

In the chart above, ABC Company moved from the 63rd percentile of their industry in 2011 to the 72nd percentile

in 2012, and to the 79th percentile in 2013. This is a good example of the amount of improvement typically seen byNBRI Clients. As important as it is to conduct survey research properly and obtain hard, valid data, it is simply anexercise in futility without targeted action that dramatically improves the organization. You have two choices: workon low scoring items that may only affect themselves; or, work on root cause perceptions that, when improved, willeffect dramatic and immediate widespread improvement across the organization.

INFERENTIAL STATISTICS

Those who work with psychological data on a regular basis, such as College Professors and NBRI OrganizationalPsychologists, apply advanced analytics to the raw data in order to pull the root causes, or primary perceptions,

out of the thinking of the survey respondents. As important as the descriptive statistics described above are, theycannot tell us the key factors driving customer and employee behavior. So, we put those aside for now, and returnto the Clients raw scores.

Most Survey Research Consultants offer their Clients correlations, which they call key factors or key drivers.This is in large part a misnomer, as shown below. A very famous correlation is that of Ice Cream Sales andSwimming Accidents. The numbers of both increase and decrease together: as one goes up, so does the other, andas one comes down, so does the other. This is the essence of a correlation: items that move together.

CORRELATION

IndustryAverage

ABC2011

ABC2012

Stretch ABC2013

Best inClass

50

90

6372 75

79

0

25

50

75

100

B e n c h m a r k

i n g P e r c e n t i

l e


20/24



I Trust Management My Morale is Good

Ice Cream Sales Swimming Accidents

HOT SUMMER DAYS

COLD WINTER DAYS

WEATHER

If your Survey Research Consultant were to tell you that these are your Key Factors or Key Drivers, what would youwork on? Unfortunately, the vast majority of Consultants simply lack the in-depth training and knowledge requiredto properly analyze psychological data.

NBRI applies two additional analyses to Client raw data: regressions and path analyses. Following our exampleabove, when we analyze the data with a Regression Analysis, we nd the Key Factor or Key Driver that is, in fact,causing the Correlation is the Weather: Hot Summer Days cause both scores (Ice Cream Sales and SwimmingAccidents) to increase, and Cold Winter Days cause both scores to decrease.

REGRESSION

Armed with this information, we now have a shortcut to improve a vast number of variables, from ice creamsales to flip flop sales to sunglasses sales, etc. The same principle should be applied to psychological data from

customer and employee surveys. An example is shown below.A major metropolitan city in the northeast United States was having high turnover; employees were dissatis ed onseveral levels; NBRI was called in to pinpoint the source of dissatisfaction. We found the following correlation.

CORRELATION

Had NBRI stopped here, Management would have been faced with a decision to work on employee trust inmanagement and/or employee morale. But advanced analytics provided Management with keen insight into thethinking of the employee population by conducting a regression analysis and path analyses on the raw data.


21/24



I Trust Management My Morale is Good

JOB PROMOTIONS ARE FAIR =NEGATIVE ROOT CAUSE

It was found that the employees overriding perception concerned job promotions being unfair. This was their ColdWinter Days, driving down the scores of employee trust and morale.

REGRESSION

Indeed, the perception that job promotions were unfair was driving down the scores of well over 40% of all surveyquestions, having far-reaching effects into perceptions of Discrimination, Diversity, Policies, Culture, and more.

ROOT CAUSE MODEL

Item# Topic Item Text Norm24 Culture Job promotions are fair. 36

2 Morale My morale at work is good. 53

7 Management I can trust what management says. 43

14 Discrimination The Company is free of discrimination. 34

16 Diversity The Company treats employees equally regardless of gender. 46

25 Policies Policies are carried out in a consistent manner. 43

43 Culture If I do a good job, I have a better chance of getting ahead. 42

54 Culture I feel free to express my opinions without worrying aboutnegative results.

37

When Management addressed the issue of favoritism that had contaminated the process of job promotions formany years, the score of the Root Cause increased dramatically, but more importantly, the scores of all itemsdriven by this Root Cause increased dramatically as well, which in turn effected huge gains in Total OrganizationScores, Employee Satisfaction and Employee Engagement, greatly reducing turnover and improving productivity.Clearly, the research had provided huge ROI and paid for itself many times over.

Once you have built a valid survey, gathered valid, representative sample data, described the body of data withdescriptive statistics, and cut to the chase with inferential statistics to pinpoint the most important issues toaddress, it is time to turn FACT into ACT, with Action Planning!


22/24



6 TURNING DATA INTO ACTION

In the olden days, it was Senior Managements responsibility to digest the survey data, debate which of the low-scoring issues should be addressed, develop their own list of initiatives to address the issues, and then consider

the budget, manpower, and timing requirements of each before implementation. This process typically requiredmonths, all while the data was aging and bene ts from improvements were on hold. More importantly, it has sincebeen proven that simply working on low-scoring items is not the key to effecting change in drivers or root causes ofcustomer loyalty and employee engagement, so minimal improvement typically resulted. While the burden is still onManagement to provide nal approvals of recommended interventions to root causes, the Action Planning Processdescribed below minimizes the time required of Management, while vastly improving the ef cacy of the process, andturning data into action in about 3 weeks!

The basic concept of Action Planning is circular.In Step One the research is conducted. In Step

Two the Root Causes are determined. In StepThree, all employee recommendations to improvethe root causes are submitted to small ActionPlanning Teams who in Step Four, develop theshort list; the short list is submitted to SeniorManagement for approvals in Step Five, and inStep Six, implementation occurs. In this manner,all minds are focused on improving the missioncritical ndings of the research study. ForCustomer Surveys, the solutions are proposedby all employees, including those closest to thecustomer which is rarely Senior Management.For Employee Surveys, the solutions areproposed by all employees, as well. After all, it isthey who experienced and reported the issuesin the rst place, and who will be responsible forimplementing the solutions.

The action-planning teams should be comprised of middle-management employees. Because there may beup to ve (5) Root Causes, there should be ve (5) action-planning teams established. Each team should havean odd number of members, such as 3, 5, or 7 individuals. The sole purpose of each team is to sift through therecommendations, and develop a short list of solutions that will go to Senior Management for approval.

The Action Planning Teams select one member from among themselves to broadcast to all employees that theyare the single point of contact to receive suggestions to improve [their particular root cause survey question].Offering prizes and rewards is, of course, optional. The goal of the communication is to create excitement and buzzamong the employees to make sound, inventive recommendations for improving the all-important root causes.


23/24



The Action Planning Teams should also be creative about ways to build excitement and collect ideas. Sendingreminder emails and counting the days to the close, putting up posters in hallways and break rooms, puttingballoons on suggestion drop boxes, sponsoring departmental contests with movie or dinner ticket awards, andother fun, celebratory ideas can be highly motivational to involve the workforce.

At the end of two weeks, each Action Planning Team meets individually. These meetings typically require no morethan a morning. When the meeting is over, it is typically not rescheduled, and a hard stop should be agreed to inadvance. Each Single Point of Contact will bring their long list of all recommendations received to improve theRoot Cause assigned to them, with a copy for every team member. The Single POC will not remove any suggestionsmade to him or her without approval of their Action Planning Team.

The team goes through the list one item at a time. As each item is mentioned, the team votes to place each item inone of the following categories:

1. unanimously agreed to be discarded, or

2. unanimously agreed to go on the short list for Senior Management approval, or3. requires further discussion.

The team members then debate, defend, and discuss the remaining items requiring discussion in an effort to gainconsensus regarding their disposition. Depending on the number of items remaining on the list after the initialvote, and the amount of time remaining for the meeting, teams may set a time limit of 5 to 10 minutes per item fordiscussion. At the end of the discussions, the nal vote takes place, and majority rules on any additional items tobe placed on the short list. The nal short list is then ready to be presented to Senior Management.

Employees will provide innovative and valuable recommendations for Senior Managements consideration.

Recommendations from the Action Planning Teams should be given high priority, and approvals from SeniorManagement should be provided as swiftly as possible.

Implementation of solutions to root cause issues should occur quickly. How it is done varies by subject matter,organizational structure, culture, manpower availability, and a variety of other factors. Senior Management,working closely with the Action Planning Teams, can lend guidance to the most ef cient and effective means ofimplementing these important solutions.

Regularly surveying and continuous improvement will ensure that your organization reaches Best in Class performance!


24/24
http://www.youtube.com/embed/arHFglXtG70?autoplay=1

how to conduct a survey - by nbri inc - america - ebook

Documents