Welcome to the December TLC Chat!

A Conversation on Qualitative Research

Your Hosts: Kelly and Roberta

Agenda

Definitions of qualitative research TLC Fellowship Report-out Tips and Tools Example of current TRG work using

qualitative research methods Current research in the field: A comment on

employee surveys

Getting Started: Speed Connecting

Find someone you don’t know that well or haven’t had the chance to connect with recently.

With your partner, discuss the following questions:

What is a recent experience that you have had working on a TRG task/assignment/project?

What did you learn from that experience?

Getting Started

“Qualitative researchers aim to gather an in-depth understanding of human behavior and the reasons that govern such behavior. The qualitative method investigates the why and how of decision making, not just what, where, when.”

What is Qualitative Research?

Investigative method which involves exploring issues, phenomena and answering questions

In-depth and holistic Used in organizations to problem solve,

understand ways of doing work, team/interpersonal/organizational dynamics

People-centered Realistic Researcher-driven Subjective

How Does This Apply to TRG’s Work?

2011-2012 TLC Fellowship Topic Interviewed 5 TRGers- Trainers, PAs and

informal conversations with others Primarily looked at four methods:

Interviews, Focus Groups, Storytelling and Online Communication

Developed Tips and Tools 1.0

What We Learned TRG is using qualitative research in client

systems like EPA, USAID, PSI and IFC Even though we rely heavily on interviews, we

also use quantitative methods for surveys, course monitoring and evaluation which strengthen our work and credibility with clients

Our own emotions, perceptions and judgments inform how we analyze the data (selecting themes, areas for concern) and what we choose to highlight as important

We have lots of questions about ways to strengthen our research methodologies

Some Qualitative Feedback “As interviewers, we definitely interpret data in our

own specific ways. This comes out in terms of how much or little we paraphrase, how we understand the emotions and voice tone of interviewees, etc. This can pose a challenge when we are working on teams on the same project/assignment.”

“Qualitative research enhances the quantitative and vice-versa. As much as we rely on qualitative methods, our clients sometimes like the numbers as well. There are ways that I have taken surveys to see how many times words were repeated and have made charts and graphs to quantify that data for our clients.”

CGIAR Assessment of Gender Research Capabilities

Lead Researcher: Roberta, OD/TS: Kelly, PM: Linda, PA: Nina

Assessing CGIAR’s environment for inclusion of gender analysis in agricultural research

Qualitative Methods Employed: Focus Groups with Gender Experts, Interviews with Respected Researchers, Survey

Using both Qualitative and Quantitative Methodologies

A comment on employee surveys

At your table

1.read the results from an USOTAT employee survey

2.as a table, come up with a quick summary that you would present to the client

a study by Poncheri, Lindberg, Thompson, and SurfaceNorth Carolina State University2008

A comment on employee surveys: negativity bias in open-ended questions

Research on organizational climate surveys

Hypotheses1. Open-ended comments produced in response to an

employee survey will be disproportionately negative in tone.

2. The length of comments will increase as they become more negative in tone.

3. The satisfaction level (i.e., positive, negative, or neutral) individuals express in their open-ended comments will be positively related to the satisfaction level they express in response to closed-ended questions.

4. The closed-ended rating produced by respondents who make comments will be more negative than the rating produced by respondents who do not make comments.

In terms of comment tone, open-ended responses tended to echo the closed- ended satisfaction ratings of the commenter.

Hypothesis 3

Results revealed that relatively dissatisfied employees were more likely to provide comments than their more satisfied counterparts.

Hypothesis 4

Length of comments increased as the comments became more negative in tone.

Hypothesis 2

Open ended comments were disproportionately negative in tone..

Hypothesis 1

What does this mean to our work?

Authors suggest

Recipients of survey feedback reports should be warned about the negative tone of open-ended comments so that they do not place undue emphasis on the comments

It is important to note that automatically presenting managers with all of the opened-ended comments provided by survey respondents may mask important group differences and trends.

authors suggest

Other survey types (e.g., 360), may not be predominantly negative but instead may be primarily positive as a result of leniency error.

Need more research on wording placement mandatory/optional

a copy of the article is posted on the VLC.