NOW THAT I’VE COLLECTED SOME DATA,HOW DO I WRITE UP MY FINDINGS?

Prepared byDr. Diane Tickton Schuster

BMR Congregational Inquiry Coordinator and

Rabbi Stacy Eskovitz Rigler Reform Congregation Keneseth Israel

Elkins Park, PA February 28, 2013

The Webinar Broadcast Has The Webinar Broadcast Has BegunBegun

• All attendees are in “Listen Only Mode” and will remain this way for the duration of the webinar. Before the presentation begins, we would like to review a few of the technical aspects of your webinar experience.

• You should all see a panel on your screen similar to our examples on the following slides. This control panel presents each attendee with various options related to our webinar presentation.

2

Helpful TipsHelpful Tips

3

Open and close your Panel

Helpful Tips

4

Asking Questions• Raise hand

• Questions Log

• Note any technical issues in the Questions Log as well

Helpful Tips

5

Audio Mode• View, Select, and

Test your audio

Today’s Agenda…A. How Your Research Fits into the BMR

Theory of ChangeB. Why Research Reports are BeneficialC. Parts of a Research ReportD. Collecting Quantitative DataE. Collecting Qualitative DataF. Analyzing DataG. Reporting What You’ve Learned

BMR Theory of Change

BMR Theory of Change Key Concepts

Impetus for Change: This is what led your congregation to participate in the B’nai Mitzvah Revolution.

Visioning: Exploring (and possibly rethinking) your beliefs about the role bar/bat mitzvah play in the life of the child, the family, and the congregation. Planning for Experimentation: The process you will go through to plan the innovations that will enable you to fulfill your vision. Experimentation: Implementation of these innovative programs and approaches. Documentation: Collecting systematic data that enable you to refine your experiments and evaluate their success. Reflection: Analyzing the data you have collected, and considering what you have learned from each round of experimentation. Community Engagement: Involving successive layers of the congregation in the BMR, so that a large part of the congregation comes to share your vision and appreciate the changes you’ve instituted.

The BMR Theory of Change assumes that there will be an ongoing interaction among the different aspects of the cycle.

For example:

• Visioning and Planning for Experimentation anticipate the research questions that drive the experiment(s)

• Documentation activities trace back to the Impetus for Change and Visioning

• Reflection builds on what’s been learned from Experimentation and leads to additional experiments

• All of these processes lead to some form of Community Engagement

Reminder: As you proceed with developing and documenting your experiment, be sure to regularly let your various stakeholders know about ongoing BMR activity and get their feedback.

Why Research Reports areBeneficial

Research reports help us to…

Continually loop back to our vision and the research questions that derived from that vision

Systematically “fill in the blanks” about what we are doing and the impact it’s having

Organize findings in ways that will be helpful for explaining—even justifying—”change” to others

Give us solid information on which to build future plans and increase community engagement

The Parts of a BMR Documenter’sResearch Report

1. Introduction2. Your congregation’s BMR

overall Research Question3. Literature Review [optional]4. Short-term Action Research

Question & Rationale

Parts of a BMR Documenter’sResearch Report (continued)

5. The Sample [who was studied]6. The Method [how the research

was conducted]7. The Findings8. Discussion/Implications/

Future Research/Community Engagement

Introduction

What is Keneseth Israel’s BMR project?

How did it come about?  

At KI, b’nai mitzvah students have long undertaken independent mitzvah projects.

The impact of these experiences has not been studied. From the perspective of the KI BMR team, typically these projects do not help the children or their families to develop a deep connection with the synagogue community as a whole.

Moreover, it is not clear how the mitzvah project is part of the child’s understanding of what it means to become a Jewish adult.

To explore these issues…

KI has decided to introduce an alternative mitzvah project experience—a BMR experiment—in which a new model of participation in social justice activity will be offered.

Called The Journey to Jewish Adulthood, this alternative mitzvah project participation will:

give both parents and children an opportunity to become more connected to the

congregation

enable students to gain an understanding of the connection between their upcoming b’naI mitzvah and the core mitzvot of Jewish adulthood

What Will Happen in 2013-2014Select pairs of 6th grade religious school children and parents will participate in a monthly service learning activity in one of KI’s existing core congregational activities:

Broad Street Ministry

KI Mitzvah Garden Rydal Park Shabbat Service KI Caring Committee One-on-One Torah Study

KI’s Research Questions

1. What impact does participation in KI’s Journey to Jewish Adulthood track have on 6th grade families’ relationship with the congregation?

2. What impact does participation in the Journey To Jewish Adulthood track have on 6th graders’ parents perception of the connection between their children’s mitzvah work and the bar/bat mitzvah?

Literature Review (optional!)

What research or writing by others might have bearing on the Journey to Jewish Adulthood project?

Note: The “literature” may be from both formal (e.g,, research journals) and informal (e.g., magazines and blogs) resources.

Potentially Relevant LiteratureArticles about the history of the mitzvah

project and its use in congregationsResearch about parents of teens and their

involvement in communityResearch about the impact of social justice

work on adolescent identityResearch about early adolescence as a time

of learning about the relationship between self and community

Short-Term Action Research Question + Rationale

As part of KI’s overall BMR research process, what did KI’s BMR team decide to study first?

In what ways did they think the findings from this inquiry might inform future behavior?

Short-Term Research Question

What can current 8th grade parents tellus about the experience of the mitzvah project for themselves and their children?

Rationale for this Inquiry

Gaining 8th grade parent feedback will providea basis for explaining to 5th grade families whywe are restructuring the KI mitzvah project processto become more closely aligned with:

a) b’nai mitzvah as a whole

b) the practice of core adult mitzvot in particular

The Short-Term Research Sample

Who was studied?

How big was the whole group?

How many/what percent of the wholegroup did this research focus on?

What characterized the sample that wasstudied? (e.g., gender, age, grade level)

The Short-Term Research Sample 8th grade parents whose children became b’nai mitzvah

from May 2011 - December 2012

53 parents (from 31 families) received email invitations to give feedback

More mothers than fathers were contacted due to the availability of their email addresses

21 parents (40%) responded to the survey

15 parents (12 families) attended 2 focus group meetings chaired by a parent volunteer

Quantitative MethodsWhat quantitative methods were used to

collect data?

How were the quantitative data documented?

How were the quantitative data organized?

How were the quantitative findings analyzed?

E-mail SurveyQuantitative Items

Checklists

Type of mitzvah project # of hours devoted to mitzvah project Duration of the project

Ratings

5-point scale about impact of the mitzvah project experience on the child

5-point scale about impact of the mitzvah project experience on the parent

Things happen….Things happen….

A similar item asking parents to rate the impact of the b’nai mitzvah project on the parents themselves inadvertently “disappeared” from the email survey.*

*Welcome to the world of research

Analyzing Quantitative Data

What was learned from the quantitative data?

Type of mitzvah project (multiple answers) Hands-On Service 16 Fundraising 8 Education/Raising Awareness 10 Collections 3 Other 2

# of hours devoted to mitzvah project 11-15 hours 6 16-20 hours 4 20-25 3 More than 25 hours 5 Other 2

Duration of the Project 3-5 individual days 4 Period of 2-3 Months 3 Period of 3-6 Months 4 Beyond 6 Months 4 Other 5

Descriptive StatisticsDescriptive StatisticsA set of numbers (sometimes referred to as coefficients) that provide a brief description of a given data set. These statistics may describe an entire population and/or a sample. The measures used by researchers to describe data sets are • Measures of central tendency (e.g., mean, median, mode) • Measures of variability or dispersion (e.g., standard deviation, skewness).

Mean

1. This project gave my child an opportunity to spend time doing something meaningful with a parent. 4.00

2. This project helped my child understand the Jewish value of Tikkun Olam (repairing the world). 3.75 3. This project helped my child learn what it means to be a Jewish adult in preparation for B'nai Mitzvah. 3.60

4. This project gave my child an opportunity to engage in conversations about morals and values. 3.40

5. This project strengthened my child's connection to the synagogue. 2.60

Qualitative Methods

What qualitative methods were used to collect data?

How were the qualitative data documented?

How were the qualitative data organized?

How were the qualitative findings analyzed?

E-Mail SurveyQualitative items

1. In your opinion, what seemed to stand out to your child about the experience with the project?

2. What stood out to you, as a parent, about your experience with the project?

3. What other impacts, if any, do you think the mitzvah project had on you or your child? Were there negative impacts, as well as positive ones?

Focus Groups Qualitative Items1. What stands out about the B’nai Mitzvah project as

part of the overall experience at KI?

2. What impact do you think the Mitzvah project had on your child’s journey towards becoming a Jewish adult?

3. What discussions did you and your child have around the Mitzvah project?

4. Should the Mitzvah project move towards more familyinvolvement?

5. What suggestions do you have about the role the synagogue should/could play?

Analyzing Qualitative Data

How do you “code” answers to open-ended questions?

Simple Steps for Coding Responses

1. Assemble all responses

1. Identify themes, repeated concepts and words/phrases

2. Generate categories

3. Code responses according to categories

4. Hold onto “uncodeable” responses for future analysis

What stood out about your child’s mitzvah project What stood out about your child’s mitzvah project experience?experience?

He could do something that he had a like/passion for, benefits others who are less needy/less fortunate

It was nice to see how much enjoyment my daughter got from working with the animals who didn't have homes and have had such a rough life

This hands-on experience gave him a sense of accomplishment, a sense of giving back and a great deal of perspective.

Project gave my child a sense of satisfaction from doing a meaningful activity

Project opened my child’s eyes(new under-standing)

Project enabled my child to “give back”

Project gave my child opportunity to demonstrate leadership, be responsible, feel successful

He could do something that he had a like/passion for, benefits others who are less needy/less fortunate

It was nice to see how much enjoyment my daughter got from working with the animals who didn't have homes and have had such a rough life

This hands-on experience gave him a sense of accomplishment, a sense of giving back and a great deal of perspective.

Project gave my child a sense of satisfaction from doing a meaningful activity

Project opened my child’s eyes(new under-standing)

Project enabled my child to “give back”

Project gave my child opportunity to demonstrate leadership, be responsible, feel successful

He could do something that he had a like/passion for, benefits others who are less needy/less fortunate

√ √

It was nice to see how much enjoyment my daughter got from working with the animals who didn't have homes and have had such a rough life

√ √

This hands-on experience gave him a sense of accomplishment, a sense of giving back and a great deal of perspective.

√ √

What about the Outliers —Responses that Can’t Readily Be Coded

Example: “Our son is a high functioning autistic

spectrum child (Asperger’s). He had fun tossing the boxes to the people in line and eating the snacks. He did connect with the bigger picture but in such a more subtle way than he can express.”

Findings

What have we learned from the data?

What has this research revealed so far?

Findings About KI’s 8th Grade Parents’ Perceptions of

Their Children’s Mitzvah Projects

1. Kids liked doing the projects. Parents liked watching the kids

take satisfaction from doing this type of meaningful activity.

2. The projects enabled the students to give back, to see how they could make a difference, to help others less fortunate They had personal connections with the projects which made the projects richer.

3. The projects opened the kids’ eyes; they came in contact with people they would not have otherwise, situations they did not know about.

4. The projects gave students an opportunity to demonstrate leadership. It was a real experience, they had to be responsible, realized the need to make a difference.

More Findings about KI’s Parents’ Perceptions….

5. In the initial mitzvah project meeting the assistant rabbi talked to the students about what they are good at and passionate about. Because of this, projects tended to focus on a passion of the student’s rather than the true need of the community.

6. Some parents reported that their families struggled with what project to do and felt they did not receive sufficient guidance after the initial meeting for making a decision.

7. Some parents expressed a need for “closure” or “debriefing” about the experience and how it connected to B’nai Mitzvah.

7. Some parents valued that the child chose a project that related to something that was important to their family (e.g., raising money for ALS because family member suffered from this)

9. Some families were able to identify a project and see it to completion while others struggled with the process.

Discussion & ImplicationsDiscussion & Implications

What do we make of the findings?

What are some of implications from what’s been learned so far?

What are some questions for future research?

How can these findings be shared with the community?

Some Implications from the Findings

The mitzvah projects had little to do with the synagogue and did not enhance the students connection to the synagogue or Judaism. Accordingly, the new structure will need to involve a change in parents’ expectations and how they need to be educated about the BMR agenda.

The kids did not understand that their work was Tikkun Olam; accordingly the new project will need to have a more significant teaching, planning and reflection process.

The process of asking students what they are passionate about creates a focus on the child rather than a focus on the world and what is needed. Accordingly, we will offer projects that the congregation has already identified as ideas for the students.

We will use these findings to explain to the 5th grade families why we are giving them an alternative b’nai mitzvah process called Journey to Jewish Adulthood.

Future ResearchFuture Research

• What questions now arise that might lead to future research?

• In what ways do these questions tie back to your institutional vision and to the original research

question?

Sample Questions for Future Research

Will a change in the process lead to a better connection between students mitzvah work and their B’nai Mitzvah?

Will a change in the process alter what parents report about the the significance of the work or the level of responsibility felt by the students and their families?

Will doing the projects with other KI families and KI congregants who lead the projects create a better connection to the synagogue in specific and to Judaism overall?

Community EngagementHow can the research findings be used to

help people in your congregation begin to share your vision and to appreciate the changes already instituted?

How might you broadcast what you have learned so far?

How will you get feedback from the congregation as you move along?

Next Steps…Your BMR Action Research Facilitator is here to help you to—

Work out details of how to collect data Develop systems for documenting data Analyze the data collected Summarize findings Develop discussion points Think about ways to share findings with your community

We look forward to hearing from you!

Diane, Josh, and Nachama