Developing innovative ways of analyzing data

Kenan DikilitaşEVO’2016

Classroom-based Research for Professional Development

Does your research aim to ......?• investigate or promote the

efficiency of your instruction? • Speaking• Grammar• Reading • Writing• Listening• Vocabulary

• investigate or promote non-linguistic factors? • Motivation to learn• Active participation in the lesson• Strategy use• Reducing anxiety• Developing confidence

What are you doing research for?:Are you investigating .....?students’ opinions, views, attitudes, or perceptions?

students’ reactions to the activities you do?

changes and developments in your teaching?

changes and developments in student learning?

changes and developments in students’ learning process?

Tools you use -1 (Observation )

•Participant observation (as a student and observer•Non-participant observation (as an observer)

Observation of actual teaching:

•Stimulated recall

Reflecting and analyzing recorded lessons

Tools you use -2 (Documentation)

Student workTasksPosters

Assignments

Learning artifacts

Lesson plans

Teaching materials

Teaching artifacts

Tools you use -3 (Reports and introspection)

Interviews

• Semi-structure• Unstructured• Structured

Narrative and biographic methods

• General stories• Specific aspects of

teaching• Specific learning or

teaching experiences

Questionnaires

•Open-ended•Likert scale

Journals (for longitudinal data)

• Particular topics• General reflection on teaching and learning

Research questions and data analysisSample RQ:How can I increase my students' participation in speaking lessons?

What kind of data do you need to answer this question?

This question can be answered once data from multiple perspectives has been gathered and analyzed, so we need to ask other questions to arrive at potential answers to this summative question/solution-based question.

Detailing the questions

1. What activities do I perform in speaking lessons?

2. What are students’ preffered activities in speaking lesson?

3. What are the students' behaviors in speaking lesson?

4. What activities do they like and dislike in speaking lessons?

Analyzing the dataWhat activities do I perform in speaking lessons? vs. What are students preffered activities in speaking lesson?

Analyze your documents you use in speaking lessons and students reports about what they prefer to explore similarities and differences.

What are the students' behaviors in speaking lesson? And why they behave so?

Analze the observational field notes that contain how they behave in speaking lessons and relate this to the next question below.

What do they like and dislike in speaking lessons? Analze the written or verbal data to understand what they like and dislike

Reflections on a research questionHow can I increase my students' participation in speaking lessons?

Now bring together all the findings and think about ways in which you can increase participation in the speaking lesson.

As you can see, this question requires analyzed data from multiple questions. 1. Consider the contrasting opinions, needs, and

expectations and make strategic decisions to come up with alternative ways of increasing participations

2. Maybe it is the way we teach, the activities we use that prevent students from participating?

3. Maybe their lack of confidence, knowledge and experience ?

4. Maybe , maybe...But you will find answers in a step-by-step questioning..

Data AnalysisTo be able to tabulate the data, numerics can be generated on the basis of the following calculations:

• Percentages• Frequency • Mean scores

Data Analysis - 1 (Likert scale questionnaire)

Data Analysis - 2 (Likert scale questionnaire)

Sample Quantified Data Figure - 1

• Esin Yüksel • Asked the participants to

reflect on the impact of the implicit-explicit vocabulary instruction.• Then she quantified the

frequency of the responses.

facilitated voc learnıng

facilitated recall improved voc learning raised awareness increased motivation 0

5

10

15

20

25

30

YESNONEUTRAL

Sample Quantified Data Figure - 2• İlknur Kurtulmuş and Canan Önal

compared the pre- and post- test results. They gave tests to the students before and after their extensive reading projects and displayed the results according to the 5 levels of word knowledge.

0 1 2 3 40%

10%

20%

30%

40%

50%

60%

70%

59%

3% 2%

8%

28%28%

2% 3%

16%

51%

Pre- Post-

Peer and teacher assessment of weekly presentation performances by Sedef Fenik

week 1 week 2 week 3 week 40

5

10

15

20

25

Presentation contentVisual aidsPerformanceGeneral

week 1 week 2 week 3 week 40

5

10

15

20

25

Presentation contentVisual aidsPerformanceGeneral

Table 4: Students’ areas of weakness after 1st and 4th self-reflection by Sedef Fenik

Students’ weakness Areas In 1st self-reflection In 4th self-reflection

Eye contact 66% 27%

Body Gestures 58% 38%

Excitement 50% 50%

Memorizing 25% 11%

Timing 25% 0%

Voice and pacing 25% 22%

Visual aids 16% 11%

Organization of information 16% 0%

Asking and answering Qs 16% 22%

Fluency 0,8% 33%

Analysing qualitative data - 1 codes and categories emerge from the data set through discovery proceduresInductive Analysis

•Identify codes and categories from the data set•Connect each open codes to form axial codes, which are then connected to form selective codes.•You start with no assumptions and try to form the theoretical basis for your research. The resulting codes help you form your own framework

codes and categories are preset on the basis of other studies through matching Deductive Analysis

•Identify relevant codes in relevant studies. For example, to analyze the data to find out reasons for lack of motivation, you can find several previously identified reasons such as parents pressure, no need for learning, negative attitudes towards the target language, failure of teachers, inefficient materials etc..•So you analyze the data against these preset codes. However, the preset codes may not be found in your data set. Deductive coding is then risky in that you end up with no matching.

Analysing qualitative data - 2• QUAL data include verbal and written sources collected from the

participants of the study. To analyze such data set,we need to generate codes and categories in the data.• There are simply three stages to follow:

Open Coding Axial Coding Selective coding

Open coding - 1

• Codes are categories (keywords or phrases) assigned to a text segment)• To do that:

• Read through your data several times

• Identify illustrative phrases or sentences

• Find tentative labels for phrases or sentences

Open coding

Open coding 2Open coding Axial coding Selective codingLearners’ active roleDiscovery and engagement in tasksLearner voice in lesson planningFreeing from structured materialsMaterials adapted for learnersPreparing learner-based materialsVarious interaction patternsGradual learning - process-oriented

Planning and work more Readiness for unexpected issuesManagement skills

Axial coding •Bring together all the codes from open coding

• Identify relationships among the open codes

• Make connections between these categories

Axial coding - 2Open coding Axial coding Selective codingLearners’ active role Physical, mental and intellectual

engagementDiscovery and engagement in tasksLearner voice in lesson planningFreeing from structured materials Generation, adaption, and

selection of materials Materials adapted for learnersPreparing learner-based materialsVarious interaction patterns Promoting interaction in the

classroomProcess -oriented instruction

Gradual learning - process-oriented

Planning and work more Careful planningManaging classroomStrategic skills

Readiness for unexpected issuesManagement skills

Selective coding - 1

• Think about the core category of each

• Figure out connections among the relationships from axial coding

Selective coding - 2Open coding Axial coding Selective codingLearners’ active role Physical, mental and intellectual

engagementLearner roles

Discovery and engagement in tasksLearner voice in lesson planningFreeing from structured materials Generation, adaption, and

selection of materials Materials

Materials adapted for learnersPreparing learner-based materialsVarious interaction patterns Promoting interaction in the

classroomProcess -oriented instruction

Instructional factors

Gradual learning - process-oriented

Planning and work more Careful planningManaging classroomStrategic skills

Teacher rolesReadiness for unexpected issuesManagement skills

Visualising the relations among selective codes

Learner centredness

Mostly Used Strategies Fostering Autonomy

Frequency(24) Responses

Guidance 3• Acting like a facilitator rather than a controller • Guiding them how to do certain tasks

Giving Responsibility 5

• Giving students more responsibility.• Letting the students choose their own materials, activities and topics.• Promoting creativity by letting them decide on what topics to choose for projects.• Giving them tasks to complete independently.• Giving students more responsibility.

Promoting Self-Study 4

• Encouraging them to go to libraries for outside class research.• Suggesting websites, blogs, podcasts.• Suggesting students films and TV series

Metacognitive Training 6

• Promote critical thinking and reflection on learning• Raising and awareness in learners’ thinking processes.• Raising awareness about learning styles and strategies• Teaching them how to learn.

Promoting Interdependence 4

• Promoting peer correction.• Promoting group works and projects in and outside the classroom.• Giving the students more interdependent duties and tasks to work collaboratively.

Promoting Discovery Learning 2

• Getting the students to answer their own questions through eliciting.• Getting them to use reference books or dictionaries so that they can answer their own questions

themselves.

Sample quantified qualitative analysis by Merve Güzel

Categirization of codes by Sedef Fenik

Self-awareness Social Skills

Recognized mistakesRecognized positive behaviors

Overcome our weaknesses

more self confidencebecome more social

Personal development Skill development

learnt to speak in front of spectator

overcome excitementimproved eye contact

grammar improvedfluency in speakingimproved presentation skills

Visualisation of categories bu Koray and Savaş

Inductive Approach Deductive ApproachStude

nts’ perception

sof the differ

ent approaches

engaginginteractive

informativecompetitive

activity-based learning-based

discovery-basednot grammar focused

kinaesthetic orientation more students talking

disengaging limited interaction passive studentslearning-orientedlittle practice memory based explicit teachinggrammar focused limited participationless student talking time

How to establish validity•Share the analyzed data with the participants to ensure we mean what they really meanMember checks

•Build trust with the participantsProlonged engagement

•Consult colleagues to re-check the codes and categoriesDebriefing•Reveal your biases and preferences in reports, field notes

Progressive subjective checks

•Use multiple methods of data analysis Triangulation

Peer debriefing • Consulting one or several impartial colleagues to give you feedback

on your codes, categories and relationships between these.• The purpose is to enhance credibility and ensure validity.

• Potential debriefing areas:• Overemphasized and underemphasized points• Unclear descriptions• Researcher biases or presumptions