analytics for profiling and promoting learners' epistemologies (apple): developing the survey...
TRANSCRIPT
Analytics for profiling and promoting learners’ epistemologies (APPLE)PHASE 1: DEVELOPING THE SURVEY TOOL
Bethany Alden Rivers, The University of NorthamptonJohn Richardson, The Open University
Epistemic beliefs are our ideas about ‘knowledge and knowing’.
(Hofer, 2002, p. 3)
‘…how students themselves think about knowledge, learning and teaching is a primary factor influencing their experience of higher education itself.’
(Richardson, 2013, p. 192).
Universities need to care more about developing learners’ epistemologies.
(Lucas & Tan, 2013)
The problem… Researchers need better ways of understanding epistemic beliefs (Schraw, et al., 2002).
‘Well-validated quantitative instruments that could be used to measure epistemological development in large samples of students are still lacking’ (Richardson, 2013, p. 201).
Knowing and Reasoning Inventory (KARI)
Measure of Epistemological Review (Baxter Magolda, 1992)
Argumentative Reasoning (Kuhn, 1992)
Reasoning Knowing+
Argumentative reasoning (Kuhn, 1991)
Used ill-structured questions to determine:
Absolutist believe that experts could know or find specific causes of problems.
Multiplist deny any certainty between experts and believe their own view is just as plausible as an expert’s.
Evaluativist knowledge is uncertain but experts do exist
Realist pre-absolutist, with an absent epistemology (Kuhn & Weinstock , 2002)
Measure of epistemological reflection (Baxter Magolda, 1996)
Role of the learner
Role of the peers
Role of the instructor
Evaluation
Nature of knowledge
Absolute knowing Transitional knowing Independent knowing Contextual knowing
Ways of Knowing
5 Domains
Theoretical frameworkReasoning
(Kuhn & Weinstock, 2002)
Knowing (Baxter Magolda, 1996)
Realist
Absolutist
Multiplist Transitional Knowing
Independent Knowing
Contextual Knowing
Absolute Knowing
Evaluativist
Creating the KARI• Converting qualitative interview protocol into
quantitative ‘statements’.• 5-point Likert scale• 3 demographic questions: age, gender, educational
background• 12 questions related to Reasoning• 20 questions related to Knowing
Examples of KARI statements for Reasoning• Experts know what causes prisoners to return to crime.
• I know what causes prisoners to return to crime.
• An expert would probably share my views on what causes prisoners to return to crime.
• It is possible to have more than one point of view as to what causes prisoners to return to crime.
• Compared to the average person, I know a lot about what causes prisoners to return to crime.
Examples of KARI statements for Knowing
• The goal of learning is to come up with my own perspective on things. (Role of the Learner)
• My classmates play a very important role in my own learning. (Role of Peers)
• The main goal of an instructor should be to help students think for themselves. (Role of the Instructor)
• I think my instructor should reward me when I think for myself. (Evaluation)
• Not everything my instructor says is true because we all have our own beliefs about things. (Nature of Knowledge)
Research questions 1. Are survey items 1-12 a reliable measure of one’s capacity for reasoning?
2. Are survey items 13-32 a reliable measure of one’s capacity for knowing?
3. Is there a correlation between reasoning and knowing?
4. Is there a correlation between demographics, reasoning and knowing?
1. Are survey items 1-12 a reliable measure of one’s capacity for reasoning?
• Cronbach’s alpha (all variables) = 0.83, which shows good inter-reliability
• Exploratory factor analysis showed two possible factors, and this was confirmed by comparison with the results of a parallel analysis of 1000 random variables.
• Factor 1 explained 41.14% of variance with salient loadings on 8 variables.
• Factor 2 explained 19.51% of variance with salient loadings on 4 variables.
• Cronbach’s alpha (Factor 1) = 0.90, which shows excellent inter-reliability.
• Cronbach’s alpha (Factor 2) = 0.75, which is also good.
2 possible constructs for Reasoning?• Factor 2 showed salient loadings on 4 variables.
• Mean ranking on Factor 2 variables was very low (M = 1.47).
• Reviewing the ‘statements’ related to these variables, it is clear that they are worded in a way where a low response (i.e. Definitely Agree) is optimal.
• E.g. ‘It is possible to have more than one point of view as to why prisoners return to crime.’
• For the next iteration, these statements may need to be reworded and another factor analysis be carried out.
2. Are survey items 13-32 a reliable measure of one’s capacity for knowing?
• Exploratory factor analysis showed four possible factors, and this was confirmed by comparison with the results of a parallel analysis of 1000 random variables.
• Factor 1 explained 20.93% of variance with salient loadings on 6 variables.
• Factor 2 explained 13.20% of variance with salient loadings on 4 variables.
• Factor 3 explained 9.88% of variance with salient loadings on 3 variables.
• Factor 4 explained 7.96% of variance with salient loadings on 4 variables.
4 possible constructs for Knowing?
Factor 1 (statements): • An instructor should work with each student to set goals for learning.
• Students and instructors should critique each other.
• The main goal of an instructor should be to help students think for themselves.
• It is important for instructors to help students apply knowledge to their own lives.
• I think my instructor should reward me when I think for myself.
• My instructor should have a way to measure how much I understand the newly acquired knowledge.
• DOMAINS 3 and 4 = THE ROLE OF INSTRUCTOR AND ASSESSMENT
4 possible constructs for Knowing?
Factor 2 (statements): • I share my views with my classmates because I often gain a different perspective
on things. Students and instructors should critique each other.
• I like to express my opinions to my classmates so we can talk about different views. It is important for instructors to help students apply knowledge to their own lives.
• My classmates play a very important role in my own learning.
• Even though I could learn just fine without my classmates, it is sometimes nice to have them around to talk to about what we're learning.
• DOMAIN 2: THE ROLE OF PEERS
4 possible constructs for Knowing?
Factor 3 (statements): • What my instructor teaches me is true.
• The role of an instructor is to transmit knowledge to their students.
• Not everything my instructor says is true because we all have our own beliefs about things
DOMAIN 5: NATURE OF KNOWLEDGE
4 possible constructs for Knowing?
Factor 4 (statements): • Learning is about obtaining knowledge from someone who knows more
than I do.
• It is necessary for me to interpret the information I receive from my instructor in order to understand it.
• The goal of learning is to come up with my own perspective on things.
• Learning is about using knowledge to think through problems.
DOMAIN 1: ROLE OF THE LEARNER
Four Constructs for Knowing 1. The Role of the Instructor and Assessment
2. The Role of Peers
3. Nature of Knowledge
4. Role of the Learner
3. Is there a correlation between reasoning and knowing?
• 2-tailed test was carried out to test correlations across all factors
• Factor 1 (from Reasoning) and Factor 1 (from Knowing) was the only relationship that was statistically significant. (p < 0.05)
• This indicates that a student’s capacity for reasoning is significantly correlated to his or her view of the role that the instructor and assessment plays in learning.
4. Is there a correlation between demographics, reasoning and knowing?
• Most of the participants reported having ‘some university experience’, A-Levels, or GCSEs.
• 77% of the participants were female.
• The mean age was 31 years.
• There was a statistically significant relationship between Education and Factor 1 (Role of Instructor and Assessment). Lower mean rankings reported by students with ‘no previous education’.
• There was a statistically significant pattern involving Education * Gender and Factor 3 (Nature of Knowledge). Lower mean rankings reported by male participants.
What’s next? Take stock of what we’ve learned so far.
Rework the KARI to address problematic components.
Second iteration of testing September to November 2014
Use of visualisations to research a case for enhancing practice…
What can an epistemological profile allow us to do?
A basis for developmentA bespoke personal development planPersonalised contentCurriculum mapping
The bigger problem?
How do we profile and promote learners’ epistemic beliefs in large online communities of practice, such as MOOCs?
References Baxter Magolda, M. (1992) Knowing and Reasoning in College: Gender-Related Patterns in Students’ Intellectual Development, San Francisco, Jossey-Bass.
Hofer, B. K. (2002) ‘Personal epistemology as a psychological and educational construct: an introduction’, in Hofer, B. K. and Pintrich, P. R. (eds.) Personal Epistemology: The Psychology of Beliefs about Knowledge and Knowing, Mahwah, NJ, Erlbaum.
Lucas, U. and Tan, P. L. (2013) ‘Developing a capacity to engage in critical reflection: students’ ‘ways of knowing’ within an undergraduate business and accounting programme’, Studies in Higher Education, vol. 38, no. 1, pp. 104-123.
Kuhn, D. (1991) The Skills of Argument, Cambridge, Cambridge University Press.
Richardson, J. T. E. (2013) Epistemological Development, Educational Research Review, vol. 9, pp. 191-206.
Schraw, G., Bendixen, L. D., & Dunkle, M. E. (2002). ‘Development and validation of the Epistemic Belief Inventory (EBI)’. In B. K. Hofer & P. R. Pintrich (Eds.),Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 261–275). Mahwah, NJ: Erlbaum.
