1 the scholarship of teaching and learning: implications for its assessment michael prosser centre...
Post on 19-Dec-2015
218 views
TRANSCRIPT
1
The Scholarship of Teaching and Learning: implications for its
assessmentMichael Prosser
Centre for the Advancement of University Teaching
The University of Hong Kong
2
1. Introduction
2. The Scholarship of Teaching, and the UK Research Assessment Exercise
3. Example of Pedagogic Research
What is pedagogic research?
How does it relate to the scholarship of teaching?
How does pedagogic research relate to other forms of scholarly inquiry in teaching and learning in higher education?
How is pedagogic research assessed in the UK RAE
3
Growing Interest in various forms of Scholarly Inquiry in Teaching and Learning in Higher Education
• Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) Program in Higher Education - USA
• Carrick Institute for Learning and Teaching in Higher Education – Australia
• Higher Education Academy – UK – The Academy
• Research Assessment Exercise – RAE – UK
Conferences, Research and Development Journals, Graduate Certificates in Higher Education, PhD programs in T&L in Higher education
The UK Professional Standards Framework for teaching and supporting learning in higher education:
‘the ability to incorporate research, scholarship and/or professional practice into those activities’
4
The Higher Education Academy in the UK supports various forms of scholarly inquiry into teaching and learning in higher education, and has developed analytical distinctions to support its aims of supporting and encouraging the various forms.
A concern is that the RAE in the UK will distort the forms of inquiry, by institutions encouraging all forms of inquiry to be consistent with and to meet the definition of research for the RAE.
5
Boyer’s four scholarships
(Scholarship Reconsidered, 1990)
Discovery – more traditional UK RAE type research
Integration – making connections across the disciplines and placing things in larger context
Application (or now often termed engagement) – goes beyond application and develops an interactions that each informs the other (research and application)
Teaching – research informed, critical, evidence based, collegial reflection on practice to improve practice
Separate, but overlapping
6
Forms of educational research and Inquiry
Outcomes include
Scholarship of Teaching and Learning
Critical, evidence based and research informed reflection on practice to improve practice
Investigative studies Recommendations for action on HE issue/problem, locally and nationally
Optionally, enhanced understanding of theoretical or conceptual underpinnings
Evaluative studies Judgement about value or worth, or actions for local enhancement
Recommendations for wider practice Optionally, enhanced understanding of theoretical or
conceptual underpinnings
Theory-driven studies Enhanced understanding of theoretical or conceptual issues for longer term thinking about policy and practice
Optionally, recommendations for action on HE issues
Analytical distinction between forms of educational research and inquiry
7
Theoretical study
Investigative or evaluative
study
Scholarship of teaching
8
Investigative practice-based and policy-based projects
Theory-driven projects
Primary Purpose developing recommendations for action on issues, policies or practices within HE.
enhancing the conceptual or theoretical understanding which underpins the topic under investigation
May also increase the conceptual or theoretical understanding which underpins the recommendations.
lead to recommendations for action on issues, policies or practices within HE.
Starting point may be
an issue, problem or set of concerns relating to HE policy or practice, perhaps articulated within a local context
conceptual or theoretical, perhaps informed by a practical concern within HE, or the project may adopt an approach leading to theory generation.
9
Definition of research for the UK RAE
‘Research’ for the purpose of the RAE is to be understood as original investigation undertaken in order to gain knowledge and understanding.
It includes work of direct relevance to the needs of commerce, industry, and to the public and voluntary sectors; scholarship*; the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction.
It excludes routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques.
It also excludes the development of teaching materials that do not embody original research.
10
Definition of pedagogic research for the RAE (generic statement)
‘60. Pedagogic research in HE will be assessed where it meets the definition of research for the RAE. It is research which enhances theoretical and/or conceptual understanding of:
• teaching and learning processes in HE• teacher and learner experiences in HE• the environment or contexts in which teaching and learning in HE take place• teaching and learning outcomes in HE• the relationships between these processes, outcomes and contexts.
Reports of studies providing descriptive and anecdotal accounts of teaching developments and evaluations do not constitute pedagogic research.Pedagogic research is firmly situated in its relevant literature, and high quality pedagogic research makes a substantial contribution to that literature.’
11
UOA 35, Accounting and Finance
16. Teaching materials and non-text output may be included in submissions where they contain original research. Such materials will be assessed in the same way as other work. Where the research aspects of submitted work may not be clear to the sub-panel, departments should describe this briefly (in up to 50 words) in the ‘Other relevant details’ field in RA2.
17. Where this is relevant to the UOA the sub panel welcomes the submission of research into the teaching and learning of the field or sub-field, at HE or other levels. This will be assessed in the same way as other work.
12
This does not imply that that theoretical and conceptual understanding cannot be developed through investigative studies or the scholarship of teaching – but that the outcomes of the study enhances theoretical or conceptual understanding – it is then the outcomes that bare judged.
13
Training of UK RAE Reviewers of Pedagogic Research in the Disciplines
The RAE and the Higher Education Academy organised a training day for members of the discipline panels responsible for pedagogic research in disciplines
The Academy identified what it considered to be high quality published papers, and asked the attendees to rate and discuss the papers in terms of the definition of high quality pedagogic research for the RAE (4*).
4 the Academy judged to have met the criterion for high quality pedagogic research
1 was more a description of the results of a survey
1 was more a description of practice
1 was an unconceptualised case study
1 was the report of an investigation
14
The attendees independently judged the 4 judged by the Academy to be of high quality, to be of high quality
The other 4 were judged to be research, but not of high quality.
The issue here is that the research needs to be well contextualised in its appropriate literature and / or field and be seen to be making a significant contribution to that literature and / or field.
15
Overview of the student learning perspective
Figure 1: Model of Student Learning
CHARACTERISTICS OF THE STUDENT (e.g. previous experiences, current understanding) STUDENTS'
PERCEPTIONS OF CONTEXT (e.g. good teaching, clear goals)
COURSE AND DEPARTMENTAL LEARNING CONTEXT (e.g. course design, teaching methods, assessment)
STUDENTS' APPROACHES TO LEARNING (how they learn e.g. surface/deep)
STUDENTS' LEARNING OUTCOMES (what they learn quantity/quality)
16
STUDENT APPROACHES TO LEARNING
Surface Approach
Intention to reproduce
- rote memorise information needed for assessment
- failure to distinguish principles from examples
- treat tasks as external impositions
- focus on discrete elements without integration
Deep Approach
Intention to understand
- meaningfully memorise information for later use
- relate new ideas to previous knowledge
- relate concepts to everyday experiences
- relate evidence to conclusions
17
Approach to learning represents the relationship between the student and the task or subject
Approaches vary between subjects
Approaches vary between tasks within subjects
Approaches not stable but variable
Surface and deep approaches describe a fundamental variation in the way students approach learning – they do not describe the rich detail in the way students approach their learning
18
STUDENTS’ PERCEPTIONS OF THE LEARNING CONTEXT
Research by Entwistle and Ramsden (1983) using interviews and questionnaires identified a number of student perceptions relating to the way they approached their studies
Student perceptions of:
Quality of teaching – including quality of feedback
Clearness of goals of course and standards of assessment
Workload so high that it was not possible to understand everything
Assessment measuring reproduction and not understanding
were found to relate to how they approach their studies and to learning outcomes (exam results and other indicators)
19
RELATIONSHIP BETWEEN PERCEPTIONS AND APPROACHES
Study of 8837 students in 51 first year subjects around Australia
(3 first year accounting courses)
Amongst the data collected were students responses to:
contextualised Ramsden's Course Experience Questionnaire and a
contextualised Biggs Study Process Questionnaire
1994-1996: Australian Research Council ; Academic Departments and the Quality of Teaching and Learning; Paul Ramsden, Griffith University, Elaine Martin, RMIT, Michael Prosser, La Trobe University, Keith Trigwell, UTS
20
Approaches to Study
Surface Approach
32. Although I generally remember facts and details, I find it difficult to fit them together into an overall picture
35. The best way for me to understand what technical terms mean is to remember the textbook definitions
Deep Approach
28. I try to relate ideas in this subject to those in other subjects, wherever possible
34. In trying to understand new ideas, I often try to relate them to real life situations to which they might apply.
Biggs Study Process Questionnaire
21
Student Perceptions Of Learning Context
Good Teaching
15. The staff made a real effort to understand difficulties students might be having with their work.
Clear Goals and Standards
1. It was always easy to know the standard of work expected
6. I usually had a clear idea of where I was going and what was expected of me in this subject.
Appropriate Workload
25. The sheer volume of work in this subject meant that it couldn't all be thoroughly comprehended (-).
Appropriate Assessment
8. To do well in this subject, all you really need is a good memory (-).
Student Independence
20. Students had a great deal of choice over how they learned inthis subject.Ramsden’s Course Experience Questionnaire
22
Factor Analysis of Perceptions of T & L Context and Approach to Study – accounting students
_________________________________________________________________
Scale Factors
____________________________
1 2
_________________________________________________________________
Perceptions of Context
Good teaching .83
Clear Goals and Standards .58 (.38)
Appropriate Workload .73
Appropriate Assessment (.36)
Student Independence .80
Approach to Study
Surface Approach -.83
Deep Approach .63 -.37
_________________________________________________________________Principal Components, Varimax Rotation, n=463
23
1. A deep approach is associated with perceptions that the teaching is good and the goals and standards are clear
2. A surface approach is associated with perceptions that the workload is too high and assessment tests reproduction
That is, variation in individual students’ perceptions of the learning context across all subjects is associated with the approaches to study (and learning outcome)
1994-1996: Australian Research Council ; Academic Departments and the Quality of Teaching and Learning; Paul Ramsden, Griffith University, Elaine Martin, RMIT, Michael Prosser, La Trobe University, Keith Trigwell, UTS
24
Overview of the student learning perspective
Figure 1: Model of Student Learning
CHARACTERISTICS OF THE STUDENT (e.g. previous experiences, current understanding) STUDENTS'
PERCEPTIONS OF CONTEXT (e.g. good teaching, clear goals)
COURSE AND DEPARTMENTAL LEARNING CONTEXT (e.g. course design, teaching methods, assessment)
STUDENTS' APPROACHES TO LEARNING (how they learn e.g. surface/deep)
STUDENTS' LEARNING OUTCOMES (what they learn quantity/quality)
25
Having looked at the way students approach their learning, how do university academics approach their teaching?
How does the approach to teaching relate to approach to learning and how does?
Prosser, Martin, Trigwell, Ramsden, Lueckenhausen (’05)
In a series of phenomenographic interview studies aimed at identifying key aspects of the differences in the way university teachers approach their teaching, the following key differences were identified:
26
A: Teacher focused, teacher activity with an intention of transferring information
B. Teacher focused, student activity with the intention of transferring information to students
C. Teacher focused, student activity with the intention of student acquiring the concepts of the discipline
D. Student focused, student activity with the intention of students developing their own conceptions
E. Student focused, student activity with the intention of students changing their conceptions
27
Intention Strategy
___________________________________________
Teacher focused
____________________________
Student focused
Teacher activity Student activity Student activity
Information Transfer
A B
Concept acquisition
C
Conceptual development
D
Conceptual change
E
28
ATI scales measuring:
Information Transmission / Teacher Focused (8 items)
It is important to present a lot of facts to students so that they know what they have to learn for the subject
Conceptual Change / Student Focused (8 items)
A lot of time in this subject should be used to question students’ ideas
29
Trigwell, Ramsden, Prosser and Martin (‘98) – individual class unit of analysis (n=55 first year courses)
Correlations between Approach to teaching and Approach to learning
Approach to Learning
Biggs (ASI)
(Mean)
Approach to Teaching (ATI
_________________________________
Information transfer Teacher focused
Conceptual change Student focused
Surface .37** -.46 **
Deep -.14 .34 *
30
Information Transfer / teacher focused approach to teaching in first year associated with student adopting more reproductive approaches to learning
Conceptual change / student focused approaches to teaching associated with students focusing more on meaning and understanding
31
Eleven of those courses were first year business studies courses.
The following scatter plots show the relationship between the two approaches for those courses
32
33
34
There is a clear relationship between the way student approach their learning in first year business courses and the way academics approach their teaching.
We know that there is a relationship between the way students approach their learning with the quality and quantity of what they learning
Is there a relationship between the way academics see their teaching and how they understand their subject matter
Substantial amount of ongoing work, including that of Lynne Leveson’s, who has addressed this issue in accounting:
More than the classroom: A study of approaches to teaching in accounting and its relationship to lecturers’ disciplinary conceptions
35
In her study Lynne interviewed 30 accounting academics about the way they see accounting as a discipline and the way they approach their teaching in first year curses.
The study was again a phenomenographic study in which the aim was to identify key differences in the way these accounting academics saw they discipline and approached their teaching
36
A Technical process for recording, measuring and presenting financial information.
B Technical processes providing financial information for decision-making.
C Processes to facilitate enhanced decision-making.
D System support for the management and control of an organisation’s resources
E Accounting as a social phenomenon shaping and maintaining social structures.
F A culturally-determined system of principles and rules which operationalise the moral principles of the right-to know in society.
Accounting Academics Conceptions of Accounting
37
The variation in the way they saw their discipline ranged from a focus in the discipline as substantially technical in orientation to seeing the discipline going beyond the technical and seeing it as being socially and culturally constituted and open to critical inquiry and analysis.
But the interesting outcome of this study was in the relationship between these ways of seeing and experiencing the discipline and that way they approached their teaching.
38
Approaches to Teaching
Conceptions of Accounting
A B C D E F
A 6
B 3 4 2 1 1
C 7 1
D 1 1
E 3
39
A relatively strong association between the way accounting academics see their subject matter and the way they approach their teaching.
Seeing accounting in terms of technical processes only is associated with more teacher focused information transfer approaches to teaching
Seeing the accounting in its broader social and cultural context is associated with more student focused conceptual change approaches to teaching
40
Is it too long a bow to draw to say that if we wish:
• to engage our student in more meaningful approaches to study,
• to enhance their perceptions of the quality of teaching and learning in accounting and
• to improve their learning outcomes,
accounting academics need
to adopt more conceptual change and student focused approaches to teaching, which may entail them seeing the discipline they teach not purely or substantially in technical terms but in its broader social and cultural context.
41
Conclusions
The UK RAE judges pedagogic research in terms of its contribution to enhancing theoretical and conceptual understandings through pure, applied, policy and practice-based research.
Accounts of such research need to go beyond being descriptive and anecdotal, and to show how it is making an important contribution to the literature of the field. A high rating would mean that it is judged to be comparable to the best work in the field worldwide.
Research in teaching and learning in higher education is highlighting the importance of the way academics see and understand their subject matter on the way they teach and, in turn, on the way their students learn.