The use of artefacts in interviews exploring mentor decisions of

student competence in practice: an evaluation

Sarah Burden RN MResSenior Lecturer in Nursing/ Teacher Fellow Faculty of Health & Social SciencesLeeds Metropolitan University

Introduction Aim To discuss the use of artefacts in semi-structured

interviews and evaluate their use in a study exploring mentor decisions of student competency in practice.

Background The author draws on experiences of using artefacts, in this instance student practice assessment documents (PADs), from their mixed methods PhD study.

Data Sources From forty four final placement decisions surveyed in student PADs, seventeen mentors were interviewed to explore their decision making processes underpinning what they had documented in the placement interviews.

Background to the study Despite increasing emphasis on skills attainment, there exists a

recurring concern that nurses are emerging from education programmes without essential clinical skills, and a prevailing view that pre-registration nurses are rarely failed in their practical assessments by mentors.

Literature review reveals that when mentor assessment decisions of their student have been examined, significant emphasis has been placed on the phenomenon of mentors failing to fail student nurses in the practice environment.

However, to date few empirical studies have examined mentor assessment decisions across the full range of ability and achievement.

Aim of study to investigate what factors underpin mentor judgements of student nurse competence in practice

and how do mentors reach a decision to pass or fail a student in practice?

Nursing decision-making research

To date most research informed by classical decision making – using descriptive, normative & prescriptive approaches.

Studies explore single event judgements & decisions using think-aloud protocols, observation and simulation.

May not capture dynamics of decision making in ‘real-world’ environments.

Naturalistic decision making

Capturing cognitive processes that occur over a period of time, in complex, uncertain environments.

Investigating how people use experience in decisions.

Studying the performance of decision makers under typical workplace conditions.

Study design

Need to address the limitations of retrospective interviews

Accuracy of the decision maker’s memory and perception of events.

Mentors engage with a student across a placement period and most likely make their final decision as a result of a relationship and multiple observations taken and developed over a period of time.

Approach required to facilitate retrospective introspection on the part of the participant and support recall of incidents over a placement period that may be superior to a simple semi-structured interview .

Stimulated Recall (SR) Methodology

A qualitative retrospective data collection method; an introspective procedure where participants are normally played video or audio recordings of their behaviour (an artefact) to stimulate recall of the event and their concurrent thinking occurring during the event.

The recorded artefact, containing as it does the participant’s ‘real world’ actions, has the potential to stimulate recall of a situation, interaction and decision which is greater than free recall and captures the complexity, uncertainty and dynamics of the situation .

Also considered to be the case when the artefact is presented in documentary form such as patient

charts, used in Chart-Stimulated Recall interviews.

Artefacts and their use in SR interviews

Objects created by an individual or detailing events where they made a contribution.

Belief that use within a SR interview brings participants closer to the moments of action, helping to overcome problems of memory and perspective in standard retrospective interviews.

Artefact examined by participant, and through use of structured recall and probing procedures by researcher, participants talk through their thinking, decisions and actions contained in the artefact.

Some similarity with ‘think aloud’ techniques may be evident in the verbalisations generated in the SR interviews and in the shared assumption that it is possible

to access internal thoughts and verbalise them to a degree.

Artefact use in the study

Use: Reviewed by researcher prior to

interview and key issues noted on interview guide.

No annotations on artefact Artefact reviewed by participant

at beginning of interview During the interview, as the

mentor worked their way through the student PAD, the topic guide was used to prompt discussion as required and ensure that all areas had been addressed

The artefact: student practice assessment document containing student written evidence records reviewed by the mentor as well as the mentor / student comments for the three placement interviews (preliminary, midpoint, final)

Topic Guide to support examination of student PAD in mentor SR interview

Artefact use: an illustration

‘LINDA’ – a final placement student where concerns were raised at the midpoint of the placement and a university lecturer attended to support development of an action plan.

Mentor comments on the aspects of practice the student needs to develop stated:

“Learning when certain aspects of a patients care need to be communicated to the nurse looking after the patient. Being aware of student limitations.”

At final interview mentor decides that the student has passed the placement and confirms that Linda ‘has achieved the required standards of proficiency for safe and effective practice for entry to the NMC register’ (p6, NMC 2008)

Mentor comments documented at final interview

“This placement has been extremely challenging for ‘Linda’ but I feel she has grown and developed well from these challenges. At intermediate interview concerns were raised regarding attitude to other members of staff, communication skills and the way she approached patients……….. Following this performance improved, attitude to staff was less abrasive and challenging and her communication within the team improved. When working with the patients she became more aware of her own self and how she came across. She needs to take the lessons learnt in this area and implement them in future places of work.”

Researcher – mentor discussion stimulated by artefact

Interviewer: So how did this student then reassure you so that you were able to pass them?

Mentor: I’m not sure they did…..

Interviewer: You’re not sure they did? Okay…… Mentor: I have some doubts, but I don’t know whether

those doubts are because…. She did irritate me. And I don’t know whether it’s because of, I didn’t know whether I was picking on her in the end and whether I was picking faults with her because like I said, everything that I asked her to do she did and the other members of staff on the ward said she had improved. Yes she had improved but I don’t know whether she had improved enough……. I do think she will get there but…. I don’t know whether we passed

her on the right conditions or not.

Researcher – mentor discussion stimulated by artefact (continued)

Interviewer: So you weren’t reassured about her?

Mentor: No. But I was reassured about, like I said she did do everything that I asked her to do and that’s the difficult one. If she hadn’t performed and did what we asked then I would not have had any problems failing her and I think it’s the fact that we asked, what we asked she did. And if she’s not done what you’ve asked then fail her clearly. But …… (looks back at the action plan documented at midpoint interview)………. Everything we have set out to do she’s met the target, even if she has only just met the target, and if she’s met the target then what can we fail her on?

Challenges of using SR interviews Researcher skills in probing and prompting to stimulate actual

recall of the event. Inappropriate probing can result in conscious censoring of the recall, resulting in a ‘sanitised’ account, a new view of an event, influenced by the use of hindsight and reflection. Mindful of this, when reviewing the interview transcripts, attention was paid to comments where mentors offered an opinion, stating that they hadn’t considered the issue before. These were then examined against the documented comments for confirmatory support or inconsistencies.

The extent to which retrospective data collection captured the concurrent reasoning of the mentors. An acknowledged threat to the validity of data gathered in SR is the time delay between the event and the process of recall, though this may be mediated to some extent in interviews using Chart Stimulated Recall.

Benefits of using SR interviews

Mentors were able to easily recall the specific student and practice assessment decision taken. Conversations were fluent and full of detail, with numerous illustrations from practice provided to support mentor comments and decisions evidenced in the student document. Other researchers have also noted these benefits in terms of increased recall and ability to articulate the rationale behind decisions and choices made.

Benefits of using SR interviews (continued)

Using an artefact in which the mentor had contributed, placed the mentor’s perspective and actions at the centre of the investigation and limited discussion focussed solely on any researcher agenda. This facilitated an acknowledgment of the experience of both parties in assessing students in practice which supported collaboration between mentor and researcher to develop a shared understanding of the assessment decision taken. Examining decisions from the insider’s perspective not only gives ownership to the participant to raise their own issues but can alleviate researcher bias and ‘hubris’ which may occur when investigating an issue they know well.

Evaluation of artefact use in mentor study - ‘Linda’

The example provided from the mentor study illustrates some of the benefits from adopting a SR approach. In discussing her doubts over the decision taken concerning Linda’s competency, the mentor articulates not only criteria, but also a process of evaluation of these criteria to inform the final decision. Reviewing comments recorded in Linda’s PAD, the approach appeared to support a more focussed discussion by the mentor of their decision processes than may have been achieved in a more usual post event interview.

Overall evaluation

Mentors were brought closer to their moments of decision and action associated with an individual student assessment.

Documented comments provided prompts which were effective in addressing difficulties of memory and recall usually associated with retrospective data collection and facilitated insight into the mentor’s thought processes.

Method provided the opportunity to discuss inconsistencies between mentor thoughts, actions and documented comments.

Reviewing the mentor comments created a point of contact between the mentor and researcher as well as conferring a sense of ownership of the mentor’s contribution to the

research.

Overall evaluation (continued)

Useful in circumstances where the researcher has previous experience, the approach allowed the mentor to discuss their thoughts and actions without the need to describe or reduce decisions for explanation.

Mentor’s perspective given prominence with the mentor and researcher working together to achieve a shared understanding of the decision making process.