Investigating the IMPACT OF LMIResearch TeamRéginald Savard, Sylvain Paquette, Céline Renald and Lynne Bezanson in collaboration with Bryan HiebertCanadian Career Development Foundation (CCDF) Canadian Research Working Group in Evidence-Based Practice (CRWG)

In partnership with New Brunswick Post-Secondary, Education and Labour

2

Outline

•Review of foundations for the study•Principal findings•Summary of results

3

Review of the Problem•Most research on LMI focuses on usability of

products:▫Readability, accuracy of information▫Ease of access, amount of use▫Most research is with students: very little with adults▫Little literature about the direct contribution of LMI▫ Savard, R., Michaud, G., Bilodeau, C. et Arseneau, S. (2007). L’effet de l’information sur le marché du

travail sur le processus décisionnel relatif au choix de carrière. Revue Canadienne de Counseling, 41(3), 158-172.

•Several questions remain unanswered▫How do people use LMI?▫What (if any) assistance would be helpful?

4

Research Questions▫If client needs are assesed and clients are given

LMI consistent with their needs:

▫to what extent does assistance by a service provider enhance their effective use of LMI?

▫and

▫to what extent is independent self-help a sufficient process for clients to use LMI effectively?

5

Review of foundations for the study• The study will take place within current service

delivery practices in the employment centres ▫so that processes can be incorporated into daily

practice if research results prove positive.•Career practitioners and clients will work in their

customary settings• LMI interventions will be isolated▫LMI is a distinct intervention, separate from

employment counselling or worksearch workshops in which LMI may be a component

▫There will be separate LMI interventions for career decision making and for job search

6

Procedures followed for the francophone sample• Translation of LMI Booklets (client and resource

centre)▫Career Decision Making (2)▫Work Search (2)

• Field testing▫Duration: 4 weeks

•Semi-structured interviews with clients ▫1 week and 4 months after entering the research

•Semi-structured interviews with research partners (career practitioners, managers) ▫several weeks after the end of the research

7

Initial Contact: Employability Assessesment Interview

• Career Decision Making

• Work Search

Être apte au travail

Choix de carrière

Dév

eloppem

ent

de

com

pét

ence

s

Recherche d’emploi

Maintien en em

ploi

Croi

ssan

ce

prof

essi

onne

lle

Dimensions de l’employabilité

8

Description of LMI Intervention4 weeks of intervention •Using the LMI booklets according to their

identified need in ▫Career Decision Making or▫Work Search

•Method of using LMI booklets:▫Independent▫Assisted▫Information and advice follow-up interviews (2)

week one and and week 3

Career Decision MakingKnow yourselfKnow the labour market

Arrive at a goal and develop an action plan

Work SearchCheck for « fit »

Get ready

Search for work

Secure employment

9

Client Sample• 52 clients between18 and 56 years of age (average = 37 years)• 25 men and 27 women• 83 % had at least a secondary school diploma• The majority worked in industry (eg.: seasonal workers, labourers)• 81 % of clients were unemployed : 19 % were in full or part time

employment

Two methods

Independent Assisted Total

Two kinds of interventions

Career Decision Making

15 17 32

WorkSearch 11 9 20

Total 26 26 52

10

Career Practitioner Sample

• 15 career practitioners• 3 men and12 women•Number of clients per career practitioner• Average = 3,47 (median = 3,87), range: between1 –

13.• Between1 and 3 clients = 12• Between 4 and 6 clients = 0• More than 7 clients = 3

11

Methodology•Post-pre self-assessment ▫At the beginning of an intervention, people do not always

know what they do not know….so there is a tendency to over or under-estimate …..« Knowing what you know now (having completed the intervention), how would you rate your knowledge when you entered the research and how would you rate your knowledge now?»

•Dependent measures▫Knowledge of LMI▫Ability to use LMI▫Personal attributes (confidence and optimism)

12

Principal Findings

1. Clients on average demonstrated significant increases regardless of the method used (independent or assisted)

2. Clients in the assisted group demonstrated greater increases over time than the independent group.

3. Assisted clients in the career decision making group demonstrated greater increases than the other three sub-groups

1. Independent career decision making2. Assisted career decision making3. Independent work search4. Assisted work search

4 sub-groups

13

First Major Finding•Clients demonstrated on average significant

increases regardless of method of intervention (assisted or independent) ▫Between pre and post intervention:

•Overall ability to access and use LMI improved:▫Knowledge re how to use LMI▫Abilities to use LMI and to take action based on

LMI▫Personal attributes: optimism, self-confidence and

by inference, motivation

Sample of Results: Post-pre questionnnaire

0) Ne s’applique pas du tout1) Ne s’applique pas du tout, mais presque2) S’applique, mais tout juste bien 3) La réponse se situe entre tout juste bien et parfaitement bien4) S’applique parfaitement

6/14 itemsNON

Sur ces 6 items (Connaissances = 1, 2, 3, 5; Compétences = 4; Attributs personnels = 6)

Descriptive Results – POST-PREConsidering all 14 items in the questionnaire

Before the intervention After the intervention

• 45 % of responses were not OK [0, 1] • 2 % of responses were not OK [0, 1]

• 55 % of responses were OK[2, 3, 4]

• 98 % of responses were OK[2, 3, 4]

• 8 % of responses were exceptional [4] • 52 % of responses were exceptional [4]

15

Descriptive Results– POST-PRE•All the mean scores on the Before Intervention

responses were in the Not OK range [0,1], with 1 exception (2,23).

•All the mean scores After the Intervention , without exception, were in the very OK or perfectly OK range [score average 3 or more].

•The amount of change was similar across all dimensions of the survey ▫We found the same degree of change for

knowledge, abilities and personal characteristics.

16

17

Second Major Finding

•Clients who were assisted by a career practitioner demonstrated greater change with time

18

•Significant improvement in overall ability to use LMI between Before and After▫These results were consistent in all sub scales

(knowledge, abilities, personal attributes)

Results Overall Score

For the group as a whole:

14 items

Post-Pre

19

Second Major Finding

•Assisted clients showed increased levels of change over time

▫We can consider that having had two information and advice follow-up interviews, although brief, allowed assisted clients to state after the intervention, greater gains in knowledge and skill related to LMI as well as gains in confidence and optimism with respect to their career futures.

20

Information and Advice Interviews• Two interviews of 20-30 minutes each• Three key steps in Information and Advice interviews:▫Re-establish the helping alliance and review the objective and

action plan agreed to▫Give advice and information related to client use of LMI as

appropriate for client need▫Come to agreement on next steps to move client action plan

forward• All Career practitioners reported following these 3 steps ▫ Importance was given to the helping alliance and to agreeing on

the next steps to be followed• Career practitioners used the second step to specifically

respond to the information and action needs of the clients

21

Third Major Finding

• Assisted clients in Career Decision Making reported the highest gains among the four groups

1. Career Decision Making (independent)

2. Career Decision Making (assisted)

3. Work Search (independent)

4. Work Search (assisted)

Results showing both the two types of intervention and the two methods

Final Question-

naire Before

Final Question-

naire After

Two Types of intervention

Two methods n

M(ET)

M(ET)

Time(p)

Method(p)

T x M(p)

Career Decision Making

Independent 15 42,00(17,78)

70,00(13,79)

278,22(< 0,01)

3,58(0,07)

19,69(< 0,01)

Assisted 17 24,65(7,09)

72,94(9,79)

Total 32 32,78(15,69)

71,56(11,73)

Work Search Independent 11 52,82(24,31)

90,27(12,25)

66,35(< 0,01)

0,03(0,86)

0,26(0,62)

Assisted 9 49,00(25,64)

91,44(13,16)

Total 20 51,10(24,32)

90,80(12,34)

22

F

23

Career Decision Making (assisted) (Suggested Interpretation)

• The unique position of clients who are trying to clarify a career direction may promote the asking of clarification questions which could increase the impact of an LMI intervention (Imel, Kerka et Wonacott, 2001) 

• The advice given and the individual assistance may have resulted in an increase in the effectiveness of LMI for the individual (Brown et Ryan Krane, 2000)

• « The impact of the help of a career practitioner working with LMI and working directly with a client trying to make career planning decisions appears to be an important finding» (Savard, Michaud, Bilodeau et Arseneau, 2007, p. 163).

Other Results

•No significant differences for: ▫Gender

Men and women responded similarly to both methods of working with LMI

▫Age▫Employment history▫Unemployment history

24

Attribution of change• To what extent would you say that the changes you

reported before and after the intervention were the result of your participation in the research?

Methods

Mainly due to other factors

Partly due to other factors

Don’t know

Partly due to this project

Mainly due to this project

Independent (n=26)

0 0 1 16 9

Assisted (n=26 0 0 0 9 17

Total 0 0 1 25 26

25

26

Attribution of change• 50 % of clients attributed their change to participating

in the LMI research and not to other factors; 48 % attributed their change partially to their participation

•Overall there appear to be no significant differences in the results of the anglophone and francophone samples with the exception that assisted francophone clients reported increased gains over the anglophone clients although both groups reported more positive change in the assisted condition.

Client Engagement (client tracking sheets)

•Clients were very engaged in working with the LMI packages ▫Clients accessed between 4 and 20 distinct LMI resources

and made use of these between 7 and 148 times. ▫On average clients accessed between 4 to 7 LMI resources;

career decision making clients accessed on average 19 resources and work search clients accessed on average 18 resources.

•Most frequently used resources

27

Work SearchLMI Booklets (90 %)Career Cruising (65 %)

Career Decision Making Career Cruising (95 %)LMI Booklets (87 %)Travailler au Canada (87 %)

General Comments from clients

• Increase in confidence▫ I doubted that I would benefit from the study based on past

experience but I became more certain and confident in my own ability to find what I needed. (214)

• Better self awareness and knowledge of possible careers ▫ It was very useful to understand myself better and use this

as a basis for searching for possibilities and making choices. (241)

• Overall a positive experience ▫ The study helped with knowing what direction to go in and I

am sure that I have benefited a lot. I am more confident and pleased to have participated. (231)

▫ This allowed me to get more clear. I think I can continue to use this information as I face future changes. (217)

28

29

Client Criticisms •The program needed to be longer given the

number of resources available. (222)

•All the information I needed was available but I could not take it with me. It would have been better to have it all instead of having to go the the Resource Centre. […]. Working full time made it very hard […]. With a little more help it would have been much easier (242)

Client Criticisms •There was too much information and that made

making a career decision more difficult […] very hard to do this alone (209)

•The information in the booklets was too broad for my region […]. This could discourage certain people (252)

• I feel I am at the same point at the end of the study (237)

30

31

Follow up interviews (1 week and 4 months after intervention)•Week 1: 39 clients (75% of original sample)•4 months: 27 clients (52% of original sample)

▫LMI overall relevant▫Clients overall optimistic and confident▫Clients continued to use LMI but less often

32

Follow up Interviews•Most useful aspects of LMI:▫Self-knowledge▫Job search techniques

•Overall weakness in service delivery:▫Action Planning

33

Follow-up interviews•Most common concerns:▫« too much » information▫Lack of computer skills▫Lack of practitioner assistance especially:

Connecting self knowledge to world of work Addressing employment barriers (confusion,

disappointment)

34

For more information

Réginald Savard, professeur et co-chercheurReginald.Savard@USherbrooke.caPaquette, professionnel de rechercheSylvain.Paquette@USherbrooke.caLynne Bezanson, CCDF, l.bezanson@ccdf.caSite du GRDC <http://www.crwg-gdrc.ca