Use  of  academic  social  research  by  public  officials:  exploring  preferences  and  

constraints  that  impact  on  research  use    Adrian  Cherney    

 ARC  Linkage  project:  “The  U9lisa9on  of  Social  Science  Research    in  Policy  

Development  and  Program  Review”  LP  100100380        

Other  team  members:  Brian  Head,  Paul  Boreham,  Michele  Ferguson,  Jenny  Povey,  Jenny  Van  der  Arend  and  Jenny  Bell.        

Where  is  the  evidence  conference  2013:  Recognising  the  value  of  grey  literature  11th  November  2013    

State  Library  of  Victoria,  Melbourne  

 

     

Presenta9on  outline  

•  Background  to  ARC  linkage  project.      •  Preferences,  access,  accessibility  and    workplace  culture  &  evidence-­‐based  policy.    

•  Results  from  public  sector  survey.  •  Concluding  remarks.    

ARC  Linkage  Partners  

•  Four  governments:  Federal,  QLD,  NSW,  VIC.    •  Line  departments  in  human  services  (educaPon,  family/community  services,  public  health).    

•  Central  agencies  plus  ‘knowledge  specialist’  agencies.    

•  Nine  funding  partners  plus  another  12  supporPng  agencies.  

Data  sources  for  ARC  Linkage  Project  

•  Targeted  survey  of  Australian  social  scienPsts  n=693    •  Targeted  survey  of  policy  relevant  personnel  in  21  agencies  n=2084  

•  Interviews  with  a  selecPon  of  academic  respondents  n=100  

•  Interviews  with  a  selecPon  of  policy  personnel  n=125  

Public  sector  agencies  and  evidence-­‐based  policy  (EBP)  

•  Perceived  gap  between  research  producPon  and  uptake  by  governments.    

•  Understanding  access  /  accessibility  important  to  closing  this  gap.    

•  Preferences  and  organisaPonal  context  impacts  on  choices  to  seek  out  and  access  research-­‐based  knowledge.    

!"#$%&'(&)*%+,-%.&/01&2"34-,-2"4%5&-+&.637%8& 93%:;& <%3,%+4&=1>&414"$&<??@&

A1;&1>&.4">>&.637%8&

5-.43-#64%5&41&

B%.21+.%&3"4%&C&

!"#$%&'()('*+,#--(..(#/+0,#--#/1234'56+ 78+ 9:;;+ !+<88+ !!78:88+=%.'"34(3/+>%"23%+#?+@'3'(.'(&.+0,#--#/1234'56++ 99;+ <8:AB+ CC9+ 9A:DE+

F"23.%"*+0,#--#/1234'56+ <9E+ D:A+ !+B88+ !+E8:CD+G2H3"'-2/'+#?+'52+!"(-2+I(/(.'2"+3/$+,3J(/2'+0,#--#/1234'56+ <B+ 8:7C+ !!E88KB88+ KKKKKK+

G2H3"'-2/'+#?+L3-(4(2.M+N#%.(/OM+,#--%/('(2.+P+Q/$(O2/#%.+=??3(".+0,#--#/1234'56+ 9D9+ <9:8A+ <<<D+ 99:78+

G2H3"'-2/'+#?+R$%&3'(#/M+R-H4#*-2/'+3/$+S#"TH43&2+U243'(#/.+0,#--#/1234'56+ ;;+ B:99+ !+<988+ !+C:EE+V%22/.43/$+N234'5+ <<9+ D:EC+ A<7+ <9:9E+

V%22/.43/$+G2H3"'-2/'+#?+,#--%/('(2.+ <88+ B:;+ W#'+T/#1/+ KKKKKK+

V%22/.43/$+G2H3"'-2/'+#?+R-H4#*-2/'M+R&#/#-(&+G2)24#H-2/'+P+Q//#)3'(#/+ CE+ E:D+ <78+ BD:7E+

V%22/.43/$+G2H3"'-2/'+#?+'52+!"2-(2"+3/$+,3J(/2'++ <;+ 8:;7+ 78+ E8:88+

V%22/.43/$+F"23.%"*++ <E+ 8:79+ W#'+T/#1/+ KKKKKK+

V%22/.43/$+G2H3"'-2/'+#?+R$%&3'(#/+3/$+F"3(/(/O++ C8+ E:E7+ 9E8+ E8:BE+

W@S+G2H3"'-2/'+#?+R$%&3'(#/+3/$+,#--%/('(2.++ 7D+ E:<9+ EAD+ <7:B7+

W@S+F"23.%"*++ B<+ <:AC+ 9CC+ <B:;8+

W@S+G2H3"'-2/'+#?+!"2-(2"+3/$+,3J(/2'++ DD+ 9:7B+ W#'+T/#1/+ KKKKKK+

W@S+G2H3"'-2/'+#?+L3-(4*+3/$+,#--%/('*+@2")(&2.++ <DB+ C:EA+ DB;+ 9;:<8+

X(&'#"(3/+G2H3"'-2/'+#?+!43//(/O+3/$+,#--%/('*+G2)24#H-2/'++ 9;+ <:EB+ <8;+ 9D:AE+

X(&'#"(3/+G2H3"'-2/'+#?+R$%&3'(#/+3/$+R3"4*+,5(4$5##$+G2)24#H-2/'++ E;B+ <;:BE+

Q/('(34+(/)('3'(#/+E89EY+'3"O2'2$+

"2-(/$2"+'#+;E;+&2/'"34+.'3??++

!"<9:C8+

X(&'#"(3/+G2H3"'-2/'+#?+N%-3/+@2")(&2.++ <89+ B:A8+ W#'+T/#1/+ KKKKKK+

X(&'#"(3/+G2H3"'-2/'+#?+!"2-(2"+3/$+,3J(/2'++ D8+ 9:B8+ <98+ B9:D8+

X(&'#"(3/+F"23.%"*++ DB+ 9:DA+ W#'+T/#1/+ KKKKKK+

!14"$& DEFG& 'EE& & &

3.9  

14.4   14.6  

5.3  

3.8  

23.6  

28.6  

5.9  7.7  

38.0  

43.1  

11.2  

0  %  

5  %  

10  %  

15  %  

20  %  

25  %  

30  %  

35  %  

40  %  

45  %  

50  %  

Senior  execuPve   Manager   Policy  officer   Data  analyst  

Figure  1:  Gender  and  current  posiPon  

Male   Female  

6.0  4.5  

34.2  

16.2  

39.2  

0  %  

5  %  

10  %  

15  %  

20  %  

25  %  

30  %  

35  %  

40  %  

45  %  

Year  12   Advanced  Diploma/Diploma  

Bachelor  Degree   Graduate  Diploma/Graduate  

Cer9ficate  

Postgraduate  Degree  

Figure  2:  Level  of  educaPon  

2.5  

19.3  

23.6  

27.1  

18.1  

8.1  

1.3  

0  %  

5  %  

10  %  

15  %  

20  %  

25  %  

30  %  

Less  than  a  year  

1-­‐5  years   5-­‐10  years   10-­‐20  years   20-­‐30  years   30-­‐40  years   40+  years  

Figure  3:  Years  in  the  public  service  

93.1  

82.5  

72.1  

71.6  

69.6  

63.9  

63.8  

56.0  

53.7  

52.1  

48.6  

37.4  

0  %   10  %   20  %   30  %   40  %   50  %   60  %   70  %   80  %   90  %   100  %  

Internal  agency  Staff  

Federal  government  agencies  

Other  state  government  agencies  in  your  state  

Professional  or  industry  associaPons  

University  researchers  

Comparable  state  government  agencies  in  other  

Interest  groups  

InternaPonal  organisaPons  

Private  consultants  

News  media  

Think  Tanks  

Local  government  

Figure  4:  Level  of  importance  placed  on  informaPon  available  from  parPcular  sources  to  inform  decision-­‐making:    

Very  important/important  

89.5  

44.2  

41.4  

32.8  

32.7  

27.5  

26.3  

22.0  

20.6  

17.5  

12.5  

12.0  

0  %   10  %   20  %   30  %   40  %   50  %   60  %   70  %   80  %   90  %   100  %  

 Internal  agency  Staff  

Other  state  government  agencies  in  your  state  

Federal  government  agencies  

Professional  or  industry  associaPons  

News  media  

University  researchers  

Interest  groups  

Comparable  state  government  agencies  in  other  

Private  consultants  

InternaPonal  organisaPons  

Local  government  

Think  Tanks  

Figure  5:  In  the  last  12  months,  how  oien  have  you  consulted  with  each  source  of  informaPon  listed  below:  Very  oien/Oien  

68.0  

47.5  

45.2  

43.0  

35.6  

32.4  

28.1  

0  %   10  %   20  %   30  %   40  %   50  %   60  %   70  %   80  %  

You  would  prefer  to  use  search  engines  on  the  web  (e.g.  Google)  

You  would  rather  consult  a  work  colleague  about  sourcing  relevant  arPcles  or  reports  

You  don’t  know  how  to  use  these  databases  

You  have  not  requested  access  because  such  a  resource  would  not  be  relevant  to  your  role  

You  don’t  have  access  from  your  work  staPon  

You  can't  download  full-­‐text  versions  of  academic  arPcles  and  and  reports  from  these  databases  

The  department,  agency  or  unit  you  work  for  does  not  subscribe  to  any  electronic  bibliographic  

databases  

Figure  6:  If  you  don’t  access  bibliographical  databases,  is  it  because:  (n  =  868)  

What  might  be  driving  these  behaviours?    

•  Trust  in  the  source?    •  Accessibility?    •  Convenience?    •  Skill  sets  of  individuals?    •  Few  partnership  opportuniPes  with  knowledge  producers?    

•  Access  to  infrastructure?    •  Nature  of  public  policy-­‐making?    •  Workplace  culture?    

55.6  

52.3  

35.7  

35.6  

19.3  

15.9  

14.6  

11.4  

0  %   10  %   20  %   30  %   40  %   50  %   60  %  

There  is  not  enough  Pme  in  the  day  or  week  to  read  relevant  research  studies  

There  is  limle  opportunity  to  build  relaPonships  with  researchers  outside  the  public  service  

My  department  has  no  formal  processes  to  translate  academic  research  into  policy  

I  lack  sufficient  decision-­‐making  power  to  ensure  policy  is  based  on  research  

The  use  of  research  evidence  is  a  low  priority  of  my  unit  

I  do  not  have  the  necessary  skills  to  interpret  results  from  staPsPcal  analyses  

Staff  are  not  encouraged  to  use  research  evidence  

I  lack  experPse  in  how  to  apply  the  results  of  research  studies  

Figure  7:  Accessing  and  using  research  evidence  in  day-­‐to-­‐day  duPes:  Strongly  agree/Agree  

Impact  of  organisa9onal  factors  •  LogisPc  regression  model  to  examine  the  relaPve  strength  of  

various  organisaPonal  factors  on  reported  levels  of  research  use.    

•  Dependent  variable  measure  of  research  use  was  based  on  quesPons  asking  respondents  whether  in  the  last  12  months  they  had  used  academic  products  or  outputs  to  understand  policies  and  programs  in  their  field.    

•  This  measure  of  research  uPlisaPon  was  divided  into  a  dichotomous  variable  (0  =  Don’t  consult  academic  research  and  1  =  consult  academic  research).      

32.9  35.3  

38.6   38.3  

17.9  16.4  

10.6   10.0  

0  %  

5  %  

10  %  

15  %  

20  %  

25  %  

30  %  

35  %  

40  %  

45  %  

In  the  last  12  months,  I  have  used  journal  arPcles  and  books  produced  by  academics  to  

understand  policies  and  programs  in  my  field    

In  the  last  12  months,  I  have  used  research  reports  produced  by  academics  to  understand  

policies  and  programs  in  my  field    

Figure  8:  ConsultaPon  of  academic  research  

Frequently   Occasionally   Rarely   Never  

Independent  variables  –  factors  influencing  research  use  

•  Items  related  to  organisaPonal  ethos  and  culture    e.g.  there  is  not  enough  Pme  in  the  day  or  week  to  read  relevant  research  studies;  the  use  of  research  evidence  is  a  low  priority  of  my  unit;  staff  are  not  encouraged  to  use  research  evidence.      

•  Judgements  about  skill  levels,  and  impediments  to  access  (difficulty  in  accessing  full  text  versions  of  academic  reports).    

•  EducaPonal  level  and  posiPon.    

!"#$%&'(&)"*+,-.&/01"*+/23&,2&+4%&5.%&,6&"*"7%0/*&-%.%"-*4&1-,75*+.

!"#$%&'()*+"#$),-(%,($./(0'/(%1("#"2/*)#(&/'/"&#.(+&%20#$'((

( ( 345(67(1%&(822'(9"$)%(

(:( ;<( =%>/&(

822'(9"$)%(

?++/&(

@#"2/*)#(&/'/"&#.(&/'0A$'("&/(#%,')2/&/2(&/A/B",$(CD(*D(#%AA/"-0/'(

EFG4HHH( IEFE3J( KFGL( MFKK( MF4E(

9/'/"&#.()'()*+%&$",$(),(*D(+&%1/'')%,"A(1)/A2( EFGNHHH( IEFE3J( KFGG( MFE3( MFNG(

="#O(&/'/"&#.('O)AA'( EFE3( IEFELJ( EF3N( KFE3( KFMG(

="#O("(&/'/"&#.(#0A$0&/( PEFQQHH( IEFKKJ( EF4L( EFGM( EF3E(

R%A)#D(*"O),-(C"'/2(%,('%0,2(/B)2/,#/( PEFE3( IEFKEJ( EFG4( EF3K( KFKK(

S)11)#0A$D("##/''),-(10AA($/T$( EFUNHHH( IEFE3J( KF43( KF3E( MFMG(

@2B",#/2(2)+A%*"( EFQN( IEFQGJ( EFUL( KFNK( MF3M(

V@(( EFN4( IEFMUJ( EF3N( KF4G( MFUQ(

W&"20"$/(2)+A%*"( EFGKH( IEFM3J( KFKU( MFEN( QF43(

R%'$-&"20"$/(2/-&//( EF3NHHH( IEFMGJ( KF4K( MF4U( NFQU(

;/,)%&(/T/#0$)B/( PEFKM( IEFMGJ( EF4M( EFL3( KF4M(

X","-/&((( PEFNQHH( IEFK4J( EFNL( EFU4( EFLG(

S"$"(","AD'$( PEFG3HHH( IEFMKJ( EFQE( EFNU( EFU3(

Y/'('$"11(A),O),-($%(&/'/"&#./&'( EFQM( IEFKGJ( EF33( KFQL( KF3M(

6%,'$",$( PNF3LHHH( IEFGUJ( EFEE( EFEK( EFEQ(

8C'/&B"$)%,'( K33U( ( ( ( (

R'/02%(9M( EFMQL( ( ( ( (

=%-(A)O/A).%%2( PGUKFGL4( ( ( ( (

#.)M(I21ZMEJ( NGUFM4( ( ( ( (

Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001 The results reported in Table 4 provide support for the argument that an overall organisational ethos

and professional culture that value research has a bearing on the uptake of academic research among

policy personnel, well above any perceived deficits in individual skills. As stated the positive

relationship observed between experiencing difficulties in accessing articles and reports and the

uptake of academic research is somewhat unusual. This can perhaps be explained by the fact that

respondents who are actively engaging academic research and are using it to inform their work

Conclusion:  Public  sector  use  of  research  evidence  &  EBP  

•  Convenience  and  expediency  mamer  a  great  deal.    •  Physical  or  electronic  availability  important  –  but  not  overwhelmingly  so.    

•  Day-­‐to-­‐day  pressures  and  constraints  reinforce  various  pamerns  of  informaPon  seeking.    

•  OrganisaPonal  ethos  and  professional  culture  help  to  generate  behaviours  that  promote  research  use.