assessing europe university based research en[1]

8/12/2019 Assessing Europe University Based Research en[1]

1/151

Assessing Europes

University-Based

Research

Expert Group on Assessment

of University-Based Research

EUR 24187 EN

EuropeanResearch Area

Science& society

E U R O P E A NCOMMISSION

Research Policy


2/151

EUROPEAN COMMISSION

Directorate-General for Research

Directorate C - European Research Area: Knowledge-Based Economy

Unit C.4 - Universities and Researchers

Contact: Adeline Kroll

European Commission

Office SDME 9/17

B-1049 Brussels

Tel. (32-2) 29-85812

Fax (32-2) 29-64287

E-mail: [email protected]

Interested in European research?

Research*euis our monthly magazine keeping you in touch with main developments (results, programmes,

events, etc.).

It is available in English, French, German and Spanish. A free sample copy or free subscription can be

obtained from:

European Commission

Directorate-General for ResearchCommunication Unit

B-1049 Brussels

Fax (32-2) 29-58220

E-mail: [email protected]

Internet: http://ec.europa.eu/research/research-eu


3/151

EUROPEAN COMMISSION

Assessing Europes University-Based

Research

Expert Group on Assessmentof University-Based Research

RTD.C4

Directorate-General for Research

Science in Society 2008 Capacities, 1.4.12010 EUR 24187 EN


4/151

LEGAL NOTICE:

Neither the European Commission nor any person acting on behalf of the Commission is responsible

for the use which might be made of the following information.

The views expressed in this publication are the sole responsibility of the author and do not necessarily

reflect the views of the European Commission.

A great deal of additional information on the European Union is available on the Internet.

It can be accessed through the Europa server (http://europa.eu).Cataloguing data can be found at the end of this publication.

Luxembourg: Publications Office of the European Union, 2010

ISBN 978-92-79-14225-3

ISSN 1018-5593

doi 10.2777/80193

European Union, 2010

Reproduction is authorised provided the source is acknowledged.

Printed in Belgium

PRINTEDONWHITECHLORINE-FREEPAPER

EUROPE DIRECT is a service to help you find answers

to your questions about the European Union

Freephone number (*):

00 800 6 7 8 9 10 11(*) Certain mobile telephone operators do not allow access to 00 800 numbers

or these calls may be billed


5/151

FOREWORD

How to create a new and more coherent methodology to assess the

research produced by European universities? This is the question experts

were asked to answer, following a 2006 Commission Communication on the

modernisation of universities1, which suggested that universities should

become more specialised and concentrate on working to their specific

strengths.

Universities rankings are increasingly popular. Today, 33 countries have

some form of ranking system operated by government and accreditationagencies, higher education, research and commercial organisations, or the

media. The most popular are the Shangai Jiao Tong Academic Ranking of World

Universities and the Times QS World University Ranking.

Rankings are used for specific and different purposes. Politicians

regularly refer to them as a measurement of their nations economic strength

and aspirations. Universities use them to define performance targets and

implement marketing activities, while academics use rankings to support their

own professional reputation and status. Students use rankings to choose theirpotential place of study and research. Public and private stakeholders use

rankings to guide their decisions about funding allocations. What started out

as a consumer product aimed at undergraduate domestic students has now

become both a manifestation and a driver of global competition and a battle

for excellence in itself.

However while there are over 17,000 higher education institutions

worldwide, rankings concentrate interest only in the worlds top 100.

In addition, if higher education is one of the engines of the economy

and a key point on the knowledge triangle, then the productivity, quality

and status of research produced by universities is a vital indicator. Hence the

importance of designing a way to evaluate it which is truly fit for purpose. But,

as always, there is no one size fits all solution! A new methodology will have

to be developed. Ideally the best would be applicable across a full range of

disciplines, including interdisciplinary research. It should assume an inclusive

notion of research, ranging from blue sky/curiosity-driven to user-led/practice-

based research. At present, some rankings include metrics on teaching and

learning, most are focused on life-science research.


6/151

Users too have their own specific needs. And, depending on what they

want to find out, they should be provided with a broad range of answers.

For example, a prospective student might look for information on a specific

discipline, on future employability, or on the fees associated with the university

of their choice. A ranking system of this kind does exist for students, but at the

moment only in Germany. The level at which the quality of research is assessed

also matters. Ranking universities as entire institutions may not be the most

appropriate way to identify where the best research is done and how it is

done. A university may be renowned for one or two departments, but may

not be excellent in all disciplines it offers. Identifying more precisely where

research is produced and disseminated should allow for a better assessment

of university-based research.I believe that the coexistence of different models to assess university-

based research is not only inevitable, but healthy. We need to design flexible

and multidimensional methodologies that will adapt to the diverse and

complex nature of research, disciplines and of our universities. In its quest

for excellence, the European Commission must and will encourage, promote

and support every effort to understand and monitor the quality of research at

universities.

I wish to end with a simple quote from someone who understoodbetter than anyone else the value of freedom, creativity and knowledge: Not

everything that counts can be counted, and not everything that can be counted

counts. This sign was hanging in Einsteins office at Princeton. Let us take the

time now to see what really counts when we rank our universities, these most

important of our knowledge powerhouses.

Commissioner Janez Potonik


7/151

1 Overview ............................................................................................................................9

1.1 )E20=/?@2 O=6631:...................................................................................................91.2 C72 *3: &-1.315 a (20-662453/?-4; ..................................................................15

2 Introduction ...................................................................................................................... 17

2.1 N4?@21;?/:[>3;25 (2;23107 ?4 /72 4-.82592 )0-4-6: .........................................172.2 C72 )=1-


8/151

g

9.10 V^+VKQC'(O '& )K'^'bVK h O'KVQ_ W)^)&VCO .................................................... 799.11 )^+[NO)( )OC))b.................................................................................................. 81

10 Appendix IV. Case Studies of the Research Assessment Experience ......................... 83

10.1 QNOC(Q_VQ ...............................................................................................................8410.2 N^Vf)(OVCi _VW() +) W(Nj)__)O a W(Nj)__)O a W)_\VNb...................................88

10.3 &V^_Q^+ kQQ_C' N^Vf)(OVCel................................................................................9110.4 &V^_Q^+ k,)_OV^`V N^Vf)(OVCel............................................................................. 9310.5 &(Q^K) .................................................................................................................... 9610.6 \)(bQê [ &'(OK,N^\O(QCV^\ kK'^+NKC)+ We *VOO)ÔK,Q&CO(QC............ 9910.7 \)(bQê [ K,) N^Vf)(OVCe(Q^ V^\G K,) ()O)Q(K,(Q^`V^\........................ 10210.8 \)(bQê [ V^VCVQCVf) &'( )jK)__)^K)..............................................................10510.9 ,N^\Q(e...............................................................................................................10710.10 V()_Q^+ .............................................................................................................11010.11 VCQ_e .................................................................................................................. 11310.12 ^)C,)(_Q^+O ...................................................................................................117

10.13 ^'(*Qe ............................................................................................................12010.14 OSQV^.................................................................................................................12210.15 O*)+)^.............................................................................................................12410.16 N^VC)+ V^\+'b ............................................................................................. 12710.17 \_'WQ_ [ *)W'b)C(VKO..................................................................................12910.18 \_'WQ_ [ QKQ+)bVK (Q^`V^\ '& *'(_+ N^Vf)(OVCV)O kQ(*Nl.................13410.19 \_'WQ_ [ C,)[RO *'(_+ N^Vf)(OVCV)O (Q^`V^\..........................................13710.20 \_'WQ_ [ S)(&'(bQ^K) (Q^`V^\ '& OKV)^CV&VK SQS)(O &'( ()O)Q(K,N^Vf)(OVCV)O ..................................................................................................................... 14010.21 \_'WQ_ a C,) _)V+)^ (Q^`V^\ ....................................................................... 143

11 Appendix V. Bibliography ........................................................................................145EU Publications .................................................................................................................145Other Publications .............................................................................................................146

*2>-913


9/151

m

"#$%#&$'

!"! !"#$%&'(# *%++,-./!01/2!3!4 05678293:! 7; ?@9*!4 8!*!98:/ 934 7; 9**!**5!32 7; ?@9*!4 8!*!98:/

2A# BCD'&'$,D $CE"&

Q;;2;;624/ -H =4?@21;?/:[>3;25 12;23107LkQNW(l 73; >20-62 3 63n-1 ?;;=2 H-1 3 .?52 13492-H ;/3F27-8521; 3/ 388 82@28;M '42 -H /72 63?4 123;-4; ?; /73/ 12;23107 2?49 3 63n-1 H30/-1 ?4 20-4-6?0 -4 O/13/29: .?/7 ?/; 6-@2 /-.315; 3 98->38 345 F4-.82592[>3;25 20-4-6:MV62 @?/388:?6-4

O/13/29: k)=1-3;25 12;23107


10/151

LI

5?@21;2 345 0-63;?; -H 3991293/25 ;0-12;M b-12-@21G 98->38134F?49; /245 /- 128: -4 D=38?/3/?@2 ?45?03/-1[>3;25 53/3G .7?07 /245 /- 73@2 34 ?4>=?8/ >?3;?4 H3@-=1 -H 7315 ;0?2402; 345 >?-;0?2402;G 345 -H )498?;7[8349=392 8?03/?-4;M C7212 ?;38;- 3 ;=>;/34/?38 830F -H 01-;;[43/?-438 0-68?;725 ?4 P=8: "IIg/- ?524/?H: /72 2 ->;21@25 ?4 12;23107 3;;2;;624/ 3; .288 3; 3438:;2 63n-13;;2;;624/ 345 134F?49 ;:;/26; .?/7 3 @?2. /-


11/151

LL

-1?24/25 120-662453/?-4;G 8245; 0125?>?8?/: /- /72;2


12/151

L"

*7?82 ;-62 -H /72 53/3 12D=?125 63: >2 1235?8: 3@3?83>82 -1 1283/?@28: 23;: /- ->/3?4G-/721 53/3 312 2?/721 4-/ 3@3?83>82 -1 -48: 3@3?83>82 ?4 8?6?/25 0?10=6;/3402;M C7?;63F2; 0-6?8?/: 301-;; =4?@21;?/?2; 345 0-=4/1?2; 5?HH?0=8/M

(D)E4/&- ,%&2-&,=$/:+ F$ +F/$%#$6 &2 :;$ 4//$//@$2: +0 72$%/&:G.F4/$6 %$/$4%-;H\=?525 >: /72 0-4@?0/?-4 /73/ /72 2 ;224 ?4 0-4/2E/G /72 \1-=< ?524/?H?25 3 4=6>21 -H >3;?0


13/151

L#

,2402G /72 07-?02G ?4/21


14/151

L$

7=!898:/031 7 03 9**!**5!32 7; ?@9*!48!*!98:/ Q9 3!R 966879:/

'4 /72 >3;?; -H /72


15/151

L%

03


16/151

LJ

_3=407 3


17/151

LU

DA2:%+67-:&+2C7?; 073


18/151

Lg

23;?8: 5?;/-1/M V4 -1521 /- 033;25 12;23107 36=8/?5?624;?-438 32 0-=4/25G 345 4-/ 2@21:/7?49 /73/

034 >2 0-=4/25 0-=4/;Md

#"# 2A# !%-CB#,E 6CD'$. :CE"&2A# P'IFCE 9H#EN,

V4 b3107 "IIIG /72 )=1-: H-0=;?49 H=45?49 -4 1282@34/-=/


19/151

Lm

Q;;2;;624/ 62/7-5-8-9?2; ;7-=85 120-94?;2 345 ;=


20/151

"I

#"$ 5#,I%-'EH SA,& :C%E&Ib34: 224 13?;25 3>-=//72?1 -@21[128?3402 -4 ?4/2143/?-438 >?>8?-62/1?0 345 0?/3/?-4 53/3>3;2;G 2M9M C7-6;-4[(2=/21; ;*. *1 A5$"'5" -1 )8;2@?21[O0-242H?/ /72 ?3; ?4 H3@-=1 -H )498?;7 8349=392 8?03/?-4;G 345 393?4;/43/?-4388:[H-0=;25 12;23107M K?/3/?-4; 623;=12 ?6=/ ?94-12/72 ?6224 13?;25 3>-=/ /72 =;2 -H


21/151

"L

12;-=102;M C72: 034 3?5 ;/13/29?0 634392624/ 345


22/151

""

Q .-1F;7-< .?/7 K-66?;;?-421 S-/-s4?FG H?H/224 626>21; -H /72 )E38 134F?49; 73@2 307?2@25 3 7?97 82@28 -H ?4/2143/?-438


23/151

"#

LlK3;2 ;/=5?2; -H 12;23107 3;;2;;624/ 345 134F?49 ;:;/26; 301-;; )=1-


24/151

"$

IC;4%4-:$%&/:&-/ +0 >$/$4%-; O//$//@$2:C7?; 0733;?0


25/151

"%

$"# 9E 0E$D%I'(# 9BB-C,$A &C 4'I$'BD'E#IC72;2 52@28-


26/151

"J

@?;=38 120-15?49;G 0-63;2;G /2074?038 513.?49;G 52;?94; -1 .-1F?496-528;G 63n-1 .-1F; ?4 ?/?-4 345Z-1 3.315[.?44?49 52;?94G


27/151

"U

0-4;?52125M C72 2E


28/151

"g

E'.$%$.20#/a \135=3/2 ;/=524/;G 2;2;/ ?4;/?/=/?-4 /- 3//245G ?408=5?49 /1245; ?4 9135=3/226


29/151

"m

C72 625?3 ?; >-/7 3


30/151

"#

"#$%& ' ()**#+, -. /0&+ 1+-)20 #34 /0&0 -. 5&0+67 800&00*&39 :#9#

/0&+ 1+-)2 ;7, 5&0+67 800&00*&39 :#9#


31/151

"T

=)&'.9%( 2%)>*-')2'/&631'?>%(?>*2*')25=%((%< %19.*-' ('+*%) 6)- *)9'()69*%)6335

@163*95 %> 626-'> 6)- 4;J .91-')9.G>>*2*')25 3'&'3H ;%F 1)-*)+:'.'6(2; *)>(6.9(1291('H 3'&'3 %> 1.' 6)- '>>*2*')25

M*)*.9(*'. %> G-1269*%)/L*+;'( G-1269*%)%( G)9'(7(*.' 6)- G LG *).9*919*%).@163*95B .1.96*)6N*3*95B ('3'&6)2' 6)- * ('.'6(2;629*&*95

:'.'6(2; .9(69'+5 -'&'3%7*'3- -696 ('A 3'&'3 %> *)9').*95B'C7'(9*.'B D163*95 6)- 2% LG .5.9'< 6)- *)-*&*-163 *).9*919*%).K99(629*%) 26762*95H ('2(1*9(%< %19.*-' ('+*%) 6)- *)9'()69*%)6335

G>>*2*')25 3'&'3H ;%F 1)-*)+:'.'6(2; *)>(6.9(1291('H 3'&'3 %> 1.' 6)- '>>*2*')25

U%263 6)- :'+*%)63 $%&'()*)' 3%263/('+*%)63 7%3*25 6)- 2%%( .91-')9.B 626-'>*2*')25 3'&'3H ;%F 1)-*)+LG K+')2*'. J'>*)' 7%3*25 6)- *)>%(< -'2*.*%). 6N%19 LG .5.9'< 6)-

LG=.

@163*95B .1.96*)6N*3*95B ('3'&6)2' 6)- * ('.'6(2;629*&*95

J'9'( LG .5.9'< 6)- *)-*&*-163 *).9*919*%).,')2; )69*%)63 2%(%< %19.*-' ('+*%) 6)- *)9'()69*%)6335

@163*95 %> 626-'> 6)- 4;J .91-')9.G>>*2*')25 3'&'3H ;%F 1)-*)+:'.'6(2; *)>(6.9(1291('H 3'&'3 %> 1.' 6)- '>>*2*')25

Q9;'( $%&'()%(*-')2'/&631'?>%(?>*2*')25

,')2; LG .5.9' *)9').*95B'C7'(9*.'B D163*95 6)- 2%


32/151

"V

891-')9 6)- K26-' 626-'> 6)- 4;J .91-')9.FB:FDF:/8G(

K26-'5 26(''( %77%(91)*9*'.=-')9*>5 ('.'6(2; 76(9)'(.=-')9*>5 N'.9 ('.'6(2; *)>(6.9(1291(' 6)- .177%(9 >%(('.'6(2;

=).9*919*%)63 6)- >*'3- -696 (' 3'&'3 %> *)9').*95B 'C7'(9*.'BD163*95B 2%%( *)9'('.9

G>/.91-')9 (69*%=).9*919*%)63 ('.'6(2; .177%(9

891-')9. =)>%(< 2;%*2' %> LG==-')9*>5 26(''( %77%(91)*9*'. =).9*919*%)63 6)- >*'3- -696 (' 3'&'3 %> *)9').*95B 'C7'(9*.'BD163*95B 2%%( *)9'('.9

:'.'6(2; 26762*95 %> *).9*919*%) 6)- ('.'6(2; 9'6(6.9(1291('896>>/.91-')9 (69*%

H@@5 ?@F( =-')9*>5 7''( LG=. 6)- N'.9 ('.'6(2; 76(9)'(. =).9*919*%)63 6)- >*'3- -696 (' 3'&'3 %> *)9').*95B 'C7'(9*.'BD163*95B 2%%(*'3- %> *)9'('.9

:'.'6(2; 26762*95 %> *).9*919*%) 6)- ('.'6(2; 9'6%(?>*2*')25@163*95B .1.96*)6N*3*95B ('3'&6)2' 6)- * ('.'6(2;629*&*95

@163*95 %> 626-'> 6)- 4;J .91-')9:'>3'29 7(*-' 6)- 26(''( 6.7*(69*%)./('71969*%)

,')2; 626-'> 6)- 4;J .91-')9.

H/LGFE CHFBFCB J'9'(%(%(? *)9').*95B 'C7'(9*.'B D163*956)- 2% LG .5.9'< 6)- *)-*&*-163 *).9*919*%).G>>*2*')25 3'&'3H ;%F 1)-*)+


35/151

"#

!"# $%&&'()$%& '()&*+ ,*-.) *&/-0123&4 +%5+ 1&6 71-68&90& 24 1- 8-10&* 92:29&9 4+*2/+8; *-= )&&*?*&:2&6&9 5*+2/8&4 +-

5*+&>5/+4 519 )*-+-+;)&4@ 519 21/8.9210 +*51485+2-14@ 4->+65*&@ &1/;/8-)5&925 &1+*2&4@

*&4&5*/% -* +&/%12/58 *&)-*+4@ 8&058 /54&4 519 =5)4A

E*56210 -1 +%& &()&*2&1/& -> *5172104 519 &(24+&1+ *&4&5*/% 544&44=&1+ &(&*/24&4@ $5 +%& 629& *510& -> 4+57&%-89&*4 519 .4&4 +- 6%2/%

21>-*=5+2-1 -1 *&4&5*/% )&*>-*=51/& 519 C.582+; 5*& *&C.2*&9 519 .4&9A G-.* 7&; )-21+45*24& >*-= +%24 5158;424H

IAB192:29.58 .4&* 0*-.)4 %5:& 5 629& *510& -> JKLD?*&85+&9 *&C.2*&=&1+4AFA$%&*& 24 -:&*85) .4&* 0*-.)4@ 2/ +- )5*+2/.85* 0*-.)4A G-* &(5=)8&@ 0-:&*1=&1+4 519 0-:&*1=&1+ 50&1/2&4 4%5*& 5

*&C.2*&=&1+ >-* 21>-*=5+2-1 +- 529 4+*5+&02/ )8511210 519 =5150&=&1+@ 6%28& 5/59&=2/4

519 *&4&5*/%&*4 651+ 21>-*=5+2-1 +%5+ /51 529 /5*&&* /%-2/&4 519 *&4&5*/% -))-*+.12+2&4M

.12:&*42+2&4 519 0-:&*1=&1+ 5*& 21+&*&4+&9 21 2=)*-:210 )&*>-*=51/& 519 C.582+; 6%28&

219.4+*; 519 &=)8-;&* 0*-.)4 651+ +-


36/151

"S

!"#$%&'()* ,)(-#'%(./01$%#2 3#%#$'45$%24 /%5)+&* &(5=21&4 +%& =-4+ 2=)-*+51+ /%5*5/+&*24+2/4 -> *&4&5*/% 544&44=&1+A B+

9&4/*2>&*&1+ +;)&4 -> 2192/5+-*4 T 4-=&+2=&4 *&>&**&9 +- 54 =&+*2/4 T 519 +%&

92>>&*&1+ =&+%-94 .4&9 +- =&54.*& *&4&5*/% 5/+2:2+;A B+ 584- &()85214 92>>&*&1/&4 21

924/2)8215*; )*5/+2/&@ 519 %-6 2192/5+-*4 /51 )-42+2:&8; -* 1&05+2:&8; 5>>&/+ +%&4&92>>&*&1/&4A U5*+2/.85* 5++&1+2-1 24 02:&1 +- 9&4/*2 +%& =-4+ /-==-18; .4&9 2192/5+-*4@ 519 +%&2* )-42+2:& 519 1&05+2:& >&5+.*&4A

+", !"#$%&'()*B192/5+-*4 -* =&+*2/4 =&54.*& +%& :5*2-.4 /-=)-1&1+4 -> *&4&5*/% 5/+2:2+;@ 21/8.9210 21).+4@

)*-/&44@ -.+).+4@ -.+/-=&4 519 2=)5/+ 519 2+4A

!"#$% 2192/5+-*4 =&54.*& *&4-.*/&4@ %.=51@ )%;42/58 519 >2151/258@ 9&:-+&9 +- *&4&5*/%A$;)2/58 &(5=)8&4 5*& +%& 1.= W5/59&=2/X 4+5>> &=)8-;&9 -* *&:&1.&4 4./% 54/-=)&+2+2:&@ )*-Y&/+ >.19210 >-* *&4&5*/%A

&'()*++ 2192/5+-*4 =&54.*& %-6 *&4&5*/% 24 /-19./+&9@ 21/8.9210 2+4 =5150&=&1+ 519&:58.5+2-1A J +;)2/58 &(5=)8& 24 +%& +-+58 -> %.=51 *&4-.*/&4 &=)8-;&9 >2/&4 -* 5>>2825+&9 50&1/2&4 +- 4.))-*+ 519 >.8>28 +&/%1-8-0; +*514>&*

5/+2:2+2&4A

,$%#$% 2192/5+-*4 =&54.*& +%& C.51+2+; -> *&4&5*/% )*-9./+4A $;)2/58 &(5=)8&4 5*& +%&1.= )5)&*4 ). )&*>-*=51/&@ -* 5/%2&:&=&1+@ >-* 214+51/& +%& /-1+*2 4/2&1+2>2/?4/%-85*8; 71-68&90&A

!-#.)% ."/ 1*"*23%+ *&>&*4 +- +%& /-1+*2 *&4&5*/% -.+/-=&4 >-* 4-/2&+;@ /.8+.*&@+%& &1:2*-1=&1+ 519Z-* +%& &/-1-=;A N&& -* >.*+%&* 924/.442-1A

$%& /%-2/& -> 2192/5+-*4 *&>8&/+4 +%& :58.& Y.90&=&1+4 519 )*2-*2+2&4 -> +%& )*-=-+&*A $%&*&

24 1- 4./% +%210 54 51 - 0*59.5+&4 >-*

85

+%& 21>*54+*./+.*&M 519 [--* +%& C.582+; -> *&4&5*/% -*5/59&=2/ &(/&88&1/&A

N-=& *&4&5*/% 544&44=&1+ &(&*/24&4 W519 *5172104X 544201 92>>&*&1+ 6&20%+2104 -* :58.&4 +-

+%& :5*2-.4 2192/5+-*4A B1 +%24 65;@ 4-=& /-=)-1&1+4 -> *&4&5*/% 5/+2:2+; 5*& :58.&9 =-*&

2=)-*+51+ +%51 -+%&* 5/+2:2+;A $%24 =5; 5>>&/+ +%& 65; 21 6%2/% )5*+2/.85* 924/2)821&4 5*&

+*&5+&9@ 54 924/.44&9 2192/5+-*4 519 6&20%+2104 24 +%&*&>-*& :2+58A $%&; =.4+ 2+?>-*?).*)-4&@

5))*-)*25+& 519 :&*2>25&*&1+ +;)&4 ->

.12:&*42+2&4A B192/5+-*4 4%-.89 >&/+&9 +%&

)5*+2&4 21:-8:&9 21 +%& *&4&5*/% 544&44=&1+ &(&*/24&A $%&; =.4+ 584- 214+28 +*.4+A B1 -+%&*

6-*94@ +%-4& 29&1/& +%5+ +%& 2192/5+-*4 5*& 5))*-)*25+&


37/151

"\

519 +*.+%>.8A $-- >&6 2192/5+-*4 /51 8&59 +- 924+-*+2-1@ 6%28& +-- =51; /51 =57& +%&

&(&*/24& +-- /-=)82/5+&9 519 /-4+8;A K8+2=5+&8;@ +%& )*-/&44 =.4+ 52*1&44 62+%

>&542 4-/258 4/2&1/&4 519 %.=512+2&4 %-6&:&* 5*& =-*& 827&8;

+- /-1429&* 244.&4 +%5+ %5:& 42012>2/51+ -* )*2=5*; 15+2-158 *&8&:51/& 519 +- ).&*&1/&4 21 +%& +2=& 4)51 -:&* 6%2/%


38/151

"`

+%& *&4&5*/% 24 *&8&:51+ 519 /2+&9 .+.*& )&*>-*=51/&A Pa89Q 5/%2&:&=&1+4 /51 :2&6@ 5/%2&:&=&1+4 =59& 21 +%&

924+51+ )54+ =5; 1-+ 2&894 -> 21:&4+205+2-1A

,$:1)*$'/ (7 *&4&5*/% 544&44=&1+ )*-/&44&4 5+ +%& .12:&*42+; 519 15+2-158 8&:&8@

>-/.4210 -1 92>>&*&1+ 214+2+.+2-158 -* /-012+2:& .12+4A $%& 9&>212+2-1 -> +%& .12+ -> 544&44=&1+

9&)&194 .)-1 +%& - +%& 544&44=&1+A

$%&*& 5*& 4.>&*&1/&4 *&4&5*/% -* 544&44=&1+A B1 +%& 82>& -* )%;42/58 4/2&1/&4@ +%& ; 4/2&1+2>2/?4/%-85*8; *&4&5*/% 5//-*9210 +- >2&89 -> 4/2&1/&A

$5


39/151

"c

,&1&*58 U*-0*5==&4

'9./5+2-1

d.=512+2&4 519 J*+4

N-/258 N/2&1/&4@ L.421&44 519 e56

N/2&1/&@ b5+%&=5+2/4 519 R-=).+210

'1021&&*210@ b51.>5/+.*210 519 R-14+*./+2-1

J0*2/.8+.*& 519 f&+&*215*; N/2&1/&4

d&58+% 519 _&8>5*&

N&*:2/&4

B1 5992+2-1@ .12:&*42+2&4 -*05124& +%&=4&8:&4 92>>&*&1+8; >-* 5 /-=)8&( 4&+ -> *&54-14@

21/8.9210 %24+-*;@ =2442-1@ >2151/&@ 58201=&1+ 62+% 15+2-158 -* *&02-158 )*2-*2+2&4@

21+&*924/2)8215*; +%&=5+2/4@ &+/A J//-*92108;@ 2+ 24 ->+&1 92>>2/.8+ +- 42=)8; /-=)5*& -*

-*=51/& -1 +%& 9&)5*+=&1+4 -* >5/.8+2&4A

$%& '()&*+ ,*-.) %54 59-)+&9 +%& /-1/&)+ -> P71-68&90& /8.4+&*Q 54 +%&

544&44=&1+A J +;)2/58 71-68&90& /8.4+&* /-=)*24&4 5 0*-.) -> *&4&5*/%&*4 4%5*210 5

/-==-1 >2&89 -> 21:&4+205+2-1A B+4 4)&/2>2/ /-=)-42+2-1 24 9&>21&9 54 +%& =-4+ 5))*-)*25+&

>2&89 -> 4+.9; >-* +%& 4)&/2>2/ ).*)-4& -> +%& *&4&5*/% 544&44=&1+ &(&*/24&A B1 4-=& /54&4@

+%& 71-68&90& /8.4+&* =5; 2&89 -> 4/2&1/&@ 528 +- *&/-0124& 1&6 -* &=&*0210 *&4&5*/% >2&894A G-* &(5=)8&@ +%& 71-68&90& /8.4+&*

=5; *&)*&4&1+ 5 1&6 -* 21+&*?924/2)8215*; 244.&?9*2:&1 /8.4+&* -* 51 21+&*?214+2+.+2-158

1&+6-*7A

$%& .12+ -> 544&44=&1+ 4%-.89 5865;4 588-6 >-* 500*&05+2-1 +- +%& 9&)5*+=&1+58 -*

214+2+.+2-158 8&:&8 54 *&C.2*&9A $%24 24 &4)&/2588; 2=)-*+51+ >-* 21+&*15+2-158 /-=)5*5&*&1/& )5)&*4X@ S]] +*59&

).*-= # )5+&1+ ->>2/&4A

$%& 7&; 2=)&92=&1+ +- &*&1+ 924/2)821&4 )*-9./& 92>>&*&1++;)&4 -> *&4&5*/% -.+).+4 6%2/% 5*& 1-+ 588 &5428; *&/-*9&9A L--74@


40/151

O]

).&*&&9 /-1>&*&1/& )5)&*4 5*& 1-+ &5428; /-=)28&9 1-* *&/-0124&9 54 &C.2:58&1+A

R*-44?924/2)8215*; 519 /-885>2/.8+ +- /5+&0-*24&A N&& 4&/+2-14 "A"

519 OAF 5-* >.*+%&* 924/.442-1 -> +%&4& 244.&4A

+"# 011) +%& 71-68&90& /8.4+&*@ 21

-*9&* +- &:58.5+& +%& =&+%-9-8-02/58 4-.191&44 -> +%& *&4&5*/% 519 +%& W)-+&1+258X

42012>2/51/& -> 2+4 /-1+*282/+ -> B1+&*&4+D&4)&/+ >-* d.=51 N.-* J12=58 N.


41/151

OI

J44&44=&1+ -> D247 519 L&1&>2+4D&4&5*/% 4%-.89 >-88-6 P0--9 )*5/+2/&Q 62+% *&4)&/+ +- 5:-29210 )85025*24=@ 5//.*5+&8;

9-/.=&1+210 *&4.8+4@ 4&/.*&8; 4+-*210 )*2=5*; 95+5@ 5/71-68&90210 +%& *-8& -> /-885&442-158 -*=58 &+%2/58 5))*-:58 -88-6 0.29&821&4 &4+5


42/151

OF

G(%$&. F1"17$'*@ &A0A 2=)*-:210 )&-)8&Q4 %&58+% 519 C.582+; -> 82>&M 4+2=.85+210 1&65))*-5/%&4 +- 4-/258 244.&4M /%510&4 21 /-==.12+; 5++2+.9&4M 21>8.&1/& .)-1 9&:&8-)=&1+4

-* C.&4+2-14 21 4-/2&+; 5+ 85*0&M 21>-*=&9 ).&+; 519 4&/.*2+;M 2=)*-:&9 4-/258 5++5/%=&1+M 519 2=)*-:&=&1+4 21 +%& 8&:&8

519 4&/.*2+; -> )-82+2/58 *20%+4A

E":$)("51"'&. F1"17$'*@ &A0J 2=)*-:&=&1+4 21 &1:2*-1=&1+ 519 82>&4+;8&M *&9./&9 654+&519 )-88.+2-1M 2=)*-:&9 =5150&=&1+ -> 15+.*58 *&4-.*/&4M *&9./&9 /-14.=)+2-1 -> >-4428

>.&84M .)+57& -> *&/;/8210 +&/%12C.&4M *&9./&9 &1:2*-1=&1+58 *247M )*&4&*:5+2-1 212+25+2:&4M

/-14&*:5+2-1 -> &/-4;4+&= 4&*:2/&4M 2=)*-:&9 )851+ 519

512=58 :5*2&+2&4M 519 595)+5+2-1 +- /82=5+& /%510&A

D4.'4)&. F1"17$'*= *>?>4.))-*+210 0*&5+&* .19&*4+519210 -> 6%&*& 6& %5:& /-=& >*-=@ 5196%- 519 6%5+ 6& 5*& 54 5 15+2-1 519 4-/2&+;M .19&*4+519210 %-6 6& *&85+& +- -+%&*

4-/2&+2&4 519 /.8+.*&4M 4+2=.85+210 /*&5+2:2+; 62+%21 +%& /-==.12+;M /-1+*2


43/151

"#

"#$%& ' ()&*)+&, -. /01+2#3-*45 #01 6-7& 8-4+3+)& 9:*-;:-3&03+#%+3+&4< #01 =&>#3+)& 92-0;%+7+3#3+-04< ?@*&4

/=A/BC"(D6 AE6BD/8"/(= 8D(;8("E="/CF/"/E6 B(=;F/G/"C"/(=6 HIC" AEJEF(8GE=" /6

DEKL/DEA

DE6ECDBI 8D(ALB"/J/"M

D&4*2N :@$%+2#3+-04 #01

-@3:@34

$ %&'() &* +',-.%/).&(0 /(1 &)234

4303/4%2 &')+')05

63+3(1.(7 &( +'4+&03 &(-8

03-3%)31 )8+30 &*

+',-.%/).&(0 %/( ,3

%&'()315

9',-.02.(7 .0 :.)/- *&4

+4&74300 .( 0%.3(%3

0%2&-/402.+5

6.**343() 1.0%.+-.(30 +4&1'%3

1.**343() )8+30 &* 4303/4%2

&')+')05

;'.)/,-3 1/)/ ,/030 *&4 / :/4.3)8 &*

1.0%.+-.(30 /(1 4303/4%2 43-/)31 &')+')0?

30+3%./--8 .( 0&%./- 0%.3(%30 /(1

2'


44/151

""


DEKL/DEA

+',-.%/).&(0 ,') /-0& )23.4 43*343(%3

-.0)05 94.(%.+/- .(13M30 /43 N3, &*

>%.3(%3? >%&+'0 /(1 Q&&7-3 >%2&-/45

/00300


45/151

"U


DEKL/DEA

/==(JC"/(= C=A 6(B/CF WE=E?/"

EX3&*0#% *&4*2N +02-7&

5

V3:3- &* *'(1.(7 /))4/%)31 ,8

4303/4%2340 /(1 '(.:340.).30 *4&&0&*#3&1 +03&%%&23@#% :*-:&*3U

9/8+-.4% )&&* *&82&E 1/3

0-421; 25)14+9

8%/7"$.4 A;; *&0&1*4% 100&005&/+ &(&*420&0 125 +- 25)*-8& *&0&1*4% )&*6-*51/4& 1/3O.1;2+:9 $%& IN 0:0+&5 20 32*&4+;: +2&3 +- *&0-.*4& 1;;-41+2-/@ E%2;& +%& Q&+%&*;1/30 0:0+&5

20 .0&3 )*251*2;: +- 25)*-8& O.1;2+:9 $%& F*&/4% 0:0+&5 *&)*&0&/+0 1/ 2/+&*&0+2/7 52(+.*& -6+%& +E-9

:.&;


52/151

MJ

$%& -)+25.5 5&+%-3 -6 *&0&1*4% 100&005&/+ 20 +- 4-5.0&* *&82&E9 $%&*& 1*& 0&8&*1;

1381/+17&0 +- +%20 1))*-14%G

=/3241+-*0 )*-823& )&&* &()&*+0 E2+% 4-/3&/0&3@ 0:0+&51+24@ 8&*262&3 1/3 S-


53/151

ML

C(D&. E

F%&')1)>.#$)"#(& -.$.(/0, @$$.$$>.#' F('/)G

$71$89:1:9:#1;?@?>A

B7#C?>A #D=

9;

?DD8@#>?8D #D=

98;?#C F:D:+?>

979>#?D#F?C?>A

#D= 9;#C:

1:9:#1;17;>71:

#GG&2,5) 1)(&H%2)(

D&0&1*4%

-.+).+d


54/151

M"

9A H"% +(#' '" %$. /.$.(/0, ($$.$$>.#' '" @IIJ!@C= -=:J2-!=:K +%&/ E%1+ 20 *&O.2*&3 20GD&7.;1* 41310+*1; 4&/0.0 -6 *&0&1*4% -.+).+0

]&&* *&82&E +- 3&+&*52/& O.1;2+: 1/3d-* 42+1+2-/ 31+1 +- 3&+&*52/& 25)14+

\-5& 5&10.*& -6 *&0&1*4% 2/6*10+*.4+.*& d &/82*-/5&/+ a &979 b2.#' '" 9N!-=@:= -=O9JN@IP!JFF2N9CQ=NO@O=F=NC@ +%&/ E%1+ 20 *&O.2*&3 20G

K1+1 -/ 4--)&*1+2-/ 17*&&5&/+0 E2+% ;-41; 7-8&*/5&/+0 1/3 -*71/201+2-/0 -6 +%& *&72-/?

K1+1 -/ 17*&&5&/+0 E2+% -+%&* ).


55/151

MM

=/3241+-*0 -6 4-55&*421;201+2-/ -6 =]?

=/3241+-*0 -6 0-421;@ &4-/-524@ 4.;+.*1; 1/3 &/82*-/5&/+1; 25)14+ 1/3


56/151

MW

O;&42GH(I&4B"# I)>)'(')"#$ (#1 2#)#'.#1.1 !"#$.R%.#0.$

I/28&*02+:>


57/151

M_

.02/7 *&0-.*4& 100&005&/+ +- 3*28& 3266&*&/+21+2-/@ 2+ 527%+ )*-62;2/79

F2/1;;:@ 2/ *&4&/+ :&1*0@ 4-/4&*/ %10


58/151

Mi

+%&2* 25);241+2-/0 6-* 042&/+2624>04%-;1*;: )*14+24& 1/3 +%&2* &4-/-524 10)&4+0 a /&&3 +- 7"/'(#' 1);;./.#0.$ (0/"$$ /.$.(/0, 1)$0)7&)#.$9 ]&&*>*&82&E&3 `-.*/1;1*+24;&0 1*& +%& )*251*: ).


59/151

M#

'/4-.*17&0 1 +%-*-.7% 100&005&/+ -/ 1 *&7.;1*


60/151

WR

P#'')4*IQ ?R #25IJI5I)( ,4* E)-S)%(3I' &0 :Q')%5 T%&H' &4 #(()((-)45 &074IJ)%(I5MNF,()* 1)(),%23

$%& 5&5MZ@ 1/3 %104--*32/1+&3 1 +-+1; -6 &27%+ 'I 3&8&;-)5&/+ 1/3 /&+E-*C )*-`&4+0@ 15-/7 +%&5 628&

+%&51+24 1/3 3200&52/1+2-/ /&+E-*C )*-`&4+0 2/ +%& 1*&1 -6 ;1/7.17&0 Y\-4*1+&0>'*105.0

]*-7*155&? H##W>JRR_Z9 e8&* +%& )10+ HL :&1*0@ %& %10 B10&3 D&0&1*4% '()&*+ ,*-.)@ 1/3 %& E10 /,55+/$("/

-6 +%& '.*-)&1/ P-552002-/T0 ^27% b&8&; ,*-.) -/ V.;+2;2/7.1;2059 [-;671/7 V14C2&E24c

0+.32&3 '/7;20% 1/3 ,&*51/ 1+ FIB 1/3 1+ +%& I/28&*02+: -6 b&&30@ 1/3 E*-+& %20 ]%K +%&020

-/ K1/2&; K&6-&T0 SD-


61/151

WH

^Aj'bNeDQ ';;&/ YD1))-*+&.*Z9 $%&0)((&% 512;G

';;&/9^1c&;C-*/m32+92&

B'D,^eFF@ \-/`1 YD1))-*+&.*Z 9&4Y, 20 1 )*-`&4+ 51/17&* 1+ +%& ,&*51/ P^' P&/+*& 6-*

^27%&* '3.41+2-/ K&8&;-)5&/+9 \%& 20 *&0)-/02


62/151

WJ

BeDD'bb>KAV=AQ@ b23219 CI*I,`-2/&3 'IA 2/ h1/.1*: JRRW 1/3 %10 2/3.0+*: 4-;;1H##i9 \%& E10 1 )-0+>3-4+-*1; *&0&1*4%&* 2/ +%&

I\ YQ-*+% P1*-;2/1 \+1+& I/28&*02+:Z 1/3 P1/131 Y$%& I/28&*02+: -6 [&0+&*/ e/+1*2-Z@ 6*-5

H##_ +- JRRR9 B14C 2/ B1*4&;-/1@ 0%& E-*C&3 6-* 1 )*281+& 4%&5241; 4-5)1/: 10 3&).+:

32*&4+-* 6-* +%& DfK 3&)1*+5&/+9 A6+&* +E- :&1*0 -6 P%12* -6 +%20 512;G )%&5);2+m.;


63/151

WL

)&*6-*51/4& -6 +%& 3266&*&/+ 14+-*0@ 512;G ,-*1/9V1*C;./3mU=QQeUA90&

Ve'K@ ^&/C9


64/151

W"

-6 I))01;1 I/28&*02+:@ +%& '81;.1+2-/ P-552++&& 6-* '0+-/21/ K&)1*+5&/+0 -6 D.0021/

]%2;-;-7: 1/3 +%& =/+&*/1+2-/1; '81;.1+2-/ ]1/&; 6-* +%& F14.;+: -6 A*+0 -6 h:8k0C:;k

I/28&*02+:9 ^& %10 H##i 1/3 4%12*&3 +%& D&0&1*4%

P-./42; 6-* P.;+.*& 1/3 \-42&+: 1+ +%& A413&5: -6 F2/;1/3 JRRH>JRRW9 ^&

4.;+.*1; 2/+&*14+2-/ 1/3 5-3&;;2/7 -6 Y520Z4-55./241+2-/9 ^& %10 ).P%1/4&;;-* -6 +%&

I/28&*02+: -6 \-.+%15)+-/ 6*-5 H##" +- JRRH 1/3 E10 )*&82-.0;: P%12*51/ 1/3 P%2&6

'(&4.+28& -6 +%& '4-/-524 1/3 \-421; D&0&1*4% P-./42; Y'\DPZ9 F*-5 H### +- JRRH@ \2*

^-E1*3 E10 ]*&023&/+ -6 I/28&*02+2&0 IN@ +%& IN JRRJ9 ^& E10 513& 1 PB' 2/ H##M 6-* 0&*824&0 +- 0-421; 042&/4& 1/3 1 C/27%+%--3 2/

JRRR 6-* 0&*824&0 +- %27%&* &3.41+2-/9 \2* ^-E1*3 E10 6-*5&*;: 1 ]*-6&00-* -6 \-42-;-7: 2/

512;G A;12/9]&:*1.


65/151

WM

2+0 3&8&;-)5&/+1; )%10&9 \.&T0 1413&524 &()&*+20& 20 2/ 4-/+&5)-*1*: A.0+*1;21/ 820.1; 1*+0

1/3 3&027/9 \%&


66/151

WW

,-8&*/5&/+ 1+ +%& P-552++&& 6-* \42&/+2624 1/3 $&4%/-;-7241; ]-;24: -6 +%& e'PK 6-* HM

:&1*09 \%& E10 +%& P%12*)&*0-/ -6 +%20 P-552++&& JRR"Z@ 0&4-/3@ 10 1 5&5512;G5)1;-51901/4%&cm.159&0


67/151

W_

\#'')4*IQ ??R TG&((,%M>)%- =)0I4I5I&4

A));2&3 D&0&1*4%

A));2&3 *&0&1*4% 20 125&3 1+ 14O.2*2/7 C/-E;&37& -* ./3&*0+1/32/7 +-

5&&+ 1 0)&42624@ *&4-7/2c&3 /&&39 ,&/&*1; &(15);&0 -6 1));2&3

*&0&1*4% E-.;3 2/4;.3& .02/7


68/151

Wi

V.;+2> @ P*-00>@ $*1/0

1/3 =/+&*>

32042);2/1*: D&0&1*4%

D&0&1*4% +%1+ &5);-:0 +%& C/-E;&37& 0+*.4+.*&0 1/3 4%1*14+&*20+24

*--5 )*14+24&9 =+

2/8-;8&0 1/ 2/+&7*1+2-/ -6 +%&-*: 1/3 )*14+24& +- &/%1/4&

./3&*0+1/32/7 -6 +%& 200.&09 =+ 20 4-55-/;: 100-421+&3 E2+% 5&3241;@

)0:4%-;-7241; -* &3.41+2-/1; 042&/4&0@ -* +%& 4*&1+28& 1*+0 1/3 5&3219

]*-3.4+282+:

$%& *&;1+2-/0%2) *&82&E&3

].


69/151

W#

]#'')4*IQ ???R =)(2%I'5I&4 &0 ?4*I2,5&%(6 BH,GI5,5IJ) ,4* BH,45I5,5IJ)

G"# -=:=@-!M 82?I9!@C9JN: @N5 J2C82C:=)(2%I'5I&46 >&5,G ;&H45 &0 53) DH-S)% &0 $HSGI2,5I&4( ,4* 853)% 8H5'H5(

e.+).+0 1*& /-*51;;:


70/151

_R

6-4.0@ 1/3 0&;&4+ -/;: 0-5& 10)&4+0 -6 *&0&1*4% 14+282+: +- .0&*0? *&0&1*4% 4-/+*2&4-/-524


71/151

_H

G": 9N59!@CJ-: JA T2@I9CQ @N5 :!MJI@-IQ 9F8@!C]R^R_ ;?>#>?8D9

=)(2%I'5I&46 ;I5,5I&4( =,5, =)%IJ)* 0%&- 95,4*,%* FISGI&-)5%I2 E),(H%)(

P2+1+2-/ 4-./+0 1*& 51/26&0+1+2-/0 -6 2/+&;;&4+.1; 2/6;.&/4&@ 10 *&)*&0&/+&3


72/151

_J

(U C,. V"%/#(& )>7(0' ;(0'"/

$%& 25)14+ 614+-* -6 1 `-.*/1; 2/ 1 )1*+24.;1* :&1* $ 20 3&62/&3 10 +%& /.5


73/151

_L

G"; 82?I9!@C9JN: 9N CJ8&'N

1,4/)*X


74/151

_"

'32+-*21; 1/3 *&6&*&&2/7 0&*824&0 6-* )*&0+272-.0 /1+2-/1; 1/3 2/+&*/1+2-/1;

`-.*/1;0d).


75/151

_M

]R"R_ =8;>81#C ;8E$C:>?8D9=)(2%I'5I&46 DH-S)% &0 $3= ,4* :aHIJ,G)45 1)(),%23 =&25&%,5)( ,4*X ,( ,''%&'%I,5)X

1)(),%23 E,(5)%( =)K%)) ;&-'G)5I&4(

I/28&*02+2&0 1/3@ 2/ 0-5& 410&0@ 7-8&*/5&/+ 17&/42&0@ 4-;;&4+ 31+1 6-* +%20 2/3241+-*9$%&

$%& E23&0)*&13 144&)+1/4& -6 +%& 25)-*+1/4& -6 *&0&1*4% +*12/2/7 20 +%& C&: ;&72+251+2/7

614+-*9 ]%K 4-5);&+2-/0 1*& *&71*3&3 2/+&*/1+2-/1;;: 10 1 4-55-/ 0+1/31*3


76/151

_W

$%20 2/3241+-* 20 .0&6.; +- 3*28& 25)*-8&5&/+ 2/ *&0&1*4% )&*6-*51/4& 2/ ./28&*02+2&0 1/3 2/

2/+&*/1; ./2+0 Y614.;+2&0@ 04%--;0@ 3&)1*+5&/+0Z9 I0&3 10 1 )*-)-*+2-/ -6 +-+1; 1413&524 0+166@

+%20 2/3241+-* 41/ 10020+ ./28&*02+2&0 2/


77/151

__

;&4LCI-I5,5I&4(

P->1.+%-*0%2) .0.1;;: 5&10.*&0 S6-*51;T 4-;;1


78/151

_i

b&8&;0 -6 &(+&*/1; 6./32/7 81*: 7*&1+;: 14*-00 32042);2/&09 F./32/7 -))-*+./2+2&0 1*& ;&00

6*&O.&/+ 1/3 1E1*30 1*& 027/26241/+;: ;-E&* 2/ +%& 1*+0@ %.51/2+2&0@ 1/3 0-421; 042&/4&0 +%1/

2/ +%& 042&/4&0 =)(2%I'5I&46 >&5,G ?4J)(5-)45 I4 74IJ)%(I5MNF,()* 1)(),%23 0%&- #GG 9&H%2)(

$%20 2/3241+-* 2/4;.3&0 1;; ./28&*02+: 1;;-41+2-/0 Y&979 2/8&0+5&/+Z 2/ *&0&1*4% 1;;-41+&3 6*-5

+%& 7-8&*/5&/+ &1*/&3 2/4-5&@ &979 2/+&*/1+2-/1;

0+.3&/+ 6&&0@ +*12/2/7@ 4-55&*421;201+2-/ 1/3 )%2;1/+%*-)24 3-/1+2-/09 I/28&*02+2&0 4-;;&4+

31+1 6-* +%20 2/3241+-* 1/3 0&;6>/-52/1+& ;&8&;0 -6 6./32/7 6-* *&0&1*4% 6*-5 4-/0-;231+&3

*&8&/.&09

$%&

=/8&0+5&/+ 2/ *&0&1*4% 20 1*7.1


79/151

_#

]R]Rh 1:9:#1;< ?D+1#9>17;>71:=)(2%I'5I&46 >3I( I4*I2,5&% -),(H%)( S,(I2 0,2IGI5I)( 53,5 (H''&%5 %)(),%23R

=/6*10+*.4+.*& 2/4;.3&0@ 2/+&* 1;21@ +%& ;2


80/151

iR

]1+&/+0 1*& 1 8&*: )--* 2/3241+-*9 $%& /.5


81/151

iH

$%20 20 1/ 25)-*+1/+ 2/3241+-* 6-* 5&10.*2/7 04%-;1*;: 2/8-;8&5&/+ E2+% -+%&* *&0&1*4%&*0

1/3 2/ +.*/ +%& &(+&/+ +- E%24% *&0&1*4%&*0 1*& *&4-7/20&3


82/151

iJ

K1+1 20 8&*2621


83/151

iL

_i #'')4*IQ ?@R ;,() 95H*I)( &0 53) 1)(),%23 #(()((-)45 :Q')%I)42)


84/151

i"

#$"#@2:C-@I9@:Q)2H5IJ) 9H--,%M

A.0+*1;21T0 '(4&;;&/4& 2/ D&0&1*4% 6-* A.0+*1;21 Y'DAZ E2;; 100&00 *&0&1*4% O.1;2+: E2+%2/

%27%&* &3.41+2-/ 2/0+2+.+2-/0 .02/7 1 4-5


85/151

iM

0.))-*+ 6-* &1*;: 41*&&* *&0&1*4%&*0 1/3 )-0+7*13.1+& *&0&1*4% 0+.3&/+0? 1/3

&823&/4& -6 04%-;1*;: &0+&&5 1/3 *&).+1+2-/9

42+1+2-/0@ /.5


86/151

iW

'DA 4-5


87/151

i_

2/4-5&@ ).


88/151

ii

#$":2N9L=-:9CZ I9?-= 5= ?-2[=II=: \ ?-2[=II=: \ ?=IO92F:Q)2H5IJ) 9H--,%M

=/ B&;72.5@ 100&005&/+ -6 ./28&*02+:>0)&42624 *&0&1*4% +&150 Y/-+ 2/32823.1;0Z

J%'(


89/151

i#

H9J9 ]*&0&/+1+2-/ -6 +%& 5-0+ 25)-*+1/+ *&0&1*4% *&0.;+0 YJ )17&0 51(Z

H9L9 e.+;--C -/ 6.+.*& *&0&1*4% 14+282+2&0 &/82017&3 YH )17& 51(9Z

H9"9 \[e$ 1/1;:020 YJ )17&0 51(9Z

H9M9 \42&/+2624 1/3d-* 0-42&+1; 25)14+ -6 +%& *&0&1*4% 14+282+: d 14+282+2&0 ./3&*+1C&/

YJ )17&0 51(9Z

H9W9 \%-*+ PU0 YH )17& 51(9Z -6 0&/2-* 042&/+20+0 Y)&*51/&/+ 1413&524 0+166Z@ 52002-/ 14+282+2&0

+%& 3-4.5&/+0 )*&)1*&3


90/151

#R

H9"9 P-/4;.02-/0 -6 +%& 5&5


91/151

#H

#$";A9NI@N5 ]@@ICJ 2N9L=-:9CQU:Q)2H5IJ) 9H--,%M

$%& *&0&1*4% &81;.1+2-/ 2/4;.3&3 0-5& 2//-81+28& E1:0 -6 .02/7 +%& )&&*>*&82&E 5&+%-39

$%& )1/&;0 E&*& 10C&3 +- 728& +%&2* E*2++&/ 0+1+&5&/+0 1/3 O.1/+2+1+28& 7*13&0 /-+ -/;: -/*&0&1*4% O.1;2+:@ 04%-;1*;: 25)14+@ 0-42&+1; 25)14+@ +%& *&0&1*4%

&/82*-/5&/+ 1/3 +%& 6.+.*& )-+&/+21; -6 +%& ./2+ 2/ O.&0+2-/9

D,-)L>I5G) &0 1)(),%23 #(()((-)45 :Q)%2I()6

D&0&1*4% '81;.1+2-/ -6 A1;+- I/28&*02+: JRR# YAAb$e '81;.1+2-/Z

$&GI2M ;&45)Q5X I42GR 2I%2H-(5,42)( H4*)% .3I23 53) )Q)%2I() 2,-) ,S&H56

$%& ^&;02/C2 I/28&*02+: -6 $&4%/-;-7: Y$NNZ@ +%& ^&;02/C2 \4%--; -6 '4-/-5240 Y^\'Z@ 1/3 +%&

I/28&*02+: -6 A*+ 1/3 K&027/ ^&;02/C2 Y$12NZ 1*& 2/ +%& )*-4&00 -6 5&*72/79 $%& /&E ./28&*02+:

E2;; 4;100

*&0&1*4% ./28&*02+:9 A1;+- I/28&*02+: 20 +-


92/151

#J

\42&/+2624 =5)14+9 \.4% O.&0+2-/0 10G A*& *&0&1*4% -.+).+0 -6 +%& I/2+ ).

&81;.1+2-/0@ &0)&421;;: 2/ 1/0E&*2/7 O.&0+2-/0 *&71*32/7 200.&0 E%24% 1*& /-+ .0.1;;: *120&3

2/ *&0&1*4% 100&005&/+0 Y0-42&+1; 25)14+@ *&0&1*4% &/82*-/5&/+@ 6.+.*& )-+&/+21;Z9 e/& 51:@

%-E&8&*@ %-)& +%1+ 4-5)2;2/7 +%& 3-4.5&/+0 .*7&3 +%&5 +- +%2/C 1


93/151

#L

#$"!A9NI@N5 ]M=I:9NW9 2N9L=-:9CQU:Q)2H5IJ) 9H--,%M

$%& 410& 0+.3: 3&04*2


94/151

#"

&81;.1+&3Z9

A/ 1332+2-/1; +1*7&+ -6 100&005&/+G 4-55&/+0 YE2+%-.+ 7*13&0Z E&*& 1;0- *&O.&0+&3 -/ +%&

3&)1*+5&/+0T 14+282+2&0 2/ 2/+&*14+2-/ E2+% 0-42&+:9

V&+%-3-;-7:G

=/ +%& ;1+&0+ DA'@ &14% 3&)1*+5&/+ )*-823&3 1 ;1*7& 0&;6>&81;.1+2-/ *&)-*+ 6-;;-E2/7 0+*24+

7.23&;2/&0G ;20+ -6 *&0&1*4%>14+28& 0+166@ 3&04*2)+2-/ -6 +%& 3&)1*+5&/+T0 *&0&1*4% )*-62;& YJ

)17&0 51(9Z@ 4-5);&+& ;20+ -6 ).


95/151

#M

%++)GddEEE9%&;02/C2962d*&0&1*4%JRRMd&/7;20%d2/3&(9%+5

?45)4*)* ,4* 74I45)4*)* ;&4()aH)42)(6

$%&*& 20 /- 3&/:2/7 +%1+ +%&*& 51:


96/151

#W

#$",A-@N!=:Q)2H5IJ) 9H--,%M

$%& 410& 0+.3: 3&04*2


97/151

#_

X.1;2+: -6 *&0&1*4% Y512/;:


98/151

#i

8S()%J,5I&4( L#**I5I&4,G ;&--)45(6


99/151

##

#$"BO=-F@NQ < AJ-:!M2NO:-@C9NO ]!JN52!C=5 ?Q^9::=N:!M@AC:-@C

:Q)2H5IJ) 9H--,%M

$%& ,&*51/ \42&/4& P-./42; D1+2/7 41**2&3 -.+ 1 )2;-+ 0+.3:

02+& 8202+09 =/ 1 5.;+2>0+&) 100&005&/+ )*-4&00@ +%& 2/0+2+.+2-/0 E&*& 62*0+ &81;.1+&3


100/151

HRR

F-* +%& 62*0+ +25&@ ./28&*02+2&0 1/3 /-/>./28&*02+: *&0&1*4% 2/0+2+.+2-/0 2/ ,&*51/: 1*&

&81;.1+&3 2/ 1 02/7;&@ 4-5)*&%&/028& &(&*420&9

=/ 1 5.;+2>0+&) 100&005&/+ )*-4&00@ +%& 2/0+2+.+2-/0 E&*& 62*0+ &81;.1+&3 ]*-5-+2-/ -6 :-./7 *&0&1*4%&*0

> N/-E;&37& +*1/06&* Y$*1/06&* +- 0-42&+:@ C/-E;&37& 320)&*02-/Z

=/3241+-*0G

X.1;2+: -6 *&0&1*4%G P2+1+2-/0 )&* )1)&*@ *1E 1/3 0+1/31*320&3? /.5)-+&/+21; )*2/42);&? 3266&*&/+ 31+1


101/151

HRH

?45)4*)* ,4* 74I45)4*)* ;&4()aH)42)(6

8S()%J,5I&4( L#**I5I&4,G ;&--)45(6

e/;: +%& )&&* &0+&&50 6-* +%& 628& 325&/02-/0 E&*& ).


102/151

HRJ

#$"NO=-F@NQ < !M= 2N9L=-:9CQ-@NW9NOK !M= -=:=@-!M-@NW9NO:Q)2H5IJ) 9H--,%M

P^' D1/C2/70 20 1 32042);2/&>0)&42624@ 5.;+2325&/02-/1; 0:0+&5 125&3 1+ )*-8232/7

2/6-*51+2-/ 6-* 0+.3&/+09 =+0 512/ &5)%1020 20 -/ +&14%2/7@ E%&*& /1+2-/1; 4-5)1*20-/0 1*&513&9 V-0+ )*-0)&4+28& 0+.3&/+0 .0& 2+ 10 -/& 0-.*4& -6 2/6-*51+2-/ E%&/ 3&4232/7 -/

E%&*& +- 0+.3:9 P^' %10 1/-+%&*@ *&0&1*4%>6-4.0&3 0:0+&5@ E%24% 20 /-+ 4-5)*&%&/028&@ )*-62+ -*71/201+2-/9 =+ E10 6-./3&3 2/

H##" )*-62+

;252+&3 4-5)1/:9 =+ 3&423&0 2/3&)&/3&/+;: -/ +%& 4%-24& -6 2+0 +-)240 1/3 )*-`&4+0 10 E&;; 10

-/ +%&2* *&1;201+2-/9

$&GI2M 8SY)25IJ)b(`6

$%& P^' I/28&*02+: D1/C2/7 20 3&027/&3 +- %&;) )*-0)&4+28& 0+.3&/+0 51C& 1/ 2/6-*5&34%-24& -6 0+.3: )*-7*15 1/3 ./28&*02+:@ E%2;& +%& P^' D&0&1*4% D1/C2/7 -6 ,&*51/

I/28&*02+2&0 133*&00&0 042&/+2624 4-55./2+2&0 -* ./28&*02+2&0 $%& ;1++&* 20 3&027/&3 +-

23&/+26: *&0&1*4%>14+28& 3&)1*+5&/+0 1+ ,&*51/ ./28&*02+2&0 2/ 0)&42624 32042);2/&09 =+ 51:

1;0-


103/151

HRL

=/3241+-*0 .0&3 6-* *&0&1*4% 100&005&/+G

A5-./+ -6 +%2*3 0+*&15 6./32/7 Y)&* *&0&1*4%&*ZG K266&*&/+ 0-.*4&0 -6 +%2*3 0+*&15

6./32/7 51:


104/151

HR"

E&


105/151

HRM

#$"OO=-F@NQ < 9N9C9@C9L= AJ- =[!=II=N!=:Q)2H5IJ) 9H--,%M

$%& =/2+21+28& 6-* '(4&;;&/4& %10 1*-.0&3 0-5& 4-/+*-8&*0: 2/ ,&*51/:


106/151

HRW

0&;&4+2-/ *-./39 $%& ./28&*02+2&0 4%-0&/ 6*-5 +%20 0+17& 0.


107/151

HR_

#$"GM2NO@-Q:Q)2H5IJ) 9H--,%M

$%& *&0&1*4% 100&005&/+ *&;1+&3 &(&*420& 20 :&1* 4-/+*14+ 2/ JRR_ +%1+ 20 +- 4-/+12/ 1 3&+12;&3 ;20+ -6 5&10.*&0 6-* 5-/2+-*2/7

1/3 100&005&/+9 $%& V2/20+*: -6 '3.41+2-/ 1/3 P.;+.*& 10 V12/+12/&* 0)&4262&3 +%&

-%&1*+&3 *&815)2/7 -6 %27%&* &3.41+2-/1; 7-8&*/1/4& 0+*.4+.*&0 ;&18&0 );&/+: -6 )-+&/+21;

6-* 6.*+%&* 25)*-8&5&/+0 1/3 ;&720;1+2-/9 $%1+ 20 +%& *&10-/ E%: +%& V2/20+*:@ +%1+ %10 1 C&:

*-;& 2/ 62/1/42/7 -6 %27%&* &3.41+2-/ 2/+*-3.4&3 +%& V12/+12/&* A7*&&5&/+ 0:0+&59 $%&

V2/20+*: E20%&0 +- %&;) +%& 2/0+2+.+&0 +- &;1


108/151

HRi

YHZ $%& V2/20+*: 10 512/+12/&* 41/ 5-/2+-* Y1/3 100&00Z 41)1

:&1* 4-/+*14+ 1+ +%& &/3 -6 JRR_ +%1+ 20 +- 4-/+12/ 1 3&+12;&3 ;20+ -6 5&10.*&0 6-* 5-/2+-*2/7

1/3 100&005&/+9 $%& V2/20+*: -6 '3.41+2-/ 1/3 P.;+.*& 10 V12/+12/&* 0)&4262&3 +%&-


109/151

HR#

B-+%@ +%& ;20+ -6 2/3241+28& 2/3241+-*0 1/3 0&;&4+&3 2/3241+-*0 6-* 5-/2+-*2/7 1/3 100&005&/+

31+& 8&*02-/ -6 [&0+ '.*-)&1/ )*14+24&09


110/151


111/151


112/151

HHJ

%10 2/+*-3.4&3 1 /&E 4-5)&+2+28& 3:/1524 2/+- +%& 0:0+&5 3&0)2+& 4-/4&*/ 1;&8&; =/0+2+.+2-/0 Y]D$b=Z 1/3 \42&/4& F-./31+2-/ =*&;1/3 Y\F=Z9

J1( 4")&,H J%'(* >)%?(/*%$H P"%&(20 /-+ 1 027/26241/+ 01;&0 3*28&*@ +:)241;;: -/;: 1332/7 ;&00

+%1/ L| -/ H9"5 01;&09 KUK0 &+4 E-.;3 3- 5.4%


113/151

HHL

#$"##9C@IQ:Q)2H5IJ) 9H--,%M

=/ =+1;: +%& &81;.1+2-/ -6 ./28&*02+:>JRRL )&*2-3 %10


114/151

HH"

K&)1*+5&/+0 E&*& 10C&3 +- 23&/+26: 1 051;; /.5WR|@

I/01+20614+-*:

JRRWZ@ 1;+%-.7% +%& P=UD E10 *&13: +- +1C& +%& 51/31+&9

D1+%&*@ 2/ JRR_ +%& /&E ,-8&*/5&/+ YV2/20+&* -6 D&0&1*4% V.002Z 1))*-8&3 +%& 4*&1+2-/ -6 1 /&EA7&/4:@ 41;;&3 Q1+2-/1; A7&/4: 6-* +%& '81;.1+2-/ -6 +%& I/28&*02+: \:0+&5 1/3 -6 D&0&1*4%

YAQUIDZ9 $%20 /&E A7&/4: E-.;3 %18& 4.5.;1+&3 +%& 4-5)&+&/4&0 -6 P=UD 6-* *&0&1*4% 1/3 -6

1/-+%&* P-552++&& 2/ 4%1*7& -6 &81;.1+2/7 &3.41+2-/ 14+282+2&0 1/3 -6 0.))-*+2/7 +%& V2/20+*: 2/

6./32/7 1;;-41+2-/ YPQU\IZ9 I/6-*+./1+&;:@ +%& 25);&5&/+1+2-/ -6 +%& /&E A7&/4: E10 326624.;+@ 3.&

+- 0&8&*1; )*-4&3.*1; 1/3 62/1/421; -


115/151

HHM

=/ 614+@ +%& K&4*&& /-9 HiR -6 Q-8&5


116/151

HHW

Q15&0 -6 &()&*+0 1*& C&)+ 0&4*&+9

P=UD +--C 2/+- 144-./+ 51/: 4*2+242050 1/3 0.77&0+2-/09

\&8&*1; 5&&+2/70 1/3 -662421; 4-/6&*&/4&0 -*71/2c&3


117/151

HH_

#$"#:N=CM=-I@N5::Q)2H5IJ) 9H--,%M

$%& +%*&& 512/ Q&+%&*;1/30 -*71/201+2-/0 *&0)-/02


118/151

HHi

A/ 25)-*+1/+ 4-/32+2-/ 20 1;0- +- C&&) +%& 1352/20+*1+28&


119/151


120/151

HJR

#$"#;NJ-^@Q:Q)2H5IJ) 9H--,%M

A /&E 5-3&; 6-* *&0.;+


121/151

HJH

0+*1+&724 3&4202-/0 1+ +%& ;&8&; -6 ./28&*02+2&0 1/3 4-;;&7&09

?45)4*)* ,4* 74I45)4*)* ;&4()aH)42)(6

=/+&/3&3 4-/0&O.&/4&0 1*&@


122/151

HJJ

#$"#!:8@9N:Q)2H5IJ) 9H--,%M

$%& *&0&1*4% -.+).+0 -6 ./28&*02+: +&14%&*0 2/ +&*50 -6 ).


123/151

HJL

'81;.1+2-/ P-552002-/ *1*&;: -8&*+.*/0 1 )*-)-01; 513&


124/151

HJ"

#$"#,:^=5=N:Q)2H5IJ) 9H--,%M

A /&E 5-3&; 6-* 1;;-41+2-/ -6 ./28&*02+:


125/151

HJM

E%24% E2;;


126/151

HJW

3266&*&/+ E&27%+0 728&/ +- 3266&*&/+ 042&/4& 62&;309

8S()%J,5I&4( L#**I5I&4,G ;&--)45(6

$%& /&E 5-3&; 6-* 1;;-41+2/7 ,IF 2/ +%& \E&320% %27%&* &3.41+2-/ 0:0+&5 41/


127/151

HJ_

#$"#B2N9C=5 W9NO5JF:Q)2H5IJ) 9H--,%M

\2/4& H#iW +%& IN /1+2-/1; 6./32/7


128/151

HJi

?45)4*)* ,4* 74I45)4*)* ;&4()aH)42)(6

=+ 20 /-+ +%& 2/+&/+2-/ -6 +%& 6./32/7


129/151

HJ#

#$"#NOIJ?@I < ^=?JF=C-9!::Q)2H5IJ) 9H--,%M

[&


130/151

HLR

&;&4+*-/24 ).


131/151

HLH

S$%20 *1/C2/7 %10 +%& ;1*7&0+ 4-8&*17& E2+% 5-*& +%1/ HW@RRR ^27%&* '3.41+2-/ =/0+2+.+2-/0

E-*;3E23& ;20+&3 2/ +%& K2*&4+-*:9T

S[&/-*51;20&3 +- H 6-* +%& %27%&0+ 81;.&9 $%&/ 6-*

&14% 3-512/@ 51(25.5 1/3 52/25.5 *&0.;+0 1*& &(4;.3&3 1/3 &8&*: 2/0+2+.+2-/ 20 10027/&3

1 *1/C 144-*32/7 +- +%& 4-5


132/151

HLJ

A4*-


133/151

HLL

%++)GddEEE9E&


134/151

HL"

#$"#OOIJ?@I < @!@5=F9! -@NW9NO JA ^J-I5 2N9L=-:9C9=: ]@-^2U:Q)2H5IJ) 9H--,%M

$%& A413&524 D1/C2/7 -6 [-*;3 I/28&*02+2&0 YAD[IZ@ 62*0+ ).


135/151

HLM

-


136/151

HLW

=I(()-I4,5I&4X I42GR 3&. -H23 I40&%-,5I&4 I( ,J,IG,SG) %)K,%*I4K *,5, ,4* -)53&*(6

K200&52/1+2-/ 20 512/;: +%*-.7% 2+0 E&< 02+&@ +%-.7% +%& 1//.1; *&;&10& -6 +%& AD[I

71*/&*0 &(+&/028& ).E23&9 $%& E&


137/151

HL_

#$"#GOIJ?@I < CM=.E10 3&8&;-)&3 0)&426241;;: +- 62;; +%20 /24%&

4-/0.5&* 51*C&+9

E)53&*&G&KMX I42GR 5I-)N0%,-)X %)(&H%2)(X 2&(5(X 5)234&G&KI)(6

$%& 5&+%-3-;-7: 6-* +%20 *1/C2/7 0.))-*+0 1 5-*& %-;20+24 1/3 0.


138/151

HLi

>,SG) _R

=/3241+-* '();1/1+2-/ [&27%+2/7

A413&524 ]&&*

D&82&E

P-5)-02+& 04-*& 3*1E/ 6*-5 )&&* *&82&E 0.*8&: YE%24% 20

32823&3 2/+- 628& 0.


139/151

HL#

1 6&E :&1*0 17-@ 1+ +%& +25& +%&0& *1/C2/70 62*0+ *&1;;: 1))&1*&39

8S()%J,5I&4( L#**I5I&4,G ;&--)45(6


140/151

H"R

#$":$OIJ?@I < 8=-AJ-F@N!= -@NW9NO JA :!9=NC9A9! 8@8=-: AJ--=:=@-!M 2N9L=-:9C9=:

:Q)2H5IJ) 9H--,%M

F2*0+ ).


141/151

H"H

6*-5 +%& '00&/+21; \42&/4& =/3241+-*0 Y'\=Z@ +- *&3.4& +%& ;20+ 6.*+%&*9 $%& ;20+ +%&/ 20 *&3.4&3

+- &(4;.3& /-/>./28&*02+: 2/0+2+.+2-/0@ 1/3 +%& *&512/2/7 2/0+2+.+2-/0 1*& 4-5)1*&3 1712/0+

-+%&* *1/C2/70@ *&0.;+2/7 2/ 1 62/1; ;20+2/7 -6 1))*-(251+&;: +%& +-) MRR 2/0+2+.+2-/09 ]D\]@

62/1;;:@ 4-5)1*&0 +%&0& ./28&*02+2&0T -.+).+0 .02/7 31+1 6*-5 =\=T0 '\=@ =(G +< 40%()0(Y[e\Z@

E%24% 2/4;.3&0 \P= 1/3 \\P=@ 1/3N+"/),# @%$,$%+) .(5+/$*YhPDZ9

$%& 6&1+.*&0 -6 +%20 )&*6-*51/4& *1/C2/7 1*& +%1+ iR| 1*& O.1;2+1+28&

2/3241+-*0 Y+%& 62*0+ +25& .02/7 %>2/3&( +- *&6;&4+ -/ +%& O.1;2+: -6 ./28&*02+2&0T

)1)&*0Z@ 1/3 +%1+ ;-/7>+&*5 1/3 0%-*+>+&5 2/3241+-*0 &14% Y%18&Z E&27%+0 .)

+- MR|@ +%&*&+&*5 &66-*+09 ^-E&8&*@

+%20 )&*6-*51/4& *1/C2/7 6-4.0&3 -/ +%& O.1;2+1+28& 1/3 O.1/+2+1+28&

)&*6-*51/4& -6 042&/+2624 )1)&*0@ 1/3 323 /-+ 2/4;.3& -+%&* 6*&O.&/+;: .0&3

./28&*02+: &81;.1+2-/ 2/324&0 0.4% 10 +&14%2/7@ *&0&1*4%@ 1/3 1352/20+*1+2-/@

/-* 323 2+ &5)%102c& -/ 1413&524 )&*6-*51/4& 2/324&0 0.4% 10 *&).+1+2-/

1/3 &(+*1-*32/1*: 14%2&8&5&/+09 $%& 2/324&0 3&027/&3 6-* +%20 )&*6-*51/4&

*1/C2/7 0+.3: E&*& 0.2+1-*2&/+&3

./28&*02+2&0 2/ /&E;: 3&8&;-)&3 4-./+*2&09 ^''AP$@ JRRi

$%& )&*6-*51/4& 5&10.*&0 1*& 4-5)-0&3 -6 /2/& 2/3241+-*0 *&)*&0&/+2/7 +%*&& 3266&*&/+

4*2+&*21 -6 042&/+2624 )1)&*0 )&*6-*51/4&G *&0&1*4% )*-3.4+282+: Y+%& /.5


142/151

H"J

144.*1+&9

$%& ]D\] 1;0- 3-&0 /-+ 0&&C +- &81;.1+& +%& -.+).+0 2/ +%& %.51/2+2&0 1/3 0-421; 042&/4&@

1/3 6-4.0&0 0)&426241;;: -/ )&*6-*51/4& 2/ +%& %1*3 042&/4&09 $%20 6-4.0 -/ %1*3 042&/4&

1;0- 0C&E0 +%& *1/C2/7 2/ 618-.* -6 ;1*7& 2/0+2+.+2-/0@ 6-* +%& 0%&&* 41)142+: 2/ 6./32/7 1/3

614.;+: +- ).


143/151

H"L

#$":#OIJ?@I \ CM= I=95=N -@NW9NO:Q)2H5IJ) 9H--,%M

$%& P&/+*& 6-* \42&/4& 1/3 $&4%/-;-7: \+.32&0 YP[$\Z 1+ b&23&/ I/28&*02+: %10 3&8&;-)&3 1

*1/C2/7 0:0+&5 -6 ./28&*02+2&0 &/+2*&;:


144/151

H""

'.*-)&G +%& HRR 1/3 +%& JMR ;1*7&0+ ./28&*02+2&0 2/ '.*-)& 6-* +%& )&*2-3 JRRR>JRR_@

[-*;3G +%& HRR 1/3 +%& JMR ;1*7&0+ ./28&*02+2&0 E-*;3E23& 6-* +%& )&*2-3 JRRL>JRR_@

$%&0& +E- 3266&*&/+ 02c& +%*&0%-;30 2;;.0+*1+& 4;&1*;: %-E 051;;&* ./28&*02+2&0 +%1+ 1*& /-+

)*&0&/+ 2/ +%& +-)>HRR Y2/ 02c&Z 51: +1C& %27% )-02+2-/ 2/ 25)14+ *1/C2/7 26 +%& 02c& +%*&0%-;3

20 ;-E&*&39

B2


145/151

H"M

__ #'')4*IQ @R FISGI&K%,'3M$%20 ;20+ -6 ).


146/151

H"W

b&8&; '()&*+ ,*-.)9 'ID JJR#M9 D&+*2&8&3 HH A.7.0+ JRR#@ 6*-5

%++)GddEEE9513*251039-*7d)*-:&4+-0&.*-)&-0d3-4.5&/+-0d3-4d*241*3209)369

F27&;@ h/ YJRRWZ S'=$G 1 /&E 5-3&; 6-* +%& C/-E;&37& +*21/7;&T9 D&+*2&8&3 JW h.;: JRR#@ 6*-5

%++)Gdd&49&.*-)19&.d4-552002-/)%?(/*%$%(*7 C)1,)0%); C"/+5(R* 'I

^27% b&8&; ,*-.) YJRR"Z S,0%); $1( 01,##();(7 J1( \%*G+) *$/,$(;H HWM_J9

D&+*2&8&3 HH A.7.0+ JRR#@ 6*-5 %++)GddEEE9)/109-*7d472d*&)*2/+dHRJd"WdHWMW#9)369

h-


147/151

H"_

V-&3@ ^9 F9 YJRR#Z SQ&E 3&8&;-)5&/+0 2/ +%& .0& -6 42+1+2-/ 1/1;:020 2/ *&0&1*4%

&81;.1+2-/T@2/01%?"' L''")+#+;%,( ($ J1(/,5%,( C35(/%'()$,#%* ]=,/*,E,^_ 8-;9 M_@ ))9 L>

Hi9

e'PK YJRRJZ S/,*0,$% !,)",#@ ]1*209

]*24&@ K9 h9 K9 YH#_RZ SP2+1+2-/ 5&10.*&0 -6 %1*3 042&/4&@ 0-6+ 042&/4&@ +&4%/-;-7:@ 1/3/-/042&/4&T@ 2/ Q&;0-/@ P9'9 1/3 ]-;;-4C@ K9N9 Y&39Z@ @+''")%0,$%+) ,'+); *0%()$%*$* ,)&

();%)((/*@ ^&1+% b&(2/7+-/ B--C0@ b&(2/7+-/@ V1009@ ))9 LaJJ9

\1;52@ h9 YJRR#Z J1( @1,##();( +< C*$,G#%*1%); =+/#&F@#,** >)%?(/*%$%(*@ [-*;3 B1/CG

[10%2/7+-/@ K9P9

\)117&/@ h9@ K2`0+&;)%?(/*%$H \(,;"(

J,G#(*@ P1/1321/ '3.41+2-/ D&)-*+ \&*2&0@ '3.41+2-/1; ]-;24: =/0+2+.+&G $-*-/+-9 D&+*2&8&3

HH A.7.0+ JRR#@ 6*-5 %++)GddEEE9&3.41+2-/1;)-;24:9-*7d)36d[-*;3>-6>K266&*&/4&>

JRRWRJHWJ9)369

U1/ D11/@ A9 F9 h9 YJRRMZ SF1+1; A++*14+2-/G P-/4&)+.1; 1/3 5&+%-3-;-7241; )*-)%?(/*%$%(*@ \%1/7%12 h21- $-/7 I/28&*02+:@ D&+*2&8&3 JJ h.;:

JRR#@ 6*-5 %++)GddEEE91*E.9-*7d*1/CdJRR_d*1/C2/7JRR_9%+59

@#,**%)*-7*155&d%-5&d9

'(4&;;&/4& 2/ D&0&1*4% 6-* A.0+*1;21 Y'DAZ `-.*/1; ;20+0 6-* ]%:0241;@ P%&5241; 1/3 '1*+%

\42&/4&0 Y]P'Z 1/3 ^.51/2+2&0 1/3 P*&1+28& A*+0 Y^PAZ9 D&+*2&8&3 HH A.7.0+ JRR#@ 6*-5%++)GddEEE91*497-891.d&*1d`-.*/1;;20+9%+59


148/151

H"i

=/+&*/1+2-/1; e.0dJRRid)17&dB14C7*-./3

X\ [-*;3 I/28&*02+2&0 YJRRiZ D&+*2&8&3 HJ A.7.0+ JRR#@ 6*-5

%++)GddEEE9+-)./28&*02+2&094-5dE-*;3./28&*02+:*1/C2/70d|JR5&+%-3-;-7:d025);&|JR-8&*

82&Ed9

D1/C2/7 \:0+&5 P;&1*2/7 ^-.0&@ =/0+2+.+& -6 ^27%&* '3.41+2-/ ]-;24:@ D&+*2&8&3 JJ h.;: JRR#@

6*-5 %++)GddEEE92%&)9-*7dD&0&1*4%d*1/C2/70:0+&504;&1*2/7%-.0&94659

D1/C2/7 [&< -6 [-*;3 I/28&*02+2&0@ P:


149/151

European Commission

EUR 24187 Assessing Europes University-Based Research -Expert Group on Assessment

of University-Based Research

Luxembourg: Publications Office of the European Union

2010 148 pp. 17,6 x 25,0 cm

ISBN 978-92-79-14225-3

doi 10.2777/80193


150/151

How to obtain EU publications

Publications for sale:

via EU Bookshop (http://bookshop.europa.eu);

from your bookseller by quoting the title, publisher and/or ISBN number;

by contacting one of our sales agents directly. You can obtain their contact

details on the Internet (http://bookshop.europa.eu)

or by sending a fax to +352 2929-42758.

Free publications:

via EU Bookshop (http://bookshop.europa.eu);

at the European Commissions representations or delegations. You can obtain

their contact details on the Internet (http://ec.europa.eu)

or by sending a fax to +352 2929-42758.


151/151

In 2008, the European Commission, DG Research

set up the Expert Group on Assessment of University-

Based Research to identify the framework for a new

and more coherent methodology to assess the

research produced by European universities. There

is no single, correct methodology. Any assessment

of the quality of university-based research will have

to take into consideration the multi-functional and

diverse nature of universities, the diverse nature

of disciplines, the level at which universities are

assessed (i.e., identify the level at which knowledge

is created and shared), the users needs and assume

an inclusive notion of research ranging from blue sky/

curiosity-driven to user-led/practice-based researchand interdisciplinary research. The key issue here is

to design evaluations that are fit for purpose. While

commercial rankings have positive and negative

impacts onto the institutions, the students and other

users, they tend to focus on only the worlds top

100 out of the estimated 17,000 higher education

institutions. Hence, there is a need to design flexible

and multidimensional methodologies that will adapt

to the diverse and complex nature of research,

disciplines and of our universities

K1-NA-24187-EN-N

assessing europe university based research en[1]

Documents