european business schools open enrolment strategy courses

European Business Schools Open Enrolment Strategy Courses

Independent Competitive Benchmarking Project

Sponsored by

Ashridge Strategic Management Centre 3 Devonshire Street London W1W 5DT

England

Prepared by: Caroline Papp

Contact details: [email protected]

Date: December 2006

Report status: FINAL

TABLE OF CONTENTS

European Business Schools Open Enrolment Strategy Courses Competitive Benchmarking Project

1. INTRODUCTION............................................................................................1 1.1 Background...................................................................................1 1.2 Objectives.....................................................................................3 1.3 Benefits ........................................................................................3 1.4 Report Organisation .......................................................................3 1.5 Reliability of Information .................................................................3 1.6 Acknowledgements.........................................................................4

2. METHODOLOGY...........................................................................................5 2.1 Approach ......................................................................................5 2.2 Selection of Business Schools & Courses............................................5 2.3 Determination and Selection of Critical Dimensions and Key Criteria......6 2.4 Data Acquisition.............................................................................6 2.5 Data Comparison & Analysis ............................................................8 2.6 Reporting......................................................................................8

3. FINDINGS ......................................................................................................9 3.1 Selection of Business Schools & Courses............................................9 3.2 Selection of Critical Dimensions & Key Criteria....................................9

3.2.1 Critical Dimensions ............................................................................ 9 3.2.2 Key Criteria .....................................................................................13

3.3 Data Acquisition...........................................................................13 3.3.1 Stage 1: Review of business schools’ websites & brochures ....................13 3.3.2 Stages 2 to 6: Initial questionnaire, clarification, detailed course information & data review...........................................................................15

3.4 Data Comparison & Analysis ..........................................................16 3.4.1 Course Offering ................................................................................16 3.4.2 Diversity of participants.....................................................................19 3.4.3 Year the courses were last revised/revamped .......................................21 3.4.4 Application process ...........................................................................21 3.4.5 Preparation of candidates ..................................................................22 3.4.6 Course content.................................................................................23 3.4.7 Case studies ....................................................................................24 3.4.8 New skills and learning......................................................................24 3.4.9 Background of Faculty .......................................................................25 3.4.10 Teaching methods ............................................................................25 3.4.11 Average age of participants................................................................26 3.4.12 Aims of candidates achieved ..............................................................26 3.4.13 Class size ........................................................................................27 3.4.14 Course duration and contact hours ......................................................28 3.4.15 Course costs ....................................................................................28 3.4.16 Customer Service .............................................................................30 3.4.17 Evaluation and Follow-up...................................................................30

4. CONCLUSIONS...........................................................................................32 4.1 Participation of Business Schools ....................................................32 4.2 Selection of Critical Dimensions & Key Criteria..................................32 4.3 Responses from Business Schools on Key Criteria .............................33

4.3.1 Variety of course offering...................................................................33 4.3.2 Diversity of participants.....................................................................34

TABLE OF CONTENTS


4.3.3 Year the courses were revamped ........................................................34 4.3.4 Application process ...........................................................................34 4.3.5 Preparation of candidates ..................................................................34 4.3.6 Course content.................................................................................35 4.3.7 Case studies ....................................................................................35 4.3.8 New skills and learning......................................................................35 4.3.9 Background of Faculty .......................................................................35 4.3.10 Teaching methods ............................................................................35 4.3.11 Aims of candidates achieved ..............................................................36 4.3.12 Class size ........................................................................................36 4.3.13 Course costs ....................................................................................36 4.3.14 Evaluation and Follow-up...................................................................37

REFERENCES TABLES Page

Table 1 Schools and courses identified at start of project 10Table 2 Business School Participation in Data Acquisition 15Table 3 Number of Strategy Courses per Business School 16Table 4 Strategy Courses by Theme 18Table 5 Male-Female % on HR & Leadership for Women Courses 20Table 6 Class sizes per school since 2001 27Table 7 Summary of Course Costs 29

FIGURES Page

Figure 1 Critical Dimensions & Key Criteria 13Figure 2 Average % of international participants by Business

School 19

Figure 3 Average Male-Female % by Business School 20 APPENDICES

Appendix A Brief biography on Caroline Papp Appendix B Financial Times Executive Education Survey 2006 criteria:

open enrolment programmes Appendix C Website Information Acquisition Record Sheet Appendix D Open Enrolment Strategy Courses Questionnaire Appendix E Critical Dimensions & Key Criteria Appendix F List of Contacts Appendix G Business School Course Information Appendix H Ashridge Strategy Courses Detailed Content Appendix I Kirkpatrick’s Four Levels of Training Evaluation


1

1. Introduction

In July 2006 Andrew Campbell, Director of Strategy, Ashridge Business School (Ashridge), commissioned Caroline Papp, independent business consultant, to undertake a study benchmarking Ashridge’s open-enrolment Strategy courses against those of other leading European business schools. (A brief biography on Caroline Papp is included as Appendix A.)

This report documents the study’s background, methodology, findings and conclusions, and provides recommendations on areas to focus on for potential participants and participants’ sponsors when considering an open-enrolment Strategy course in terms of critical dimensions and key criteria for course selection.

The rationale behind commissioning an external consultant was to ensure that the research was carried out in an objective manner and presented without bias towards any one business school.

1.1 Background

All companies require an understanding of business strategy in order to remain competitive and grow. One of the most popular ways of ensuring that executives gain this understanding is to enrol them on a course. Strategy courses, like many executive education programmes, fall into two main categories: ‘open enrolment’ and ‘customised’ courses.

‘Open’ enrolment courses, as their name suggests, receive participants from more than one company and their content is designed to convey learning that will be applicable not only to various companies but also across a range of industries. By contrast, ‘customised’ courses are tailored by business schools to meet the specific needs of a particular organization.

Recent research suggests that “the world of executive education (EE) is changing. Increased competition between business schools and other providers, reductions in the corporate workforce and lower corporate education budgets have all affected the environment for non-degree EE programmes”. The current trend is for clients to be “much more selective in the executives they send, where they send them to and the programmes they choose … time, money and relevance appear to be major pull factors”. [1]

“This has led to a market-driven provision of corporate EE programmes” and those providers who have stayed with open enrolment programmes are “now spending more time and resources attracting customers, improving client service, customising programmes and marketing their capabilities”. [1]


2

Although many schools have pulled out of the EE market, those who have stuck with open enrolment programmes and become specialists are now seeing a revitalised sector. In fact, “overall, the number of participants on the programmes of the [Financial Times’ Annual Executive Education Survey] top 45 providers has increased over the past year by 7.5%”. [2]

When selecting an open strategy course, potential participants and sponsors (HR/Talent Managers or Training budget-holders) will typically review publicly available information before making a shortlist and perhaps contacting the schools for further details. Such information may include business schools’ websites and brochures, as well as comparative reviews such as the annual Financial Times Executive Education Survey (FTEES) of global non-degree executive education programmes, which ranks open and customised programmes. Whilst this may appear to be straightforward, there are certain factors that make direct comparison difficult, including the following:

• There are many business schools and courses from which to choose: a total of 51 schools participated in the Financial Times Executive Education (open programme) Survey for 2006, whilst the 9 European business schools invited to take part in this project offer a total of 42 courses.

• Whilst the information from business schools is often comprehensive, it is understandably tailored to highlight what they consider to be their strengths.

• Business schools’ own literature does not follow a consistent format. The FTEES addresses this to some extent by defining some critical criteria with which to compare schools, and ranking them based on participants’ and the schools’ answers. Questionnaires are completed online by open enrolment participants and statistical data are submitted online by each participating business school. However, the FTEES ranks schools, rather than individual courses.

• The FTEES 2006 only considered participant responses for general management programmes in its assessment of open enrolment programmes; strategy programmes were not included. A total of 4,275 course participants responded to the 2006 survey. Ashridge therefore sponsored this research in part to highlight any discrepancies in ranking between its strategy and general management programmes.

It can be seen from the above that there is a need for an improved method, or tool, to aid businesses in the selection of an appropriate open strategy course.


3

1.2 Objectives

The overall aim of this project is to provide a guide to enable potential clients to obtain all the information they require to select the open strategy course that best suits their needs.

The primary objectives are as follows:

i) To identify the critical dimensions which clients are (and should) be focusing on when making their assessment of what open strategy programme to purchase from which particular business school.

ii) To gather appropriate information on strategy courses from specified business schools in Europe. Note that this study does not include business schools in the USA.

iii) To make a comparative study between the differing schools and courses they offer.

iv) To present the analysis of the information in a way that clients will find useful and usable when selecting an open strategy course.

1.3 Benefits

The principal benefit of this study is that it will give clients wishing to select an open strategy course an understanding of the critical dimensions that they need to be considering and an insight into the different content and qualities of a selection of courses from the top business schools in Europe.

A secondary benefit is that the report will potentially help the business schools to focus their efforts on those critical dimensions that are seen as important to their customers.

1.4 Report Organisation

This report is divided into the following sections:

1. Introduction 2. Methodology 3. Findings 4. Conclusions

1.5 Reliability of Information

Whilst this report and the opinions contained herein are correct to the best of the author’s knowledge and belief, the author cannot guarantee the completeness or accuracy of any descriptions or conclusions based on


4

information supplied by third parties.

The draft report was sent to the main contact at each business school whereby they were given the opportunity to request changes for any discrepancies found. Requests were acknowledged and alterations made.

The project has been undertaken over the summer months (July to September), a period during which business schools, like other academic institutions, update information for the forthcoming academic year. As a result of this, since the start of the project (July 2006) changes have been made to a number of the business schools’ websites. Although all efforts have been taken to identify such changes, readers should be aware that there may be some discrepancies between the information reported in this document and information that is currently on the schools’ websites.

The author’s liability to the purchaser and their successors in title, pursuant to the terms of the appointment of the author by Ashridge, is strictly limited to the work undertaken and the matters contained and specifically referred to in this report.

1.6 Acknowledgements

An expression of gratitude is extended to all the business schools that agreed to take part in this exercise as competitive sensitivity can often stifle the free flow of information. The author is also very appreciative of the help offered by Della Bradshaw of the Financial Times, Kate Charlton at Ashridge and all those involved in the consultation process. It is hoped that all will find some value in this report and that the information given may help to generate some new ideas.


5

2. Methodology

2.1 Approach

The approach adopted in this study is based on the principle of competitive benchmarking, i.e. the systematic comparison of organisational operations, processes and performances with those of direct competitors in order to create new standards and/or improve processes [3].

The project comprised the following tasks:

1. Selection of business schools & courses 2. Determination and selection of critical dimensions and key criteria 3. Data acquisition 4. Data comparison & analysis 5. Reporting

2.2 Selection of Business Schools & Courses

For the purpose of this benchmarking study the focus has been those European business schools that provide open enrolment strategy courses and are listed in the FTEES 2006 rankings. The FTEES uses its own criteria (see Appendix B) to assess business schools worldwide providing open enrolment courses and publishes a list of its top 45.

Business schools were short-listed through discussion with Andrew Campbell (Ashridge), the project sponsor, while the author made the final selection. The European benchmarking partners chosen were rated by the Financial Times as performing on general management programmes at least as well as, or better than, Ashridge; this approach is customary in benchmarking studies.

Three programmes that are recorded as strategy programmes on the business school websites have been excluded from the main analysis, although course details are presented in Appendix G. These three programmes are:

Cranfield – The Director as Strategic Leader IESE – Mission Critical Leadership IMD – Strategic Leadership for Women

The reason for excluding these is pragmatic. They were judged to be as much about leadership as about developing or implementing strategy. This suggested the creation of a category of programme ‘strategy and leadership’. However, if this category had been included, it would have been necessary to research a number of additional courses not originally identified, which would have involved a further request for information from all the schools. This would have widened the scope of the study and


6

delayed production of the report. Hence the decision was to leave out the three programmes to be fair to all schools.

2.3 Determination and Selection of Critical Dimensions and Key Criteria

Determination of the key comparison criteria is a critical step in any benchmarking project [3]. In this study, the comparison criteria were determined through the following steps:

1. Discussion with A Campbell, Ashridge (project sponsor). 2. Review of relevant literature. 3. Review of FTEES criteria and discussion with FTEES staff. 4. Consultation with HR professionals and Training budget-holders

(hereafter referred to as ‘sponsors’) in small/medium and large organisations.

5. Discussion with Kate Charlton, member of Ashridge faculty and co-author of a recent study on “Measuring return on investment in executive education” [4].

Two levels of assessment were determined: critical dimensions and key criteria:

• Critical dimensions are the broad aims that any potential client should be using as the primary filters for selecting an open course.

• Key criteria, or sub-sets of the critical dimensions, have been selected to help clients focus on what each business school does prior to, during and post-course to ensure that the critical dimensions are achieved. These key criteria have been used in this project as the basis for all data acquisition and analysis.

Diverse stakeholders will have varying views on what is most important to them so, although the critical dimensions will stay the same, the relative importance of key criteria may change for each stakeholder as they decide which are most important for the participant, the sponsor and the company.

2.4 Data Acquisition

This involved focusing research on the business schools’ activities to identify what they do within each of the key criteria to ensure that they meet the critical dimensions identified above. Data collection was undertaken in following stages:

1. Review of business schools’ websites and brochures 2. Initial ‘Open Enrolment Strategy courses questionnaire’ 3. Tabulation and review of data from initial questionnaire 4. Clarification of data from initial questionnaire and request for missing

information 5. Requests for information on detailed course content


7

6. Final information update 7. Draft report to business schools Stage 1: Review of business schools’ websites and brochures

Prior to contacting the business schools, a thorough review of their websites and brochures was undertaken and course information extracted where available. This information was compared with the key criteria to assess the degree to which potential clients can obtain essential information without a specific enquiry to the business schools (example of website information record sheet is included as Appendix C).

Stage 2: Initial ‘Open Enrolment Strategy courses questionnaire’

A questionnaire was compiled to gather general course information and details of the key criteria. The questionnaire was emailed to the course contacts listed on each business school’s website with an explanatory note covering the background to the work. Where required a follow-up telephone call was made.

See Appendix D for example of questionnaire.

Stage 3: Tabulation & review of data from initial questionnaire

All information received was tabulated in a spreadsheet. The use of a spreadsheet helped rapid visual identification of data gaps and facilitated comparison between schools, with regard to both quantity and depth of information.

Stage 4: Clarification of data from initial questionnaire and request for missing information

Following review of the initial data, clarifying questions were formulated based on the information originally sent in order to ensure, as far as possible, a consistent level of detail.

Tabulated information was returned to the course contacts to ensure that the original information was correct, together with clarifying questions.

Business schools were invited to provide clarification either by email or telephone, depending on the preference of the course contacts.

Stage 5: Requests for information on detailed course content

Details of key course content provided by Ashridge were used as the basis for a request for additional information from all the business schools. The key course content was defined as any topic that is given more than two hours of course time. For example, if the course description said that the participants would learn strategy tools and concepts, the school was asked to clarify what these tools and concepts were and the amount of time allocated to each, as this information is essential when making a detailed examination of course content to fit specific business needs. This


8

request was appended to the original tabulated information that was returned to the course contacts for checking (Stage 4 above).

Stage 6: Final information update

Responses from Stages 4 and 5 were cross-checked against initial information and all new data tabulated and prepared for final analysis.

Stage 7: Draft report to business schools

The draft report was sent to course contacts requesting them to check that the relevant information was correct and to offer an opportunity to provide additional comments, although inclusion was not guaranteed.

Responses were collected, amendments were made and the final report produced.

2.5 Data Comparison & Analysis

Data comparison and analysis were conducted concurrently with data acquisition, as the former both relied upon and informed the latter. Responses from each business school were reviewed to identify best practice in meeting the key criteria.

2.6 Reporting

This report gives the reader an understanding and overview of how the work was carried out (this section) and highlights key findings and conclusions (following sections).


9

3. FINDINGS

3.1 Selection of Business Schools & Courses

From the FTEES top 45 list, 9 European Business schools were selected and invited to take part in this study. All the schools in this study have strategy programmes of between 3-5 days’ duration and are open to employees from any interested companies.

A list of schools and courses identified at the start of the project is presented as Table 1 (page 10).

3.2 Selection of Critical Dimensions & Key Criteria

3.2.1 Critical Dimensions

Review of literature and consultation with HR professionals, industry contacts and Ashridge provided a significant degree of consistency regarding the most important overall factors to consider when selecting an open strategy course. These were found to correspond well with Kirkpatrick’s Four Levels of Training Program Evaluation (Appendix I and [5]), although their emphasis is different: Kirkpatrick’s model focuses on post-course evaluation, whereas this study has concentrated on pre-course evaluation factors for the selection of strategy courses. The factors were defined as critical dimensions and were found to be:

• Relevance of new skills and learning (Kirkpatrick Level 1 and 2) • Acquisition of new skills and learning (Kirkpatrick Level 2) • Transfer and application of new skills and learning

(Kirkpatrick Levels 3 & 4) • Value for money (Kirkpatrick Level 4)

As with Kirkpatrick’s levels the critical dimensions follow a hierarchy, i.e. the learning must be:

• Relevant before it can be successfully acquired • Acquired before it can be effectively transferred • Transferred before it can be considered value for money

Relevance of new skills and learning

This includes aspects such as the selection process and course content, i.e. putting the right person on the right course: will the course meet the overall business need? Relevance of learning forms part of the first and second of Kirkpatrick’s Four Levels of Training Program Evaluation [5].

Although much less than the cost of a customised programme, sending an employee on an open programme still represents a significant investment for companies, so it is important to them that the content is right. As a result, the historical distinction between open and customised programmes is blurring, and “recent years have increasingly seen open enrolment providers begin to add customised elements to their programmes, with courses tailored to individual participant needs”. [2]


10

Table 1 Schools and courses identified at start of project

Business School FT 2006 Ranking

Country Strategy Courses

Ashridge 34 U.K. 1. Leading a Group of Businesses 2. Making Strategy Happen 3. Making Successful Acquisitions 4. Mastering Cost Reduction 5. Organisational Design and Internal

Collaboration 6. Strategic Decisions 7. Strategic HRM 8. Strategy and Finance 9. Strategy in Service Businesses

Cranfield School of Management

20 U.K. 1. Achieving Strategy through Business Process Change

2. Breakthrough Strategic Thinking 3. Delivering Strategic Change 4. Developing Deliverable Strategies 5. Strategic Consulting Skills 6. The Director as Strategic Leader+ 7. Understanding Finance to Influence Strategic

Decisions HEC Paris 23 France 1. Managing Organisational Change

2. Strategies for Competitive Growth Henley Management College

35 U.K. 1. Developing Business Acumen 2. Strategic Procurement 3. Strategic Toolkit

IESE Business School

2 Spain 1. Getting Things Done 2. Mission Critical Leadership+ 3. New Game Strategies -Breaking Competitive

Stalemates IMD 10 Switzerland 1. Driving Strategic Innovation

2. Strategic Finance 3. Strategic Leadership for Women+

INSEAD 7 France 1. Building the Business Strategies for Asia Pacific 2. Business Strategy for HR Leaders 3. Competitive Strategy 4. Managing Partnerships & Strategic Alliances 5. Strategic Issues in Mergers and Acquisitions

Instituto de Empresa 7 Spain 1. Building Winning Innovative Global Strategies 2. Cross Border Mergers and Acquisitions 3. Developing Strategies that Create Value 4. Doing Business In China 5. Leading Across Boundaries

London Business School

13 U.K. 1. Achieving Strategic Agility 2. Developing Strategy for Value Creation 3. HR Strategy in Transforming Organisations 4. Mergers and Acquisitions 5. Strategic Talent Management

+ The course has not been included for analysis in this report; please see section 2.2 for the rationale behind this decision.


11

At the Olin School at Washington University, for example, even one-day programmes are now partly customised:

“Faculty write to participants to ask what they want to get out of these one day programmes and respond to that”. [2]

The importance of accurately meeting the business need is supported by the consultation in this project, in which 80% of sponsors rated programme content as the most important factor in selecting a course.

Acquisition of new skills and learning

This covers aspects such as preparation of participants, the selection of teaching methods and case studies used, and quality of the teaching faculty. Acquisition of learning is the second of Kirkpatrick’s Four Levels of Training Program Evaluation [5].

Learning is highly individual, and consequently schools must ensure that their approaches meet all needs. Professor Harker, Dean of Wharton Business School, summarises Wharton’s approach, where a variety of methods is used, including lectures, cases, simulations and action learning, as follows:

“We want to focus more on learning and less on teaching … our focus is along the lines of: what are the most important things students need to learn and what is the most effective way for them to learn those things? It’s freeing; you break out of old ways of doing things”. [6]

The importance of pre-course preparation (and follow-up: see next section) is explained by Ken Bardach, Associate Dean for Executive Education at the Olin School:

“We have to get away from the mindset that it is all about what happens in the classroom. It’s about what happens after the classroom, and that depends on what happens pre-course.” [2]

The quality of the faculty, which influences both teaching and course content, is addressed by the Financial Times, which states that business schools are moving away from research that meets the requirements of “esoteric publications” and are “revising the way they approach faculty to put companies’ needs on the research agenda”. Top business schools, such as Darden, at the University of Virginia, are putting “executive education more firmly into the setting of the research agenda”. [2]

In the consultation undertaken as part of this project, 60% of respondents/sponsors rated preparation as their third highest priority, whilst 50% placed new skills and learning as their fourth highest priority.


12

Transfer of new skills and learning into the workplace

This looks at whether the individual’s and organisation’s aims were achieved and how business schools approach ensuring that participants apply the learning that they have acquired. Application of new skills or behaviours is the third of Kirkpatrick’s Four Levels of Training Program Evaluation, while the impact on the business corresponds to Level Four [5].

Transfer and application of new skills and learning to the workplace to improve the business is ultimately what any of the schools involved in this project must have as their primary objective for course participants:

“What companies are buying is the impact that happens after the programme.”

Whilst this quote by Jon Spector, Director of Executive Education at Wharton [6] was made in connection with custom programmes, it is fair to say that it is equally applicable to open courses.

This view is supported by recent research at Ashridge into measuring return on investment in executive education [4]:

Questionnaires were distributed to HRD professionals and sponsors, most of whom were CEOs, managing directors or general managers. When given options to describe what kind of return on investment they were looking for from executive programmes, 88% of sponsors indicated that either “I want individual leaders to perform effectively” (45%) or “I want the organisation to be in better shape to deliver our strategy” (43%) most closely matched their wishes.

This is a clear indication that business schools should be looking for ways to help ensure that participants apply their learning.

Ashridge’s research is supported by this project’s consultation, in which 50% of respondents/sponsors rated follow-up in their top four priorities. As one consultee remarked: “it is not so much about academic and managerial expectations being met but the applications of the learning demonstrated”.

Value for Money

This covers the sponsor’s view of whether the course was worth its cost.

This is often more qualitative than quantitative, as shown by the recent research by Ashridge into return on investment in executive education [4], referred to above. This research found that only 8% of sponsors considered that their views on return on investment were best matched by the phrase “I want a clear financial benefit, where gains from the programme exceed the costs”.

This shows that other factors are being taken into account. There is no point in an individual having a good reaction to a course and an increased capability if nothing changes back in the workplace.


13

Nonetheless, direct cost is a significant factor for companies, of whatever size: 80% of participants/sponsors in this project rated value for money as their second most important selection factor.

3.2.2 Key Criteria

The critical dimensions described above are broad aims that can be difficult to assess without more detailed definition. Accordingly, the critical dimensions were sub-divided into key criteria to help potential purchasers of open strategy courses focus on what each business school does prior to, during and post-course to ensure that the critical dimensions are achieved. The key criteria provide a framework to allow potential purchasers/sponsors and candidates to select the course that best meets their needs.

The key criteria are similar to, but more comprehensive than, the FTEES criteria. The consultation process highlighted some differences, especially concerning the FTEES weightings, which potential purchasers of open strategy courses should bear in mind. (The FTEES criteria and weightings are presented as Appendix B).

The FTEES weightings are based on the views of course participants, which may be different from those of purchasers/sponsors. For example, course participants in the FTEES 2006 placed ‘Follow-up’ and ‘Preparation’ as the 7th and 8th (out of 10) most important factors, whereas this project’s survey and review of literature rated them within the top four.

Figure 1 (page 14) shows graphically how the key criteria considered in this study correspond to the critical dimensions. This is also shown in tabular form in Appendix E. The rationale behind the selection of each of the key criteria is presented in Section 3.4, together with the information provided by the business schools.

3.3 Data Acquisition

3.3.1 Stage 1: Review of business schools’ websites & brochures

All business schools had web pages for their courses with comprehensive information including contact details of key course personnel.

The level of information available on business school websites and in brochures was found to vary considerably and much of the information required on key criteria had not been addressed or was limited. In general, the business schools’ approach to information on their courses is to provide more detailed information in their brochures than on their websites. Ashridge, IESE, INSEAD and London Business School (LBS) offered the most comprehensive information on their websites and covered more of the key criteria without the need to download the course brochures.


14

KEY CRITERIA Number/variety of strategy courses

Diversity: - % of international candidates & % male v female

Year the course was revamped/ revised

KEY CRITERIA Aims of candidates achieved

Min/max class size Average class size

KEY CRITERIA Course cost Cost per day

Accommodation incl. Meals incl.

Customer Service

KEY CRITERIA Evaluation/Follow Up

KEY CRITERIA Length of course

Total no. of contact hours

KEY CRITERIA Application process

Preparation of candidates Course Content

Case studies New skills & learning

Background of strategy faculty Teaching methods

Average age of candidates

CRITICAL DIMENSION Value for Money

CRITICAL DIMENSION Transfer of Learning

CRITICAL DIMENSION Relevance of Learning

Figure 1 Critical Dimensions & Key Criteria

CRITICAL DIMENSION Acquisition of Learning

Relevance & Acquisition of Learning

Acquisition & Value for Money


15

3.3.2 Stages 2 to 6: Initial questionnaire, clarification, detailed course information & data review

The lack of data on key criteria available from the business schools’ websites led to a requirement to contact the business schools directly to obtain further information.

It was at this stage that the issues of annual leave and commercial confidentiality first arose, which caused delays in obtaining information from some of the schools.

It was also found that the consistency of information varied between schools and depending on the method of data acquisition (i.e. questionnaire completed by school or by telephone interview), which was partially addressed by returning tabulated information to the course contacts.

The schools’ responses at each stage of data acquisition is summarised in Table 2. A list of contacts is included as Appendix F.

Table 2 Business School Participation in Data Acquisition

Business school Stage 2 Stage 4 Stage 5

Questionnaire Clarification Course content

Ashridge M

Cranfield School of Management P

HEC N/A N/A

Henley Management College

IESE Business School + M*

IMD M

INSEAD ++ P **

Instituto de Empresa (IdE) P

London Business School M

+ Questionnaire completed by email for all courses except Mission Critical Leadership which was completed by telephone interview.

++ Information provided on 2 out of 5 courses, one by telephone interview (Managing Partnerships & Strategic Alliances), one by emailed questionnaire (Building the Business: Strategies for Asia Pacific).

*IESE also volunteered a similar depth of information on 6 other courses, which were unfortunately outside the scope of this study.

** INSEAD said that they do provide this information to participants / sponsors on request.

M = Majority

P = Partial


16

3.4 Data Comparison & Analysis

A wealth of information was obtained from the 7 business schools that agreed to take part in Stages 2 and 4 of this project. However, as is common with projects of this type, the consistency of information varied in both breadth and depth, owing to a number of factors. Some of these were within the control of the participants and author, whilst some were not, and included commercial confidentiality and absence of key personnel (annual leave).

All the responses received from the business schools have been tabulated and are presented in full in Appendix G. The sections below highlight any major differences or uniformities in the context of best practice and qualify numerical answers where appropriate.

3.4.1 Course Offering

Number & Variety of Strategy Courses

The selection of the right course is essential to meet both the participant’s and the organisation’s needs. For a given length of course, there will always be a balance between breadth of subjects and depth of coverage. The schools approach this in different ways, with Ashridge offering the largest number of courses (9) and IESE and IMD the smallest (2).

Table 3 Number of Strategy Courses per Business School

School No. of

Strategy Courses

Ashridge 9 Cranfield 6+ IESE 2+ IMD 3+ INSEAD 5 IdE 5 London Business School 5

+Strategy and Leadership programmes excluded

Programme Themes

The number of courses offered by the schools, and the different approaches adopted to address the balance between breadth of subjects and depth of coverage as mentioned above, means that comparison can sometimes be difficult. In Table 4 (page 18), courses have been grouped, where possible, based on the information obtained through websites, brochures and questionnaires.


17

Readers should note that the nature of research on diversity is currently changing “from gender and race to national culture and functional culture” (D. Newkirk, chief executive of executive education, Darden, in [2]). This is reflected in Table 4 as follows:

• National culture: two courses have been grouped under the heading ‘Strategy & National Culture’: INSEAD’s Building the Business: Strategies for Asia Pacific and IdE’s Doing Business in China. Both of these reflect the current shifts in global economic power.

• Functional culture: Ashridge (Strategy in Service Businesses, Strategic HRM, Strategy and Finance) has the highest number of functional culture courses, with INSEAD (Business Strategy for HR Leaders), LBS (HR Strategy in Transforming Organisations), IMD (Strategic Finance) and Cranfield (Understanding Finance to Influence Strategic Decisions) focusing on these aspects to a lesser extent.

Readers should bear in mind, however, the lack of detailed information on course content provided by most of the schools, and hence the inherent difficulty of this type of exercise.


18

Table 4 Strategy Courses by Theme

Business School

Course Title

Competitive Strategy for Value Creation Ashridge Strategic Decisions Ashridge Strategy in Service Businesses Cranfield Developing Deliverable Strategies IESE New Game Strategies – Breaking Competitive Stalemates INSEAD Competitive Strategy IdE Shaping Strategies for Value Creation LBS Developing Strategy for Value Creation

Group Level Strategy Ashridge Leading a Group of Businesses

Strategy & Innovation Cranfield Breakthrough Strategic Thinking IMD Driving Strategic Innovation IdE Building Winning Innovative Global Strategies LBS Achieving Strategic Agility

Executing Strategy Ashridge Making Strategy Happen Ashridge Mastering Cost Reduction Cranfield Achieving Strategy through Business Process Change Cranfield Delivering Strategic Change IESE Getting Things Done

Strategy and Finance Ashridge Strategy and Finance Cranfield Understanding Finance to Influence Strategic Decisions IMD Strategic Finance

Strategy and HR Ashridge Strategic HRM INSEAD Business Strategy for HR Leaders LBS Human Resource Strategy in Transforming Organisations LBS Strategic Talent Management

Strategy and Organisation Ashridge Organisation Design and Internal Collaboration IdE Leading Across Boundaries

Mergers, Acquisitions and Alliances Ashridge Making Successful Acquisitions INSEAD Managing Partnerships and Strategic Alliances INSEAD Strategic Issues in Mergers and Acquisitions IdE Cross Border Mergers and Acquisitions LBS Mergers and Acquisitions

Strategy and National Culture INSEAD Building the Business Strategies for Asia Pacific IdE Doing Business in China

Miscellaneous Cranfield Strategic Consulting Skills


19

3.4.2 Diversity of participants

The issue of diversity and its relative importance in different subjects is complex. For example, leadership courses are clearly culturally, functionally and gender dependent, and hence the diversity of course participants is critical. The content of strategy courses, however, is more focused on the tools and rigorous frameworks of economics, money and competitiveness, and thus it could be argued that such courses are less subject to cultural nuances and hence that a diverse delegate group is less important. Nonetheless, it is certain that a thorough understanding of ethnicity, cultural background and gender issues is essential in the global business environment. An ethnically and culturally diverse group of participants with a near 50-50 gender split would be more likely to generate potentially different perspectives and a richer learning environment, hence the questions on the % of international students and male-female student ratio.

Average Percentage of International Participants by Business School

A breakdown of the average % of international students by business school is presented as Figure 2. From this it can be seen that the 3 schools that attract largest proportion of non-local/international participants are:

1. INSEAD 2. IMD 3. LBS

0

10

20

30

40

50

60

70

80

90

100

%

Ashrid

ge

Cranfie

ldIESE

IMD

INSEAD IdE LB

S

Figure 2 Average % of international participants by Business SchoolFigure 2 Average % of international participants by Business School


20

Average Male-Female Percentage by Business School

A breakdown of the average male-female ratio by business school is presented as Figure 3. Numbers are obtained by taking the mean of the male-female percentages for each course excluding the two HR Strategy courses (provided by Ashridge and LBS), as figures for these two courses are significantly different from the other strategy courses at each of these schools and would skew the analysis. The male-female percentages on these two courses are as follows:

Table 5 Male-Female % on HR Courses

Business School / Course Male % Female %

Ashridge: Strategic HRM 50 50 LBS: HR Strategy in Transforming Organisations 58.5 41.5

Figure 3 Average Male-Female % by Business School

0

10

20

30

40

50

60

70

80

90

100

%

Ashrid

ge

Cranfie

ldIESE

IMD

INSEAD IdE LB

S

Figure 3 Average Male-Female % by Business School

MaleFemale


21

3.4.3 Year the courses were last revised/revamped

It is important that the business schools regularly review their programmes to ensure that they are teaching the most up-to-date material in the most appropriate ways. All schools were found to review and make minor revisions to their courses on a yearly basis but up to 5 years can elapse before courses undergo a major overhaul (revamp).

3.4.4 Application process

It is important to understand the participant selection process as this will help determine the appropriate managerial/academic level, the diversity of both international and gender mix and the relevance of the participants’ issues.

The application process is the first step in checking the relevance of the new learning for the participant and in starting the acquisition of new knowledge.

Most of the business schools undertake some form of selection procedure in addition to an application form to ensure, as far as is possible, that the course will meet the applicant’s business needs and that the applicant is at an appropriate level to benefit suitably from the course. However, the nature of this additional selection varies considerably:

Ashridge

Applicants are provided information about the programmes – extensive websites, articles describing content, detailed brochures etc Ashridge makes every effort to ensure the right people are on the right course to ensure the group dynamic works.

Leading a Group of Businesses – 100% of applicants are interviewed by telephone and invited to a ‘taster’ event to check for suitability. Between 20% and 50% of applicants for other courses are also given telephone interviews, depending on the course.

IESE

All applicants complete a “basic on-line questionnaire to ensure the right level” (quote from IESE response to initial questionnaire).

INSEAD

Applicants for two of the five courses included in this project (Managing Partnerships & Strategic Alliances and Building the Business: Strategies for Asia Pacific) undergo a “selection/screening process to target learning at right level and depth” (quote from INSEAD response to initial questionnaire). No information was given regarding the nature of the selection/screening or the remaining 3 courses.


22

LBS

Applicants for all courses undergo “screening – applications are reviewed to ensure that (a) there is the right mix of participants and (b) programme suits needs of individual and organisation” (quote from LBS response to initial questionnaire). No further details were given regarding the nature of the screening.

3.4.5 Preparation of candidates

It is important to know how the business schools prepare the candidates in advance of the course as this will aid the ease and speed of the learning for the participant.

Appropriate pre-course work will start to focus the participants’ minds on the subjects to be covered and begin to accustom them to the school’s style of learning so that they are fully prepared at the start of the residential part of the course.

If undertaken correctly, preparation of candidates, together with the application process, will ensure that the right delegates are accepted onto the right courses and hence maximise the relevance and acquisition of new skills and learning.

All schools involve participants in some form of pre-course preparatory work although the nature of this preparation varies considerably, between schools and between courses run by the same school.

Ashridge

Participants in all courses are sent course materials at least 4 weeks prior to the commencement of the course. In addition, participants in Leading a Group of Businesses receive articles, a task and a DVD before the course, at 8 weeks and 6 weeks respectively. Participants in the Strategic HRM course are required to complete a 3600 review, present on their own HR function and read relevant articles in preparation for the course.

Cranfield

Participants in all courses are sent case studies in advance, so they are ready to start work on them when they arrive, together with background reading specific to the particular course. In addition, participants in Delivering Strategic Change are given leadership profiling questionnaires to be completed by peers and colleagues.

IESE

No preparation is required for Getting Things Done. Preparatory work is involved in New Game Strategies - Breaking Competitive Stalemates but IESE did not give any details.


23

IdE

Participants on all courses are sent reading materials and exercises about 4 weeks before the start of the programme, which they must read and complete as preparation for the courses.

IMD

Participants are sent some preparatory reading materials before the start of the programmes, which must be completed in order to fully benefit from the course.

INSEAD

Participants on Managing Partnerships & Strategic Alliances and Building the Business: Strategies for Asia Pacific are provided with a case study, articles, suggested readings and other resources in order to prepare for the programme.

The INSEAD website also gives information on articles that correspond with the different courses.

LBS

LBS state that they work with participants and their organisations before they attend the programme to ensure that expectations and objectives are set - both for the participant and their colleagues/line managers etc - and that they will receive support to implement their ideas and solutions when they return to work. For example, participants may be required to bring details of a strategic problem relating to their business, or to complete a pre-programme questionnaire, which is used during the programme to analyse their organisation's capabilities. Participants are also given access to the on-line Virtual Learning Environment in advance of attending the programme, where they can access materials and view details of other participants.

3.4.6 Course content

It is essential to know what specific tools and methods are being taught in order to make an informed choice of the most suitable course for both participant and organisation. The course content should reflect what the participant will be able to put into practice.

Generic information under broad headings, such as ‘an integrated approach to implementing strategy’, ‘understand and interpret new approaches’ or ‘competitor analysis and positioning’ can be found on all the business schools’ websites and in their course brochures. Additional broad information on case study topics and teaching methods (see Sections 3.4.7 and 3.4.10 below) was also obtained from all the schools participating in this project via the initial questionnaire.


24

However, only Ashridge supplied detailed information on course content at the level that is required to enable a client to make a fully informed decision on selecting the right course to meet his/her business needs.

The information provided by Ashridge can be found in Appendix H.

3.4.7 Case studies

Case studies should be relevant to the professional activities of course participants, as learning improves with increasing relevance. However, this does not necessarily mean that they must be from the participants’ own industries or sectors, but they must illustrate transferable issues and learning.

Case study information is presented in the individual school tables in Appendix G.

Most business schools provided industry-specific detail on case study content, with Ashridge, Cranfield, IESE, IMD and LBS being the most comprehensive in their responses to the questionnaires. IdE state that “all” industries are used in case studies for all their courses.

For its Strategy & Finance course, Ashridge does not use pre-prepared case studies, stating that, “all the techniques in strategy are applied to the businesses of the participants, working in rotating groups”. Other schools also work on participants’ issues to varying degrees throughout their programmes.

INSEAD state that they offer 1 case study per subject area and aim to “provide what the market wants” but did not specify further. The school tries to bring in a speaker from inside the company in question to ensure a live/real case study event for participants to listen to and understand differing worldviews. INSEAD did not provide any information in their questionnaire response although details of case studies for some courses were obtained from the website.

3.4.8 New skills and learning

New skills acquired by participants must be relevant in their workplace and pragmatic to implement. In addition, participants and their sponsoring organisations should be seeking something extra, to give them an edge in their business, so it is important to understand how each course encourages new ways of thinking, through academic research and leading business practice, and how it ensures participants to interact and critique the material they are learning.

According to Blair Sheppard, Chief Executive of Duke Corporate Education, companies do not want purely academic research:

“They want contextually-driven research. They need insight into current issues.” [2]


25

Although all the schools were keen to state that they are at the forefront of both academic research and business practice and that they incorporate this into their courses, it was difficult to ascertain the differentiating factors, or unique selling points, of the courses.

The most specific information came from the following:

Ashridge – All of Ashridge’s strategy programmes (except Strategic HRM) are directed by Ashridge Strategic Management Centre. This centre conducts research on strategic management issues and builds programmes around this research. Hence Ashridge state that the strategy programmes contain new material which is often unique to Ashridge.

IESE – emphasised daily updating of knowledge, benchmarking key business schools and constant contact with industry, other business schools and Universities, media, alumni and professional, industry and academic bodies.

3.4.9 Background of Faculty

The background of the faculty, i.e. those members of staff who are involved in developing and teaching the courses, determines the quality of the course and its balance of academic/business/industrial/consulting content. Most schools have a mix of faculty – career academics, practitioner-lecturers and visiting speakers - to ensure that all aspects are covered.

It should be noted that the background of the whole faculty is important here, rather than just those staff teaching on one particular course. Thus, when reviewing the data (Appendix G), readers should examine other strategy courses run by the same school.

The schools were asked to clarify their answers by stating the % of the faculty that had either an academic, consultant or manager background.

Cranfield is the only school where the faculty are 100% practitioner-lecturers.

At Ashridge, almost all of the lecturers in the strategy faculty have had extensive experience with leading strategy consultancies such as Boston Consulting Group or McKinsey.

IMD, INSEAD, IDE and LBS did not specify the backgrounds of their faculty members.

3.4.10 Teaching methods

Learning styles can differ considerably between individuals so a programme should apply a variety of learning techniques/methods to accommodate the differing styles rather than be tied to a single pedagogy. Schools should utilise a range of appropriate teaching methods and materials, both traditional and contemporary, such as


26

lectures, case studies, simulations and action learning.

All schools provide the participants with a variety of teaching methods (see individual school tables in Appendix G). Whilst all use the traditional methods of lectures, case study and group work, there is increasing variance between the schools in the application of other techniques such as action learning and role play. Ashridge and INSEAD use action learning to work to on participants’ own workplace problems. Only Cranfield does not provide an on-line resource for any of its courses.

IdE is the only school that takes participants on an industrial excursion, on its Doing Business in China course.

3.4.11 Average age of participants

The average age of participants can be a quick indicator to determine the managerial level of the course by the seniority of participants.

The average age of participants on all courses over the last 5 years ranges from 40 (IESE and LBS) to 45 (IdE), which is consistent with the business schools’ target attendee profile of senior managers and company directors.

3.4.12 Aims of candidates achieved

This relates to the degree to which both academic and managerial expectations are met in-course. If the aims of participants are achieved then it is more likely they will be able to transfer and apply their learning back in the workplace. At Olin School (Washington University), for example:

“They [faculty] stop half an hour before the end of the class and ask participants how they will go away and do things differently.” [2]

Details of the schools’ methods of ensuring that participants’ aims are achieved are given in the individual school tables in Appendix G.

All business schools offer some sort of continuous evaluation of progress towards learning objectives throughout the course. These evaluations take various forms, including work on participants’ workplace issues, informal discussions with course directors over lunch and in the evenings (for example, Cranfield), or more formal arrangements, such as IESE’s ‘Red Thread Review’, which takes place at the end of each day’s teaching. During the ‘Red Thread Review’ participants reflect on the day’s learning, considering the relevance to their issues and how they can apply the learning to change their situation.

The approach to assessment also varies between the schools, with most offering assessment on some courses. Only IdE assesses participants on all courses, whilst LBS does not undertake any course assessments.


27

All schools also ask participants to complete an evaluation form at the end of each course, which includes a question about whether the participant’s aims have been achieved.

Formal assessments of participants’ acquisition of new knowledge should correspond to Kirkpatrick’s Level 2, although it was not clear what the schools do formally to assess participants’ prior level of knowledge and hence the increase during the course. All other methods of assessing whether participants’ aims have been achieved correspond to Kirkpatrick’s Level 1: reaction of the student [5].

Achievement of post-course aims, i.e. those relating to transfer of learning into the participants’ organisations, is addressed below under ‘Follow Up’.

3.4.13 Class size

Class size can have a significant effect on both the type of learning and the participant’s ability to learn. A balance must be struck between providing a class large enough to ensure adequate breadth of issues, examples discussed and networking opportunities whilst not restricting discussion of individual issues and personal attention from faculty staff.

Detailed data on minimum, maximum and average class size per course over the last 5 years is presented in the individual school tables in Appendix G. This is information is summarised in Table 6.

Table 6 Class sizes per school since 2001

School Minimum Maximum Average

Ashridge 6 30 16

Cranfield 6 24 14

IESE 12 40 32

IMD 20 45 32

INSEAD* 27 35 25

IdE 12 25 22

LBS 25 50 33

*INSEAD’s figures are based on figures for only one course.


28

3.4.14 Course duration and contact hours

The length of course may be a critical factor for sponsoring organisations as time on a course is time away from the workplace. It will also be a factor in determining value for money and should reflect the level of knowledge and understanding required to be passed on to participants.

There is often, but not always, a direct correlation between length of course in days and total contact hours, as the length of teaching days may vary between schools and evening activities may also be arranged.

Details of course duration and total number of contact hours are given in the individual school tables in Appendix G.

All courses are between 3 and 6 days in length, with a typical 8-9 hour study day (although some finish early on the last day), and a comparison of courses of the same types (see Section 3.4.1 and Table 4) shows the majority to be of similar length.

Only Ashridge, Cranfield and INSEAD provide on-campus dedicated accommodation, which potentially offers participants ready opportunities to interact with each other and sometimes with faculty staff outside the classroom, thus helping to increase learning. Other schools have arrangements with leading hotels to ensure that participants have suitable surroundings. However, the degree of formality of this will vary, with only LBS reporting that it organises some evening activities.

3.4.15 Course costs

Course costs may cover teaching only, teaching and meals, or teaching, meals and accommodation. Costs will also reflect course duration, so it is important to compare like with like, for example cost per day.

Ashridge and Cranfield provide on-campus accommodation included in the course fee, whilst INSEAD can arrange on-campus accommodation at extra cost.

Details of course costs are given in the individual school tables in Appendix G and summarised in Table 7 (page 29). As some courses include accommodation and meal costs, whilst others do not, a per night cost of £100 has been assumed for those that do not include accommodation/meals in order to facilitate comparison.

On this basis, per-day course costs range from £725 per day (Cranfield, Developing Deliverable Strategies and Achieving Strategy Through Business Process Change) to £1412 per day (London Business School, Mergers & Acquisitions). The mean cost per day is £993 with over half of the courses charging more than £950 per day. All Cranfield and IESE courses are less than £960 per day, whilst all LBS and INSEAD (except Building the Business: Strategies for Asia Pacific) courses are above £1100 per day.


29

Table 7 Summary of Course Costs

School Course Length of

Course (days)

Fee ex-VAT (£)

Accom/meals cost (£)

Cost per day incl accom &

meals (£) Ashridge Leading a Group of Businesses 5 £ 5,650.00 0 £ 1,130.00 Making Strategy Happen 4 £ 4,400.00 0 £ 1,100.00 Making Successful Acquisitions 5 £ 4,900.00 0 £ 980.00 Mastering Cost Reduction 4 £ 4,400.00 0 £ 1,100.00

Organisation Design and Internal Collaboration 4 £ 4,400.00 0 £ 1,100.00

Strategic Decisions 5 £ 4,900.00 0 £ 980.00 Strategic HRM 5 £ 3,900.00 0 £ 780.00

Strategy & Finance 5 £ 4,550.00 0 £ 910.00

Strategy in Service Businesses 4 £ 4,150.00 0 £ 1,037.50 Cranfield

Achieving Strategy Through Business Process Change 5 £ 3,625.00 0 £ 725.00

Breakthrough in Strategic Thinking 3 £ 2,400.00 0 £ 800.00

Delivering Strategic Change 4 £ 3,200.00 0 £ 800.00 Developing Deliverable Strategies 5 £ 3,625.00 0 £ 725.00 Strategic Consulting Skills 3 £ 2,400.00 0 £ 800.00

Understanding Finance to Influence Strategic Decisions

5 £ 4,200.00 0 £ 840.00

IESE Getting Things Done 3.5 £3,041 300 £ 954.44

New Game Strategies - Breaking Competitive Stalemates 3 £3,041 200 £ 1,080.18

IMD Driving Strategic Innovation 6 £4,885 500 £ 897.52 Strategic Finance 5 £4,673 400 £ 1,014.54 INSEAD

Building the Business: Strategies for Asia Pacific 5 £ 3,234.15 400 £ 726.83

Business Strategy for HR Leaders 5 £ 5,168.92 400 £ 1,113.78 Competitive Strategy 3 £3,817.57 200 £ 1,339.19

Managing Partnerships & Strategic Alliances

5 £5,168.92 400 £ 1,113.78

Strategic Issues in Mergers & Acquisitions 5 £5,168.92 400 £ 1,113.78 Instituto de Empresa

Building Winning Innovative Global Strategies

3 £2,432 200 £ 877.48

Cross Borders Mergers & Acquisitions 3 £2,432 200 £ 877.48 Doing Business in China 4 £3,311 0 £ 827.70 Leading Across Boundaries 3.5 £3,041 300 £ 954.44 Shaping Strategies for Value Creation 3 £2,432 200 £ 877.48 London Business School Achieving Strategic Agility 5 5,350 400 £ 1,150.00 Developing Strategy for Value Creation 5 5,350 400 £ 1,150.00 HR Strategy in Transforming Organisations 5 5,350 400 £ 1,150.00 Mergers and Acquisitions 4 5,350 300 £ 1,412.50 Strategic Talent Management 4 4,950 300 £ 1,312.50


30

3.4.16 Customer Service

Business schools are teaching best business practice, so they should lead by example. This criterion looks at how schools ensure good customer service and what potential purchasers of their services should be expecting.

All schools place great importance on customer service. They adopt various methods to ensure that participants’ needs are met, for example with dedicated teams and relationship managers. All schools request that participants complete feedback questionnaires at the end of course.

3.4.17 Evaluation and Follow-up

Evaluation and Follow-up is the business schools’ principal mechanism of finding out whether they are meeting one of their clients’ key criteria of transfer of learning to the workplace. It is common practice in almost all disciplines to issue feedback forms at the end of a training course, often with a question about ‘aims achieved’.

However it is also important to ask what schools do to ensure that learning is transferred to and applied in participants’ workplaces post-course. This is especially important in relation to strategy courses which, by their nature, deal with medium- to long-term matters: it is only once delegates are back in the workplace that the application of new skills and learning – the most important aims to achieve – can be properly assessed.

All schools recognise that transfer of learning into the workplace is essential. However, their methods of ensuring this vary considerably, with some placing more importance on formal follow-up than others.

Ashridge

Following completion of their course, participants are invited to Ashridge Alumni events, which include networking and presentations on particular issues. Participants are also invited to other free Ashridge activities, such as research presentations or days devoted to specific topics, e.g. strategy. Participants are also given tutor email addresses and encouraged to contact tutors as they use tools and concepts on the job. No structured follow-up is offered.

Cranfield

Cranfield Faculty staff are available to talk to delegates post-course and discuss particular issues they may be facing back in the workplace. Cranfield state that many relationships are long-term and that it is not uncommon for delegates to call a course director some years after a course to discuss issues. No structured follow up is offered.


31

IESE

IESE employ a “100 day follow up” on Getting Things Done. This is an IT platform, through which participants can keep in contact with academics as well as each other. Third parties, such as a participant’s line manager, can also have access. The aim is to encourage sharing of best practice, group work and sustainable networking. On New Game Strategies - Breaking Competitive Stalemates, IESE issue participants with reading material.

IMD

IMD do not undertake any structured post-course follow up, stating that “the level of seniority of our participants makes this redundant as each manager comes to our programs with a clear learning agenda which is scrupulously followed during the course of the program”.

INSEAD

On Managing Partnerships and Strategic Alliances, INSEAD sends an email to participants after 3 months to remind them of the commitments that they made whilst on the course.

‘Issue-teams’ may also be set up during the course, whereby participants with similar matters to resolve group together and are facilitated by a professor. These may continue post-course, but this is at the individuals’ discretion rather than being formalised.

IdE

Participants on all courses are given on-line access through which they can continue the topics on-line in chat rooms and forums. Contact with program teachers and staff is available. Additional relevant documentation is posted on the topics discussed. No structured follow-up is offered.

LBS

Participants on LBS courses complete a questionnaire at the end of the course. Apart from that, no structured follow up is provided. LBS’ intention is to set up support within a participant’s organisation prior to the course that will continue post-course.

All the methods employed by the business schools of assessing transfer to and application of learning in the workplace appear to correspond to Kirkpatrick’s Level 1: reaction of the student [5].


32

4. CONCLUSIONS

A competitive benchmarking study of open-enrolment strategy courses provided by nine top European business schools has been undertaken. The study comprised the following steps:

1. Selection of schools & courses 2. Selection of critical dimensions and key criteria 3. Data acquisition 4. Data comparison & analysis

Key conclusions from the study are presented below.

4.1 Participation of Business Schools

Nine leading European business schools that provide open enrolment strategy courses were selected from the FTEES 2006 rankings and invited to take part in this project. Of these:

• One school (Ashridge) participated fully in all three stages of data acquisition.

• Six schools (Cranfield, IESE, IMD, INSEAD, IdE and LBS) provided information for two of the three stages of data acquisition.

• One school (HEC) provided information for one of the three stages, even though it will not be running these particular open strategy courses in academic year 2006/07 as it is in the process of completely overhauling its courses for 2007/08.

• One school (Henley) did not take part in the study.

In general the schools were willing to take part and provide general information but commercial sensitivity did somewhat restrict their openness and impede the sharing of more in-depth course information.

4.2 Selection of Critical Dimensions & Key Criteria

Review of literature and consultation with HR professionals, industry contacts and Ashridge provided a significant degree of consistency regarding the most important overall factors to consider when selecting an open strategy course. These were found to correspond well with Kirkpatrick’s Four Levels of Training Program Evaluation [5], although their emphasis is different: Kirkpatrick’s model focuses on post-course evaluation, whereas this study has concentrated on pre-course evaluation factors for the selection of strategy courses. The factors were defined as critical dimensions and were found to be:

• Relevance of new skills and learning • Acquisition of new skills and learning • Transfer and application of new skills and learning • Value for money


33

The critical dimensions were sub-divided into key criteria to help potential purchasers of open strategy courses focus on what each business school does prior to, during and post-course to ensure that the critical dimensions are achieved. The key criteria were used to formulate questionnaires to obtain information from the business schools. The key criteria are similar to, but more comprehensive than, the FTEES criteria, and provide a framework to allow potential purchasers of courses and candidates to select the course that best meets their needs.

Diverse stakeholders will have varying views on what is most important to them so, although the critical dimensions will stay the same, the relative importance of key criteria may change for each stakeholder as they decide which are most important for the participant, sponsor or company.

4.3 Responses from Business Schools on Key Criteria

This section highlights the most important conclusions from the analysis of information presented by the business schools in the context of best practice in their ability meet the key criteria and hence fulfil the critical dimensions.

4.3.1 Variety of course offering

The business schools’ approaches to addressing the broad scope of the strategy field varied considerably, with the number of courses offered by the schools ranging from 2 (IESE and IMD) to 9 (Ashridge). Most of the courses could be grouped relatively easily into similar themes.

Particular note should be made of those courses that reflect the changing nature of research on diversity “from gender and race to national culture and functional culture” (D. Newkirk, chief executive of executive education, Darden, in [2]).

With regard to national culture, two courses have been grouped under the heading ‘Strategy & National Culture’: INSEAD’s Building the Business: Strategies for Asia Pacific and IdE’s Doing Business in China. Both of these reflect the current shifts in global economic power.

With regard to functional culture, Ashridge (Strategy in Service Businesses, Strategic HRM, Strategy and Finance) has the highest number of functional culture courses, with INSEAD (Business Strategy for HR Leaders), LBS (HR Strategy in Transforming Organisations), IMD (Strategic Finance) and Cranfield (Understanding Finance to Influence Strategic Decisions) focusing on these aspects to a lesser extent.


34

4.3.2 Diversity of participants

In general the business schools based in continental Europe (IESE, IMD, INSEAD and Instituto de Empresa) have a higher percentage of international participants than the British schools, although London Business School is the exception to this, coming third out of seven at 70% non-UK participants.

The male-female ratio at all schools is biased towards male participants, with only IdE reporting more than 30% female attendees. The exceptions to this trend are the HR Strategy courses offered by Ashridge and LBS. The reasons for the low female percentage were not explored in this project. However, possible causes could be that the ratios reflect the relative lack of women in strategic roles in industry, or that those women in these roles are not receiving their ‘strategy’ development, or at least not at the business schools that took part in this project.

4.3.3 Year the courses were revamped

All the business schools review and/or make at least minor modifications to all courses on an annual basis, thereby demonstrating that they recognise the importance of regularly updating their courses to ensure that they incorporate the latest research and business practices.

4.3.4 Application process

Most of the schools indicated that they undertake some form of screening process to ensure the optimum match between participant, participant’s and organisation’s aims, and course selection. From the level of detail received, it appears that the approach adopted by Ashridge on its course ‘Leading a Group of Businesses’, i.e. to conduct a telephone interview with all applicants and invite them to a ‘taster event’, is setting the standard in best practice.

4.3.5 Preparation of candidates

All schools involve participants in some form of pre-course preparatory work for at least one of their courses.

Some schools’ preparatory work concerns the participant’s own company (e.g. LBS), whilst other schools prescribe topics (e.g. IdE). Some set tasks to complete (e.g. Ashridge, IdE) whereas others require reading (e.g. Cranfield, IMD).

INSEAD, IESE and LBS also engage participants’ line managers or sponsors to ensure that all parties’ expectations and objectives are aligned and that the participant will receive the support that is needed to ensure transfer of learning to the workplace.

From the information obtained in this study, it would appear that a combination of work on the participant’s own company’s issues, involving


35

not only the participants but also their companies, together with a subject chosen by the school and an exercise to complete, would best help to prepare participants for the material to be covered in-course, the learning style of the school and the application of learning post-course.

4.3.6 Course content

“You have to really get under the skin of a business school if you want to work out what distinguishes it from its competitors. At first glance they all seem to offer the same thing; it takes closer research to reveal the differences” [7].

The generic descriptions on schools’ websites and brochures, such as ‘industry analysis’, generally do not give enough information to be able to distinguish between courses. Although this project has attempted to obtain further details of course topics from all the schools, only Ashridge provided the information requested so no conclusions can be drawn.

4.3.7 Case studies

Ashridge, Cranfield, IESE, IMD and LBS provided quite comprehensive information whilst IdE stated that “all” industries are used in case studies for all their courses; INSEAD did not provide subject information although limited subject information can be found on their website. On its Strategy & Finance course, Ashridge does not use pre-prepared cases, basing all studies on the businesses of the participants, while other schools use a combination of both.

Information on case studies is very important for sponsors and participants in the process of selecting a course. A combination of up-to-date pre-prepared cases and studies based on the participants’ businesses will maximise relevance to their specific business needs.

4.3.8 New skills and learning

Although all the schools were keen to state that they are at the forefront of both academic research and business practice and that they incorporate this into their courses, it was difficult to ascertain (apart from the specific example Ashridge gave) of the differentiating factors, or unique selling points, of the courses and hence no real conclusions can be drawn.

4.3.9 Background of Faculty

All schools except Cranfield have faculties of mixed academic and practical background, suggesting that they will be well placed to deliver the “contextually-driven research” identified as a need by Duke Corporate Education (Section 3.4.8 [2]).

4.3.10 Teaching methods

All schools adopt a variety of teaching methods, including lectures, case studies, simulations and action learning, in accordance with best practice. It will be up to the participant and sponsor to examine the school’s


36

methods and compare with the participant’s learning style and individual preference to identify which approach will best suit their needs.

4.3.11 Aims of candidates achieved

This relates to the degree to which both academic and managerial expectations are met in-course. Achievement of post-course aims, i.e. those relating to transfer of learning into the participants’ organisations, is addressed below under ‘Follow Up’ (Section 4.3.15).

All business schools offer some sort of continuous evaluation of progress towards learning objectives throughout the course, some more formal than others.

The most formal is IESE’s ‘Red Thread Review’, which takes place at the end of each day’s teaching, and during which participants reflect on the day’s learning by considering the relevance to their issues and how they can apply the learning to change their situation.

The approach to assessment also varies between the schools, with most offering assessment on some courses. Only IdE assesses participants on all courses, whilst LBS does not undertake any course assessments.

All schools also ask participants to complete an evaluation form at the end of each course, which includes a question about whether the participant’s aims have been achieved.

Formal assessments of participants’ acquisition of new knowledge should correspond to Kirkpatrick’s Level 2, although it was not clear what the schools do formally to assess participants’ prior level of knowledge and hence the increase during the course. All other methods of assessing whether participants’ aims have been achieved correspond to Kirkpatrick’s Level 1: reaction of the student [5].

4.3.12 Class size

Average class size ranges from 14 (Cranfield) to 33 (LBS). As with teaching methods (see Section 4.3.10 above), this may be a significant factor in determining the course’s fit with an individual’s learning style.

4.3.13 Course costs

Course costs quoted by schools are not always given on the same basis, as some include accommodation and/or meals, whilst others are for tuition only. Ashridge and Cranfield include on-campus accommodation within the course fee, whilst INSEAD can arrange on-campus accommodation at extra cost.

Course costs were found to vary both between schools and between courses offered by the same school. Using an assumed £100/night cost for accommodation and meals there is a difference of nearly £700 per day between the most expensive course (£1412, London Business School, Mergers & Acquisitions) and the least expensive (£725, Cranfield,


37

Developing Deliverable Strategies and Achieving Strategy Through Business Process Change).

Although cost is the bottom line for any business, participants and their sponsoring organisations should remember that it is only one component in determining value for money, especially given the far-reaching nature of the learning acquired on strategy courses.

4.3.14 Evaluation and Follow-up

This relates to the schools’ processes for helping to ensure that participants’ learning is transferred to and applied in the workplace.

Most of the schools offer follow-up to some degree, but it is reactive, with participants being relied upon to get in contact if issues arise rather than the schools pro-actively monitoring the application of learning.

The exceptions are IESE, INSEAD and IdE, all of which offer pro-active support of some sort. INSEAD sends an email to participants after 3 months to remind them of the commitments that they made whilst on the course. IESE and IdE give participants access to an on-line forum through which they can keep in contact with faculty as well as each other. Third parties, such as a participant’s line manager, can also have access to the IESE forum.

A proper evaluation of whether learning has been transferred, applied and is of benefit to the organisation would correspond to Levels 3 and 4 of Kirkpatrick’s Four Levels of Evaluating Training Programs [5]. However, the methods employed by all the schools only correspond to Level 1. Research from Ashridge [4] suggests that this may need to change:

“85% of our HRD respondents reported that they thought evaluation of executive education would become a more important issue over the next three years. 85% of HRD respondents and 75% of sponsors agreed that ‘it is the strategic importance of executive education that makes evaluation vital’ … one respondent summed this up: ‘it is critical that learning is useful and applied’.”


References

1. Anonymous (2004) Back to school for executive learning, Development and Learning in Organizations. Bradford: 18, pg 23

2. Bradshaw, D (2006) Research is back on the menu, Financial Times: London, May 15

3. ten Have et al. (2003) Key Management Models, Benchmarking, pg 22. Prentice Hall: Harlow

4. Charlton, K & C Osterweil (2005) Measuring Return on investment in executive education: a quest to meet client needs or pursuit of the Holy Grail? Ashridge Journal: Autumn, pg 6-13

5. Kirkpatrick, D (1994) Evaluating Training Programs: the Four Levels. San Francisco,CA. Berrett-Koehler

6. Knight, R (2006) Wharton has abolished the phrase: ‘We don’t do that’ University of Pennsylvania’s business school is not resting on its laurels.’ Financial Times: London, May 15, pg 12

7. Carrington, L (2003) Schools of thought, Human Resources: London, pg 32


APPENDICES


APPENDIX A Brief biography on Caroline Papp

Caroline Papp is a self-employed independent business consultant and coach. Her main areas of expertise include: Executive Coaching, Ethical Leadership, Learning and Development, and Talent Management.

Caroline has 10 years’ experience of working within the Human Resource arena. She has a proven track record of adding value to organisations through success in leading, designing and implementing organisational change across a range of industries. These include: Aviation, Education, Oil (downstream) and Retail.

She has gained both the theoretical understanding and practical know-how of career development and succession, coaching, corporate social responsibility, learning and development, performance management and recruitment.

Caroline has completed numerous benchmarking exercises within her corporate roles in her search for best practice covering areas such as career management, performance management, recruitment and training & development.

She has also provided organisations with a portfolio of learning and development activities and has previously benchmarked UK business schools and training providers against one another in terms of Leadership Development and Executive MBA.

Caroline’s postgraduate educational background includes: PgDip HRM (1999), MBA (Lancaster University, 2004) and she is currently finishing her PgCert in Coaching (2006).


APPENDIX B Financial Times Executive Education Survey 2006 criteria: open enrolment programmes

(after Financial Times, 2006)

The first 10 criteria are based on responses given by course participants, the final six on a survey of the business schools. All criteria are presented in rank form, apart from women participants (%). The top school in each criterion is ranked number 1.

Weights in brackets

1. Preparation (7.8): Users rate the provision of advanced information on the programme content and the participant selection process

2. Course design (8.8): Assesses the flexibility of the course and appropriateness of the structure and design

3. Teaching materials (8.3): A rating of how contemporary and appropriate the teaching materials were, and whether the mix of academic rigour and practical relevance was appropriate

4. Teaching faculty (8.9): The quality of the teaching and the extent to which teaching staff worked together to present a coherent programme

5. Quality of participants (8.0): The extent to which other participants were of the appropriate managerial/academic level, the international mix of the class and the interaction between course participants

6. New skills (8.8): The relevance of new skills in the workplace, the ease with which they were implemented and the extent to which the course encouraged new ways of thinking

7. Follow-up (7.9): The level of follow-up offered once participants returned to their workplace and networking opportunities with other participants

8. Aims achieved (8.6): The degree to which academic and managerial expectations were met

9. Food and accommodation (6.4): The quality of food and accommodation

10. Facilities (7.4): The quality of teaching accommodation and IT and library facilities

11. Women participants (2.0): The proportion of female participants

12. International participants (3.0): The proportion of international participants and the number of different nationalities among all open programme participants

13. Repeat business / growth (5.0): An index of growth in income and proportion of repeat business

14. International location (3.0): Measure of courses run internationally or outside of the business school’s region (e.g. North America, Europe, Asia etc.).

15. Partners (3.0): Number of programmes taught in conjunction with other business schools

16. Faculty diversity (4.0): Measure of the nationality and gender mix of the faculty.


APPENDIX C Website Information Acquisition Record Sheet

Critical Dimensions

*

Key Criteria

R Variety of strategy courses? A, C Length of Course (days) A, C Total Number of contact hours A Min/Max Class size A Average class size (over last 5

years) R, A Average age of candidates R What year was course

revamped/revised? C Fee C Fee per day C Does fee include

accommodation? C Does fee include meals? R Diversity: % of international

candidates (non-UK) over last 5 years

R Diversity: Male/female as % R, A Background of strategy faculty

as % academic, consultant and manager

R, A Application Process R, A Preparation of candidates: what

pre-course work is given? T Follow-up: what post-course

contact is there, e.g. assignments, on-going liaison etc?

R, A Course content A Aims of candidates achieved R, A What teaching

methods/materials are used on each course?

R, A What industries do the case studies focus on?

R, A New skills and learning Key

R = Relevance of new skills and learning A = Acquisition of new skills and learning T = Transfer and application of new skills and learning into the workplace C = Cost


APPENDIX D Open Enrolment Strategy Courses Questionnaire

Course no. 1 2 3

Length of course (in days) Total number of contact/study hours?

How many times a year does programme run?

Min/ Max Class size? Average class size too?

Min Max

Min Max Min Max

Average age of candidates What year was course last revamped/ revised?

Does fee include accommodation?

Yes/ No Yes/ No Yes/ No

Does fee include meals? Yes/ No Yes/ No Yes/ No Is fee exclusive of VAT? Yes/ No Yes/ No Yes/ No % of international candidates Male/ Female as % Background of strategy faculty more focused on academic or practical experience?

Academic/ Practical

Academic/ Practical

Academic/ Practical

Regarding preparation of candidates is pre-course work given?

Yes/ no Yes/ no Yes/ no

Regarding follow-up is post course work given?

Yes/ no Yes/ no Yes/ no

What teaching materials used on course – please list

What teaching methods are used on each course? Please delete/add to list as necessary.

Case Study Group Work

Lectures Action

Learning On-line

resources Video

PowerPoint Presentation

Role Play Simulations Assessment


Lectures Action

Learning On-line

resources Video




Lectures Action

Learning On-line

resources Video



What industries do the case studies focus on? Please list


I have a few questions regarding the FT ranking which may require a longer explanation and I am more than happy to call you to discuss. My questions are Preparation – Please explain exactly what is done to prepare the candidates and what is unique about the way the course has been set up? New Skills and Learning – How do you ensure relevance in the workplace? What are the new ways of thinking on this course re: strategy? Follow-up –What does the school do for follow-up on the strategy courses? Aims achieved – How do you ensure that aims are achieved both academically and managerially? Faculty – How diverse is your faculty on the strategy courses in terms not only of experience but also gender and ethnicity? Customer Service – How do you ensure good customer service? What KPI’s do you have e.g. response times, ratings of courses etc? Finally is there anything else that you would like to add? E.g. on your courses’ uniqueness, image, quality, service or accessibility etc.


APPENDIX E Critical Dimensions & Key Criteria

Critical Dimensions

*

Key Criteria

R Variety of strategy courses? A, C Length of Course (days) A, C Total Number of contact hours

A Min/Max Class size A Average class size (over last 5 years)

R, A Average age of candidates R What year was course revamped/revised? C Fee C Fee per day C Does fee include accommodation? C Does fee include meals? R Diversity: % of international candidates (non-UK) over last 5 years R Diversity: Male/female as %

R, A Background of strategy faculty as % academic, consultant and manager

R, A Application Process R, A Preparation of candidates: what pre-course work is given?

T Follow-up: what post-course contact is there, e.g. assignments, on-going liaison etc?

R, A Course content A Aims of candidates achieved

R, A What teaching methods/materials are used on each course? R, A What industries do the case studies focus on? R, A New skills and learning

Key

R = Relevance of new skills and learning A = Acquisition of new skills and learning T = Transfer and application of new skills and learning into the workplace C = Cost


APPENDIX F List of Contacts

Name of Institution Website Address Initial Contact Follow up

Contact Main Contact Details

Financial Times 16-28 Tabernacle Street London EC2A 4DD ... England

www.ft.com Della Bradshaw Ursula Milton Della Bradshaw Business Education Editor E-mail: [email protected] Tel: +44 (0)207 873 4673

Ashridge Strategic Management Centre 3 Devonshire Street London W1W 5DT England T:+44 (0)20 7323 4422 F:+44 (0)20 7323 0903

www.ashridge.com Andrew Campbell Angela Munro Stephen Bungay David Sadtler Mary Kennedy Arno Haslberger Kate Charlton

Andrew Campbell Director of Ashridge Strategic Management Centre E-mail: [email protected] Tel: +44(0)207 323 4422

Cranfield School of Management Cranfield Bedford MK43 0AL England T: +44 (0)1234 751122 F: +44 (0)1234 751806

www.som.cranfield.ac.uk Jane Roberts Jane Roberts Jane Roberts Client Service Manager E-mail: [email protected] Tel: +44 (0) 1234 754351

HEC Paris 1, rue de la Libération 78351 Jouy en Josas cedex Tel : + 33 (1) 39.67.70.00 Fax: + 33 (1) 39.67.74.40

www.hec.edu

Sean Kilbride Joshua Kobb Joshua Kobb Director of International Programmes E-mail: [email protected] – Tel: +33 (0) 1 39 67 70 19


Name of Institution Website Address Initial Contact Follow up Contact

Main Contact Details

Henley Management College Greenlands, Henley-on-Thames, Oxfordshire, RG9 3AU, England T:+44 (0) 1491 571454 F:+44 (0) 1491 571635

www.henleymc.ac.uk Angela Beckett n/a Angela Beckett Executive Education Enquires E-mail: [email protected] Tel: +44 (0)1491 418767

IESE Business School - Barcelona Campus Avenida Pearson, 21 08034 Barcelona Spain T: +34(0) 93 253 4200 F: +34 (0) 93 253 4343

www.iese.edu Tricia Kullis Fernanda Anguiano Mark Wuijten

Mark Wuijten Mark Wuijten Programme Director E-mail: [email protected] Tel: +34 93 253 4200

IMD International Ch. de Bellerive 23 P.O. Box 915 CH-1001 Lausanne, Switzerland T: +41 (0)21 618 0111 F: +41 (0)21 618 0707

www.imd.ch [email protected] Victor Lolev Victor Lolev Programme Advisor E-mail: [email protected] Tel: +41 21 618 03 42

INSEAD Europe Campus Boulevard de Constance 77305 Fontainebleau T : 33 (0)1 60 72 40 00 F : 33 (0)1 60 74 55 00

www.insead.edu Allison Wheeler Liz Ciccarella Jaclyn Mah

Liz Ciccarella Liz Ciccarella Assistant Director - Open Enrolment Programmes E-mail: [email protected] Tel: +33 1 60 72 40 30


Name of Institution Website Address Initial Contact Follow up Contact

Main Contact Details

Instituto de Empresa María de Molina, 11 28006 Madrid SpainT: +34 (0) 915 689 600 F: +34 (0) 915 689 61

www.ie.edu Marina Tirado Marina Tirado Marina Tirado International Programme Director E-mail: [email protected] – Tel: +34 91 782 17 15

London Business School Regent's Park London NW1 4SA United Kingdom T:+44 (0)20 7000 7000 F:+44 (0)20 7000 7001

www.london.edu Tim Pearson Michelle Guest

Caroline Buckley Amy Gatrill

Any Gatrill Business Development Executive E-mail: [email protected] – Tel: +44 207 000 7377


APPENDIX G BUSINESS SCHOOL COURSE INFORMATION

(included as a separate spreadsheet)


APPENDIX H Ashridge Strategy Courses Detailed Content

Topic

Lead

ing

a G

roup

of

Bus

ines

ses

Stra

tegi

c D

ecis

ions

Stra

tegy

& F

inan

ce

Stra

tegi

c H

RM

Mak

ing

Stra

tegy

Hap

pen

Mak

ing

Succ

essf

ul

Acq

uisi

tions

Mas

terin

g C

ost R

educ

tion

Org

anis

atio

n D

esig

n an

d In

tern

al C

olla

bora

tion

Stra

tegy

in S

ervi

ce

Bus

ines

ses

4 rules of acquisitions X* X* X* X 5 Forces X X 7 tests of organisation design X X Acquisition integration X BCG matrix X Benchmarking X Blocks to change framework X Collaboration/ synergy framework X X Communicating strategy framework X Corporate /parenting advantage X X Cost/ benefit analysis X Decision Grid X Decision process framework X Discounted cashflow X X Due diligence X

Employee Engagement X

Financing Structures X

GE Matrix X X X Growth Traffic Lights X X Hierarchy value analysis X Incremental Value X X

Major tools and concepts

of the Programme

(only list if they take up 2

hours plus of the timetable) Job briefing X

Language of organisation relationships X Mission Analysis X X Negotiation X Operational excellence tools X Parenting opportunity analysis X Parenting skills analysis X Performance measurement X X Personal value propositions X Portfolio analysis X Power/ interest grid X Process engineering X Process of org design X RAPID framework X Ratio Analysis X X Strategy Triangle X Strategy trade offs X Trust analysis X Valuation methods X Value Chain X X X Value Drivers X X Value Line X Valued Customer X

APPENDIX I Kirkpatrick’s Four Levels of Training Evaluation (Kirkpatrick, 1994)

[http://www.businessballs.com/kirkpatricklearningevaluationmodel.htm]

This grid illustrates the basic Kirkpatrick structure at a glance. The second grid, beneath this one, is the same thing with more detail.


Level Evaluation type (what is measured)

Evaluation description and characteristics

Examples of evaluation tools and methods Relevance and practicability

1 Reaction

• reaction evaluation is how the delegates felt about the training or learning experience

• e.g., 'happy sheets', feedback forms

• also verbal reaction, post-training surveys or questionnaires

• quick and very easy to obtain • not expensive to gather or to

analyse

2 Learning

• learning evaluation is the measurement of the increase in knowledge - before and after

• typically assessments or tests before and after the training

• interview or observation can also be used

• relatively simple to set up; clear-cut for quantifiable skills

• less easy for complex learning

3 Behaviour

• behaviour evaluation is the extent of applied learning back on the job - implementation

• observation and interview over time are required to assess change, relevance of change, and sustainability of change

• measurement of behaviour change typically requires cooperation and skill of line-managers

4 Results

• results evaluation is the effect on the business or environment by the trainee

• measures are already in place via normal management systems and reporting - the challenge is to relate to the trainee

• individually not difficult; unlike whole organisation

• process must attributing clear accountabilities

APPENDIX I (cont) Kirkpatrick’s Four Levels of Training Evaluation (Kirkpatrick, 1994)


Kirkpatrick’s four levels in detail





1 Reaction

• reaction evaluation is how the delegates felt, and their personal reactions to the training or learning experience, for example:

• did the trainees like and enjoy the training?

• did they consider the training relevant?

• was it a good use of their time?

• did they like the venue, the style, timing, domestics, etc?

• level of participation • ease and comfort of

experience • level of effort required to

make the most of the learning

• perceived practicability and potential for applying the learning

• typically 'happy sheets' • feedback forms based on

subjective personal reaction to the training experience

• verbal reaction which can be noted and analysed

• post-training surveys or questionnaires

• online evaluation or grading by delegates

• subsequent verbal or written reports given by delegates to managers back at their jobs

• can be done immediately the training ends

• very easy to obtain reaction feedback

• feedback is not expensive to gather or to analyse for groups

• important to know that people were not upset or disappointed

• important that people give a positive impression when relating their experience to others who might be deciding whether to experience same








2 Learning

• learning evaluation is the measurement of the increase in knowledge or intellectual capability from before to after the learning experience:

• did the trainees learn what was intended to be taught?

• did the trainee experience what was intended for them to experience?

• what is the extent of advancement or change in the trainees after the training, in the direction or area that was intended?

• typically assessments or tests before and after the training

• interview or observation can be used before and after although this is time-consuming and can be inconsistent

• methods of assessment need to be closely related to the aims of the learning

• measurement and analysis is possible and easy on a group scale

• reliable, clear scoring and measurements need to be established, so as to limit the risk of inconsistent assessment

• hard-copy, electronic, online or interview style assessments are all possible

• relatively simple to set up, but more investment and thought required than reaction evaluation

• highly relevant and clear-cut for certain training such as quantifiable or technical skills

• less easy for more complex learning such as attitudinal development, which is famously difficult to assess

• cost escalates if systems are poorly designed, which increases work required to measure and analyse







Examples of evaluation tools and methods

Relevance and practicability

3 Behaviour

• behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour, and this can be immediately and several months after the training, depending on the situation:

• did the trainees put their learning into effect when back on the job?

• were the relevant skills and knowledge used

• was there noticeable and measurable change in the activity and performance of the trainees when back in their roles?

• was the change in behaviour and new level of knowledge sustained?

• would the trainee be able to transfer their learning to another person?

• is the trainee aware of their

• observation and interview over time are required to assess change, relevance of change, and sustainability of change

• arbitrary snapshot assessments are not reliable because people change in different ways at different times

• assessments need to be subtle and ongoing, and then transferred to a suitable analysis tool

• assessments need to be designed to reduce subjective judgement of the observer or interviewer, which is a variable factor that can affect reliability and consistency of measurements

• the opinion of the trainee, which is a relevant indicator, is also subjective and unreliable, and so needs to be measured in a consistent defined way

• 360-degree feedback is useful method and need not be used before training, because respondents can make a judgement as to change after training, and this can be analysed for groups of respondents and trainees

• assessments can be designed around

• measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

• simple quick response systems unlikely to be adequate

• cooperation and skill of observers, typically line-managers, are important factors, and difficult to control

• management and analysis of ongoing subtle assessments are difficult, and virtually impossible without a well-designed system from the beginning

• evaluation of implementation and application is an extremely important assessment - there is









change in behaviour, knowledge, skill level?

relevant performance scenarios, and specific key performance indicators or criteria

• online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

• self-assessment can be useful, using carefully designed criteria and measurements

little point in a good reaction and good increase in capability if nothing changes back in the job, therefore evaluation in this area is vital, albeit challenging

• behaviour change evaluation is possible given good support and involvement from line managers or trainees, so it is helpful to involve them from the start, and to identify benefits for them, which links to the level 4 evaluation below









4 Results

• results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

• measures would typically be business or organisational key performance indicators, such as:

• volumes, values, percentages, timescales, return on investment, and other quantifiable aspects of organisational performance, for instance; numbers of complaints, staff turnover, attrition, failures, wastage, non-compliance, quality ratings, achievement of standards and accreditations, growth, retention, etc.

• it is possible that many of these measures are already in place via normal management systems and reporting

• the challenge is to identify which and how relate to to the trainee's input and influence

• therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training, so they understand what is to be measured

• this process overlays normal good management practice - it simply needs linking to the training input

• failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

• for senior people particularly, annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

• individually, results evaluation is not particularly difficult; across an entire organisation it becomes very much more challenging, not least because of the reliance on line-management, and the frequency and scale of changing structures, responsibilities and roles, which complicates the process of attributing clear accountability

• also, external factors greatly affect organisational and business performance, which cloud the true cause of good or poor results





This grid illustrates the Kirkpatrick's structure detail, and particularly the modern-day interpretation of the Kirkpatrick learning evaluation model, usage, implications, and examples of tools and methods. This diagram is the same format as the one above but with more detail and explanation.

Since Kirkpatrick established his original model, other theorists (for example Jack Phillips), and indeed Kirkpatrick himself, have referred to a possible fifth level, namely ROI (Return On Investment). In my view ROI can easily be included in Kirkpatrick's original fourth level 'Results'. The inclusion and relevance of a fifth level is therefore arguably only relevant if the assessment of Return On Investment might otherwise be ignored or forgotten when referring simply to the 'Results' level.

Learning evaluation is a widely researched area. This is understandable since the subject is fundamental to the existence and performance of education around the world, not least universities, which of course contain most of the researchers and writers.

While Kirkpatrick's model is not the only one of its type, for most industrial and commercial applications it suffices; indeed most organisations would be absolutely thrilled if their training and learning evaluation, and thereby their ongoing people-development, were planned and managed according to Kirkpatrick's model

european business schools open enrolment strategy courses

Documents

data acquisition

data comparison analysis

course offering

course duration

course content

course costs

diversity of participants

average age of participants