Exploring Models and Partnerships for e-learning in SMEs

Disclaimer

This publication has been produced with the support of the European Commission.

Whilst every attempt has been made to ensure that the information contained within this booklet are correct, neither the authors, the European Commission nor the University of Stirling will be liable for any errors or omissions contained within it.

2

The contents of this publication do not necessarily reflect the position of the European Commission or the University of Stirling, nor do they involve any responsibility on their part.

3

Exploring Models and Partnerships for e-learning in SMEs

4

5

Table of Contents

i. Acknowledgemnts 1

ii. Foreward 2

iii. Overview 2

1. Developing new pedagogies and e-learning in SMEs.Graham Attwell 1

2. Epistemic Activity, Organisations and Learning: towards a framework for analysing the use of e-resources in Small and Medium Enterprises.

David Guile 1

3. Localization of software and learning material for SMEs: how is it possible?Bernard Blandin 1

4. E-learning Content and Software Localization.George Bekiardis 1

5. Prospects for the development of software that truly supports collaboration and learning in SMEs.Graham Attwell and Mike Malloch 1

6. A framework for the evaluation of e-learning.Jenny Hughes and Graham Attwell 1

7. Practical evaluation interventions in understanding informal learning within SMEs.David Slater 1

8. Evaluating the effectiveness of e-learning strategies for SMEs.Eduardo Figueira 1

9. E-learning challenges in Austrian SMEs.Klaus Reich and Freidrich Scheuermann 1

10. Assessing the application of online learning in a work-based setting.John Munro and Philip Crompton 1

11. A holistic vision of the future of e-learning.Kees Schuur 1

Acknowledgements

6

This publication is the result of two seminars held in autumn 2002 and Spring 2003. The seminars were supported by a grant from the European Commission DG Education and Culture and were coordinated by KnowNet, a small research and development company based in Wales. The main theme for the seminars was "Exploring Models and Partnerships for e-learning in SMEs." There was a particular focus on the issues of pedagogies for e-learning and evaluating e-learning.

Around 14 people attended each seminar drawn from ten different European countries and including researchers, trainers and e-learning providers.

The seminars were intended to develop a discourse for and between research, policy and practice in this field. We believe this aim has been achieved. The next tasks will be to both deepen the research approach and to initiate pilot programmes to support SMEs in developing e-learning as part of a learning culture.

May I thank all those who took part and contributed to the seminars and helped shape this publication. In particular I would like to thank Philip Crompton from the University of Stirling for his support and Claire Middlleton from Knownet who undertook cheerfully the not inconsiderable task of administering the project.

7

Foreward

Graham Attwell, Knownet (UK)

Professor Duncan Timms, University of Stirling (UK)

Digital technologies have been a major driving force behind the profound changes in work organisation, production and society over the last twenty years. These changes have led to what social scientists characterise as the “knowledge-based economy”, “the knowledge society” or the “information society”, forms in which the knowledge of individuals and organisations is critical to innovation and to economic and social development. Whilst in previous societal forms initial education and training were seen as providing the basic skills and knowledge required for work and for participation within society, the new social and work forms require continuous updating of personal and collective skills and knowledge – or lifelong learning (Attwell, forthcoming). The knowledge society is also the learning society.

Lundvall (1996) argues that the ‘learning economy’ is a more appropriate term than the ‘knowledge-based economy’ in articulating today’s agenda where specialised and codified knowledge has a very short life-span. It is the capability to learn how to create new knowledge and adapt to changing conditions that will increasingly determine the performance of individuals, firms, regions and countries (see Lundvall 1996). A continuous process of re-learning and extending knowledge is an essential pre-requisite for membership in the knowledge society.

The need for an extension of life-long learning, stimulated by the changes in economy and society, has occurred at the same time that developments in communications and information technologies offer the potential for the need to be met. Just as digital technologies have been a key force in driving economic and social demand for new knowledge, so they have also been hailed as the key for developing a lifelong learning infrastructure. E-learning, the application of Communication and Information Technologies to curriculum and pedagogy, has been seen as providing universal access to information and as providing a new flexible and ubiquitous learning environment open to all regardless of the constraints of time and place.

The promise of e-learning is a long time in actualization. Despite spawning a whole new industry, not to say numerous government and European sponsored initiatives and programmes, progress to-date has been less than convincing. The development of e-learning has been dominated by the metaphors of the virtual classroom and the virtual university, an over-obsession with technologies and a focus on distance applications of existing learning opportunities, rather than the diffusion of learning in wider societal activities and forms. Some have gone so far as to claim that the term e-learning has become devalued to the extent where it might be more properly seen as a marketing word rather than a description of pedagogic and learning practices (Attwell, forthcoming).

8

Most of the attention paid to e-learning has been concerned with its application to the formal education system of schools and universities. There has been very limited attention to vocational and occupational learning and the development of e-learning environments in less formal learning contexts. Billet (2001) points out:

“For many workers, perhaps most, the workplace represents the only or most viable location to initially learn and or/develop their vocational practice.”

Billett suggests that informal learning is the primary form of vocational learning – both initial and ongoing – for most people in the workforce today. Therefore, learning from an economic, human and social point of view has to be embedded in the fabric of all work organisations. If e-learning is to make a contribution to changing the learning paradigm, e-learning must be embedded in the work organisation. E-learning has until now a limited application in small enterprises – and has tended to focus on providing networked access to virtual classroom type environments or to distance learning supported by computer-based materials. The present volume is an attempt to redress the lack of attention that has so far been paid to the implications of e-learning for workers in small and medium-sized enterprises.

9

Overview

The focus of the present volume is on the implications of Information and Communications Technologies (ICT) for learning in European SMEs. The papers were produced as part of a project, funded by the European Commission, DG Education and Culture, and co-ordinated by Knownet, a small north Wales based research and development company, with support from the University of Stirling in Scotland. The main tool used to gather the raw material for the project was two seminars, drawing together a range of experts from across the EU. The seminars were intended to build on the experience and outcomes of existing European research in the field in order to focus on the issues for developing the use of ICT for learning in SMEs and produce policy recommendations for the European Commission and Member States. The overall title for the seminars was Exploring Models and Partnerships for e-learning in SMEs. Whilst the themes for the seminars were broad, the seminars focused more tightly on issue of pedagogies for learning in SMEs and on efforts to evaluate learning using ICT in SMEs. These were seen as critical to developing the theory and practice of using ICT for learning in SMEs and as a subject for policy advice and development.The first seminar was held in Stirling in November 2002 and the second in Brussels in February 2003. Invitations to the seminars reflected two primary concerns. The first was to ensure a dialogue between participants in different countries in Europe, in recognition of different European traditions, cultures and economies affecting the development, organisation and operation of small and medium-sized enterprises and the differing patterns of technology diffusion and use in different countries in Europe. The second was the desire to encourage dialogue between different disciplines, including researchers from sociology, computer science, communications theory and education along with specialists in pedagogy and evaluation. The aim of the seminars was not to solve problems as such, but to begin to map the range of research and policy issues and to lay the basis for a longer going research and development effort relating to the effective use of e-learning in the small and medium-sized workplace. The papers included in this volume reflect both the diversity of background of the participants and the aim to scope the field.

The first paper, ‘Developing new pedagogies and e-learning in SMEs’ by Graham Attwell provides a general account of the background, discourse and conclusions to the working sessions on pedagogy held at the two seminars. The first seminar looked at the main issues involved in developing pedagogies for the use of information and communication technologies in SMEs. These include definitions of both e-learning and SMEs, managers’ attitudes to learning, the supposed cost benefits which might accrue from e-learning and issues around standards and technology and content.Towards the end of the Stirling group discussions, participants started looking at the problem of forecasting future developments and what the implications might be for SMEs. They agreed that the real challenges were to stop thinking within existing paradigms of learning and e-learning and to develop new ones. This raised a number of issues.

10

1. One was the nature of informal (or tacit) knowledge as opposed to formal knowledge.

2. The second issue was the tension between individual learning, which some commentators believe tends to be encouraged by e-learning, and the social nature of adult learning as reflected in social constructivist theory.

3. The third issue followed on from this, stressing the need to ensure that e-learning can support the sharing of knowledge and the development of new knowledge which are vital for SMES. If e-learning and e-resources are to be a medium for doing this, there is a need to look at the social processes involved in online interaction and to develop new processes and solutions which can support collaboration and co-operation.

4. The fourth issue, arguably more fundamental than the others, was the question of what problem is e-learning trying to solve. It is difficult to evaluate the benefit of e-learning to SMEs if it is unclear what problem it is trying to solve and whether technical developments are going to radically change its characteristics.

The second seminar concentrated on recommendations for policy and an attempt to identify future research needs. The aim was to transcend the gap between research, policy and practice.Participants identified a wide range of potential policy issues. Three main themes were identified as warranting further research:

1. The first was to research how SME managers and SME employees are using ICT within their everyday work and to use this as a basis for developing environments and opportunities for e-learning.

2. The second was to recognise an ongoing paradigm shift from e-learning to knowledge management and to initiate research on knowledge management including ‘How does it happen?’ - ‘What technologies support it?’ - and ‘Chaos management, complexity management, and the management of uncertainty”.

3. The final need was for more research into social networks in and around SMEs and their impact on knowledge sharing, development and learning.

The second paper, by David Guile, is entitled ‘Epistemic Activity, Organisations and Learning: towards a framework for analysing the use of e-resources in Small and Medium Enterprises’. Guile says that discussions in EU policy about the ‘knowledge economy’ have tended to emphasise the contribution the technological output of knowledge makes to economic development and the use of ‘e-learning’ to widen access to education and training so that more people acquire the qualifications needed for ‘knowledge work’. By focusing on the increase in knowledge-based products and services and on the demand for qualifications, policymakers and educational researchers have tended to overlook issues relating to the concept of the knowledge economy and its relationship to learning and the use of e-resources.The first issue is that the knowledge economy does not just hinge on the increased presence of technological and informational products of knowledge and on the volume of qualifications. It also means that ‘knowledge cultures’ (Knorr Cetina, 1997) have spilled and woven their tissue into the fabric of society and workplaces. A knowledge economy is not just an economy of more experts, more technological gadgets, more specialist interpretations, any more than lifelong learning is primarily concerned with the accumulation of more qualifications.

11

The second issue is that although the level of educational qualifications workers hold is an important issue, qualifications are not the determinants of ‘knowledge work’ (Guile, 2001). Knowledge work presupposes distinctive forms of social organisation and social practices that enable people to learn to produce, circulate and use knowledge in workplaces. Knowledge work also presuppose constant renewal and transformation, demanding an individual and collective capacity for learning.The aim of the paper is to provide a framework for analysing the symbiotic relationship between knowledge cultures, learning and e-resources. In order to do this, the paper introduces the new concept of ‘epistemic activity’. The concept refers to participation in social practices, including the use of their associated artefacts, which facilitate the production and application of new knowledge. The paper starts by identifying the premises that inform the concept of epistemic activity; it then proceeds to use the concept to analyse the symbiotic relationship between knowledge cultures and the production of new knowledge. Having done so, the paper concludes by outlining a framework to analyse one particular expression of epistemic activity; namely the deployment of e-resources in SMEs designed to foster learning and knowledge creation.

The third paper, by Bernard Blandin addresses the topic ‘Localization of software and learning material for SMEs: how is it possible?’ The paper revisits research undertaken ten years ago through the COMETT programme (Auvinen, Blandin et al., 1994), which aimed to identify cultural differences that have an impact on learning in order to facilitate the localization and adaptation of learning material across boundaries. This approach appears to be still valid, but, says Blandin, should be deepened, since further research conducted in the meantime and sociological concepts such as the “social world” (Blandin, 2002a) and “learning culture” (Blandin, 2003) are likely to help refine the approach. It is not enough to adapt content: the “form” of the learning material has also to be localized. Designing effective learning environments has to take into account social and cultural factors, but also learning software and learning material usability has to be considered as “situated” (Blandin, 2003). Occasions and conditions for the use of learning material arise directly out of the context of learning activities which are implemented.Blandin asserts that the localization process as well as the design process for software and/or learning material cannot be isolated from the design or the adaptation of learning situations in which the software or the learning material is to be used. The paper presents background research on cultural aspects of learning and their particular relevance to SMEs, then goes on to examine how the “situated usability” of software and learning material impact on these results. Finally, it proposes a two-tier development process for the successful design and/or localization of software and learning materials. The paper looks at learning styles, identifying an ‘Entrepreneurial City’ learning culture and learning style for SME managers and employees. Blandin concludes that it is of the utmost importance that e-learning materials are inserted in the right context of use, which has to take into account the learning culture and the learning style of the ‘Entrepreneurial City’.

The importance of local context is taken up in the fourth paper, ‘e-learning Content and Software Localization’, by George Bekiaridis. The paper says that besides technology itself, the major factors that impact on the successful implementation of vocational e-learning programmes are content and software. The majority of e-learning content produced globally is in English. However, especially for technical subjects there is demand for content in native languages. It is obvious that content localization is becoming ever more necessary in Europe.

12

The paper explores a number of the issues that influence the localization of vocational e-learning content and proposes methods for the effective localization of e-learning. Bekiaridis says localization is the process of adapting a product or service to a particular language, culture, and desired local ‘look-and-feel’. Ideally, a product or service is developed so that localization is relatively easy to achieve - for example, by creating technical illustrations for manuals in which the text can easily be changed to another language and allowing some expansion room for this purpose. This enabling process is termed internationalisation. An internationalised product or service is therefore easier to localize. The process of first enabling a product or service to be localized and then localizing it for different national audiences is sometimes known as globalisation. The paper looks at the different stages in localisation of software including technical translation and cultural adaptation.

The fifth paper by Graham Attwell and Mike Malloch considers ‘Prospects for the development of software that truly supports collaboration and learning in SMEs’. Most attempts to develop ICT-based learning have failed to develop appropriate new models of pedagogy and curriculum development. Indeed, it might be argued that most ICT-based learning has been regressive in promoting more rigid, course-driven learning. Why should this be so? The authors suggest that the major problem lies in a failure to understand the relationship between the media being employed - predominantly learning platforms and groupware technologies - and the issues of pedagogy and curriculum. All software platforms have an inherent pedagogy built into their fundamental design constraints and decisions. Different platforms support (or fail to support) different ways of learning. In many cases ICT is used most effectively simply for information exchange, and when learning does take place it does so outside the special ICT based e-learning technologies provided.There is a need to look more closely at how different software platforms and applications support learning, and to develop a better understanding of the pedagogic processes which take place when learners engage with these technologies. In the paper Attwell and Malloch examine the use of collaborative environments for knowledge sharing, and ask how existing software environments (especially software used for communication within enterprises) support collaboration. They go on to discuss what kind of systems might be developed to strengthen knowledge sharing and learning within workplace environments. They say they are not suggesting that the development and implementation of new software can by itself transform the workplace into a rich learning environment. But they are suggesting that there is a need for software developers to work together with SMEs and SME associations to iteratively develop and test new software tools and systems that can facilitate collaborative knowledge sharing and development in real-world work contexts. They contend that the focus on e-learning software and platforms (and the hype around e-learning) has diverted attention away from the learning potential of email and groupware products that are being used to a greater or lesser extent in larger corporations and in some SMEs. They secondly argue that because of the ‘distance’ between developers and users there has been little attention paid to the evaluation of software - not in a technical sense, but in terms of its potential for enabling knowledge development and learning.

The sixth paper, ‘A Framework for the Evaluation of E-learning’, by Jenny Hughes and Graham Attwell looks at different theories, strategies and practice in the evaluation of e-learning. The authors point out that e-learning has been an area that has attracted considerable research and development funding. If this investment is to be maximised, it

13

is imperative that robust models are generated and used for the systematic evaluation of e-learning. Evaluation tools are required, which are flexible in use but consistent in results. Hughes and Attwell quote the American Society for Training and Development (2001) in saying:

“Although recent attention has increased e-learning evaluation, the current research base for evaluating e-learning is inadequate…Due to the initial cost of implementing e-learning programs, it is important to conduct evaluation studies.”

The paper is written from the perspective of professional evaluators and evaluation researchers, rather than VET researchers or even e-learning practitioners. It is underpinned by a set of assumptions about the evaluation process and also based on personal research evidence that suggests that standards are more likely to be improved by diversity, flexibility and experimentation than through standardisation. The first section outlines the principles and assumptions on which their evaluation work is based. The second section is a review of over 200 evaluation reports on e-learning with the focus on identifying the type of report rather than its content. This is followed by a proposed framework for reporting on and classifying e-learning evaluation together with some lessons they have learned from their own evaluations. The overarching conclusion reached by Hughes and Attwell is that the evaluation of e-learning is fundamentally the same as the evaluation of any other learning, but with particular groups of variables playing a more prominent role and the impact of others differing significantly from their impact in traditional learning. Specifically, they suggest that political factors are crucial because the nature of e-learning challenges conventional theory and established benchmarks.

The seventh paper, ‘Practical evaluation interventions in understanding informal learning within SMEs’, by David Slater, continues the evaluation theme, focussing on approaches and methodologies for evaluating informal learning in SMES. Slater starts by quoting Billett (2001) who suggests that informal learning is the primary form of vocational learning – both initial and ongoing – for most people in the workforce today. This is presumed to be especially true within smaller organisations. The paper looks at the application of the work of Argylis (1965) when evaluating company culture and internal interactions. It goes on to examine the work of Kleiner and Roth (1997) in mapping organisational, as opposed to individual, ‘Learning Histories.’ Slater examines the practicality and relevance of these theories and approaches to evaluating informal learning in SMEs. He proposes there are three main stages that can be explored in attempts to increase the role of informal learning in SMEs:

1. Identify those barriers currently in place. It is suggested that this be done through exploring the ‘theories-in-use’ and ‘espoused theories’ currently governing behaviour. However, other methods may be more appropriate depending on circumstances.

2. Explore interventions that will serve to overcome those barriers. These should be reviewed on an ongoing basis and moderated as appropriate.

3. Put in place mechanisms for the ongoing review of informal learning within the SME.

14

It is recognised that an SME cannot normally be expected to spend considerable time or resources on these activities, save as a response to recognised and acute difficulties. However, where relatively simple mechanisms can be put in place, requiring the minimum of external support, SMEs can considerably boost the potential for informal learning to take place among their staff.

The eighth paper, by Eduardo Figueira, is on ‘Evaluating the effectiveness of e-learning strategies for SMEs”. Figueira says that the development and use of any e-learning programme or strategy represents a considerable individual, organisational and social investment. For this reason, the effectiveness of any e-learning provision should be evaluated. Without knowing the efficacy of e-learning strategies, one cannot know if it is worth using them or not. So, measuring effectiveness constitutes a very useful tool for basing decisions on the adoption of any e-learning strategy. Built-in programme evaluation allows trainers and others responsible to monitor e-learning programmes and provision and make changes for improvement.

Globally, programme effectiveness can be evaluated using five types of approach:

1. Based on the programme goals,

2. Based on the decision-making process,

3. Goal-free,

4. Based on an expert’s knowledge, and

5. Naturalistic approach.

Figueira proposes a global framework for evaluation comprising a conceptual matrix to guide development of criteria and indicators for the evaluation of the effectiveness of e-learning. The framework, intended to be utilized for measuring the effectiveness of e-learning strategies, is based on a revised version of Bennett’s hierarchy and measures three main levels: participation, reactions and results.

In the final part of the paper Figueira proposes criteria to evaluate effectiveness of e-learning. He says it appears that measuring the effectiveness of e-learning should take into consideration the nature, characteristics and objectives of the e-learning strategies. For this reason, approaches to the evaluation of the effectiveness of e-learning should take into account the following e-learning dimensions:

1. Organisational & Management dimensions,

2. Pedagogical dimension,

3. Technological dimension,

4. Ethical considerations,

5. Learning assessment and certification, and

6. Evaluation Strategy.

Paper nine by Klaus Reich and Friedrich Scheuermann presents a case-study of ‘E-learning challenges in Austrian SMEs’. They say e-learning is changing the way enterprises gain competitive advantage through improved human performance. Small and medium-sized enterprises have the problem that e-learning technologies, methods and strategies have mostly been developed to meet the needs of large enterprises and cannot

15

be exactly transferred to their needs. Based on a survey by the authors of e-learning in Austrian SME’s (Scheuermann & Reich, 2002), the current state of ICT based learning in Austria is analyzed and suggestions for better implementation and use of e-learning are made, with a special focus on cooperative e-learning approaches. If better e-learning solutions are to be found, Reich and Scheuermann suggest that groups of SMEs have to be identified. Cooperation and partnership are essential because it is too expensive to develop solutions for individual SMEs. SMEs have to develop new organisational structures for the implementation of cooperative and collaborative forms of learning. At the moment most SMEs implement isolated learning solutions that fail to meet the needs of cooperative and collaborative forms of working and learning. New approaches are needed in terms of understanding the broad context of learning. Pedagogical approaches in Austrian SMEs are often inadequate and do not meet the needs of e-learning. It is also suggested that there is a lack of flexible learning solutions available to SMEs. There is still much to be discovered about how people learn using different technologies, particularly in relation to interactivity, and how materials can be developed and structured to enable all learners to make effective use of them. Learning at the workplace should partly replace ‘old’ teaching in classrooms and face-to-face courses away from the enterprise, but this is difficult to implement. Integration is important: learning should not take place besides work, but with and through work. Learning at the workplace is different from learning at school and in a classroom. A problem-based approach is required: rather than using a pedagogic models based on linear progression through subject knowledge it is more relevant to base learning in the solution of problems arising from practice. Flexible learning media are needed.

Paper ten, by John Munro and Philip Crompton, is entitled ‘Assessing the Application of Online Learning in a Work-based Setting’. Munro and Crompton say globalisation has increased the pressures on companies and national economies towards greater competitiveness. Much faith has been put by governments and educational policy makers in the development of knowledge economies where information becomes the key component for creating competitiveness. This has led to increasing interest in the development of work-based learning, lifelong learning and the application of Web-based learning, or e-learning as it has come to be known, as mechanisms for bringing learning into the workplace, the home and the community.

Munro and Crompton provide a case study of the e-learning experience of a medium sized enterprise in Central Scotland. The purpose of the investigation was to assess the impact of online learning in a work-based setting and to determine if the course content was appropriate to the learning needs of the employees. The company piloted a course for 51 employees based on the Finance Module of the Certificate in SME Management (CSMEM). The module was provided in online format by the University of Stirling. The case study identifies several pertinent issues relating to the role of e-learning in the workplace, including the practicalities of learning by ICT, time and workload pressures, company culture, and the learning preferences of the employees.

The final paper, ‘A holistic vision of the future of e-learning’, by Kees Schuur, provides a wide-ranging discussion of the future of e-learning. In his introduction, Schuur says e-learning environments have become readily available during the last 5 years. In addition there is a great deal of ‘hype’: if we believe the claimed advantages of e-learning, for example cost savings, Just-In-Time learning, at the learners’ own pace, at any place, any time, flexible learning styles, etc, the future looks bright. Despite this, only a small

16

percentage of learners, especially in industry, are currently using e-learning environments.

Schuur explores several factors that influence the use of e-learning environments from the point of view of developments in different disciplines and in society, and provides a vision of the future for ICT based learning. It outlines for development of e-learning environments in the future. It is not intended to be an exhaustive list, but rather to show the direction of development and to be a starting point for further discussion. Examples of different systems and platforms are included to illustrate the discourse. What kind of impact will new technologies have on society? Where are we heading? How can we shape the development of technology? And what will the impact be on our learning and the instruments we use for learning, such as e-learning?Attempts to predict the future are always perilous. Schuur points out that there are at least three ways of forecasting:

(1) One way to create a picture of the future is to identify present technologies and based on this to describe possibilities for the near future.

(2) A second way is to look back at historical developments and to extrapolate from these into the future.

(3) A third way is to combine elements from different disciplines and view different combinations as pictures of the future.

Common Themes:There are a number of common themes that emerge from the papers and the discussions that took place in the Stirling and Brussels seminars:

The importance of informal learning and the need to take this into account in the design and implementation of e-learning.

The implications of constructivist theory with its emphasis on collaborative learning.

The need to integrate e-learning with other social and cultural processes, implying local adaptation.

The requirement to evaluate the effectiveness of e-learning before embarking on its whole-scale implementation.

The nature of the knowledge required in SMEs and the significance of a move to knowledge management.

Disappointment with what appears to be the relatively slow rate of uptake of e-learning (but we tend to forget how recent a phenomenon the Web is).

The need to follow the logic of partnership in the design and implementation of e-learning strategies. Individual SMEs are unlikely to be able to afford the investment required to develop effective e-learning

17

strategies by themselves: what is needed is a community of learning practitioners.

The papers published here are intended to be the first step in the development of such a community.

ReferencesAmerican Society for Training and Development (2001)

Argyris, C. (1965) Organisation and Innovation, Homewood, III: R.D. Irwin

Attwell, G. (forthcoming) The challenge of eLearning in small enterprises: issues for policy and practice in Europe

Auvinen, A.M. Blandin, B. Chapman, P. Dondi, C. Evans, R. Goldstone, L. Sharratt, R. edited by Johnstone, A. (1994). Cultural impact on learning : A practical guide to managing the effective adaptation of learning materials across international boundaries.

Billett, S. (2001) “Participation and continuity at work : A critique of current workplace learning discourses. Context, Power and perspective : Confronting the Challenges to Improving Attainment in Learning at Work”. Joint Network / SKOPE / TRLP International workshop 8-10th November 2001, Sunley Management Centre, University College of Northampton. Available at the informal education archives: http://www.infed.org/archives/e-texts/billett_workplace_learning

Blandin, B. (2003) Usability Evaluation of Online Learning Programmes: a Sociological Standpoint”, in Ghaoui, C. (ed). Usability Evaluation of Online Learning. Idea Group Inc. (PA), pp.313-330.

EU (2000), The Lisbon strategy for economic, social and environmental renewal, http://europa.eu.int/comm/lisbon_strategy/index_en.html

Guile, D. (2001) “From ‘credentialism’ to the ‘practice of learning’: rethinking learning for the knowledge society”, Policy Futures in Education, 1, 1.

Kleiner, A. and Roth G. (1997) Learning Histories. A New Tool for Turning Organisational Experience into Action. http://ccs.mit.edu/LH/21wcp002.html

Knorr Cetina, K. (1999) Epistemic Communities, Boston: Mass:Harvard Education Press

Lundvall, B.-Å. “The Social Dimension of The Learning Economy,” DRUID Working Papers 96-1, DRUID, Copenhagen Business School, Department of Industrial Economics and Strategy/Aalborg University, Department of Business Studies . http://ideas.repec.org/p/aal/abbswp/96-1.html

Scheuermann, F. and Reich, K.(2002 in press): ELearning in Austrian SMEs. Innsbruck, 2002.

18

19

Graham Attwell, KnowNet (UK)

Developing new pedagogies and learning in SMEs

European seminarsIn mid 2002 KnowNet, a small research company based in North Wales, were awarded a grant by the European Commission DG Education and Culture to host two seminars around the theme of ICT and learning In Small and Medium enterprises. The seminars were intended to build on the experience and outcomes of existing European projects and research in the field in order to focus on the issues for developing the use of ICT for learning in SMEs and produce policy recommendations for the European Commission and Member States.The first seminar was help in Stirling, Scotland in November 2002 and the second in Brussels in February 2003Whilst the themes for the seminars were broad they the seminars focused in particular at the issue of pedagogies for learning in SMEs and at the evaluation of learning using ICT in SMEs. This included the following issues:

How can we measure the effectiveness of e-learning in SMEs? How can we measure the effectiveness of different models for e-learning in SMEs What constitutes a rich learning environment for e-learning? How do different social factors impact on learning in SMEs? What pedagogic models are being used for e-learning in SMEs? What pedagogies can support the use of ICT for e-learning in SMEs What is the relationship between different pedagogies and different technological

platforms? What is the role of the teacher or trainer in supporting learning using ICTs in

SMEs? What is the relationship between changing business practice and e-learning in

SMEs?

In total twenty-six researchers attended the two seminars, drawn from nine different European countries. They were selected for their previous work in this area. The seminars were planned with a dual agenda. Although the programme provided for the formal presentation of research papers, this was accompanied by exploratory workshop activities designed to encourage reflection on the issues and problems and to develop policy recommendations.

Two parallel workshop strands were developed – one around pedagogies and e-learning in SMEs and the second around evaluation and e-learning in SMEs. This paper provides an account of the workshop activities on pedagogies and SMEs. It is not intended as an academic or research paper as such, but rather as providing access to the broad ranging discussions that took place.

Definitions of e-Learning and SMEs

The issue of what comprises an SME is not just a semantic issue. The major issue raised in discussion was that the needs of SMEs depend on a number of different factors of

20

which one of the most important is size. The capacity for building the infrastructure and the capability to develop an appropriate learning cultures to support e-learning will be very different in micro enterprises (<50 employees) than in larger SME. Similarly, there will be differences between enterprises of around 50 – 100 employees and enterprises between 100 employees and the official upper level of 350 employees. The feeling of the group was that the greatest need for research and development was in the micro SME and in those with between 50 and 100 employees as these are the groups who have the most limited resources and thus experience the greatest problems in providing training for employees.

The definition of e-learning is very important in this context. The development of e-learning has been driven predominantly through the application of the virtual classroom paradigm for academic learning in the higher education sector. Thus, it has taken on many of the connotations of traditional learning in this environment. e-learning has tended to be seen as the use of digital media (including CD-ROMs, the World Wide Web and intranets), for the delivery of structured programmes which are materials based. This definition is problematic.

Many SMEs make extensive use of digital media for accessing technical manuals and for web searches and increasing numbers are involved in business to business (B2B) e-commerce applications. This is often taken as evidence that they are engaging in e-learning. However, we would challenge the idea that the use of ICT for information storage and retrieval necessarily constitutes learning. Rather, we would contend that for real learning to take place, this new information has to be applied in a way which then develops new mental models and schema, be they tacit or explicit. For collective knowledge development to take place, these new mental models have to be made explicit and shared. There is no doubt that CD-ROMs and the use of the web are useful and convenient ways of storing and retrieving information but unless that information is transformed through use it comprises neither learning nor knowledge development.

Nevertheless, e-learning in SMEs may be most powerful when it is integrated in company business processes through networks and systems for commerce and business development. The tools or software systems used in learning will often not be dedicated e-learning platforms but everyday business systems and software. In fact, email may be the most common learning tool for sharing information and new practices within the workplace, leading to knowledge creation. Critically, the learning materials may well not be bought in or prepared by e-learning specialists, software houses or multi media publishers but the products of employees’ documented and shared enquiry into their own practice.

This requires a far wider definition of e-learning than is used currently. It means taking into account all the electronic and digital resources – both formal and informal – used by the enterprise as potential learning and knowledge development tools. It also means looking at the totality of business processes in designing the e-learning environment. In this respect, e-commerce may be a considerable driver for change. For example, even the smaller SMEs are increasingly using computers and software systems as part of their business organisation for a range of different tasks and processes, including the procurement of materials, logistic organisation, sales and marketing and process control. It is possible that e-learning may be most effective when it is integrated with these e-commerce practices and with the use of digital media, computers and networks within work organisations and structures as it then becomes part of the culture of the enterprise.

21

Such a perspective breaks down the divide between what has tended to be seen as e-learning and what has been categorised as `knowledge management’ or management development. Pedagogically, it integrates the acquisition and practice of skills and knowledge with work processes, developing what some researchers have referred to as work process knowledge (Rauner and Bremer, 2001)

Barry Nyhan (2003) talks of “building organisations in which people have what can be termed ‘developmental work tasks’.” These are challenging tasks that ‘compel’ people to stretch their potential and muster up new resources to manage demanding situations. In carrying out ‘developmental work tasks’ people are ‘developing themselves’ and are thus engaged in what can be termed ‘developmental learning'.” The challenge for developing e-learning in enterprises is the integration of ICT in such a way that it supports developmental work tasks, rather than merely electronically cataloguing routine roles and tasks.

Participants in the Stirling seminar said that e-learning is “Always combined with other learning”. This is important. E-learning, whether in SMEs, large corporations or in schools cannot be viewed as an entity in itself. This is a critical idea when it comes to the design of learning environments for SMEs.

Managers attitudes

The analysis above is taken primarily from a research perspective. The scene changes when we move to the perspective of the SMEs themselves - or more accurately the SME manager or owner. Participants in the Stirling seminar spent some time discussing exactly what SMEs wanted from e-learning and concluded that because there is very little empirical research, much of this discussion was anecdotal.

Firstly, from our own personal experiences, SMEs want to break away from institutions and institutional based training. Despite the much-flaunted move towards flexibility in vocational education and training, VET institutions in general are still unable to meet the needs of SMEs. Managers are often looking for not `just-in-time’ but `absolutely-last-minute’ training. In order to provide this they need flexible learning materials and flexible methods of delivery. Current technologies and processes for developing and delivering materials are problematic in this respect. Arguably, this sort of training for SMEs is not a core function for VET institutions and we discussed the need for processes and systems, which allowed SMEs to develop and share their own materials (see below).

Thirdly, SMEs want to save money. They perceive e-learning as a potentially cheap and effective form of last-minute training. Given the present economic recession in many countries in Europe and increased competition, many SMES say they can no longer afford face-to-face training. The perception that e-learning is cheap is not without problems (see section on `costs’ below).

The difference between `needs’ and `wants’ is particularly important when it comes to SMEs. Few SMEs carry out systematic training needs analyses; neither do they have training officers. Few have full time qualified trainers. The CEDEFOP study (Attwell, forthcoming) on e-learning and SMEs concluded that the attitude of individual managers is the single most decisive factor in influencing the development of ICT for learning in SMEs. Managers tended to be pragmatic in their wishes with regard to training rather than following any scientific method of assessing needs.

The CEDEFOP study found little support by managers for introducing e-learning. More surprisingly there was little support for SMEs or SME managers in developing e-learning

22

in their enterprise. A number of the companies in the, admittedly small, sample were using e-Learning because of their role in supply chains. In Italy, regional organisations were trying to stimulate the use of e-learning. In Birmingham, in the UK, the local government was promoting e-learning through an EU financed ADAPT project, In the Austrian Tyrol, the Chamber of Trade has some involvement. However, in each case the support available was both quite specific and quite limited. There was no natural point to which managers could turn for help, nor did the use of e-Learning appear to be a major issue for the different support agencies and networks in which the SMEs were involved.

Cost benefits

The question of who pays is perhaps the most pressing, unresolved and complex issue in the development of e-learning for SMEs. The issue is significant at a number of different levels. E-learning has been widely promoted as a cheap or cost effective answer to the policy issue of engendering and promoting Lifelong learning within SMEs. Yet, most researchers, and many consultants, are adamant that e-learning cannot be seen as a cheap solution. There may be economies of scale to be reaped in the future but even this is open to doubt. Despite the many ROIs – or Return on Investment Studies – it is unclear whether e-learning is at present competitive when compared with traditional learning methods. E-learning must, therefore, be justified in its own terms – in opening up access to Lifelong Learning or in providing richer learning environments for learning.

It was generally agreed that the prime task was to develop the efficiency and effectiveness of e-learning and promote the benefits to SMEs. However, there were two counter posed views, expressed within the Stirling seminar, on how this issue should be addressed in the future. One was that as e-learning is too expensive for SMEs and is likely to remain so for the foreseeable future, the state should subsidise e-learning by fostering new ideas and supporting the implementation of new technologies in order to provide access to new learning infrastructures and cultures. The second was that, despite present low rates of take up, e-learning should be developed through the market.

Standards and technology and content

Access to learning materials is a major issue in the development and implementation of e-learning in SMEs. As the European Commission notes in a recent document

“the development of the `digital economy’ and the wider use of the Internet and of computer and networking equipment has raised accessibility to multimedia to unprecedented levels, thus enhancing opportunities for producers and consumers…However, this rapid evolution does not seem to have been matched in the new educational content sector.”

The consensus was that there is a lack of European educational multimedia content coming from institutional, professional and industrial sources in the education, publishing and educational software sectors. After an initial phase of enthusiasm, often described as “hype”, there are growing doubts about the real demand for educational e-content, and about its relevance for improving learning”1. Our studies would, in general, bear out this assessment.

1 European Commission Directorate-General for Education and Culture, Open invitation to tender No DG EAC 21/02 for the provision of services concerning the carrying out of studies in the context of the eLearning initiative.

23

An examination of an extensive catalogue of e-learning materials available in Italy has revealed an extremely limited range of subject area provision. Most materials are for technologies, mainly the use of standard software packages, and for networking. The next largest category was learning materials for managers and for management activities such as marketing and thirdly, materials for language learning. Beyond this the provision is very limited. Obviously these materials are largely targeted at technical, professional and management employees (or white collar workers). The situation is further exacerbated in Europe because of the diversity of languages. E-learning was pioneered in Europe in the university sector and most learning materials were provided in English. Whilst this may be acceptable in a higher education environment, most vocational learners require learning materials in their own language. Dondi and Zucchini (1998) have drawn attention to the problems of market-led software and materials being provided in different language versions, especially those languages serving smaller population groups. Furthermore, whilst universities could draw on very broad discipline areas and on common discipline structures between countries, the learning needs of employees in SMEs are often quite narrow and very specific, with limited international (or even national) markets. There is a growing debate on the issue of globalisation and localisation in software and learning materials and the group agreed that the task of translating materials to different languages is not a straightforward technical issue but involves significant cultural transformations.There have been many policy initiatives and measures to stimulate market provision of e-Learning materials. A considerable number of projects have encouraged universities to provide learning programmes and materials for SMEs. However, it is open to question as to how effective these initiatives have been – and to the sustainability of project led developments.One of the key debates revolves around standards, both for learning materials and learning platforms. Standards mean that learning materials and systems would be interoperable – and therefore more sustainable. Standards would enable and facilitate the development of learning materials within an individual enterprise or cluster of enterprises. Within a wider vision of e-Learning, that is, e-learning as a contribution to knowledge management or collective knowledge development, a learning ‘object’ or discrete piece of learning material is seen as the fusion of the outcomes of learning together with the knowledge created, at all its different stages and in all its different forms. Learning objects or materials could be created by the learner themselves as a result of their engagement in business activities. The primary role of the computer based learning platform would not be in the delivery of the materials but in facilitating the transformation and communication of ideas as knowledge. The computer or ICT based learning environment would be primarily a process tool to support the creation and transformation of knowledge. The overwhelming advantage of this approach is it allows learning materials to reflect and support the different contexts in which learning takes place in SMEs.The development and implementation of learning platforms is yet another difficult issue. In universities early development tended to be through the use of platforms which were locally produced and maintained. More recently, the trend has been towards adoption of (increasingly technically sophisticated) platforms developed by private (mainly American) e-Learning technology specialists. Over the last year, as the market situation has become tighter, there has been a spate of takeovers and mergers, accompanied by ‘downsizing’ in those remaining companies.

Participants in the Stirling seminar were sceptical that technological advance has been accompanied by improvements in the pedagogies that these platforms facilitate. The cost

24

of server software applications, let alone the difficulties in installing and maintaining server-based systems, is beyond the reach of most SMEs. This leaves them a limited range of options. CD-ROMS, although useful in some contexts, do not allow communication between learners. Buying off-the-shelf courses from providers operating their own platforms and servers restricts choice and there is little or no opportunity for SMEs to develop their own learning materials or to manage and develop their own knowledge systems. Once again, the obvious solution is for clusters or groups of SMEs to collaborate on a regional basis, with umbrella organisations or networks offering software platforms as well as learning materials. We have nothing against standardised commercial learning platforms. Our major point is that the learning technologies and systems should form part of wider knowledge sharing and development networks. It is also very important that they support the emergent standards for e-learning materials in order to allow interoperability and sharing of development efforts, and to facilitate localisation.

New paradigms

Towards the end of the Stirling group discussions, participants started looking at the problem of forecasting future developments and what the implications might be for SMEs. We agreed that the real challenges were to stop thinking within existing paradigms of learning and e-learning and to develop new ones. Whilst this discussion was inconclusive and a little fragmented, it raised a number of interesting issues.

One was the nature of informal (or tacit) knowledge as opposed to formal knowledge. E-learning until now has concentrated on formal learning and the reproduction of traditional courses through digital media. Yet most studies suggest that, at least for skilled work, it is the tacit knowledge that is most powerful within SMEs. The question is how e-learning systems and architectures can be used to support the development and sharing of tacit or informal knowledge.

The second issue was the tension between individual learning, which e-learning tends to be, and the social nature of more traditional learning. The social dimension is recognised as an important motivator and driver for many learners and a positive social environment can improve the effectiveness of learning as well as allowing a wider range of methodologies.

Thirdly, if the sharing of knowledge and the development of new knowledge are important future developments for SMES and if e-learning or e-resources are to be a medium for doing this, we need to look at the nature of the interactions in e-learning and to develop new processes and solutions. We also agreed on the need to move towards a new paradigm of continuous learning based on the application of new working principles.

Fourthly, but arguably more fundamental, was the question of what problem is e-learning trying to solve. Is it of benefit for SMEs at the present stage of technical and pedagogic development? Clearly SMEs are using ICT technologies for access to information but they see that as very different to signing up for e-learning ‘courses’. It was said that some SMEs think they need e-learning but they do not necessarily know why! In many cases it is being driven by the need to employ new information technologies but the lack of any integration with e--earning is holding back progress.

Fifthly, there is a separate issue around how to persuade employees themselves, rather than their bosses, to use new technologies for learning and how are individual learning needs reconciled with collective ones. That is, how can we resolve the PSP paradox - providing pleasure, sustainability and profit all at the same time! We were unconvinced that we knew the answer and doubted there was one.

25

Brussels seminarThe second EC sponsored seminar was held in Brussels in February 2003. The seminar was designed to build on the work of the first meeting in Stirling. Following a revue of the outcomes of the first seminar, participants were asked to develop a series of policy recommendations, if possible directed at local, national and European level. The aim was to transcend the gap between research, policy and practice.

Learning culturesThe first session reviewed the Stirling outcomes. It was felt that not enough attention had been paid to the learning culture in SMEs. The learning culture in SMEs differed significantly from that in larger organisations. SMEs had a different value system. SME managers, and employees in SMEs had a stronger identification with the enterprise and with the role the enterprise plays in local and regional economies. Whilst larger companies were driven by maximisation of profit, other goals, such as developing local employment, were important to SMEs. Knowledge is developed and shared in different ways in SMEs. Whilst in larger companies there are formal procedures and processes for the sharing of information and knowledge within the organisation, informal processes and peer group networks and local communities were often more significant in SMEs.The culture of SMEs is characterised by strong interpersonal networks, both within the enterprise and between different enterprises. Clubs and associations play a key role in the culture of local SMEs and are frequently a central source of ideas and new knowledge. At the same time it was recognised that there are important barriers to the sharing of knowledge between SMEs. Knowledge may be seen as providing competitive advantage in local economies, particularly between companies in the same sector. This was seen as a complex issue. Undoubtedly there are some issues and challenges, which are shared between different SMEs and enterprises, are happy to work together to respond to these issues. However, in other cases sharing knowledge would be seen as helping competitors.In this scenario informal learning plays a far greater role than in the more formally structured larger organisations. Learning is more likely to take place as a result of challenges to practice and equally more likely to take place within peer group interactions or through communities of practice. This has significant implications for the design of e-learning systems for SMEs.

Regulation and certificationThe issue of regulation and certification caused considerable discussion. Certification of learning was seen as a cultural issue. In some sectors in some occupations in some countries certification of learning was a vital issue. But in other sector and countries certification of learning was not seen as a pressing issue, more important was the learning and experience gained. The regulation of learning and training could lead to a formal ‘e-market’ for learning – but at the same time could prescribe the form and nature of learning taking place and fail to recognise and support the importance of informal learning.

Possible solutionsThe second session in the Brussels seminar began to consider possible solutions to the issues that had been identified. It was seen as important to create “Cultures of Learning” within and between SMEs. In this regard it is necessary to identify and involve the different stakeholders who may come from quite diverse contexts and backgrounds. A major question was as to whether it is possible to introduce e-learning from outside the culture of the local and regional SMEs. Whilst many universities and training organisations in Europe had tried to promote the virtues of e-learning, they were coming in from the outside and often did not relate to the culture of the target groups. It was often

26

difficult for those outside the SME culture to understand the needs and concerns of SMEs. Even language could be a problem with SME managers unable to relate to the language and use of language by researchers and ICT based learning developers.In looking at potential solutions it was important that e-learning was not approached as an issue or solution in itself. The discussion over e-learning for SMEs was only an ‘amplification’ of the overall question of the provision of learning and training in SMEs. E-learning would not always be the most appropriate form of learning and would nearly always need to be complemented by other forms of learning using more traditional technologies. It was not enough just to adapt learning materials for use in SMEs. We need to adapt the entire learning environment and the learning models we deploy – what one participant called “engineering of the form”.

Actions for pedagogyThe final session of the workshop programme focused on actions and recommendations for actions to develop new pedagogies and promote the use of e-learning in SMEs. Once more, the discussion was wide ranging.

Promote Cultures of LearningThe most critical issue was to promote a culture of learning in SMEs. This entailed taking actions to involve a wide range of stakeholders including training organisations and providers, local and regional government organisations, SME associations, regional economic development organisations, sector organisations, employers organisations, trade unions and professional bodies, supply chains and community bodies – including vitally those clubs and societies to which SME managers and employees belonged – as well as SMEs themselves. This was seen as a long-term strategy but one without which no amount of e-learning directed initiatives would succeed.

Mobilize SME managersAs part if such an approach it was particularly important to mobilize SME managers through the development and support of sector networks, regional networks and regional learning networks.

Focus on organisational developmentE-learning could not be viewed as an issue or aim in itself. In fact the present focus on learning and training was even seen as unhelpful, particularly given the vague and unfocused nature of discourses on lifelong learning. Instead e-learning had to be addressed within the context of organisational development, including more support for SMEs in introducing new forms of work organisation and new technologies within the workplace.

Support networks as a new developmental paradigmNetworks should be seen as a new developmental paradigm. Present approaches to learning from policy makers and planners still reflected an ”industrial age model” with lifelong learning merely seen as “more of the same for more people”.

Develop a differentiated approachSuch a new approach needed to be more culturally differentiated – “one size does not fit all”. Due recognition of the different needs of different sectors and regions is necessary – rather than a blanket approach to introducing e-learning in SMEs.

27

Recognise triggers for changeIt was also important to identify the different triggers that acted as a catalyst for change and to provoke the promotion of learning and a learning culture. These would vary again between sectors and regions and over time. When these trigger points were recognised structures were need to take immediate short-term actions to capitalise on opportunities.

Policy supportThere were many actions that could be taken by local, national and European policymakers to support the use of ICT for learning.Participants proposed a change in present funding policies to promote and support larger numbers of small projects based on communities of practice and running more intensively over shorter time periods. These projects should be focused on work based learning rather than of the provisions of distance learning courses or on virtual classrooms and should seek to develop and capitalise on informal processes of learning. Rather than look to external learning materials developers or training bodies to supply learning programmes they should focus on SME employees as providers (or senders) of knowledge. The development of competence in SMEs was seen as predicated on the knowledge, skills and attitudes of the SME employees themselves. Present programme and project based evaluation was too slow and focused in the wrong areas. Instead policy makers should be developing the skills and culture of on-going, continuous self-evaluation based on communities of practice.At a European level inter-regional networks and projects were seen as particularly valuable. Similarly, projects support for clusters of SMEs offers great potential.For a number of years researchers and developers have been drawing attention to the issue of standards and interoperability. The European Commission should urgently address this issue. One possible policy would be to insist that all publicly funded projects must be based on Open Standards software and outcomes must be available through the General Public Licence. A further change in policy and funding could be to provide more support for the different industrial associations – chambers of commerce and sector organisations that represent SMEs – although there was some concern that they might prove too bureaucratic to lead the changes we sought to promote.The final discussion was around the future priorities for research in this area. Three main themes were identified.The first was to research how SME managers and SME employees are using ICT within their everyday work and to use this as a basis for developing environments and opportunities for e-learning.The second was to recognise and ongoing paradigm shift from e-learning to knowledge management and to initiate research on knowledge management including ‘How does it happen?’ - ‘What technologies support it?’ - and ‘Chaos management, complexity management, and the management of uncertainty”.The final need was for more research into social networks and SMEs and their impact on knowledge sharing, development and learning.Participants recognised the need for new forms of research and in particular to promote ‘accompanying research’ – with research taking place alongside developmental activities. This could help overcome the present divides between research and practice and researchers and practitioners. It was suggested that if the European Commission was to see fit to support future seminars of this type, it could prove fruitful to bring representatives of industrial associations and SME managers to participate along with researchers from the field.

28

References

Attwell G. (forthcoming), The challenge of e-learning in small enterprises: issues for policy and practice in Europe. Luxembourg: Office for Official Publications of the European Communities.

Dondi C. and Zucchini I., “Economic and Organisational Issues in the Trans-National Development and Delivery of ODL and eLearning Courses”, in Research Perspectives on Open Distance Learning, SCIENTER, Bologna, 1998Nyhan, B.; Cressey, P.; Tomassini, M.; Kelleher, M.; Poell, R. (2003) Facing up to the learning organisation challenge, Volume I – key issues from a European perspective. Luxembourg: Office for Official Publications of the European Communities.Rauner, F. & Bremer, R, (2001) Experten-Facharbeiter-Workshops als instrument der brufswissenschaftlichen Qualifihationsforschung’ ‘Mensch-Maschine-Inetraktion’ Arbeiten und lernen in rechnergestuetzen Arbeitessystemen in Industrie, Handwerk und Dienstleistung, Eicker, Friedhelm A; Patersen W. (eds), Baden-Baden: Nomos

29

30

David Guile, University of London

Epistemic activity, organisations and learning: towards a framework for analysing the use of e-resources in small and medium enterprises

The Learning economyDiscussions in EU policy about the ‘knowledge economy’ have tended to emphasise the contribution the technological output of knowledge makes to economic development, and the use of ‘e-learning’ to widen access to education and training so that more people acquire the qualifications needed for ‘knowledge work’ (EU 1996). This focus is certainly understandable since policymakers, along with educational researchers, have a longstanding concern with the relationship between qualifications, employment and economic success. However, by focusing on the increase in knowledge-based products and services and on the demand for qualifications, policymakers and educational researchers have tended to overlook certain issues as regards the concept of the knowledge economy and its relationship to learning and the use of e-resources2.The first issue is that the knowledge economy does not just hinge on the increased presence of technological and informational products of knowledge and on the volume of qualifications. It also means that ‘knowledge cultures’ (Knorr Cetina 1997) have spilled and woven their tissue into the fabric of society and workplaces. A knowledge economy is not just an economy of more experts, more technological gadgets, more specialist interpretations any more than lifelong learning is primarily concerned with the accumulation of more qualifications. On the one hand, knowledge economies presuppose the presence of knowledge cultures, that is, the ‘whole set of structures and practices that serve knowledge and unfold with its articulation’ (Knorr Cetina (1997:70). On the other hand, they also presuppose constant renewal and transformation, in other words, they presuppose the creative use of material artefacts (e.g. computers) and symbolic artefacts (e.g. languages) to facilitate learning and knowledge creation.The second issue is that although the level of educational qualifications workers hold is an important issue, qualifications are not the determinants of ‘knowledge work’ (Guile 2001). Knowledge work presupposes the distinctive forms of social organisation and social practices, which enable people to learn to produce, circulated and use knowledge in workplace. They also presuppose constant renewal and transformation, that is, they depend upon an individual and collective capacity for learning.The aim of this paper is to provide a framework for analysing the symbiotic relationship between knowledge cultures, learning and e-resources. In order to do so, the paper introduces a new concept - ‘epistemic activity’. The concept of epistemic activity refers to participation in social practices, including the use of their associated artefacts, which facilitate the production of and application of new knowledge. The paper starts by identifying the premises that inform the concept of epistemic activity; it then proceeds to use the concept to analyse the symbiotic relationship between knowledge cultures and the

2 The paper uses the term e-resources instead of Information and Communication Technology to reflect the idea that such technologies constitute, in theory, a single resource which can be deployed to enhance working and learning.

31

production of new knowledge. Having done so, the paper concludes by outlining a framework to analyse one particular expression of epistemic activity; namely the deployment of e-resources in Small and Medium Size Enterprises to foster learning and knowledge creation.

The concept of epistemic activity

Background to the concept.The concept of ‘epistemic activity’ is based on recent work in two very different branches of the social sciences. The first is work in the Sociology of Science/Social Theory, which has turned the conventional argument about knowledge societies, namely that knowledge is a component of economic and technological paradigms on its head. Knorr Cetina (1999, forthcoming) has argued that the dehisence of knowledge has resulted in the growth of ‘epistemic (i.e. knowledge) cultures’ in an increasing range of contexts throughout advanced industrial societies: a development that implies that social, economic and organisational life may become part and parcel of particular knowledge cultures. Moreover, she claims that this development implies a more ‘inclusory’ approach as regards access to knowledge in all types of organisations. The second is recent work in Cultural Historical Activity Theory which has argued that, in order to understand the practice of knowledge creation, it is important to understand how ‘culture weaves things together’ (Cole 1996). The next section of the paper discusses the contribution of both theoretical traditions to understanding the symbiotic relationship between knowledge cultures and knowledge production. It concludes by arguing that their respective assumptions constitute the theoretical basis for the concept of epistemic activity.

The idea of epistemic cultures. Historically, epistemic cultures have always been associated with science. One of their defining features was the existence of an ‘experimental system’ that facilitated a process of reasoning which result in the production of new knowledge (Rheinberger (1992). Experimental systems, according to Rheinberger, were characterised by two distinct, but related, activities: ‘question generating’ and ‘answering providing’ activities (Rheinberger 1992). The former refers to the practice of identifying an object of inquiry, in other words, the ‘epistemic’ thing’ to be investigated through the research. The latter refers to the ‘technological object(s)’, that is, the stable moments of experimental research that are achieved through the establishment of criteria to guide the investigation, to structure the allocation and use of those artefacts involved in the investigation, normally in laboratories, and to determine the pattern of working. The distinction between the ‘question generating’ and ‘answering providing’ activities is an analytical and not a material distinction. It does not define fixed parts of a system, rather it reflects ‘places’ within it and how the practice of ‘question generating’ and ‘answering providing’ may change places with one another from time to time (Rheinberger 1992).According to Knorr Cetina, Rheinberger’s idea’s about epistemic things, which originated in the History and Philosophy of Science in an attempt to identify the epistemological significance of experimental systems, can be extended to cover all artefacts and activities involving expertise, irrespective of the context of activity. She feels that the defining hallmark of scientific inquiry - ‘question generating’ and ‘answering providing’ - is no longer restricted to science because there is increasing evidence of similar patterns of knowledge production in other contexts. Furthermore, she points out that, in light of

32

today’s software-based artefacts, Rheinberger’s original equation of artefacts with technological instruments is highly problematic. Knorr Cetina (1997:10) defines software-based artefacts (e.g., e-resources) as:

simultaneously things-to-be-used and things-in-a–process-of-transformation: (since, added emphasises) they undergo continual process of development and transformation…. These objects are both present (ready-to-be-used) and absent (subject to further research)

She argues that artefacts of this kind can be included in the category of epistemic things for the following reasons. They are subject to further development, they do not presuppose a specific pattern of use and they can be used to facilitate a more iterative relationship between question generating and answering providing activities3. Knorr Cetina, however, introduces a note of caution into her general argument about the emergence of epistemic cultures more widely throughout society. She acknowledges that these cultures are not emerging simply because they are a natural and progressive development of the application of knowledge within the economy. Knowledge cultures imply:

an ‘epistementality’ of particular beliefs about, for example, the correct distribution of knowledge, the naturalness of access to it, the particular ways knowledge should be handled and inserted into personal and organisational life. Such epistementalities also take form as particular organisational arrangements or roles and agencies (Knorr Cetina forthcoming).

This suggests that knowledge cultures have the potential to impact on political, economic and social life in radically different ways. From her standpoint, since e-resources can be deployed in a variety of ways and, moreover, depending on the mode of deployment embed and restrict or enhance inquiry, organisations have to address a new challenge. They have to re-think organisational beliefs about who will contribute to the process of determining how artefacts can be used to transform working and learning, how knowledge about this process of transformation is to be shared and so forth. A development that, as was noted earlier, presupposes a more inclusive approach as regards participation of workers in the production, circulation and application of knowledge.One of Knorr Cetina’s major achievements has been to reconceptualise our understanding of the emergence of and the implications of knowledge cultures in the social sciences. Her analysis, however, stops short of explaining how human mediate external symbolic and cultural systems and artefacts. For this reason, it is helpful to relate her insights about culture and artefacts to the concern in Activity Theory, a body of theory that has specifically addressed the relationship between activity, learning and knowledge production.

3 Knorr Cetina’s observations about the epistemic possibilities of ICT have some affinities with Zuboff’s (1988) distinction between the ‘informating’ possibilities of ICT. The chief difference between the two writers appears to be that whereas informating refers to the capaacity of ICT to provide access to information, epistemic things refers to the more iterative relationship between question generating and answering activities.

33

The role of activity within epistemic cultures.One of the central premises in AT, a premise that is an implicit feature of Knorr Cetina’s work, is that the subject-object relation has been transformed through insertion of artefacts. This means that the relationship between human beings and the external world is no longer predicated on either the idea of cause and effect or stimulus and response (Bakhurst 1991). Instead the subject-object relation is ‘mediated’ by artefacts and this enables humans to come to terms with the world, inscribe value in the world and, ultimately, to transform it.The cornerstone of the concept of mediation is the idea that individuals are active agents in their own psychological, cultural and social development (Cole 1996). At first sight, this can sometimes appear to be a rather commonplace statement; yet it provides, as Cole (1996) argues, the underpinning for the following assumptions that the concept of mediation rests upon. The first assumption is that development is not biologically pregiven; it occurs as a result of a reciprocal immersion in and engagement with the social, historical and social context, especially the artefacts, for example, concepts, ideas, schema and computers, available at a particular period in time. The second assumption is that human development is shaped and influenced by the practical activities in which human beings are engaged. The third assumption is that artefacts are both material and ideal; they are produced by human beings and the very act of production leads humans to inscribe significance and value into the design/construction of artefacts. These observations about artefacts constitute an important link between Activity Theory and Knorr Cetina’s ideas about epistemic cultures. Instead of viewing artefacts as an off-shoot of culture, Cole, along with Knorr Cetina, view them as the fundamental constituent of culture; they are created by human beings and used by humans to help them to coordinate their relationship with the world and with one another. This implies that artefacts do not exist in isolation as elements of culture: the view that predominates currently in much of the policy and research literature about e-learning. On the one hand, artefacts can not only be conceived in terms of a hetarchy of levels that includes cultural models, for example, question generating/answer providing models, but also in terms of their materiality and ideality. This is an important distinction because artefacts’ materiality and ideality afford different possibilities for their future development or application. It is possibility that according to Knorr Cetina, has enabled e-resources to become epistemic objects. On the other hand, artefacts exist as such only in relation to an activity, the activity system of cultural context in which they are used. Thus, epistemic possibilities are realised either in relation to the way in which they are currently utilised or in relation to yet to be imagined ways of use.From this perspective, all human activities are mediated through contact with artefacts and, as a result, have ‘mutlidirectonal consequences’ because activity modifies individuals in relation to others as well as the medium they are jointly engaged in (Cole 1996). This implies that it is important to remember that all forms of activity are rooted in culture or ore precisely in the distinctive form of social practice associated with particular cultures4

Cole illustrated this issue by drawing on Brofennbrenner’s (1979) ideas about the ecology of human development, Cole points out that human beings develop cognition through participating in different activities where they have to use symbolic and material artefacts

4 The terms ‘activity’ and ‘practice’ are sometimes seen as synomous, especially since writers in Actiivty Theory and Socio-Cultural Theory who employ these terms share a number of assumptions about the link between human activity/practice and the development of the mind (Lave 1993). In this paper, I use activity to refer to generic issues, such as, knowledge production and social practice to refer to the processes that influence or shape knowledge production.

34

and simultaneously engage in semiotic reasoning. Thus, from Cole’s standpoint, in order to understand how knowledge is produced, it is essential to identify how the cultural context ‘weaves artefacts and activities together’.The idea of weaving is a helpful notion when considering the relationship between epistemic activity and culture for a number of reasons. First, it alerts us to the importance of identifying those social practices that enable people to use artefacts and to engage in forms of ‘dialogic inquiry’ (Wells 1999) and, in the process, to foster learning and development and facilitate the production of new knowledge. Second, it sensitises us to the need for more inclusive frameworks as regards access to knowledge, the rules for the application of knowledge and so forth (Knorr Cetina forthcoming). Third, it indicates that the learning of the individual and the system are intertwined with one another (Engeström 2000).Taken in combination, the above discussion suggest that it is possible to formulate a tentative characterisation of the concept of epistemic activity. Epistemic activity implies engagement in ‘question generating and answering providing social practices, involving the use of artefacts that facilitate the production of new knowledge.

Conceptualising epistemic activity in organisationsThe concept of epistemic activity introduces a new way of conceiving of the knowledge economy/society debate. In contrast to the prevailing view in much of the policy literature that the increase presence of technological and informational products of knowledge, and, as a corollary, the primary educational concern is to use e-resources to increase access to existing educational resources, it presupposes a rather different spatial and temporal conception of the relationship between knowledge and economic and social development.Spatially, as we have seen, the concept presupposes the presence of knowledge cultures that unfold with its articulation in and between contexts. Temporally, it alerts us, on the one hand, to the longstanding existence of knowledge work in industrial society, albeit in the field of science since the eighteenth century. On the other hand, it also alerts us to the existence of a more pluralistic conception of knowledge work; a conception based on the emergence of new combinations of theoretical and everyday knowledge sometimes referred to as ‘polycontextual’ (Engeström et al 1995) or ‘transdisciplinary knowledge (Gibbons et al 1994) in modern work environments. The emergence of these new forms of knowledge presuppose that one of the new challenges in modern societies is to identify the distinctive form of social organization, social practices and artefacts, which enable people to learn to produce, circulate and use knowledge in workplaces. One of the most interesting attempts to conceptualise the evolving role of knowledge in economic activity has been advance by Victor and Boynton (1998). They present the relationship between economic activity, e-resources and organizational learning in terms of four knowledge-based responses that orgnaisations can make to the series of accumulating tensions, which manifest themselves in their production systems. Victor and Boynton define these responses as the shift from ‘craft’ to ‘mass production’ to ‘process enhancement’ to mass customisation’; they stress, however, that these different forms of production do not automatically emerge from one another. The transition from one mode of production to the next only occurs as enterprises respond to specific possibilities for development that surface in the existing mode of production.Their starting point is a modern conception of ‘craft work’, which they define as, the ‘invention and creation of high-priced and novel products’. They typify the form of knowledge that is central to economic activity as ‘tacit’ knowledge (i.e. residing with workers) and the deployment of e-resources as ‘customised’ design in this system of

35

working. According to Victor and Boynton, the progress from craft to mass production occurs at the point when enterprises begin to inquire how to leverage tacit knowledge, that is, to capture and codify it to enhance the volume of production or to extend the reach of customized systems to meet increased demand from customers.In contrast to craft work, systems of mass production have the following characteristics. They rely on the application of ‘articulated knowledge’ (i.e. highly codifed and procedural) and the use of pre-specified and automated e-resources to ensure the efficient and effective delivery of products and services. For Victor and Boynton, the paradox of mass production is that in following instructions without introducing any significant modifications, enterprises accumulate a new type of knowledge. They define this as ‘practical knowledge’ (i.e. knowing where the instructions are effective and where they are not). Capturing this practical knowledge to gain market advantage introduces process improvements; this development however involves enterprises choosing to compete on product and service differentiation rather than low costs. Furthermore, this strategic choice also presupposes the adoption of a more inclusive approach towards their workers than has normally been the case with mass production. It is not inevitable that all enterprises will chose to respond to these challenges, some will prefer to remain wedded firmly to mass production.One of the main challenges of process enhancement is to integrate production and innovation through the creation of work teams and workplace cultures that foster intensive and reciprocal communication flows within teams, to facilitate constant transformation in product and service delivery. Another challenge is to deploy e-resources to facilitate cross-functional flows of information to support micro-transformations of products and services, normally within a single organizational context. Although they employ different terminology, Victor and Boynton argue that process enhancement presupposes that management link different sections of the workforce, for examole, procurement, production, distribution, marketing in question generating and answer providing activities to enhance work flow.The final shift enterprises can make has come about as customers move beyond quality and seek products and services that offer exactly what they want. This pressure on enterprises can, in theory, result in a form of production that Victor and Boynton (1988) chartacterise as ‘mass customisation’. Rather than the sequenced line of mass production or the continuous improvement of process enhancement, ‘work is now organized as a complex, reconfigurable product and service system’.The transition to such a system is possible because process enhancement tends to generate a body of ‘architectural knowledge’ about process activities throughout an enterprise. This form of knowledge reveals to an enterprise the structures of its work processes, their interconnections, and the possibilities for building them either on each other or onto customers’ work processes to achieve new combinations or sequences. Achieving these goals involves a radical extension of enterprises’ conception of e-resources: they have to be deployed to integrate constantly changing network information processing and communication requirements to service the production of short-lived applications, unpredictable product/service requirements and to facilitate a new form of innovation. According to Victor and Boynton, this can be referred to as ‘co-configuration’, its hallmark is the integration of customers into enterprises’ business systems. A number of conclusions can be drawn from the proceeding analysis of the relationship between knowledge, e-resources and organizational learning. The first conclusion is that it is important to acknowledge that the epistemic activity that spurs the movement from one mode of production to the next results in the production of different forms of

36

knowledge and involves different forms of learning. This suggests that it may be valuable to identify a framework for analyzing the range of manifestations of epistemic activity enterprises might be engaged in. The second conclusion is that, in developing such a framework, it is important to avoid adopting am ‘anthropological’ conception of learning (Young 1999). In other words treating the relationship between culture and activity as though it were a naturally recurring process. This implies the need to differentiate more carefully between the purposes and outcomes of learning. The third conclusion is that it is important to identify the way in which the ideality of e-resources predisposes them to serve different purposes, in which case, it may be helpful to differentiate between such artifacts.

Conceptualising learning, artefacts and epistemic activityOne of the most promising suggestions for differentiating between the purpose and outcome of learning has been provided by Engestrom (1994). He has developed a typology which distinguishes three types of learning - ‘adaptive’, ‘investigative’ and ‘reflexive5’ - and three contexts for learning - ‘discovery’, investigation’ and ‘critique’ - in terms of the different relationships between learner and context that are presupposed by each type of learning. His ideas have a number of uses for any attempt to clarify the contribution that e-resources might make to knowledge creation and the embedding of epistemic cultures in enterprises. First, they represent a continuum reflecting the extent to which learning is related to but separate from discovery. There is no discovery of new knowledge produced in ‘adaptive’ learning because it takes contexts as given and unchangeable and the intention is to come to terms with the existing stock of knowledge. This is not to dismiss adaptive learning; it is the social bedrock of the other types of learning and formed the basis for how the highly sophisticated cultures of pre-industrial societies developed and were reproduced from generation to generation. Second, ‘investigative’ learning, the desire to explore the implications of an issue or topic, can generate discoveries as long as opportunities are provided for new forms of participation in accordance with the principle of question generating and answer providing within existing ‘communities of practice’. Such opportunities are likely to generate discoveries that are bounded by the parameters of the organisation in which they are produced, not unlike the conditions of Kuhn’s ‘normal science’. ‘Reflexive’ learning, the third category in the typology, seeks to go beyond the immediate context in which a learner is located. This form of learning tends to emerge only takes place when investigative learning is unable to deal with the problems experienced by those in a particular, and their is a felt need to for the context within which the learning is located to be challenged.For Engestrom, the starting point for learning is the contradictions that exist within activity generally (and workplaces, in particular). Crisis points arise when ‘communities of practice’ confront contradictions that are not immediately resolvable within the contexts in which they have arisen and require the transformation of existing ‘communities of practice’. This idea has some affinities with Victor and Boynton’s argument about the pressure points that spur enterprises to chose to take advantage of the knowledge that has built up inside them. In both cases, for this to happen two conditions

5 I use the term reflexive learning instead of ‘expanded learning’. Engestrom’s concept of expanded learning is inextricably bound up with a particular pedagogic approach , which her refers to as, the ‘expansive cycle of learning’. It is my contention that the principles that underpin the concept of expanded learning have a broader application than is acknowledged by Engestrom, for this reason, I have preferred to call this type of learning ‘reflexive’ (Guile and Young 1999).

37

have to be met. First, it must be possible for members of a ‘community of practice’ to call into question the context of learning- in other words the ‘community of practice’ itself. Second, new artefacts, for example theories, ideas, schemas etc need to be available to support reflexive learning and act as ‘mediating resources’ to help postulate new relationships between the context and practice.The second issue raises the question as to how to categorise artefacts; an issue that has generated a considerable literature on its own (Wartofsky). Since the focus of this paper is e-resources that can be used to facilitate learning, I have restricted my self to a brief discussion of those artefacts. As was noted earlier, artefacts are both ideal and material. Put simply, artefacts are produced by human being and the very act of production leads humans to inscribe significance into them. Hence artefacts so not exist in isolation as elements of culture/.On the most helpful attempts to categories e-resources has been provided by Bates (1995). He has distinguished between Computer Mediated Resources (i.e. CD Roms/web sites) and Computer Mediated Communication (i.e. audio, textual/video-conferencing). The former are primarily concerned with enhancing the presentation of learning material since they offer access to simulations, provide opportunities for problem solving, they also can be used to present feedback on performance. The latter offer quite different and distinctive features. The can facilitate continuous communication and interaction unencumbered by spatial or temporal considerations. Consequently, they support dialogue between members of local or distributed communities of practice as much as between potential members of communities of practice or the possibility of setting up new communities.It is the inscription of human values into the design of artefacts that results in Computer Mediated Resources and Computer Mediated Communication having different design features and, moreover, having different pedagogic implications. Most commercially available multimedia programmes and many web sites are characterised by a pedagogically determined learning process, since instructional objectives have been built into software programmes. These instructional objectives often restrict users to responding within pre-determined limits. In contrast, Computer Mediated Communication provides a more flexible ranger of resources, which can be used to support collaborative activities and self-directed learning. Therefore, they allow, in theory, participants to shape, through negotiation and collaboration the goals of working and learning.

Towards a framework for analysing the relationship between learning, e-resources and knowledge creationThe preceding discussion of learning and artefacts has highlighted a number of crucial issues to be taken into account if we are to understand the contribution that e-resources make to epistemic activity. For the sake of brevity, they can be summarised as follows.

38

Diagram 1. Conceptualising learning, ICT and epistemic activity.

Types of learning

Types of ICT Epistemic outcomes for SMEs and Individuals

Computer-accessed resources

(CD Rom, on-line Library, Websites etc)

Computer-mediated communication

(Email, file exchange, video conferencing etc)

Adaptive learning.

e.g. information retrieval

e.g. use information to accomplish set task(s)

e.g. acquire information from expert(s)

e.g. use information to accomplish set task(s)

little change in working and learning practice

‘additive’ conception of e-resources

Collaborative learning.

e.g. information retrieval

e.g. discuss information in community of practice

e.g. use information to address agreed problems

e.g. establish community of practice

e.g. agree questions for exploration/conduct dialogic inquiry

e.g. develop ideas to reconfigure work/ learning practices

reconfigured working/learning practices

network e-resources for specific community of practice

opportunities to:- participate in

discussions

- solve problems

- develop knowledge/skill

Reflexive learning.

e.g. information retrieval

e.g. identify contradictions in information/ problem at hand

e.g. use information to introduce new working and learning practices

e.g. establish internal and external community of practice

e.g. agree question for exploration/dialogic inquiry

.g. identify contradictions with community of practice

e.g. model solutions/ identify obstacles

e.g. agree strategies to transform working/learning practice

transformed working/learning practice

network e-resources to expanded community of practice

opportunities to:

- resituate knowledge/skill

- relate knowledge/skill to

- other communities of practice

- further transform working/learning practice

There are different types of learning and each type presupposes a different type of outcome. There are different types of e-resources and each type presupposes a rather different from of communication and interaction. Taken in combination, these two observations point towards a more complex set of issues as regards the use of e-resources in general, let alone, their contribution to epistemic activity. It is the contention of the paper that it is possible to use the analytical distinctions raised in the preceding discussion of learning and e-resources as criterai to formulate a typology for analysing the relationship between learning, artefacts and knowledge production.

39

ConclusionsThe first conclusion which can be drawn from the typology is that in order to use e-resources to support epistemic activity, it is necessary to begin to create inclusive cultures inside SMEs that provides increasing opportunities for workers to:

reflect on experiences and to try our new ideas in practice; participate in and discern the rules and protocols of knowledge

producing communities; apply new knowledge to develop working and learning practices.

The second conclusion is that establishing new modes of working and learning involves workers in ‘horizontal’ and ‘vertical’ forms of re-situation (Guile and Young, forthcoming). The former occurs when individuals carry out a known activity in a new context. The latter can lead individuals to develop new goals, new action and new strategies in order to grasp the connections between different activities. This process may take two forms. First, new patterns of activity emerge from the original context which constitute a modification of the original activity rather than an alternative realisation of that activity. Second, it may only be possible to resolve the original problem through contact with artefacts that lie outside the immediate context. Reconfiguring or transforming working and learning may involve exploring the value of concepts and ideas that are external to the enterprise, even though they may have been developed and debated over a period of time in relation to many similar practical problem. This involves learners mediating between theoretical knowledge and everyday knowledge in an attempt to interpret new situations in workplaces in light of the new knowledge they are developing as well as to deal with counter interpretations.The third conclusion is that typology of e-resources and epistemic activity is applicable for analysing similar forms of learning and knowledge production because the concept of epistemic activity is not restricted spatially or temporally. Rather it reflects ‘places’ within enterprises and the extent to which the practice of ‘question generating’ and ‘answer providing’ are actively encouraged or discouraged. Nevertheless, a caveat has to be introduced: these implications have to be set in the context of the stage of development (i.e. Victor and Boynton’s spectrum of development) o epistemic activity within specific SMEs. As ever, context is critical to practice.

ReferencesBates, A.W. (1993) Distance Learning and Technology SRHE/OU, Buckingham, UK

Bell

Brofenbrenn

Castells, M. (1996) The Rise of the Network Society, Vol 1. MacMillan, London

Cole, M. (1996) Cultural Psychology Cambridge University Press, Cambridge

Daniels. H. (2001) Vygotsky and Pedagogy Routledge, London

Engeström, Y (1993) Training for Change, ILO, Geneva, Switzerland

Engeström, Y. Engeström, R., and Karkkainen, M. (1995) ‘Polycontextuality and boundary crossing in expert cognition: Learning and problem solving in complex work activities’ Learning and Instruction 5. 1. pp. 319-336

Engeström, Y. (2000) Expansive Learning at Work: Toward an activity-theoretical reconceptualisation In: Journal of Education and Work, 14, 1, pp.

Gibbons, M. et al (1994) The New Production of Knowledge Sage, London

40

Guile, D. From ‘credentialism’ to the ‘practice of learning’: rethinking learning for the knowledge society, Policy Futures in Education, 1, 1.

Knorr Cetina, K. Epistemic Communities, Harvard Education Press, Harvard 1999

Knorr Cetina, K. (forthcoming) Five Transitions Towards a Knowledge Society

Lave, J. (1993) The Practice of Learning In Chaicklin, S. and Lave, J. (eds) Understanding Practice Cambridge University Press, Cambridge

Rheinberger, J. (1992) Experiment, Difference and Writing in Studies in the History and Philosophy of Science Vol 23, No 2. 305-21

Victor, B. and Boynton, A. Invented Here Harvard Business School Press, Harvard

Wells, G. (1999) Dialogic Inquiry, New York: Cambridge University Press.

41

42

Bernard Blandin, CESI (France)

Localization of software and learning material for SMEs:how is it possible?

IntroductionThis paper revisits research undertaken ten years ago through the COMETT programme (Auvinen, Blandin & al., 1994) aiming to identify the cultural differences which have an impact on learning in order to facilitate the localization and adaptation of learning material across boundaries. This approach appears to be still valid, but should be deepened, since further research done in the meantime and sociological concepts such as the “social world” (Blandin, 2002aand “learning culture” (Blandin, 2003) are likely to help refine the approach.It is not enough to adapt content: the “form” of the learning material has also to be localized. Designing effective learning environments has to take into account social and cultural factors, but also learning software and learning material usability has to be considered as “situated” (Blandin, 2003). Occasions and conditions for the use of learning material arise directly out of the context of learning activities which are implemented.As a result of these findings, it appears that the localization process as well as the design process for software and/or learning material cannot be isolated from the design or the adaptation of learning situations in which the software or the learning material is to be used.In this paper I will first present background research on cultural aspects of learning and their particular relevance to SMEs, then I will examine how my standpoint on “situated usability” of software and learning material impact on these results and finally, I will propose a two-tier development process as a condition for the successful design and/or localization of software and learning materials.

Culture, Learning and SMEsIt is not common in discussions of ‘Learning’ to look at anthropology or sociology. This is why, in my opinion, cultural aspects of learning are not sufficiently explored. Differences in national cultures and national value systems and their impact in the field of management has been discussed by several authors, including Hofstede (1991), Hampden-Turner and Trompenaars (1993).Hofstede identified the following areas which impact on the managerial relationship:

- Social inequality, including the relationship with authority,

- The relationship between the individual and the group,

- Concepts of masculinity and femininity and their social implications,

- Ways of dealing with uncertainty, relating to the control of aggression and the expression of emotions,

- Time orientation (short-term or long-term).

43

Hampden-Turner and Trompenaars identified the following ‘dilemmas’ which might create tension when national value systems are in confrontation:

- Universalism / particularism,

- Analysing / integrating,

- Individualism / communitarianism,

- Inner-directed orientation / outer-directed orientation,

- Time as sequence / time as synchronisation,

- Achieved status / ascribed status,

- Equality / hierarchy.

In the field of education and training, our “Cultural Impact on Learning” research pointed out obvious differences from one country to another in terms of structure, administration, curricula, regulations, etc. at every level6. But, beyond the cultural factors, there are also individual factors which impact on the way people learn and their learning strategies, such as what psychologists call ‘Learning Styles’. Different researchers have identified several components of learning styles. The most advanced work comes from Canada, with the work of Hill and his follower, La Montagne. The components of learning styles, which partly overlap cultural factors, are the following (La Montagne, 1985):

- Learning support (relationship with authority, with the group, with the individuals),

- Decoding of information (practitioner / theorist),

- Processing of information (inductive / deductive).

Another author, E.T. Hall, explored the cultural differences related to the organisation and the use of space which impact on communication and interpersonal relationships (1966). He identified three “spheres” around a person, the dimensions of which differs according to culture. These were:

- Intimate space: the closest ‘bubble’ of space surrounding a person. Entry into this space is allowable to only the closest friends and intimates.

- Social and consultative spaces: the spaces in which people feel comfortable conducting routine social interactions with acquaintances as well as strangers.

- Public space: the area of space beyond which people will perceive interactions as impersonal and relatively anonymous.

Hall developed his theory of “proxemics”, arguing that human perceptions of space, although derived from sensory apparatus that all humans share, are moulded and patterned by culture.There is no need to demonstrate that landscape, environment, and human settlements vary according to location, climate and other geographic parameters. Scenery and the environment to which people are used also constitute cultural references, which label implicitly the background of any scene as “familiar”, “from elsewhere”, or “exotic”.Based on this initial research, and on eight case studies, we proposed, as an output of the “Cultural Impact on Learning” research, a set of recommendations concerning the

6 More recent reports have added to this work. See, for example, the European Commission Study Group on Education and Training report ‘Accomplishing Europe through Education and Training’, and in particular the Annex (Part VI).

44

adaptation of the content of learning material across borders. The main findings were as follows.“Do not only translate, but take into consideration, and be prepared to modify the following:

- Scenic elements and lifestyles,

- Economic and institutional systems,

- Social relations (authority, gender, interpersonal contact etc.),

- Language (categories, structures etc.),

- Value systems and beliefs,

- Learning styles and learning strategies.”

Learning material had to be adapted for use in any particular country, to take account of national cultures. I will now demonstrate that even in a given country, there is no single culture. W have already seen, with learning styles, that we might be obliged to take into account more individual characteristics than culture.E.T. Hall, in some way, expressed the same idea as Marcel Mauss (1950) with his concept of “body techniques”, arguing that the way you walk, the way you sit, the way you swim etc. are cultural, not natural, and that they are learned. Furthermore, recent approaches in linguistics make the assumption that our categories (Lakoff, 1987) and our ways of thinking, though framed by common schemata based on body structure, are also moulded and patterned by culture and language (Lakoff & Johnson, 1999). This reinforces the hypothesis by Sapir-Whorf that conceptual models are formatted by language and categories, and are specific to a particular population. Lakoff provides numerous examples, including the Dyirbal (an Australian aboriginal tribe) classification of the universe into 4 categories, a classification lost when younger generations began to learn English and the Japanese ‘Hon’ category, of regrouping long objects (hairs as well as sticks or pencils, candles, etc.) which has no equivalent in any other language (Lakoff, 1987). Even what could appear to us as a basic human categorisation, based on human physiology, such as colour, is a cultural product, and the ability to distinguish between two colours like blue and green appears to be related to the fact that these categories can be named in a given language, as was illustrated by Kay and Kempton’s experiment comparing English speakers and Dani-only speaking speakers. The Dani are a New Guinea tribe whose language distinguishes own two colours and for whom blue and green are the same (Kay & Kempton, 1984). The epistemological consequences of such an experiment are analysed by Varela, Thompson & Rosch (1992). I have demonstrated that objects also contribute to these moulding and patterning of our minds, since the use of objects requires “social schemata of uses”, which are themselves cultural and vary from one country to another (Blandin, 2002b). Finally, it is now becoming evident that almost every behavioural or communication skill or thinking capacity is shaped by our culture and through our language.I will also argue that the word ‘culture’ should not implicitly be taken as synonymous to ‘national culture’. It can also refer to smaller groups: one of the first sociologist to use the expression “social world” to describe the culture of a small group was Howard S. Becker in his famous study on outsiders (Becker, 1963). This notion was used in interactionist sociology, and re-invented by the French sociologists Boltanski & Thevenot, who tried to

45

understand the judgement process7, and the existence of different “value measurement systems” through which groups of people evaluate the world in which they live (persons, objects, event etc.). These “measurement systems”, determine different “Cities” (“Cités”, in French) or “social worlds” within French national culture (Boltanski & Thévenot, 1991). These “Cities” are rooted in the history of social bodies and in their value systems:

- the “inspired” world (or the world of artists),

- the “domestic” world (or the world of “we-relationships”),

- the world of “fame”,

- the “civic” world,

- the world of “merchants”,

- the “industrial” world.

For Boltanski & Thevenot, the consequence of these sub-cultures is that social relations between people from different “worlds” take the form of conflicts or compromises.Using Boltanski & Thevenot’s approach, it is possible to demonstrate that there are more “social worlds” than the six they described. In particular, SMEs founders, for me, belongs to a specific “social world”. SMEs are generally analysed from an economic standpoint. But some research has shown that SMEs founders share common sociological characteristics, defined by the term “entrepreneurship”. Most of the research distinguishes between the “entrepreneur” and the manager of a large company: entrepreneurship is defined as “the pursuit of a discontinuous opportunity”(Carlton, Hofer & Meeks, 1998), whereas management is the maintenance of the continuity of an organisation. For these authors, “management” starts when “entrepreneurship” finishes, when the organisation reaches self-sustainability. According to them, this means entrepreneurship, or establishing a SME, is a specific behaviour.The traits most frequently cited as characterising entrepreneurs include the desire for independence, locus of control, creativity, a risk taking propensity and the need for achievement (Carlton, Hofer & Meeks, 1998). French research (Aumont & Mesnier, 1992) presents similar results: entrepreneurship is the achievement of a project and the entrepreneur is characterized by his or her need to face challenges, the desire for independence, a risk taking propensity, the need for action and creation, the need for achievement and the ability to anticipate and to seize opportunities. If we consider “Entrepreneurship” as the “Common Principle”, and the “Entrepreneur” as the “Grand State”, “Independence” as “Personal dignity”, “Success” as the main “Judgement criterion” etc., “Entrepreneurship” can clearly be seen as a particular “City” or a particular “social world”, as meant by Boltanski & Thevenot. It means that SMEs, in a given country, have their own sub-culture, rooted in the “Entrepreneurial City”, which is different from the “Industrial world” of the managers of big firms, or the “Merchant world” of capitalists and shareholders. My hypothesis is that this particular culture of SME owners frames both their own learning strategies and their training policies as employers. This raises a number of paradoxes. SMEs are generally considered as lacking a training culture8, and all statistics show that SME employees participation in training is low compared with larger organisations. On the other hand, “undertaking”, which is the core of entrepreneurship, is

7 Their question was: How can people evaluate things and share their evaluation; how is it possible to have a common appreciation of things?8 See the Cedra ICT Research Network report (Attwell, forthcoming), or, for France, the survey on training policies in very small enterprises (Bentabet, Michun & Trouve,1999).

46

considered, together with “researching”, a process which generates and supports the learning process and the desire to learn (Aumont & Mesnier, 1992). Self-directed learning requires some of the entrepreneur behavioural characteristics: autonomy, a feeling of responsibility, self-management and self-monitoring of learning process and learning paths, viewing problems as challenges (Mardzia Hayati, 2001). And entrepreneurs, when asked about their learning strategies, appear to be, in the majority, self-directed learners or self-educated learners, or to use another term, “autodidacts” (Le Meur, 1998).Duplaa’s recent research in France (2002) reinforces these findings. First, he confirms what was pointed out in the Cedra ICT Research Network report (Attwell, forthcoming): the attitude of SMEs “managers” emerged as the single most decisive factor in influencing training policies. This in turn implies managers have to be involved in all decisions about training and about the design and localisation of. Duplaa (2002) suggests the need for alternative solutions to formal training within SMEs including associations or clubs to share experience and good practices between entrepreneurs and coaching and on-the-job training for employees. In fact, informal learning situations appear to be the rule in the “Entrepreneurial City”. People learn in SMEs, but not within formal training systems: some entrepreneurs interviewed by Aumont & Mesnier (1992) even say that they learn all the time. A common learning style, described by Bentabet, Michun & Trouve (1999), has the following characteristics:

- Learning support: the owner of the company is considered as the one who knows. Interpersonal relations are very important since value systems are transmitted together with knowledge and know-how.

- Decoding of information is by the practitioner. Practical solutions are required, theory is considered as not useful or far-fetched.

- Processing of information is inductive. Working situations and problems encountered must be the starting point.

At this stage, it appears that SMEs belong to a particular social world, the “Entrepreneurial City”, with a particular “learning culture” (Brown, Collins & Duguid, 19899) generally associated with a specific learning style. This might explain the difficulties in transfering learning material and training models from training institutions or large companies to SMEs. Local adaptation of the content of learning material, as proposed in the research on the “Cultural Impact on Learning”, might not be enough. Before discussing this issue, we need to explore another aspect of the use of learning material and learning software relating to its “form”, which is generally embedded within theories of design as “Usability”.

Situated UsabilityThis section is based on my chapter in a book dedicated to the evaluation of the usability of online learning (Blandin, 2003). In this paper I will only provide an outline of the ideas. I will define the word “usability” and then summarize why usability has to be considered as situated, and finally examine the conditions which might facilitate the use of learning material as self-study material.

9 Brown, Collins & Duguid (1989) consider the differences between learning process as implemented in school and in activity-based learning processes as naturally developed by practitioners in their professional life or “Just Plain Folks” in their daily life as a consequence of two different “Cultures of Learning”.

47

In its broad sense, “usability” addresses the relationship between tools and their users. Usability depends on a number of factors measuring how well the functionality of the tool fits user needs. For software, this may include how well the flow through the application fits user tasks, how well the response of the application fits user expectations. Generally, this relationship between a human being and an artefact or an object is considered as independent of any contextual, social or cultural aspects: usability criteria relate to “Human Factors” considered as universal.From a designer’s viewpoint, usability is seen as a relationship between a human being and an artefact which measures the productivity of a user using the artefact (Nielsen, 1994). This may appear confusing, since it does not make a distinction between two usability-related concepts, “ease of use” and “usefulness”. Therefore, I prefer to use the extended notion of usability proposed by Notess, following Norman and the “User-Centred Design” School of University of California, which distinguishes clearly between the two usability-related concepts, but takes tboth into account into its notion of “usability” (Notess, 2001). This also answers the questions of the differenced between “Utility” and “Usability” raised by authors like Grudin (1992) or more recently Tricot & Tricot (2000) and Tricot & Lafontaine (2002).Donald Norman (1988) proposes the following basic principles for good design for usability:

Rule 1: Provide a good conceptual model,

Rule 2: Make things visible,

Rule 3: Map the controls, their movements and their results in the real world,

Rule 4: Provide feed-back for any actions.

Norman’s rules and ser-centred design principles, named “Usability Engineering” by Notess (2001), apply to all software including learning software. Elaboration on these rules led Nielsen, the father of software usability, to propose his “Ten Usability Heuristics” (Nielsen, undated), in which we find the same principles. Usability, when applied to software design and engineering, is synonymous with “ergonomic issues”. It attempts to define rules to design the application in order to match users behaviour to various types of equipment and different languages. For example, Nielsen, in his book on web usability, focuses on page, content, site and intranet design, addressing issues such as viewing pages on various monitor sizes, as opposed to writing concise texts for “scanability”; or paying attention to users with disabilities or international users (1999). Quinn (2001) describes an example of widespread usability problems concerning learning software:

- Counter-intuitive reading order of on-screen material” (breaking Rule 1),

- Failure to relate to the real world experience of the user” (breaking Rule 3),

- Poor presentation of key information” (breaking Rule 2),

- Lack of accessibility, even in most basic sense” (breaking simultaneously all the rules, as shown in the examples given in the following pages of Quinn’s paper).

I would argue that to apply these types of rules to designing learning software would never guarantee that they would be “usable”. I will, in the following section, make the case for a strong dependence of “usability” on social and cultural aspects because using an object is a social activity, implicitly implying social relations belonging to different registers (Blandin, 2002b). Universal “Human Factors” on which to build usability criteria do not exist but, instead, we have to take into account contingency factors, for

48

which we need appropriate conceptualisation. My arguments, which are summarized here, take into account both cognitive and sociological standpoints (Blandin, 2003).The main cognitive standpoints are based on the research presented in the first section, which leads to the conclusion that since our conceptual models are cultural in essence, there is no good conceptual model. This means that the usability of a software tool, even in a restricted sense, has to be related to the cultural context in which the tool has to be used. As an example, the “intuitive reading order” rule proposed by Quinn (2001) cannot produce the same screen layout in Western Countries, in Japan, or in Arabic Countries, simply because reading order in these cultures is different!The second point I discussed is the notion of “real world”, which also appeared in the second of Nielsen’s Usability Heuristics (undated). From a social constructivism viewpoint, the “real world” appears as a social construction which arise from the situation confronting the subject (Blandin, 2002b). The “Situated Cognition” approach is rooted in similar considerations: concerning tools, Brown, Collins & Duguid (1989) state that “the occasions and conditions for use arise directly out of the context of activities of each community that uses the tool, framed by the way members of that community see the world. The community and its viewpoint, quite as much as the tool itself, determine how a tool is used. Thus, carpenters and cabinet makers use chisels differently.” Again, this leads us to the conclusion that social and cultural context has to be taken into account to evaluate usability.Thirdly, I pointed to evidence from the Sociology of Uses. Punie’s survey of “Non Uses” of several ICTs applications in Flemish Households (1997) reveals that there is a pre-condition to use a tool: the tool must respond to a particular need. “No need” is the main reason people say they do not use devices like videotape recorders, handheld computers or Pay-TV. “Usefulness” appears as a necessary prerequisite to use a tool. “No need” does not mean no relation at all with the object: there could be some cognitive or emotional relation, or the object could be considered as an indicator of social status, etc. But this type of relation does not mean the tool will be used, because it takes place in a register that is different to the “utility register” (Blandin, 2002b).If we agree “Usability addresses the relationship between tools and their users” (Usability First, undated), “usefulness” has to be considered as a component of “usability” in its broader sense. This is also applies learning software. Recent papers on e-learning usability (Quinn, 2001; Notess, 2001) have shown e-learning programs are not used because of users’ lack of motivation. This has to be taken into account in assessing their usability (Notess, 2001). Technology museums are full of products that people found easy to use, but which have never been used (Jennings, 2001), and many current e-learning programs seem to be good examples of such products, according to Quinn (2001)!That e-learning programs are not used is also one of the findings of Elliott Masie’s survey (2001. The survey, based on American companies, shows that the key factors to successfully implement corporate e-learning – i.e. to have it used by employees – are determined by the organisation in which the programs are implemented, and not, or very little, by the intrinsic characteristics of the programs themselves. According to Masie, the following ‘motivators’ determine the acceptance of e-learning:

- good advertising and championing within the company,

- time and support provided during working hours,

- creation of an e-learning culture within the company,

- provision of incentives such as peer recognition and career advance.

49

The theory of the “Seven Pillars of Self-Directed Learning” proposed by Philippe Carre (1992) for the successful implementation of Self-Directed Learning within an organisation – whatever the material – shows that most of the conditions for success are dependent on organisational decisions (5 pillars out of 7!). This suggests that to successfully implement e-learning, organisational support for the motivation of learners plays the main part. Thus, from the viewpoint of the Sociology of Organisations, social and cultural contexts have to be taken into account to set up criteria for usefulness and therefore for usability criteria. Again, this highlights the importance of cultural and sociological context in determining the “usability” of tools. The existence of “Social Schemata of Uses” associated with the use of tools (Blandin, 2002b) make also the case for encompassing factors describing the environment of uses in any usability assessment. An example of this is my usability problems in North America in using domestic electric appliances like hair-dryer or boilers complying to the “User-Centred Design” recommendations (Norman, 1988). These appliances have a “On” button and an “Off” button, instead of an “On-Off” switch as we have in Europe. Despite cognitively knowing this, my fingers simply refuse to use a second button to switch off the device and my reflex action, always repeated, is to try to press again the “On” button. Another illustration is the “Accountant case”, I cited in my paper (Blandin, 2003). A woman was using CBT software to learn algebra. She was systematically making errors in the exercises provided on the bottom line of the screen, though was successfully completing similar exercises on different parts of the screen. It turned out She was an accountant, used to adding a column at the bottom of the column, and this prevalent schema prevented her from adding numbers in a different way.This is why I am inclined to propose the notion of “Situated Usability” to name a set of “heuristics” that account for describing how the environment impacts on usability factors. These heuristics are the following (Blandin, 2003):

(1) Social Schemata of Uses: to be used within a given community a tool should embed common Social Schemata of Uses of this community;

(2) Type of Action: to be used in a given situation a tool should correspond to users needs and purposes in this situation, and allow the performance of a given action;

(3) Culture of the Users: to be used within a given community a tool should convey representations and practices which are considered as “common sense knowledge” by the user;

(4) Culture of the Environment: to be used within a given community, a tool should convey representations and practices which are considered as “common sense knowledge” by the community;

(5) Tool Efficiency in a given situation: to be used in a given situation a tool should have proven efficiency in such a situation;

(6) Ability of the User to use the Tool: to be used in a given situation, a tool must be mastered, to some extent, by the user which also means that an object does not become a tool immediately or even within a short time;

(7) Motivation of the User to use the Tool: to be used in a given situation, a tool should interest the user sufficiently to use the tool rather than acting in a different way.

Users do not live or act in an abstract world alone with the tool they are using. Users, as human beings, live and act in a world which is at the same time social and material. Use of tools is deeply rooted in users’ culture, which means that it is rooted in previous learning from their social and material environment. As a consequence, usability must take into account the user’s social and material environment. This is why usability has to

50

be “situated”. This simply means that usability has to take into account user experience, in its broad sense, and this is certainly one of the best Design principles! But it is also probably the most difficult to implement. “User experience”, and in particular the relationship between the user and an object or an artefact is context-dependent, and therefore requires a sociological standpoint to be analysed and fully understood.If we now return to learning software and learning material, I assume that the epistemological stance of the teacher or trainer, the motivation of the learner, the organisational learning culture and environmental factors strongly interact in determining the conditions for their use. I will conclude this section with a brief outline of the conditions which facilitate the successful use of learning software or learning material for adult learning. My hypothesis is that configurations which work should embed what Malcolm Knowles in his book Self-Directed Learning (Knowles, 1975) and others (Long, 1995) consider the main characteristic of adult learning. In other words, they should support the development of Self-Directed Learning. The motivators determining the acceptance of the learning material will be the same as those listed above as the “Seven Pillars of Self-Directed Learning” (Carre, 1992).However “Self-Directed Learning” is a concept with no place within learning theories such as Behaviourism or Instructionalism which deny the learner as the locus of control and refuse the idea the learner may acquire knowledge or know-how without the help of someone else who knows. Constructivism appears as the only current theory of learning in which learner’s activity is considered to play a major part, and therefore it appears as the only theory compatible with the implementation of Self-Directed Learning. This is why I assume that there could be a correlation between the effective use of learning material – in particular learning software – and learner’s motivation and his or her Self-Directed Learning Readiness (Guglielmino, 1977). When Self-Directed Learning is implemented within an organisation, then usability of learning software also depends on the following criteria:

-sufficient support and appropriate to match the learner’s ranking on the Self-Directed Learning Readiness Scale;

-an appropriate learning culture and support environment to match the expected motivation of learners.

Several conditions are necessary in the use of a tool in a given environment. These conditions can be analysed using my “Situated Usability Heuristics”. In the case of learning software, or more generally of learning material, these heuristics can also be used with the word “culture” being replaced by “learning culture”. If we consider a particular “social world”, then the “community” designates this “social world”. The reader will easily transpose the heuristics in order to determine the conditions to use effectively learning software in the “Entrepreneurial City”.

Designing and Localizing Learning MaterialWe are now able to assemble the pieces of the puzzle to answer the question posed in the title: how is it possible to design and localize learning material for SMEs? This section describes a general process, and its relation to the “Entrepreneurial City”.The evaluation of the usability of learning material cannot separate the material from the situations in which this material is used. This implies that both design and localization processes have to focus at the same time on the material and the conditions of its use by a given target group, with its particular cultural characteristics. The distinction which I made between the content and the form appears to be very useful here, since it helps to rationalise the design and localization processes through a two-tier model, each level dealing with specific objects as detailed in the table below (Table 1).

51

The process for each tier follows the five classical phases of instructional design: 1) Needs Analysis, 2) Design, 3) Development,4) Implementation5) Evaluation.

But since we have two different domains, which can be processed independently, it is necessary to cross-check the compatibility between the form and the content, at least during the design phase, and to take this into account in the evaluation.

Context of use Learning materialContent-related objects

- Skills and knowledge standards - Objectives and evaluation modes- Didactical programme and learning

activities- Learning culture of the actors (learners,

tutors, trainers etc.)- Learning styles and learning strategies

(learners)- Motivation (learners)

- Discipline, level, type of content

- Learning activities in which it is used

- Social, economical, institutional systems

- Social relations, value systems and beliefs

Form-related objects - Pedagogical approach- Types of learning activities- Rhythm, duration of activities- Learning place- Learning space architecture and

organisation- Technological system- Support system and communication

modes

- Type of material (paper-based, audiotape, videotape, software etc.)

- Scenic elements and lifestyles

- Language

Adaptation and localization of learning material is necessary, though not sufficient, and often both the form and the content of learning material will need adaptation in the localization process. Thus, when the learning material is software or is digital, localization will be greatly facilitated if the material is built around a database and if texts, pictures, sounds, films etc. are considered as parameters and described in the database. In that case, the content-related or the form-related objects can themselves be described as metadata to help identify the material10.Finally, as a conclusion, I must point out some assumptions about learning materials and how to adapt them to SMEs. From Table 1, we can assume that:

- in terms of content, if the subject is not aforeign language or foreign culture, the material has to refer to national socio-economical context and to the “Entrepreneurial City” value system and relational models;

- in terms of form, if the subject is not a foreign language or foreign culture, the material has to be in national language, and must feature local scenes and “ordinary” lifestyles.

10 Though it is not my purpose here to discuss the issue of metadata standards, it is obvious that the proposed approach challenges the current models, like AICC, SCORM, or the ISO SC36 proposals because, even if elements of the context of use can be described in some models, form and content are never distinguished.

52

There are no other conditions for the material. But, it is also of the utmost importance that the materials be inserted in the right context of use, which has to take into account the learning culture and the learning style of the “Entrepreneurial City”, including:

- interpersonal relations between the learner and tutor from the same world,- a group of peer learners from the same geographical area, but avoiding

competitors,- activity-based learning, with a practical inductive approach rooted in case

studies, whenever possible provided by the learners themselves,- ‘side activities’, including social events, contests etc. to foster motivation,- short sequences (1 or 2 hours), which could take place at the beginning or

end of a f working day,- if travel is needed to the learning place, it should be in the close

neighbourhood (typically not more than a ten minutes trip).These appear to be common sense recommendations. But they have still to be confronted with the “real world”, and, in such matters, there is but one question: if we build it, will they come?

ReferencesAumont, B. & Mesnier, P.M. (1992). L’acte d’apprendre. PUF, Paris (France).

Auvinen, A.M. Blandin, B. Chapman, P. Dondi, C. Evans, R. Goldstone, L. Sharratt, R. edited by Johnstone, A. (1994). Cultural impact on learning : A practical guide to managing the effective adaptation of learning materials across international boundaries. Contact, Manchester (UK).

Becker, H.S. (1963). Outsiders: Studies in the Sociology of Deviance. The Free Press, New-York (NJ).

Bentabet, E. Michun, S. & Trouve, P. (1999). Gestion des hommes et formation dans les très petites entreprises. Etude n°72. CEREQ, Marseille (France).

Blandin, B. (2003). Usability Evaluation of Online Learning Programmes: a Sociological Standpoint”, in Ghaoui, C. (ed). Usability Evaluation of Online Learning. Idea Group Inc. (PA), p313-330

Blandin, B. (2002a). Les mondes sociaux de la formation, in Les TIC au service des nouveaux dispositifs de formation. Education Permanente n° 152, 2002-3, p 199-211.

Blandin, B. (2002b). La construction du social par les objets. PUF, Paris (France).

Brown, J.S. Collins, A. & Duguid, P. (1989). Situated Cognition and the Culture of Learning, in Educational Researcher Vol.18 n°1, 32-42.

Carre, P. (1992). L’autoformation dans la formation professionnelle. La Documentation française, Paris (France).

Carton, R.B. Hofer, C.W. & Meeks, M.D. (1998). The Entrepreneur and Entrepreneurship – Operational Definitions of their Role in Society. Paper presented at the annual International Council for Small Business conference, Singapore. Retrieved January 29, 2003: http://www.sbaer.uca.edu/Research/1998/ICSB/k004.htm

Cedra ICT Research Network. (2002). Joint Cedefop-European Commission Study on e-learningin SMEs. Cedefop, Thessaloniki (Greece).

Duplaa, E. (2002). Vers un dispositif FOAD adapté aux toutes petites entreprises. Cesi, Paris (France).

53

European Commission Study Group on Education and Training. (1996). Accomplishing Europe through Education and Training. Retrieved January 29, 2003, at the following URL: http://europa.eu.int/comm/education/reflex/en/homeen.html

Grudin, J. (1992). Utility and Usability: Research issues and development context, in Interacting with computers, Vol 4, n°2, 209-217.

Guglielmino, L. (1977). Development of the Self-Directed Learning Readiness Scale. Doctoral Dissertation, University of Georgia, Athens (Georgia).

Hall, E.T. (1966). The Hidden Dimension. Doubleday, New-York (NJ).

Hampden-Turner, C & Trompenaars, F. (1993). The Seven Cultures of Capitalism. Currency Doubleday, New-York (NJ).

Hofstede, G. (1991). Culture and Organizations: Software of the Mind. McGraw-Hill, New-York (NJ).

Jennings, T. (2001). Dead Media Project Working Notes by Categories. Retrieved August, 25, 2001, from http://www.deadmedia.org/notes/index-cat.html

Kay, P. & Kempton, W. (1984). What is the Sapir-Whorf Hypothesis, in American Anthropologist Vol. 86, n°1, 65-79.

Knowles, M. (1975). Self-Directed Learning: A Guide for Learners and Teachers. Association Press, New York (NJ).

Lakoff, G. (1987). Women, Fire and Dangerous Things. What Categories Reveal about the Mind. The University of Chicago Press, Chicago and London.

Lakoff, G. & Johnson, M. (1999). Philosophy in the Flesh. The Embodied Mind and its Challenge to Western Thought. Basic Books, New-York (NJ).

La Montagne, C. (1985). Le profil d’apprentissage, vue d’ensemble. IRPA, St-Hubert (Quebec).

Le Meur, G. (1998). Les nouveaux autodidactes. Chronique sociale, Lyon (France).

Long, H. (ed.) (1995). New dimensions in Self-Directed Learning. University of Oklahoma, Norman (Okl).

Marzia Hayati, A. (2001). Self-Directed Learning. ERIC Digest. Retrieved January 30, 2003: http://www.ericfacility.net/ericdigests/ed459458.html

Masie, E. (2001). E-Learning: If we Build It, Will They Come ? American Society for Training and Development, Alexandria (Va).

Mauss, M. (1950). Sociologie et anthropologie. Quadrige / PUF, Paris (France).

Nielsen, J. (1999). Designing Web Usability: the Practice of Simplicity. New Riders Publishing, Indianapolis (In).

Nielsen, J. (1994). Usability Engineering. Morgan Kaufmann Publishers, San Francisco (Ca).

Nielsen, J. (no date). Jacob Nielsen Online Writings on Heuristics Evaluation. Retrieved August, 25, 2001, from http://www.useit.com/papers/heuristic/

Norman, D. (1988). The Psychology of Everyday Things. Basic Books, (USA).

Notess, M. (2001). Usability, User Experience, and Learner Experience. E-Learn Magazine In-Depth Tutorials. Retrieved August, 25, 2001 from the following URL: http://www.elearnmag.org/subpage/sub_page.cfm?section=4&list_item=2&page=1

Punie, Y. (1997). Imagining “non uses”. Rejection of ICTs in Flemish Households, in Imagining Uses. Proceedings of the 1st International Conference, May, 27-29,1997. Bordeaux (France), 165-176.

54

Quinn, A. (2001). Why people can’t use e-learning. What the e-learning sector needs to learn about usability. Retrieved August, 25, 2001, from the following URL: http://infocentre.frontend.com/servlet/Infocentre?access=no&page=article&rows=5&id=163

Tricot, A. & Tricot, M. (2000). Un cadre formel pour interpréter les liens entre utilisabilité et utilité des systèmes d’information, in Actes du colloque Ergo-IHM. Biarritz (France), 195-202.

Tricot, A. & Lafontaine, J. (2002). Evaluer l’utilisation d’un outil multimédia et l’apprentissage, in Apprentissage des langues et technologies : usages en émergence. Clé International, Paris (France), 45-56.

Usability First (no date). Introduction to usability. Retrieved August, 25, 2001 from http://www.usabilityfirst.com/intro/index.txl

Varela F., Thompson E. & Rosch E. (1992). The Embodied Mind. Cognitive Science and Human Experience. MIT Press. Cambridge (Mass).

55

George Bekiardis, ERGONKEK (Greece)

E-learning Content and Software Localization

IntroductionFew would dispute that the rapid developments in today’s information technology have revolutionized traditional learning environments. Indeed, the concept of e-learning was born from the fusion of information and communication technologies and it has mirrored the achievements of such technology by providing users with ever-greater learning opportunities. The implementation of e-learning should expand the range and numbers of skilled workers through vocational training and education. But, besides the technology itself, the major factors that impact on the successful implementation of vocational e-learning programmes are content and software.The majority of e-learning content produced globally is in English. What happened in non-English speaking countries in Europe? Some European countries (France, Italy, Greece etc) are proudly insisting on their native languages. In these countries there is less willingness to use English even for technical subjects. It is obvious that content localization is becoming ever more necessary in Europe.The purpose of this paper is to explore the issues that influence the localization of vocational e-learning content and to propose methods for effective localization of e-learning.

What is LocalizationThe best definition for localization is from whatis.com (http://www.whatis.com):“Localization is the process of adapting a product or service to a particular language, culture, and desired local ‘look-and-feel’. Ideally, a product or service is developed so that localization is relatively easy to achieve - for example, by creating technical illustrations for manuals in which the text can easily be changed to another language and allowing some expansion room for this purpose. This enabling process is termed internationalisation. An internationalised product or service is therefore easier to localize. The process of first enabling a product or service to be localized and then localizing it for different national audiences is sometimes known as globalisation.In localizing a product, in addition to idiomatic language translation, such details as time zones, money, national holidays, local colour sensitivities, product or service names, gender roles, and geographic examples must all be considered.”A successfully localized service or product is one that appears to have been developed within the local culture.

E-learning content and software localization for a specific country and a specific target groupBefore starting the localization of e-learning content and software we must exactly specify the following:

56

Target country and market (e.g. Greek vocational training institutes)Target audience (e.g. trainees and trainers)Use of content and software (e.g. informal continuous training)Localized content and software has to meet the specifications and objectives of the context within which it will be deployed.

Technical Translation The localization process is largely based on technical translation as described above through cultural adaptation. Technical translation is in contrast to literal translation. It covers not only the translation of technical or technological texts but all the texts that : a) belong to human knowledge (science, arts, economy etc) b) usually contain specific terminology.Technical translation is a kind of interpretation for different contexts and different target groups.

Localization MethodologySome years ago localization of software was a simple process, concerned mainly with the translation of manuals and the user interface. Translators with average skills were able to manage localization projects. Today things are more complex. Multimedia applications, including e-learning applications, are more sophisticated and there has been a dramatic increase in the number and diversity of users of multimedia applications. High quality applications and content are produced now by local software companies in small countries (e.g. Greece) and the market has become more cautious of localized software and content. Translation is a major part of the localization process but in most cases a merely “translated” product does not appear to have been developed within the local culture.

Stages in localization methodologyTechnical translation is the first step for localization. Technical translation must begin with a detailed reading of content, this reading must be :

Dynamic. The reader (translator) must understand the meaning instead of a simple reading.

Part of the appropriation process of the terms and relations of the terminology included in the content.

Dynamic reading is not just an understanding of the content but can be extended to include a search of related documentation and a complete understanding of the subject of the content. For example you cannot localize elearning content for total quality management without an understanding of the theory and practice of management.

Knowing only the language that the original content is written in is not enough for localization. The key issue is not the words and phrases but the meaning. In technical translation the literal translation of words and phrases is not important. What is important is the transfer of information to a specific target group that has no access to the original content. For example the English phrase “b Cep type variable stars” could be used in Greek using the term “Kifides”, a term that preserve the meaning of original phrase.

57

The following diagram illustrates this process.

TerminologyIn technical translation terminology is an important issue. The use of the right terms can be a nightmare for translators. There are two key points to consider regarding terminology:

Often a term in the source language corresponds with more than one term in the

target language or vice versa.

Some terms, especially in Information Technology, have been recently ‘invented’

and there is not an equivalent in the target language.

What is localized The localization of el-earning content is applied to different kinds of contents including:

Texts

Video narratives or subtitles

Descriptions of images

User interface commands and menu items

Set up information and instructions

Help information

Error messages

Documentation

Attention must be given to:

Terminology: use of official dictionaries is imposed

User interface commands and menu items: translation must provide the term

corresponding in the appropriate function

Compatibility: the same terms must be used in all parts of the content (text, help

text, documentation etc)

58

Source Language

Target Language

Meaning

Style: the style must be consistent throughout the content

Culture AspectsCultural adaptation is one of the most important issues in e-learning content localization. Cultural adaptation is not only an issue of translation but is something more. In some cases e-learning content must be differentiated to meet the cultural preferences of the target group and language. Attention must be given to:

Idioms

Humour

Gestures

Pictures

Sounds

Fashion

Religion

Values and symbols

History

Laws

Colours

Ethics and morality

The list above is only illustrative. Every type of content has its own needs for cultural adaptation during localization.

Technical Aspects Beyond translation and cultural adaptation there are some technical aspects must be considered in el-earning content localization.

Character encoding

Fonts

Dates and numbers format

Measures

Rearrangement of shapes, images, texts

Replacement/modification of images including text

Rerecording of narrations, lip-synching

Indexing and sorting of data

ReferencesFolaron, D. (2002). Taking an E-Learning Project Across Borders

59

Dunn, P. and Marinetti, A. (2002). Cultural Adaptation : Necessity for Global eLearning. Retrieved from LineZine (http://www.linezine.com)

Moyer, L. (2001). Is Digital Learning Effective in the Workplace?

Eveland, JD. (2001). Cultural Adaptation : Design effective eLearning across national boundraries. Retrieved from LineZine (http://www.linezine.com)

“Effective Localization SDL International”, Polylang (http://www.polylang.com)

“Localization process overview”, CLOCKWORKS, (http://www.clockworks.com)

“How does localization differs from straight translation?”, Localization Industry Standards Association (http://www.localization. org )

60

Graham Attwell and Mike Malloch, KnowNet (UK)

Prospects for the development of software that truly supports collaboration and learning in SMEs

The Learning economyDigital technologies have been a major driving force behind the profound changes in work organisation, production and society over the last twenty years. These changes have led to what economists characterise as the knowledge-based economy in which the knowledge of individuals and organisations is critical to innovation and economic and social development. Whilst previously initial education and training were seen as providing the basic skills and knowledge required for work and for participation within society, the new social and work forms require continuous updating of personal and collective skills and knowledge – lifelong learning.Lundvall and Bórras (1999) argue that the ‘learning economy’ is a more appropriate term than the ‘knowledge-based economy’ for economies in which specialised and codified knowledge has a very short life-span. It is the ability to learn how to create new knowledge, and how to adapt to changing conditions, that will increasingly determine the performance of individuals, firms, regions and countries (see Lundvall and Bórras, 1999: p. 31).

E-learningJust as digital technologies have been a key force in driving economic and social demand for new knowledge, they have also been hailed as a potential means for addressing the problems of continuous re-skilling by developing a lifelong learning infrastructure. e-learning, the application of Information and Communication Technologies to curriculum and pedagogy, has been seen as providing universal access to information through flexible and ubiquitous learning environments open to all.Despite spawning a number of new technology companies and numerous government and European sponsored initiatives and programmes, the uptake and efficacy of learning using these new technologies has been less than convincing. The development of e-learning has been dominated by the metaphors of the virtual classroom and the virtual university, an over obsession with technologies, and a focus on distance applications of existing learning opportunities; much less attention has been paid to the diffusion of learning through new processes in wider social contexts. There has been very limited attention to vocational and occupational learning, or to the development of e-learning environments in less formal learning contexts. Research suggests that most learning - for good or for bad - takes place in everyday life and work social situations (Nyhan et al, forthcoming). In other words, most of our learning is informal, and takes place in a variety of social contexts. Furthermore, anecdotal observations point to the likelihood that e-learning may be at its most powerful in less formal and more situated learning environments.Work carried out in various social settings - particularly in small and medium-sized organisations - plays a very important part in people’s lives. Learning, from an economic,

61

human and social point of view, has to be embedded in the fabric of all work organisations. If e-learning is to make a contribution to changing the traditional learning paradigm (institution-based; phase and stage-related), e-learning must become embedded in the work organisation. The use of ICT (in a broad sense) for learning is considered a major factor in implementing the paradigm of lifelong learning, and in providing staff from SMEs in particular with access to continuing vocational training. ICT-supported learning is expected to help in achieving the following objectives:

1. to increase access to learning opportunities through increased flexibility of delivery modes (flexible learning) and/or by overcoming geographical barriers to participation (physical access);

2. to enhance the quality of the learning experience in terms of content and/or teaching

3. to increase the efficiency of the organisation by reducing costs and/or increasing productivity

However, existing studies show that, whereas many major companies have initiated programmes and facilities to deliver e-learning for continuous vocational training, in particular for professional development, e-learning has until now had only a limited application in small enterprises. Within SMEs, e-learning projects have tended to focus on providing networked access to virtual classroom type environments or to distance learning supported by computer-based materials. These small-scale studies have drawn attention to a troubling lack of reliable knowledge about the current position regarding the use of e-learning in enterprises, particularly in SMEs.

E-resourcesDavid Guile (forthcoming) has suggested the need to move away from the present concept of e-learning – based largely on the virtual classroom and the production of e-learning materials - to the idea of e-resources. E-resources points to the learning potential of the ICT technologies increasingly being used throughout the workplace, even in SMEs. ICT may be used for communication with customers, for stock control, for progress checking, or for ecommerce (increasingly used in business to business transactions (B2B) and less so for business to consumer transactions (B2C)). Such an approach would embed learning firmly within the social processes of the workplace. Work based learning should cease to be an independent activity taking place (by location) in the workplace, and become part of the process of work.This approach demands a paradigm change in approaches to curriculum design and understanding. Although many researchers have promoted a change from course driven learning to individual learning programmes, or from individual learning to the learning organisation, they have generally failed to address the need for new methods in curriculum design and development. Most attempts to develop ICT-based learning have similarly failed to develop appropriate new models of pedagogy and curriculum development. Indeed, it might be argued that most ICT-based learning has been regressive in promoting more rigid, course-driven learning. Why should this be so? We suggest that the major problem lies in a failure to understand the relationship between the media being employed - predominantly learning platforms and groupware technologies - and the issues of pedagogy and curriculum. We have argued elsewhere (Attwell, 1999) that all software platforms have an inherent pedagogy built into their fundamental design constraints and decisions. Different platforms support (or fail to support) different ways of learning. In many cases ICT is used most effectively simply for information exchange,

62

and when learning does take place it does so outside the special ICT e-learning technologies provided.There is a need to look more closely at how different software platforms and applications support learning, and to develop a better understanding of the pedagogic processes, which take place when learners engage with these technologies. In this paper, we attempt to examine the use of collaborative environments for knowledge sharing, and to ask how existing software environments (especially software used for communication within enterprises) support collaboration. We examine the strengths and weaknesses of such environments for collaboration and learning. We then go on to discuss what kind of systems might be developed to strengthen knowledge sharing and learning within workplace environments. We are not suggesting that the development and implementation of new software can by itself transform the workplace into a rich learning environment. We are suggesting that there is a need for software developers to work together with SMEs and SME associations to iteratively develop and test new software tools and systems that can facilitate collaborative knowledge sharing and development in real-world work contexts.We contend that the focus on e-learning software and platforms (and the hype around e-learning) has diverted attention away from the learning potential of email and groupware products that are being used to a greater or lesser extent in larger corporations and in some SMEs. We secondly argue that because of the ‘distance’ between developers and users there has been little attention paid to the evaluation of software - not in a technical sense, but in terms of its potential for enabling knowledge development and learning.

What is wrong with existing collaboration software?Before delineating the limitations of existing software, let us praise what is good about it.

What is right about existing collaboration toolsIt is evident that existing tools for online working and knowledge sharing have made a wide impact, if not a deep one. We divide our discussion of these successes into two categories:

Ubiquitous, standards-based toolsAlmost everyone in the industrialised world now has some access to tools and content-delivery environments based on industry standards: email and web pages are familiar to everyone, and have changed many people’s working practices.The great merit of these tools and environments is that they are based on thorough suites of interoperability standards, and so their scope reaches far beyond individuals’ organisations to allow the spontaneous emergence of virtual communities of interest and, to some extent, practice.To a lesser extent, yet to be standardised but widely available, peer-to-peer instant messaging, chat, video conferencing, internet telephony and shared whiteboard applications are used by many people to enhance synchronous communication at a distance.

63

Some more recent standards, like XML, RDF, SOAP, RSS, and emerging lightweight standards for web-logging communities, are beginning to enable solutions like shared newsfeeds, interchangeable representations for online content, and distributed commentary, linking and description of content from many sources.

Groupware and knowledge management toolsFor more than a decade, special commercial environments have existed to aid people in organisations to communicate and to organise knowledge resources.Groupware is a term used to describe products, like Centrinity’s FirstClass and Lotus Notes, which provide a suite of collaboration tools such as shared conferences (shared email folders), calendars, resource spaces and chat rooms.Non-commercial and open-source products also exist for shared bulletin boards and topic-centred discussion arenas, but these are less functional, with less powerful administration, customisation and moderation tools than the commercial products. A standards development or open-source movement to create free, highly functional, standard environments and tools for groupware would be a very useful step forward.Knowledge Management is a more recent term applied to environments in which companies can organise corporate document repositories and present a front end combining access to these with groupware functionalities, often also allowing users to customise their view of these resources.Content Management is a related term applied to tools for expressing, archiving, versioning, managing and delivering large repositories of documents.These environments provide a significant step forward from ‘standard’ email and web pages, mostly by organising messages and content so that users can easily access shared versions when they need them. Standard email does not allow shared copies of messages, and forces each user to undertake the task of organisation independently. Standard web pages make very poor document repositories, since they are designed for appearance rather than content, and are hard to edit or version-manage ( though we note that these failings can to some extent be obviated by increased adoption of ‘modern’ web-standards based techniques for ‘semantic’ markup in html or xhtml, well-separated from appearance-control using Cascading Stylesheets)We ourselves are avid users of such systems and would not deny their usefulness for certain purposes if carefully maintained and moderated.

64

In addition to groupware and knowledge management environments, there has recently been a proliferation of ‘virtual learning environments’ (there are many similar terms, such as ‘managed learning environment’ and ‘e-university’). These are, essentially, a combination of some groupware and some knowledge management functionality, generally not thoroughly implemented, along with some basic tools for generating course notes. Our experience indicates that a great deal of human intervention is required when employing these tools in educational settings, and that very little value is added to the learning process regardless of the effort expended; what is useful in disciplined work environments seems to be much less so in learning.

Why is online collaboration and learning still so sparse?In spite of the universal availability of email and web pages, and even with the careful deployment of groupware, knowledge management and ‘virtual learning’ environments, communities of users do not enjoy the rich interactive experience we feel is essential before the internet can become a medium for knowledge work and learning.We briefly list below some of the problems as we see them.

The ‘place’ metaphor is weakWhether exhibited in ‘site’, ‘location’ or ‘discussion space’, the overwhelmingly dominant metaphor for engaging with web content and messaging is place: ‘Where’ do you want to go today? ‘Where’ should I send this report? ‘Where’ should I look for this piece of information? ‘Where’ did I put that message about last month’s meeting?Note that, as natural as this sounds when thinking about the physical activities associated with knowledge work and learning, it is in fact profoundly unlike the terms that naturally describe the activities of knowledge work and learning themselves.What, why, who, how… these are more natural terms. In fact, when analysed closely, we lack explicit terms for most of the cognitive and inter-subjective motifs of topic-setting and shifting, contextualising, assertion, reflection, consensual edifice-building, etc. This should not be surprising – people are individually and collectively too embedded in these processes to be aware of them. But we must not allow the spurious salience of the place metaphor to stop us from reflecting on the absurdity of applying it to the design of environments for manipulating knowledge in a non-located information-based mediumPart of the problem with the place metaphor is that it forces us away from seeking ‘what’ we want to know, or expecting appropriate background resources to be made salient to us by the system when we express ‘what’ we are engaged in with respect to ‘what’ issue. We may know that the system is too dumb to do that in existing technologies, and that our best bet is to look a ‘place’ where we can start tunnelling through hyperlinks to get at a fact or opinion. But this need not always be the case.Another problem with the place metaphor is that knowledge – even as expressed in web pages or documents – is always distributed, and thus never to be found in one ‘place’. If knowledge could be expressed so that a dumb computer could assemble what we want to look at from distributed ingredients, instead of people having to repeat this task with long sessions of mouse-clicking, life would be much easier. If authors could express themselves in the context of such a system, so that background and related materials were automatically available to their readers, a lot of repetition and circumlocution could be avoided.

65

It is not just with respect to documents and crafted content that the place metaphor is inappropriate. Electronic messaging, though it currently seems naturally about ‘sending’ to a ‘place’ (or an ‘account’), could be much more powerful if notions like ‘about’ and ‘assertion’ could organically create the system’s links between ‘messages’ (brief ad-hoc contributions to a conversation), content and activities, as well as notions like ‘to whom’ and ‘in reply to which item in someone’s in-box’.Please note that we are not implying any reliance on the representational techniques and complicated inference engines of Artificial Intelligence. Sadly, we also have bitter direct experience of the futility of that approach. The alternatives we propose are simply to allow interfaces for people to structure and contextualise their writing, resource-collation and messaging in meaningful but sharable categorical terms that can be processed by computers, to allow messaging and content to be discoverable and collatable automatically from their semantic properties, and to allow content and discourse to have common interfaces in each others’ context.

Expression of meaning is still textualWeb pages and email messages are both based on simple standards for the transmission and display of text characters. This is why universal standards have succeeded for them; no-one had to agree about how to express meanings, just characters. The html standard has a very few, very broad categories like heading level and list-item, and the email standard allows a subject separate from body, but there are no conventional ways to denote more specific semantic types or gestures.It would be very hard to reach agreement on standard ways to ‘tag up’ meanings, and we do not propose that standards should exist for this, but we note that there are some useful ingredients in place for at least standardising a non-textual medium for expressing meanings:

- the XML standard allows the exchange of arbitrary tree structures, and of the grammars on which they are based, in a widely accessible way

- the Dublin Core metadata standard allows the expression of basic descriptive assertions

- various domain-specific sub-standards are in development (usually based on XML) for expressing meaningful kinds of gesture, for instance chemical formulae and mathematical equations

- technology exists that could allow a standardised platform for expressing, sharing, and discussing a variety of community-specific semantic structurings while using common tools for searching, organising and displaying structured content and metadata

What might it be like if there were conventions for expressing meanings directly, rather than as strings of text characters? Computers are very bad at processing meanings, but very good at storing and processing abstract structures. We would not expect computers to do anything with meanings (outside of restricted areas like simplifying formulae and querying databases), but they can display and organise structures and categorical types to human users on the basis of rules, and they can allow humans to make searches, organise content and specify relationships in terms of abstract types and structures. Fine-grained expression of meaning would be cumbersome in such terms, but broader-grained expression could be more convenient than prose to generate (think of the difficulty authors face in turning the ‘mind-map’of ideas and relationships they are trying to convey into linear prose or bullet-points).We list below some of the consequences of using text characters (organised into files or messages) as the standard for conveying meaning.

66

Hard to find appropriate contentThe text-as-standard becomes an obvious problem for most people when they are trying to search the web. Standard web searches must be based on strings of characters found in the target documents; as we all know, thinking of useful search strings is a dark art and not always a useful one.‘Metadata’ is a term used for categorical or semantic assertions made about pieces of content. Many web users are becoming familiar with the term, as frustration with character-string searches grows.Of course, services have emerged which attempt to apply some categorical structure to web content:

- many pages have been sorted into broad categories by search engine staff, or by volunteers of the open directory project

- link sites, which painstakingly attempt to provide a common entry point for content related to a particular issue

- many authors and organisations now embed Dublin Core metadata in their web pages, sometimes alongside other metadata terms

- community or organisation metadata repositories, like link sites, can apply semantic terms to web resources)

These services suffer from being based on structures that are either shallow (large-scale categorising efforts, Dublin Core) or sparse (community repositories). It would be much easier to leverage the human effort of categorisation and semantic tagging if documents were expressed with some semantic structure in the first place. For many documents or messages, structured authoring could provide enough ‘self-metadata’ to allow powerful searching.Note that the same problem applies to email or groupware messages. In heavy traffic, important message content quickly becomes lost, and text-string search is often incapable of retrieving it (think of the head-scratching you’ve done to try to remember a phrase or exact term from a message so that you can search for it in a folder or conference).

Content does not interoperate meaningfullyWeb content is widely used for educational purposes, but a constant problem in the design of collaborative learning environments is that web pages are monolithic and stupid. Monolithic in that they can only be delivered all at once, replacing whatever content had previously been displayed. Stupid in that they cannot send information to their environment (whether about user actions, or even just about their own properties). It is possible to impose special constraints on web-page designers to make it somewhat easier to assemble them into learning sequences and environments, but this method defeats the point of standards, and is in any case fraught with technical difficulties.A second problem is familiar to those who have created educational web content. Whereas the dream of hypertext was to allow a universe of content to interact, putting an omniscient tutor over every student’s shoulder as she acted on your content, the reality is that linked or background content has to be hand-crafted (or at least hand-linked) by the author.How much better it would be if authors had only to create that part of their content which is unique to their purpose, with content from other sources accessible on the basis of its semantic properties, and able to be woven into displays and sequences on the basis of its structure.

67

How much more interactive the user experience could be if small chunks of content from whatever source could be woven into an integrated and managed client environment so that their actions within the content could be processed by the user’s choice of software in the context of the user’s preferences, peers and other activities being undertaken in the session.

Hard to link related content and conversationA common problem in the design and use of groupware systems for learners is that it is very hard to sustain electronic discussions around resources.In face-to-face communication, even by telephone, humans have a wealth of techniques for indicating topic and for ‘pointing’ at objects, resources, themes or goals. In electronic messaging, these techniques are not available, and other devices must be found, exploiting the graphical user interface. Embedded URLs are very useful for pointing to a resource in the first instance, but very bad at tracking the resources or versions involved in a sustained discussion. URLs are given the same salience in groupware systems as any other text within a message body; relationships and links are not foregrounded in the user interface.If it is difficult to point to a web resource from a discussion, it is even more awkward for users to point to- one discussion thread from another- one piece of content from another- a discussion thread from a piece of contentTwo-way links are, at present, almost impossible to create.

‘re;’is a weak basis for discourse structuringThe ‘reply button’ is, of course, the generator of most electronic message traffic, as the frequency of the prefix ‘re:’ in email subject headings attests.Unfortunately, the progress of discussions is often lost in this succession of ‘re:’s’. The reason is that, while there is only one type of ‘reply button’, users will in fact be expressing many different kinds of response, some of which move the discussion forward or away from the original thread; this variety and movement is lost in a homogeneous chain of ‘re: re: re…’.This makes it hard to follow threads in heavy-traffic sites, and very hard to return to old threads or enter them as a newcomer.Note that it is technologically possible to design groupware that provides special ways of replying with conversational gestures.A second source of information loss in the reply chain is that, while messages typically contain many and diverse ideas and points, it is only possible to reply to the entire message, not its sub-content,This makes it impossible to follow the strands of threads as they branch.Structured messages and structured linking of new message content to existing content and discussion would allow individual points to be organised transparently within message content and responded to individually. Instead of homogeneous linear threads, it would be possible to create diverse branching trees and webs of message content and links.

68

Content creation, deployment and management is technicalBecause HTML is designed to describe the appearance, as opposed to the meanings, of content, authoring web pages is to some extent a technical process, even when graphical tools are used. Deployment (posting to the web) and management (editing, controlling navigation, version-control) are even more technical; usually authors and editors have no direct control over their content at all once it enters the deployment workflow.This creates many problems for authors. It also separates those with knowledge of content from control over the navigation and link structure of larger content blocks, and makes flexible organisation of structure and content very difficult.Readers, too, lose control over content needlessly. If content were authored, stored, organised and delivered in terms of its meaningful properties, users could flexibly control the way they want to view it. Currently, users have very little control even over the appearance of a web page they are viewing.

Structure is sparse, shallow and mostly accidentalContent and discourse on the internet is, of course, structured already. Roughly, the categories are site, page, groupware conference, discussion thread, etc.But this structure is

- inflexible- largely accident of history of content and message creation, and thus

uninformative- shallow both in tree-depth and in semantics – there is little meaningful

range of structuring decision that can be applied to standard web content and messages

-

The ‘action’ is missing from ‘interactive’Learners and knowledge workers can look at content, but what can they do with it?Because content is monolithically unstructured, and its link-structures are painstakingly hand-assigned during authorship, there is nothing in the nature of the content itself that invites action from a user, or that can be actively related to, or aggregated with, other content. ‘Interactivity’ has come to mean either action on remote databases through data entry forms, or action on highly expensive islands of animation or branching movies. If content comprises vertical books, users prefer to read it on paper.It is the constant frustration of workers trying to elicit usership of e-learning platforms that their users find so little to do with the content they are presented with that they engage very shallowly and infrequently.

Discussion divorced from contentConventional systems make a sharp distinction between conversation and resource (the popularity of listserver archives as a public technical resource reveals a low-tech example of the internet community spontaneously disagreeing with this distinction). We believe that a large part of a community's 'know how' is knowing what people talk about when they talk about a field or a work practice.If both content and messaging were authored and accessed in structured ways, the distinction would naturally blur. Content could grow in small chunks with rich relations to networks of discussion and content.

69

Key issues and ideas

Sharing & structuring: Metadata, XML & semantic expressivenessWe feel that the key requirement for structured authoring, metadata and messaging to succeed is an architecture for sharing and reconciling diverse structural ‘grammars’ and semantics.Our experience with metadata for online learning resources makes it very clear that there is no ‘correct’ metadata schema. Users will only feel comfortable and empowered when they can directly influence, and immediately override, the structures they use to express or describe their meanings.This of course creates a problem for interoperability. But that is only a technical, architectural problem – we can design systems to help maximise both coverage/interoperability and richness/expressiveness. Such systems will not be automatic. They will have to provide good interfaces for users to discover, discuss, refine, reconcile and decide on schemas, and for information professionals to design mappings and canonical schemas for common purposes. Tools must be able to seamlessly deal with a wide range of schemas (for instance our own advanced metadata tagging and search tools configure themselves on the basis of XML schemas chosen by users).Users must have a wide range of structured editing tools present to hand in their appropriate contexts, together with a rich ability to find and link in related content.

Structured communicationElectronic communication is changing the way people work, enabling remote collaboration and accessible, self-documenting discussions. As uptake and usage of email and electronic groupware increases, though, the limitations of current protocols and tools are creating real problems for sustained collaboration.One problem concerns the unstructured nature of email. While text messages can carry important meanings when their context is shared between writer and reader, they are harder to understand outside of that immediate context. Nor is it straightforward to search for those meanings as time passes after their initial receipt, or to combine meanings from multiple messages into new ideas or assertions, or to link those meanings to other electronic environments. At present, everything depends on text, and on hierarchical organisation imposed on message storage (whether in email clients or groupware environments).This problem only exhibits itself at larger scales of usage and higher expectations of sharing and building on an organisation or community's discussions. For simple, lightweight use, simple email is effective. At slightly larger scales and slightly higher functionality expectations, conventional groupware makes a big difference, but, even when very carefully moderated and maintained, groupware systems become less useful and accessible as they grow in success.Structured messages could be very useful in overcoming the frustrations users experience with large numbers of text messages. By 'structured', we mean that different aspects of the communicative act are represented distinctly in the messaging and message-browsing system. In particular, structured messaging could replace the concept of 'thread' (which organises messages by a sequence of 'reply' acts) with richer, multidimensional concepts like 'raising issues', 'solving problems', 'tracking progress', and 'linking related material'. Structure in the messages could be represented semi-graphically to message readers and

70

composers; more importantly, the relationships among messages - and between messages and other resources and documents - could be graphically represented.By providing an environment in which many small components and content-nodes were already available for re-use, embedding and cross-linking, and in which simple 'gestures' could replace textual elaboration, message composers could eliminate much of the longwinded re-expression and periphrasis that makes context-setting so difficult in existing email.Readers of structured messages could avoid having to scan long passages of redundant or familiar text; they could get the 'gist' from graphically depicted structure and relationships, and 'drill in' to the aspects of content and context that they needed.Perhaps most usefully, message structure would automatically create a form of structured metadata allowing rich querying and alternative virtual organisation of communities' valuable discussions.

Freeing users to create webs of knowledge, practice and discourseThe key problem in any attempt to experiment with new internet architectures is ‘where will all the content come from’? The web exploded with content because of its very simple standards – anyone could create a web page (though it was very much like early desktop publishing in requiring naïve authors to attend to fine-grained page design). If we are to expect interestingly structured, interoperable content, authors are required to change the way they write; we have found this to be a very difficult learning curve for most professionals.Our key idea is to design the environment to reduce the average ‘chunk-size’ authors must create, and maximise the value of small structured bits of knowledge in the right context. This is why we feel it is key to

- provide a structured messaging environment, so that message content can become core content

- provide a very thorough and responsive interface for finding materials and relating them richly to new content, whether message content or ad-hoc ‘nuggets’ of knowledge.

- provide a transparent system for sharing structures, so that users can explore rich semantic structurings for their own uses without locking themselves into non-interoperable modes of searching and organising.

Building new knowledge on practiceThe aim of these systems would be to allow employees in SMEs to develop and build new knowledge and learning through everyday work practices and through communication within communities of practice. Communication and learning could become synonymous. This is not to suggest that all communication leads to learning, but we do believe that learning and knowledge can be derived from the communication processes which take place in the work environment. A further aim would be to support the social processes of learning and overcome (at least in part the problems of atomised and individualistic approaches to knowledge development which bedevil many e-learning platforms.Three key steps would be required to move forward towards the approach we advocate.

71

Steps for the future1) Research: More research is needed into the nature of learning and knowledge

development in SMEs. This should include an examination of the links between work organisation, technology and learning, and of how technologies and work organisation can be optimally designed to support learning processes.

2) Standards: New standards are needed for interoperability of software and for curriculum design. The present nascent standards in the education and training sectors have focused on the specification and description of learning materials, rather than on pedagogies and learning processes. Such standards derive from the course-driven paradigm, focusing on number of hours, levels, outcomes etc. By their nature, the standards exclude the use of many software applications in enterprises. Concepts of functionality and interoperability spaces based on pedagogy – on the learning process – could support far wider notions of learning and curriculum development. We note that progress towards such aims will not be swift or easy since it will have to involve both innovation and co-operation on a large scale.

3) Co-development: We need new forms of partnership between software developers and educational researchers and practitioners. Much of the global investment in e-learning over the past five years has been expended in the production and implementation of stable but limited e-learning content-delivery platforms, with much less investment in exploring new ways of harnessing ICT within the learning process. Too much e-learning software is written because it ‘can be’, while very little production-quality effort has been applied to creating the software that ‘must be’. Software developers need to be involved in the practice of learning. At present there is a wide skills gap - widely different occupational profiles - between those who develop learning platforms and those who support learning practice.

In Europe, at least, there is considerable public funding being spent on e-learning. A refocus towards workplace learning (and away from the virtual classroom), and increased emphasis on research and practice in learning processes at the workplace, together with a concerted attempt to build deeper and earlier working relationships between developers and practitioners, could support a first step towards the paradigm change we call for.

72

Jenny Hughes, CRED (UK) and Graham Attwell, KnowNet (UK)

A framework for the evaluating of e-learning

IntroductionThe development of e-learning products and the provision of e-learning opportunities is one of the most rapidly expanding areas of education and training. Whether this is through an intranet, the Internet, multimedia, interactive TV or computer-based training, the growth of e-learning is accelerating. However, what is known about these innovative approaches to training has been limited by the shortage of scientifically credible evaluation. Is e-learning effective? In what contexts? For what groups of learners? How do different learners respond? Are there marked differences between different ICT platforms? Does the socio-cultural environment make a difference? Considering the costs of implementing ICT based training, is there a positive return on investment? What are the perceptions of VET professionals? What problems has it created for them?E-learning is also one of the areas that attracts the most research and development funding. If this investment is to be maximised, it is imperative that we generate robust models for the systematic evaluation of e-learning and produce tools which are flexible in use but consistent in results.

“Although recent attention has increased e-learning evaluation, the current research base for evaluating e-learning is inadequate…Due to the initial cost of implementing e-learning programs, it is important to conduct evaluation studies.” (American Society for Training and Development 2001).

The Capitalisation report on the Leonardo da Vinci 1programme, one of the biggest sponsors of innovative e-learning projects in European VET, also identified the lack of systematic evaluation as being the major weakness in e-learning projects. However, whilst some have been desperately seeking answers to the question `What works and what doesn’t work?’ and looking for ways of improving the quality of e-learning, the response by a large sector of the community of e-learning practitioners and by the technocrats in particular, has been a growing preoccupation with interoperability and regulation of platforms and models. This almost certainly stems from the bureaucratic confusion between standardisation and standards and a belief that tightening the first will somehow improve the second. This paper is written from the perspective of professional evaluators and evaluation researchers, rather than VET researchers or even e-learning practitioners, although we have been at various times involved - and are still involved – with both of these fields. It is underpinned by a set of assumptions about the evaluation process and also based on personal research evidence that suggests that standards are more likely to be improved by diversity, flexibility and experimentation than through standardisation. The first section outlines the principles and assumptions on which our evaluation work is based. The second section is a review of over 200 evaluation reports on e-learning with the focus on identifying the type of report rather than its content. This is followed by a proposed framework for reporting on and classifying e-learning evaluation together with some lessons we have learned from our own evaluations.

73

Basic Principles

We recognise that the purpose of evaluation may be primarily developmental, concerned mainly with improvement and learning, or may be concerned with accountability and justification and more inspectorial in focus. However, we believe that in both cases

Evaluation is an essential element in the design and planning of any e-learning programme or innovative process

Evaluation is integral to e-learning activities and not `bolted-on’

Evaluation should span the whole lifecycle of the programme and should be formative as well as summative.

Evaluation should be client centred, based on a non-dependency relationship and leading to long term client autonomy and sustainability

Evaluation should recognises the diversity of stakeholders and respond to their different needs by offering a wide range of evaluation products, tools and processes.

Evaluation is a skilled intervention and a specialist field of knowledge and practice

Evaluation should be ethical, professional and responsible

Evaluation should be informed by a range of different approaches and theoretical perspectives to ensure congruence between the evaluation process and the policies, processes and practices being evaluated. Thus, the evaluation of e-learning should make use of the opportunities created by the technologies.

A taxonomy of e-learning evaluation A web-based literature survey of American, European and Australian e-learning evaluation reports in English, French and German revealed that the existing research is very limited and can be grouped under seven main headings.

Case studies of specific e- training programmes For the most part these are descriptive rather than analytic or predictive, predominantly American, mainly located in a Higher Education rather than vocational training environment and focussed on the ‘virtual classroom’ model. They also tend to be restricted to particular subject areas, in particular I.T, languages and engineering disciplines. (This is not necessarily to say that e-learning is restricted to these areas, rather that they are over-represented in evaluation reports.)

Comparisons with traditional learning There are some (but surprisingly few) systematic studies that compare e-learning effectiveness with traditional learning and which are empirically robust. Those that exist are mainly small-scale studies, often using a matched pairs design and are frequently of very specific instances of e-learning in which the e-learning methodologies are idiosyncratic and the conclusions are non-generalisable

Tools and instruments for evaluation of e-learningThere is an abundance of literature detailing tools for the evaluation of e-learning. However, these are mainly divided into two types. Firstly there are many on-line data gathering instruments for assessing, typically, the user interface characteristics of

74

software (e.g. student perception questionnaires) or secondly, there are devices to record and analyse useage by duration and frequency of log-in, pages accessed, user profile etc. Many of these are sophisticated in their design and ingenuity but lack guidance on interpretation and analysis.

Return on Investment (ROI) reportsThere are surprisingly few ROI reports, considering the huge investments into e-learning at all levels. The majority of those that exist draw mainly from industry based examples and are written from an HRD perspective. The conclusion is inevitably that the investment was cost-effective and represented value-for-money but often the savings are defined in efficiency rather than effectiveness with no long-term impact analysis that takes account of unintended outcomes and consequences. It is also difficult to compare figures across reports because the distinctions between net and gross costs, capital and revenue costs, displacement of existing funds, costs over time etc. are often blurred or missing. Many ROI type evaluation reports appear to be justifying investment rather than evaluating it and more geared to an audience of shareholders rather than researchers.

Benchmarking models There have been several attempts to generate sets of criteria for quality assuring e-learning. However, these tend to be skewed towards proposing quality standards for e-learning systems and software which often disregard key variables in the wider learning environment or are based on criteria associated with evaluating traditional learning processes (and which disregard the technology) or criteria associated with measuring learner achievement through traditional pedagogies. An additional problem is that the designers of these benchmarking systems are often locked in to a particular model of e-learning which limits their transferability.

Product evaluation By far the greatest number of `hits’ on evaluation of e-learning are reports describing (and extolling the virtues of) particular education software. The vast majority of these reports are commissioned or published by the software developers. This is not to question the usefulness of these reports or necessarily to doubt their validity but evaluation of `de-contextualised’ software is not an acceptable substitute for the rigorous evaluation of e-learning systems.

Performance evaluation Scrivens (2000) in the USA, uses the term `performance evaluation’ for what would, in European terms, be called student assessment. Whilst it is true that an examination of student performance is a powerful indicator of the effectiveness of e-learning, it is by no means the only one. Moreover, a survey of reports on performance evaluation in the context of e-learning were mainly concerned with on-line tools and instruments for examining knowledge-based learner performance and could therefore be categorised under that heading.

Analysis of results of literature search – the gaps in the `research market’.Although the existing literature is limited, there are, nevertheless, some significant findings and key issues to address. Not surprisingly, the validity and usefulness of the research into the evaluation of e-learning is often limited by the agencies that drive it. Firstly, most of the current evaluation at the business impact level has been driven by clients or buyers of e-learning (those who are funding the project) rather than by the

75

designers, developers and front line deliverers and there is evidence in many cases that they are seeking to justify their investment. The evidence also suggests that this is even more the case in the public sector than in the corporate sector and more the case in large companies than in SME. (Changing Technologogical Management : The Evaluation of the Further Education National Learning Network, 2000 DfEE UK)Examining the explicit and implicit political and economic agenda of key agencies in the evaluation of e-learning is a major area of investigation. Is the purpose of evaluation about justification and valorisation or about improvement ? Do we really want answers and if so, what ‘currency’ would the answers have ? How would funding agencies react if evaluation reports were published claiming that e-learning was not effective, that e-learning projects had actually failed ?Secondly, available evidence, thus, far suggests that traditional classroom instruction yields a more favourable learner response than e-learning solutions. (Kirkpatrick 1959 : Level 1 Evaluation). This issue represents a perplexing problem for proponents of e-learning. (e.g.Scrivens M 1999) It also raises the question of whether evaluation of e-learning compared with traditional learning should be the real issue or is it evaluation of e-learning within itself ? (And similarly, between different e-learning platforms). Further research is needed to explore these two fundamentally different perspectives and generate reference materials which will look at the strengths and limitations of norm-referenced, criteria-referenced and ipsitive-referenced models of e-learning evaluation.Thirdly, e-learning has been shown to be as effective as traditional face-to-face learning over a parallel series of comparative studies with American university students over a range of science based subject disciplines (U.Ohio Centre for Evaluation Studies 1998 -2000). While recipients of face-to-face learning have expressed more satisfaction (`Level 1’ evaluation - Kirkpatrick 1959) with traditional learning solutions, the learning outcomes (Level 2 evaluation) are not different for participants of e-learning programmes. Another key area for investigation should be whether these results are replicated in a European context, for non-university students and in vocational rather than in academic subjects. Fourthly, many researchers have claimed that the same evaluation strategies and processes utilised in other types of evaluations can be applied to e-learning programmes. This may or may not be the case. However, a re-examination of widely used models and benchmarks is warranted (e.g. the ‘1-5 Levels of Evaluation’ proposed by Kirkpatrick) and any inconsistencies and limitations in an e-learning evaluation environment identified particularly for their applicability and potential for adaption or refinement for the evaluation of e-learning in European VET.Fifthly, the ROI studies indicate a positive return for companies implementing e-learning programmes. (Costs of Networked Learning, 2000-1 funded by JISC). Although most studies show a positive return based on cost reduction alone, (although often expressed in terms of revenue rather than capital expenditure) ROI studies need also to include analysis of benefits and ‘hidden’ costs for example, training the trainer staff costs, time away from the workplace costs. New research is needed which attempts a broader analysis and, in particular, will apply the ROI model to VET providers in the public sector.Sixthly, there is evidence of a growing practice of building evaluation into an e-learning process through the use of on-line tools that assess students’ perception and performance based on the belief that this can save time as well as money. This notion should be examined from the perspective of the pedagogical assumptions underpinning it and the robustness and usefulness of the data generated in this way.

76

Finally, most of the credible, holistic evaluation of e-learning has been based predominantly on an evaluative approach based on systems theory or using a positivist-rationalist approach. This is actually the case in most evaluations of VET programmes but the limitations of `systems theory evaluation’ (feedback and error detection) may be more significant in e-learning than in traditional learning. The relevance of this approach, particularly at policy level should be challenged and alternative theoretical bases explored using some of the models generated by Van der Knaap (1998 –2001) and by Elliott Stern at the Tavistock Institute on policy evaluation. However, there is some good news. Useful empirical work is being undertaken by Sheffield Hallam Telematics in Education Research Group (EU CI funded MOE and REMiT projects) although this group does not focus specifically on evaluation research nor on VET but has produced interesting models. The Oriente network for the evaluation of ICTs in education has also generated outcomes and ideas that could be adapted to VET rather than general education. Similarly, the Leonardo EVAL 2 Leonardo 2 pilot project is building a virtual support environment for evaluators and an on-line database of evaluation resources and the Capitalisation and Evaluation Research Network (Leonardo 2 network project.) brings together evaluation and VET professionals to work on common themes. All of these projects have identified the evaluation of e-learning as a major issue but have insufficient resources to allocate to in-depth work on this specialist area. What is lacking is a theoretical basis and a coherent research framework. There is little systematic research into broad based issues and concepts, or the generation of transferable models and processes of evaluating e-learning or into the design of tools for analysing, rather than collecting, data. Furthermore, there are few papers written which collate the results of the existing research and classify it in an accessible way. Nor is there substantial evidence of work that extrapolates and tests generalisable principles arising from the case studies and surveys or which comments on the implications or application of these in a European VET arena.Moreover, the gaps in knowledge impact on every level of the VET infrastructure. Policy makers and policy influencers need greater awareness of the implications of particuar e-learning strategies and models to make informed decision on e-learning policy and funding. There need to be improved links between research and evaluation so that evaluation outcomes inform the research agenda and researchers can improve the validity of field observation. The skill base of E-learning evaluators need to be increased and they need tools and instruments which will increase their ability to make more analytic and interpretive evaluations of e-learning, using a greater range of methodologies. E-learning managers, staff and other VET professionals need better evaluation products so that the design and delivery of e-learning programmes is improved.

A New FrameworkFrom a baseline of practice of attempting to evaluate many e-learning programmes, one of the biggest problems has proved to be handling the number of variables which potentially impact on the effectiveness of the programme and deciding what constitutes dependent, independent and irrelevant variables in a given situation.Over several e-learning evaluation projects, five major clusters of variables have emerged; individual learner variables, environmental variables, technology variables contextual variables and pedagogic variables. Each of these can be disaggregated into more precise groups and further disaggregated until individual variables can be identified and isolated.

77

Individual learner variables include physical characteristics (e.g age, sex, physical abilities) learning history, (negative / positive experience, level of attainment, duration, recency etc)learner attitude (positive / negative)learner motivation (high / low)familiarity with the technology

Learning environment variables includethe immediate (physical) learning environment the organisational or institutional environmentthe subject environment the

Contextual variables include socio-economic factors (e.g. class, gender,)the political context (e.g. who is funding /paying for the e-learning and for what reason ?)cultural background (e.g. how highly is learning / e-learning valued ?)geographic location (e.g. country, language, urban/rural)

Technology variables include hardwaresoftware, connectivity, the media mode of delivery,

Pedagogic variables includeLevel and nature of learner support systems accessibility issues.MethodologiesFlexibilityLearner autonomySelection and recruitmentAssessment and examinationAccreditation and certification

We are currently undertaking a survey of practitioners to test out the validity of the above taxonomy, which is also being informed by a search of existing literature and thus is constantly changing. The intention is threefold.Firstly we are seeking to build a robust classification system with clearly identified levels of aggregation, (which themselves may be context determined.) for mapping and coding existing work into the effectiveness, efficiency and economy of e-learning irrespective of whether this is an evaluation or an independent research study. Methodologies are cross-referenced against the variables being studied and major areas of omission can be identified that in turn will suggest a future research agenda.Secondly we are using the clusters of variables for proposing and testing hypotheses. For example, at the micro level, part of the iLab Project is testing the hypothesis that the effectiveness of different e-learning pedagogies will depend on particular individual learning histories. Another survey is exploring whether the effectiveness of particular

78

technologies depends on gender. At a macro level we are also interested in whether the presence (or absence) of some individual variables or clusters of variables are more significant than others in determining the effectiveness of e-learning and, if so, can they be weighted in some way? Is the profile of the learner more significant than the nature of the learning environment? Is the effectiveness of the technological solution outweighed or enhanced by particular environmental variables? Which is more important – getting the software right or the learner support right? Can we use multi-variance statistical techniques (such as factor analysis) to see which variables ‘cluster’ together and the extent to which they impact on each other?Thirdly, we have found it a useful framework for evaluating and researching the effectiveness of specific e-learning projects and programmes. The evaluation of e-learning, and research into the evaluation of e-learning, has been dominated by descriptive ethnographic studies, rather than interpretation and analyses and there is a predominance of ethnomethodological approaches, in particular, heavily contextualised case studies. The relatively small number of empirical studies have focussed on a limited number of variables. The best of these have controlled for variables other than those under study; the worst have simply discounted them. As the databank of research results is built up, particularly as the different variables are `weighted’, it becomes easier to identify the irrelevant variables and allow for the impact of others. It also allows predictions to be made which can short circuit the search for an appropriate evaluation methodology.

ConclusionsOur overarching conclusion is that the evaluation of e-learning is fundamentally the same as the evaluation of any other learning but with particular groups of variables playing a more prominent role and the impact of others differing significantly from their impact in traditional learning.Specifically, we have found that the political factors are crucial because the nature of e-learning challenges conventional theory and established benchmarks. For example, the completion rates are notoriously low in e-learning. E-learning is by definition easy to sign up to and, by the same token, easy to drop out of. This threatens institutional models. E-learning promotes learner autonomy and learner choice about how, when and where to study. Is it surprising, therefore, that learners make decisions about when they feel they have learned enough for the moment? Is this a bad thing? In the majority of cases we have looked at, `dropping out’ has not been to do with the quality of the course as perceived by the student, but a conscious decision that they wanted to stop. Quality standards for e-learning programmes often rate student choice highly and yet still persist in using completion rates as an indicator of effectiveness thus contradicting their own implicit value judgements.The social control model in e-learning is also diminished; there is usually no peer pressure, unlike traditional learning environments and the teacher-student relationship is weaker, a situation which is often at odds with the institutional culture.Interestingly, most of the e-learning programmes we have looked at, particularly the most recent, use less variety of media than much conventional teaching. For example, there is often a sense of `failure’ on the part of programme organisers if material cannot be provided on-line, even when recommending a book or sending an audiotape through the post would be more effective. Much e-learning has lost the concept of `multi-media’ in the search for technological sophistication. Similarly, access to a diversity of source materials type is harder to effect on-line although quantity of material is increased.

79

There are many other examples; the test is whether we can harness these observations and thousands of others like them generated by evaluators of e-learning in VET in Europe and elsewhere. Can we map them scientifically and record them so that other evaluators and practitioners have easy access to them? Can we generate tools to measure their significance? Can we identify the critical success factors not simply describe yet more `examples of good practice’? And as evaluators can we influence the research agenda in order to test the validity of the observations empirically and to make useful generalisations and predictions for the benefit of future practice?

ReferencesAlliger G M and E A Janak (1989) Kirkpatrick’s Levels of Training Criteria: Thirty years later, Personnel Psychology (42)

American Society of Training and Development 2001

www.astd.org/virtual_community/comm_elrng_rdmap/e_lrng_comm_top_page.html

Carnevale A and Schulze E (1990) Return on Investment: Accounting for Training, Training and Development Journal, Alexandria VA ASTD Publishing

Kaufman R, Keller J and Watkins R (1995) What Works and What Doesn’t: Evaluation beyond Kirkpatrick, Performance and Instruction 35 (2)

Kirkpatrick D (1975) Techniques for Evaluating Training Programmes Alexandria VA ASTD

Changing Technological Management (2000) Report by LSDA and Sheffield Hallam on the evaluation of the Further Education National Learning Network, Sheffield UK Learning and Skills Council.

Scrivens M

http://eval.cgu.edu

Stern E (2002) Editorial, Evaluation: International Journal of Theory, Research and Practice, Vol 08 Issue 02, Sage UK

80

David Slater, ??? (Ireland)

Practical evaluation interventions in understanding informal learning within SMEs

Introduction

“For many workers, perhaps most, the workplace represents the only or most viable location to initially learn and or/develop their vocational practice. Workplaces are becoming even more salient, as the responsibility for maintaining the currency of vocational practice is now being increasingly transferred to workers in the current reformulation of lifelong learning policies and practices. In this context, opportunities to engage in work, the kinds of tasks individuals are permitted to participate in and the guidance provided become key bases to understand and evaluate how and what individuals learn through their work. It is important to understand how workplaces afford individuals or cohorts of individuals these opportunities.” (Billett, 2001)

Billett suggests that informal learning is the primary form of vocational learning – both initial and ongoing – for most people in the workforce today. This is presumed to be especially true within smaller organisations, for a number of key reasons:

There is unlikely to be a training division or visiting training unit, as one might expect to find in a larger organisation.

In smaller organisations, there is less likely to be a strict demarcation of roles and responsibilities. A worker is more likely to have to ‘fill in’ for a colleague whose job he or she is not entirely familiar with.

Workers are often initially selected by smaller companies on the basis of their flexibility and diversity of experience. Likewise, there is a personality type that lends itself more to work in SMEs than larger or State organisations.

The ‘entrepreneurial spirit’ often pervades smaller organisations. This encourages people to attempt tasks or activities, with the support of others, of which they have little experience. SMEs are less likely to be risk-adverse than larger organisations.

Often all staff will be known to one another and the hierarchy is relatively flat. This allows informal learning to be initiated and sustained on an ad hoc and unstructured basis.

The interdependencies of fate and task between staff members are more obvious and transparent.

The organisation, as a whole, may have to respond quickly to changing external circumstances. These could be market, technology or competitor driven.

81

However, as we shall discuss later these are the assumed characteristics of SMEs with regard to their amenability to effective informal learning. Further, these are the characteristics that many SMEs insist, when asked, apply to themselves. In some cases some or all may apply. In others, the SME may actually exhibit qualities that make informal learning far more problematic then the above picture would suggest.Taking a broader view, SMEs are under increased pressure to innovate if they are to survive. This applies not only in the ‘new’ sectors but across all fields of economic activity. A study of 34 SMEs in the manufacturing sector in an Objective 2 region (Atwell and Hughes, 2002) reviewed the motivators toward innovation that exert themselves on the planning and strategic thinking processes. The definition of ‘innovation’ used in this study was:

“…any purposeful change in product, process, work organisation, technology or work environment which is novel in a specific work context…”

This definition of innovation, change within the individual company rather than a technological or other advance per se, matches closely with and will be accompanied by organisational learning and adaptability. The work identified three types of motivator:

Stimulus. A specific and particular action or event which provokes a specific and particular response. Typically, one-off or isolated events which precipitate change.

Catalyst. The presence of a factor or factors ‘in the background’ which speed up the rate of change. Typically these factors will be present over a period of time rather than being one-off events.

Imperative. The ‘must-do’ situation – an event or series of events that make change inevitable and usually urgent with identifiable negative consequences in the event of failure to change.Each of these motivator types may be internal or external to the organisation. The table below provides an example of each type.

Stimulus. Catalyst. Imperative.

InternalA new manager who is ‘environmentally conscious’.

Falling profits, shrinking markets, increasing materials waste.

Company going bankrupt. Severe Industrial Relations Problems.

ExternalA rival business opens targeting the same clients.

Growing public awareness of environmental issues.

Change in primary legislation on environmental issues.

While an effective response to any of these challenges will inevitably contain a learning element, the nature of that learning and the circumstances under which it can take place will vary greatly. For example, where the motivator is a catalyst, learning can be planned, structured and take place over a medium or longer term. Where the company is facing an immediate crisis, learning may be emergency and heuristic in nature.From an evaluation perspective, the factors which define the culture, attitudes and structures of the organisation determine the extent to which it is a suitable setting for

82

effective informal learning to take place. Where the organisation is not ‘informal learning ready’, the role of the evaluator is to assist it to move toward that state.The factors which motivate change and innovation have a strong bearing on the nature and function of any informal learning that does take place. The effectiveness of the learning will be judged, ultimately, by its capacity to meet the challenge faced.In this paper, we will take a brief look the application of the work of Argylis when evaluating company culture and internal interactions. We will then look at the work of the Art Kleiner and George Roth in mapping organisational, as opposed to individual, ‘Learning Histories.’ This work was undertaken at MIT’s Centre for Organisational Learning. What we have done here is to take this work and see how it might realistically be applied within an SME.

Moving toward a ‘Learning Organisation’.Before attempting to evaluate informal learning at the workplace, it is essential that the core characteristics of an organisation that is amenable to informal learning be understood. The nature and effectiveness of informal learning, the processes driving and enabling it and the helping or hindering factors within an SME will reflect and be shaped by the internal culture of that SME. Further, and with regard to evaluating such activity, it is important to note that within micro-organisations this culture may often be an expression of the personalities of the founding entrepreneur or entrepreneurs.The culture of an organisation – and in particular the manner in which colleagues interact and the practical implications of the hierarchical structure – has a very profound effect on the processes that will accompany informal learning. This is not necessarily true for formal learning activities, which can be ‘bought in’ from external bodies and may often take place in a setting physically removed from the organisation itself. In such an instance, the issue for the evaluator may often be the discovery of the extent to which what is learned transfers, in a sustainable manner, back from the place of learning to the place of work.In the evaluation of informal learning, however, the evaluator must first establish the extent to which the organisation represents fertile ground for such learning to take place. In an action-oriented evaluation, barriers to learning should be identified and measures put in place to address the underlying reasons those barriers are present. This has to be done before any attempt is made to evaluate the learning process itself, or its outcomes. Indeed, it can be argued that this must be done before any attempt at learning takes place at all. A number of theoretical approaches, which can be directly mapped to actual intervention, have been developed. One is described in some detail here. We can then go on to look at the evaluation of the informal learning process. Of interest here is the work of Chris Argylis and Donald Schon who examined the impact of organisational structures on the behaviour of individuals within those organisations. While the scope of their work extended far beyond looking at learning within SMEs, it is highly relevant in this context.Argylis and Schon (1974) argue that people have mental maps with regard to how to act in situations – including learning at the workplace with their colleagues. This mental map determines the way that people and groups plan, implement and review their actions. Within an SME, this mental map will be determined to an extent by external factors, such as the society within which the SME is placed or the economic sector it operates in. However, it is far more likely to be shaped, or at least heavily moderated, by internal factors. In particular, it is highly dependant on the nature of the relationship between the group and its individual members. Critically, few organisations are aware of the nature of the maps they use – or even that they are using one at all. For example, even where an

83

organisation is convinced, or at least management is convinced, that it is has an open and supportive learning environment the reality for intended recipients of that learning may be at variance with this view.For the evaluator, the identification of these contradictory perceptions of the learning dynamic within the company – and their resolution through a consensual ‘roadmap’ for change within the organisation - is the core task.In exploring the nature of these relationships, Argylis developed the notion of ‘Theories of Action’ (Argylis 1957, 1962, 1964). Smith (2001) provides a good overview of these ideas and principles for action. In short, Argylis proposes that there is a split between theory and action. In particular, individuals have a theory as to how they would act or behave in a particular situation. This is referred to as their ‘espoused theory’. In many cases within SMEs these ‘espoused theories’ may be explicitly defined through company procedures or other documents. A number of people within the organisation – and often people with decision making or executive authority – may also believe these theories to be true, or at least a ‘reasonable fit’, to the reality in the office, on the shop floor or vertically through the company hierarchy.However, these theories often do not explain, or in conflict with, what actually happens. This is governed by and in turn reflects the ‘theories-in-use’.

“When someone is asked how he would behave under certain circumstances, the answer he usually gives is his espoused theory of action for that situation. This is the theory of action to which he gives allegiance and which, upon request, he communicates to others. However, the theory that actually governs his actions is his theory-in-use” (Argyris and Schon, 1974: 6-7)

While theories-in-use may be more powerful predictors of actual behaviour they are often tacit and are seldom explicitly stated. In an SME, their existence may only rarely be recognised or their nature explored. Often they will have evolved in an organic or unplanned fashion, and will be ‘there because they are there’. In other cases they will reflect the relationships between individuals, units or departments within the organisation. In a micro-organisation they may be largely derived from the personality or approach of the founding entrepreneur.Argylis argues that effectiveness results from developing and maintaining congruence between the two. The challenge for the evaluator is to assist SMEs with whom they work in recognising areas of incongruence and in identifying suitable interventions to address those areas. Ideally, the evaluator should also be able to put in place procedures or measures that allow the organisation to continue this work on an ongoing basis with minimal, or ideally no, outside assistance.As opposed to the evaluation of learning per se, this involves an audit and review of dynamics within the organisation. Critically, the concept of double-loop learning applies here not only to the learning process but also to the evaluation process. Double-loop learning addresses not only the immediate learning objectives but also the governing variables. Similarly, the evaluation has to address not only simpler ‘strategy-consequence’ questions with regard to learning within the organisation but the ‘theories-in-use’ that govern the interactions making informal learning effective (or even possible) or otherwise.Before proceeding, it is worth noting that a double loop evaluation approach presents an inherent difficulty for the evaluator. In fact, any evaluation or audit process whose objective is the improvement of the effectiveness of informal learning will at some time face this dilemma. Edmondson and Moingeon (1999:60) state that:

84

“The underlying theory, supported by years or empirical research, is that the reasoning processes employed by individuals in organisations inhibit the exchange of relevant information in ways that make double-loop learning difficult – and all but impossible in situations in which much is at stake. This creates a dilemma as these are the very organisational situations in which double-loop learning is most needed.”

If we re-phrase this in relation to the double-loop evaluation of the suitability of a company to effective informal learning, it suggests that the more amenable a company is to such an evaluation approach the less likely it is to need it and the more ‘informal-learning ready’ it actually is.However, reality for an evaluator operating on the ground will often not be so simple. There are a number of considerations here:

The strategy adopted by the evaluator should tend to allow slow and interactive discovery of the principles and practical benefits of a holistic approach to the evaluation process. Within an SME exhibiting strong Model I characteristics (see below) caution and patience will be essential.

Often, and particularly in smaller and medium enterprises, the identification of a ‘champion’ will be critical to success. This person, or persons, will serve to assist the evaluator in overcoming resistance to the approach.

External forces may be driving the process. These might include market forces, micro or macro economic factors or the behaviour of competitors.

These are the ‘motivators’ described above.In any case, for the evaluation to succeed the evaluator must ensure that the process is ‘owned’ by the organisation and individuals within it. It may well be that for informal learning to succeed within an organisation, structural or attitudinal change will be necessary. It must be recognised that in almost all cases such change will be viewed as a threat by some individuals who will be resistant to change.Most usefully for evaluators in this regard, Argyris developed two models of theories-in-use. The first, and in his experience the prevalent, is one where learning is inhibited. The second is one where it is supported. The role of the evaluator is to assist organisations in moving from model I to model II. It is useful here to outline the characteristics of the theories-in-use in each model type:

Model I - Theory-in-use characteristics.The governing values are:

Each actor sets out to achieve the purpose as the actor sees it. In the context of work within an SME, the extent to which this applies to decision makers or the founding entrepreneur(s) will be of critical importance.

Win, do not lose. Suppress negative feelings. Communication is poor. This is clearly a

barrier to informal learning. Emphasise rationality.

85

Primary Strategies are: To gain control of the environment and tasks unilaterally. Protect self and others unilaterally.

Operationalised by: Unillustrated attributions and evaluations. Advocating courses of action which discourage inquiry. Considering ones own views to be obviously correct without any

apparent reflection. Making covert attributions and evaluations. Often this will result in a

climate of rumour within the organisation. Face-saving moves such as leaving potentially embarrassing facts

unstated.Consequences include:

Defensive relationships. Low freedom of choice. Reduced production of valid information. Crucially, there will be little public testing of ideas.

A model I organisation is predominantly characterised by defensiveness and protectiveness. Model II organisations are characterised by a very different set of theories-in-use:

Model II - Theory-in-use characteristicsThe governing values of Model II are:

Valid information. Free and informed choice. Internal commitment.

Strategies include: Sharing control. Participation in the design and implementation of the action.

Operationalised by: Attribution and evaluation illustrated with relatively direct observable

data. Conflicting views will surface. Encouraging the public testing of evaluations.

Consequences should include: Minimally defensive relationships. High freedom of choice. Increased likelihood of double-loop learning.

Argylis points to an increased likelihood of double loop learning as a possible consequence of an organisation moving from Model I to Model II. In reality the consequences for an SME, particularly one operating in an innovative sector or approaching traditional economic activities in an innovative manner are deep-reaching and will affect almost every area of the company’s operations. Such an organisation also exhibits the characteristics most likely to make informal learning work. This may occur as an indirect impact of its increased capacity to introduce innovative working methods.

86

Indeed, it can be argued that moving an organisation toward Model II will have the inevitable, even where unintended or unplanned, outcome of greatly enhancing the informal learning process.The role of the evaluator is to help the organisation move from Model I to Model II – or at least as far toward those elements of Model II that are most closely implicated in assisting effective informal learning as practicalities, time and the organisational culture allow. It should be noted that it is not anticipated that all aspects of the organisation should conform to the Model II type - or that it would be the goal of the evaluator in this work. The nature of commercial organisations within capitalist economic systems will produce inherent hierarchies of power and control. Organisations are political and it is important to recognise this.The key elements of Model II with regard to enhanced informal learning are:

Internal Commitment.

Participation in the design and implementation of action.

Encouraging public testing of new ideas.

Minimally defensive relationships.The objective of the evaluation is to identify the main ‘theories-in-use’ that apply in each case and to explore their levels of congruence with espoused theory. Interventions must then be put in place to change the default theories-in-use. The work follows six broad phases. At all stages, the criteria specified in the Model II description should serve as the core indicators of progress.

Phase 1. Understanding the current situation.In this phase of the work, the current theories-in-use and espoused theories are explored. The word ‘situation’ rather than ‘problem’ is used carefully here as this work must refer to more than specific recurring instances where difficulties are encountered.This phase of the work will involve intensive consultation with individuals from the organisation. In some cases, this may involve one-to-one interviews with key personnel - for example, the Human Resources manager or founding entrepreneur(s). It may also involve focus groups or other information sources such as questionnaires. The emphasis in this work is to identify the theories-in-use that dominate, and where possible to trace their histories and origins.This work may also involve looking at other aspects of the organisation such as its structure, operational procedures, ownership or staff benefits. This will give a clearer picture of the SMEs espoused theory. As noted above, this will often be explicitly available from documentary or other sources.On the basis of this, a first outline or ‘map’ of the learning dynamic within the organisation can be drawn up.

Phase 2: Agreeing the ‘Map’ with clients.The next step, which will to an extent overlap both chronologically and in tasks with the first, is to agree the ‘map’ with the client. This is the most difficult, sensitive and important phase of the work. Initial conclusions can be presented back to the client. As the nature of this work may be less confidential than work on the first phase larger and more ‘mixed’ – in terms of roles and responsibilities within the organisation – groups should be involved.The objective here is not just to reach agreement with the client, but to ensure that the ‘map’, its origins and impact on the operation of the organisation and the working lives of

87

individuals within it are fully accepted and internalised. This may take some time, and require considerable honing of the initial ‘map’.This phase will determine the overall success of the initiative. It should serve in itself as a mini pilot of Model II behaviour within the organisation. Discussions should be frank and open, people should be unafraid to speak their minds and responsibility for previous mistakes or misunderstandings should be taken. This phase, while ostensibly serving to agree the ‘map’ of the current situation, is a core element of the learning process itself. Also, it should be undertaken in such a way as to allow ‘ownership’ of the process to pass into the hands of people working within the organisation.As part of this process, objectives should be identified where possible. These could be broad or narrow in focus, short or long term in duration. Setting objectives is important for two main reasons. It serves to make concrete the discussions and remove any impression that they may be strictly theoretical or exploratory. By doing so, it may help to reveal facets of the real situation that are pertinent in this work. Secondly, it provides a framework for the later joint evaluation of the outcome and future needs.

Phase 3. Validate the agreed ‘Map’ and identify Interventions.Once the ‘map’ has been agreed it can be verified by deriving testable predictions. These can be done by looking to practice and history in conjunction with members of the organisation or by communally asking ‘what if’ questions. Methods to use here might include focus groups or role playing sessions.If the map appears to properly reflect the reality of the organisations culture and activity the evaluator can begin to formulate interventions. Much of the ‘what if’ work will already have begun this task. Examples of interventions might be:

Changing the physical layout of the workplace to facilitate cross-communication between different units or workgroups.

Introducing a ‘mentor’ system, where more experienced staff would work in a structured fashion with others.

Making paid time available each week for workers to explore their own learning needs. For example, the company might agree to make other staff available to them during this period if those staff could meet particular learning needs.

Establish a ‘planning learning group’ drawn from all departments to plan future developments.

These examples chosen at random illustrate the scope of possible interventions. It is also important to note that informal learning can take place in a structured manner and actions can reflect this. Some of these examples may not at first glance appear specifically learning-related at all. Through discussion, the most appropriate interventions are selected. These may be selected on the basis of:

Match to objectives agreed during Phase 2.

Match to overall organisational objectives.

Match to other activities or initiatives planned.

Acceptance. Is the initiative broadly accepted across the different elements of the organisation?

Benefit. Expected benefit of the intervention.

88

Simplicity. At first, easier interventions may be more appropriate. Not only does this allow for gradual development of informal learning capacity, it may also facilitate the evaluation process.

Scope of involvement. The extent to which the intervention involves a broad range of individuals, functions and departments.

The expected capacity of the intervention to illuminate the internal processes helping or hindering its introduction. To what extent does it touch on the more sensitive areas of the agreed ‘map’.

Cost, both in terms of direct investment and any labour hours required.

The decision as to which intervention(s) are to be implemented should be taken in as co-operative a fashion as possible.

Phase 4: Implementation of Identified Intervention(s).There can be no prescriptive outline of this phase of the work. It will be different in each instance, dependant on the nature of the intervention and that of the organisation. However, a number of factors should be borne in mind during this phase:

It must be agreed in advance that sufficient resources for full implementation will be available.

The evaluator should pass as much control over to participants as is feasible and technical and management considerations allow.

Where a small team or sub-group is involved this should be as internally democratic and representative as possible. It will still, of course, be answerable to and part of company structure.

Regular review should form part of the intervention. This should allow for ongoing change. It should also serve to act as information and feedback for a later full review.

The intervention should be introduced on the basis of an understanding that if it does not meet the originally stated objectives, or if those objectives change or become obsolete or less relevant, it will be discontinued. When this happens, resources will be passed to other or new interventions.

Phase 5: Review, Evaluation and Future Planning.In evaluating the effectiveness of the informal learning initiative, a dialogical evaluation approach would appear to be the most appropriate in the majority of cases. This is partly because the process that led to the introduction of the intervention should have been dialogical in nature, but also because it reflects the nature of a Model II organisation. If there are difficulties in conducting a relatively open, reflective and interactive review, this in itself is indicative of the success or otherwise of the project.The role of the evaluator, then, is to facilitate this dialogue and to draw out conclusions and recommendations. However, there are a number of aspects to the work that the review should address (Jeffs and Smith, 1999).

Interactions. What were the characteristics of these? To what extent were they educative or informative? How were they initiated and were they formally or informally initiated? Have they been sustained? If not, why not? Do they reflect the sort of values we are trying to encourage?

89

Focus. What issues and topics formed the core of the informal learning that took place? Was this what was originally planned? If not, why not and was the actual focus more appropriate? Was what was learnt applicable in the workplace? Was this directly applicable in peoples work or did it serve more to change the nature of their interaction and relationship with the organisation?

Setting. Was the setting appropriate? If a variety of settings were tried, which was most effective? Did this vary with the nature of the interactions and focus?

Aims. What were the aims from the outset? In hindsight, were they realistic and achievable? Did they change and if so why? Did conflicts of interest arise with regard to aims and objectives? How were they resolved?

Resources. Were the resources sufficient? If not, why not? Was there considerable overspend, and if so why? If more resources were needed, were they forthcoming?

Strategies. How did our original learning strategies change? How, if at all, were they altered and who altered them?

Outcomes. What were the outcomes for different participants? What were the direct outcomes for the organisation? Were there any unexpected or unintended outcomes? Were there any negative outcomes?

Next Steps. What main lessons have we learned? What are the key barriers to moving toward a situation where informal learning might be more successful? What is needed to move informal learning in the organisation toward Model II? Who needs to be involved? What structures need to be in place? Who needs to know about all this?

Learning Histories - Turning Experience into Action.Even where an initial pilot or extended pilot to set in place the structures and interactions needed to ensure effective informal learning is successful, measures must be taken to ensure the sustainability of the intervention. Further, when an organisation undergoes change as a result of one of the innovation ‘motivators’ described in the introduction to this paper, it is important that lessons can be learned from those events and that those lessons are gathered, presented and disseminated in such a way as to influence future behaviour. A mechanism for the on-going monitoring, feedback and correction of internal processes is required. Ideally, this should be one that requires little external support from external evaluators or experts.The ‘Learning History Research Project’ at MIT developed a set of tools and techniques to address these and other issues. In particular, the project recognised that within ‘learning organisations’ a number of key information, evaluation and assessment needs should be addressed (MIT Centre for Organisational Learning, 1997). These are:

The leaders or champions of all learning and change efforts will eventually have to internally justify their accomplishments and show that their work represents a valid investment.

Companies need to know the value of their learning experiences to date. They also need to understand how successes in one part of the company can be replicated in others.

90

Participants need to be able to judge the value of their past experiences in informal learning and make a concrete input into future plans or actions.

It should be possible to assess past actions and provide feedback to ongoing processes in a way that does not encourage a reversion to defensive, Model I type, behaviour.

To properly plan future actions, people within the organisation should be able to understand past events from other people’s perspectives.

The learning history is a structured process, with structured outputs, which address these issues. A Learning History (Kleiner and Roth, 1997) is described as:

“A ‘learning history’ is a document, or a series of documents, possibly in audiovisual or other format, that is disseminated in a deliberately structured manner. The document, and the dissemination, are both designed to help the organisations to become better aware of their own learning and change efforts.The learning history presents the experiences and understandings of participants, people who initiated, implemented and participated in organisational transformation efforts, or some collaborative learning experience, as well as non-participants who were affected by those efforts.The learning history tells the story in the participants own words and in a way that helps the rest of the organisation move forward without having to re-invent what a small group of learners have already discovered.A learning history thus represents the organisation talking to itself, in a safe and carefully structured way, about the things it needs to hear but hasn’t yet listened to.”

Much of the work of Kleiner and Roth looked at the implementation of change within organisations and how lessons from such processes could be learned. A rigorous procedure for implementation of the method has been developed. In this context, we might look at using parts of this method in order to provide ongoing review of informal learning within an SME. In particular, we will look at how this might be done with minimal external support, by people within the SME.We assume here that the SME has undertaken the groundwork as described above in order to position itself as a ‘learning organisation’ or at least one where informal learning is supported and its importance recognised. Intensive work has taken place to identify the theories-in-use and espoused theories, which govern behaviour and this map has been agreed and tested. Interventions have been put in place to improve informal learning and exchange of ideas. Some of these may have been done on a pilot basis, and it is important to extend and continue the successful trials. On a continuous or intermittent basis, external or internal motivators are moderating the circumstances under which this learning is taking place.Essentially, the ‘Learning History’ approach supports reflection and the exchange of opinion. Interviews take place with all those directly concerned. This would be an onerous and very expensive task within a large organisation but can be managed by a relatively small team within an SME. Critically, this team would use, and be trained to use, the ‘jointly-told’ story technique that lies at the heart of the learning histories method.The history is separated into ‘short stories’, each with a particular focus. For example, these might refer to a particular learning initiative, internal project or production run. Each of these is presented in a three-part format. First, the known, agreed or observable

91

facts are presented. This section, essentially an introduction, is given using the whole width of the page under a header describing the episode or event being described.The page is then split into two columns. On the right hand side is given transcripts or excerpts of transcripts from the relevant interviews undertaken. Roles, but no names, are shown. On the left comments or opinions from the ‘learning historians’ are shown. In the original MIT schema, this would be a mixed team drawn from outside experts and key company personnel such as the Human Resources Manager or the Head of Training. These would have an overview of all documentary evidence and together would build up the story line. It is unlikely that this approach would be effective in an SME, partly due to the expense of retaining such persons on an ongoing basis. A small team would be assembled to do the work on an occasional basis – or in response to a particular event or situation – with some access to outside support if needed.In addition to these elements of the document, full width interludes are included are added as required to describe background events or a change in the focus of the interviews, for example that we are now moving to a different department or geographical site. Here is an example of a ‘jointly-told story’ page. The content is purely illustrative.

Decision to house graphics and software design teams together.As a result of the last review, it was decided to move the software and graphic design elements of the web team so that they shared the same room. This was done because it was generally felt that communications between the two were poor, which was affecting project delivery. Also, an ‘us and them’ culture was developing between the two teams – even to the extent that social relations were beginning to break down.It was felt that much of the antipathy existed because each team had no real grasp of what the other was doing, or the difficulties they faced.A mistake appears to have been made here in that the two groups were not consulted in full and together before the move. Only the leaders were consulted, who felt they were fighting their corner in the discussions.

Graphic Designer : Strangely, although I felt that the move was beneficial to both teams I must admit that I still resent the way that it was done. There was little consultation, it was presented to us as a fait accompli. I have learned more about the software side of the development – but to be honest I enjoy my work less now.

This is a difficult issue to resolve. It would appear that the two working together does present operational problems but has solved many of the deeper seated issues that were destroying cohesion in the company as a whole.

Head of Software Team : This move has not been beneficial for us because the graphics team have a different way of working. For a start, they make a lot more noise and talk a lot more than we are used to. Not banter, it’s work related talking but it just appears that they have to communicate a lot more as they go about their business. I have learned a lot – and in particular about what they need from us in terms of requirements – but productivity has definitely fallen.

After these interviews were conducted, it was decided that the weekly production meetings would take place in the room and would involve all the members of both teams rather than just the group leaders.

92

Using this technique, the learning historian narrates and annotates events and processes within the SME. A number of core phases would be involved in this process.

Phase 1 : Planning and Scope.It is not expected that this activity would be constantly ongoing, but would be conducted occasionally in response to specific events or as part of planned, regular review of interventions or actions within the SME. Before proceeding, the areas that need to be addressed need to be identified – the ‘titles of the episodes’. This may entail an initial consultative process to agree the focus of the work.

Phase 2 : Interviews and Data Gathering.The ‘learning historian’ should gather the perspective of every significantly involved or affected person. These interviews should encourage the interviewee to be reflective about events and be open to consideration of future actions. The interviews are both an input to and an outcome of the process. Their reflective nature should give the interviewee space to think about broader issues within the organisation in a way that he or she might not normally have time to.

Phase 3 : Identifying Key Issues.When the interviews have been completed, the information collected is gathered and emerging lessons collated. These may relate to the individual themes or areas addressed or may relate to the company in a more holistic manner.In this phase of the work the services of an outside person, familiar with the process and preferably working with the SME in this role on an ongoing basis, may be used. With the support of that person where required, the ‘learning history’ document is drawn up.

Phase 4 : Participant Consent.Once the ‘learning history’ has been completed participants are shown the extracts relating to their interviews, and the attached comments, before they are circulated to any other person. This is critical to maintaining trust in the process.

Phase 5 : Reflective Feedback.In workshops, people from every part of the organisation, having read and studied the document, discuss its implications. How accurate was the document? Do people now have things they wish to add, or take away, from it? What are the next steps forward? What has been learned at the level of the SME as a whole? What was successful or otherwise about the whole process?Ideally, the ‘learning history’, which creates an ongoing narrative of the culture and dynamic within the organisation, will serve as a powerful instrument for understanding the nature of interactions, including informal learning, within it.

Conclusions.Most SMEs recognise the need to improve communications and informal learning processes internally. While they would aspire to becoming ‘learning organisations’ there are formidable barriers in their way. Not least may be current practices which hinder such progress but go unrecognised and unchallenged.To maximise the effectiveness of their internal and informal learning processes, SMEs may first need to:

93

Identify those barriers currently in place. It is suggested here that this be done through exploring the ‘theories-in-use’ and ‘espoused theories’ currently governing behaviour. However, other methods may be more appropriate depending on circumstances.

Explore interventions that will serve to overcome those barriers. These should be reviewed on an ongoing basis and moderated as appropriate.

Put in place mechanisms for the ongoing review of informal learning within the SME.

It is recognised that an SME cannot be normally expected to spend considerable time or resources on these activities, save as a response to recognised and acute difficulties. However, where relatively simple mechanisms can be put in place, requiring the minimum of external support, SMEs can considerably boost the potential for informal learning to take place among their staff.

ReferencesAnderson, L (1997) Argyris and Schons theory on congruence and learning. [On Line]. Available at http://www.scu.edu.au/schools/sawd/arr/argyris.html

Argyris, C. (1957) Personality and Organisation, New York: Harper Collins.

Argyris, C. (1962) Interpersonal Competence and Organisational Effectiveness, Home wood III.: Dorsey Press.

Argyris, C. (1964) Integrating the individual and the Organisation, New York: Wiley

Argyris, C. (1965) Organisation and Innovation, Homewood, III.: R.D. Irwin.

Argyris, C. and Schon, D (1996) Organisational Learning II : Theory, method and practice, Reading, Mass: Addison Wesley.

Argyris, C. and Schon, D. (1974) Theory in Practice : Increased Professional Effectiveness, San Francisco: Josey-Bass

Billett, S. (2001) Participation and continuity at work : A critique of current workplace learning discourses. Context, Power and perspective : Confronting the Challenges to Improving Attainment in Learning at Work. Joint Network / SKOPE / TRLP International workshop 8-10th November 2001, Sunley Management Centre, University College of Northhampton. Available at the informal education archives: http://www.infed.org/archives/e-texts/billett_workplace_learning

Jeffs, T. and Smith, Mark K. (1999) Informal Education. Conversation, democracy and learning, Ticknall: Education Now Books.

Kleiner, A. and Roth G. (1997) Learning Histories. A New Tool for Turning Organisational Experience into Action. http://ccs.mit.edu/LH/21wcp002.html

MIT Centre for Organisational Learning. (1997) Field Manual for the Learning Historian. http://www.ccs.mit.edu/LH/intro.html

Smith, Mark K. (2001). Chris Argylis: Theories of Action, double loop learning and organisational

94

Eduardo Figuiera, Academus (Portugal)

Evaluating the effectiveness of e-Learning Strategies in SMEs

IWhat is e-learning?How individuals live and work has suffered significant changes lately due to innovations introduced to and carried on by the information and communication technologies (ICT). In fact, ICT is the basis for the society of today and, in addition, it is “helping” the society to move rapidly towards a Society of Knowledge. In this context, e-learning will play a particular and relevant role for individuals once it will give more power to them in terms of choosing their learning preferences. E-learning strategies will not only allow individuals to choose the contents, time and pace for learning but also produce a true revolution in the education and training systems. This is very important because learning plays a central role in knowledge creation which by its turn is essential for making the transition from Society of Information to Society of Knowledge. However, given the relevant role that ICT plays for that transition, it is also important to understand how different e-learning is from learning. In other words, e-learning has to be defined. In general and simple terms, e-learning may be defined as learning opportunities delivered and facilitated by electronic means. That means that what qualifies the e-learning mode of learning is the fact of the learning contents being delivered by electronic technology. However, in this line of reasoning, two questions are in order: 1) – Is the means of delivering (electronically in that case) important enough to qualify a mode of learning?2) - Can training/teaching contents delivered by electronic means be considered an e-learning strategy?According to one adult educator working in community development, e-learning strategies are like cars; they are just a means to get faster the knowledge (place) you want to get. So, that adult educator assumes that e-learning differs from “traditional” learning only by the way learning contents are delivered. This may be not true. The reason for that is because people approaches learning from different ways and perspectives. Traditionally, people approaches learning through a third party since they usually need a guide, a tutor, or a facilitator to learn more effectively. Only people who are mature may not need such a third party. However, e-learning may introduce a new strategy for approaching learning since it integrates the three needed components for learning: (1) contents, (2) strategies, and (3) tutoring. In addition, people learn in different ways, that is, individuals own specific learning styles which may vary across contexts, tasks and age. So, it may be possible that e-learning makes individuals to create a new learning style and may help academics to understand better the learning process. In fact, e-learning strategies may introduce changes in the way people learn due to fundamentally the increasing effectiveness of interoperability and security, (Rosenberg, 2001), appropriateness of time and pace, opportunity and rapidness.It is important to note that a general idea exists that e-learning corresponds to all educational and training sets that uses ICT, including distance education. This is not

95

correct once the term e-learning indicates that the emphasis is on learning not on teaching, one of the main characteristics of the education systems. That does not mean, however, that e-learning should be goal free and disconnected to an organized system being conducted completely on the learner’s responsibility. What means is that the e-learning strategies should have individual’s learning as the system basis and its main aim and assume that responsibility for learning must gradually be transferred from a tutor or facilitator (a third party) to learner in function of his/her level of maturity. In addition, e-learning strategies should also integrate a system for individual’s assessment based on the process of assessing learning outcomes. Assessment of outcomes constitutes a more effective and rigorous indicator for individual’s learning than the traditional assessment still conducted by many educators in the education and training systems. Furthermore, learning does not happen in an empty space or outside time, without definition of goals. Learning is contextual since it is influenced by history and memory, by cultural and community identity and values (Magli, 2003).Unlike many of web-based ideas for businesses and other activities, e-learning has got through the economic crisis of the two last years. However, before e-learning reaches the fullest potential, it needs to go through three other changes (Reeves, 2003). First of all, a mental model for e-learning has to be developed and expanded in individuals’ minds. Mental models are cognitive structures built by individuals’ minds which influence the way they react to change and solve problems (Reeves, 2003). Like learning styles, usually, individuals construct different mental models in function of the experience they have with the object about which they construct the mental model. For this reason, people related to marketing of e-learning products often describe e-learning as media events using terms such as “full motion video” and “dynamic learning objects” (Reeves, 2003). Instructional programmers and trainers probably see e-learning as training activities analogous to “presentations” and “group discussions”, for example (Reeves, 2003). Subject matter specialists, by contrast, conceptualize e-learning as an alternative (electronic) strategy for delivering content. Second, quality of assessment within the e-learning products must be improved. Outcomes show if and how much learning has occurred, performance has changed, and results have been attained (Reeves, 2003). In fact, assessment of outcomes has been considered essential for education and training, reflecting the emphasis on learning followed lately by most educators (Huba & Freed, 1999; Marzano, Pickering, & McTigh, 1993, cited by Reeves, 2003). This is much more evident in the business world once what is important is performances being correlated and connected to company’s goals. And third, rigorous evaluation strategies of e-learning initiatives should be taken as a serious business (Reeves, 2003). This is important because everyone needs to know if outcomes have been achieved, i.e., certification of outcomes has a very important role to play in e-learning.The aim of this paper is to provide a framework to measure effectiveness of e-learning strategies or programmes. This will be done without referring to any particular e-learning programme or strategy.

Evaluating the effectiveness e-learning as a strategy to see its worth Development and use of any e-learning programme or strategy represents an individual, organisational and social investment. For this reason, effectiveness of any e-learning offering should be evaluated. Without knowing efficacy of e-learning strategies one cannot know if it is worth of using them or not. So, measuring effectiveness can constitute a very useful tool to base one’s decision about use of any e-learning strategy. In

96

addition, built-in programme evaluation allows trainers and other formative responsible to monitor e-learning programmes and offerings and make changes for improvement. In fact, given the potential for changing of the e-learning strategy, mandatory evaluation represents a relevant characteristic since it permits to know how much worth for people and organisations are the e-learning offerings.Globally, programme effectiveness can be evaluated following five types of evaluation approaches (Table 1): (1) Based on the programme goals, (2) Based on the decision-making process, (3) Goal-free approach, (4) Based on an expert’s knowledge, and (5) Naturalistic approach (Pietro, 1983). Evaluation approach based on the programme goals is fundamentally oriented for verification and quantification of execution of the programme goals. For this purpose, experimental or quasi-experimental research models are used in a 4-step approach: (1) Identification of the programme goals, (2) Translation of the programme goals into quantitative data, (3) Data collection from individuals related to the programme, and (4) Comparing established goals with reached goals. Evaluation approach based on the decision-making process takes this process as the main element of the evaluation model. Although theoretically this approach appears to be of simple execution, in practice its implementation is less simple than expected due to the complex nature of the decision-making process. Assuming that reality and taken evaluation as a process of designing, collecting, and furnishing useful information for judging alternative solutions, Stufflebeam has proposed the CIPP approach (Context, Input, Process, Product) (Stufflebeam et al., 1971). The CIPP approach is based on the identification of five programme aspects (Pietro, 1983): (1) the decision-making process, (2) the scenarios in which the decision-making process occurs , (3) the appropriated decision-making models to each scenario, (4) the basic types of decision that are taken, and (5) evaluation types that best fit each one of the decisions.The goal-free evaluation approach is based on the assumption that evaluation should be oriented to assess not only the established goals but all consequences that a programme has produced.The evaluation by an expert approach is an evaluation conducted by an individual or a team specialised on the object being evaluated. The basic principle of this approach is based on the human judging.The naturalistic approach is based on the assumption that there rarely exist a true that is accept by all people involved in a programme (Pietro, 1983). For this reason, the naturalistic evaluator try to respond to the major number possible of questions posed by the all individuals involved in the programme.

97

Evaluation Approaches

MODEL MAIN AIM TYPICAL QUESTIONS METODOLOGY

Programme Goals

Estimating how goalshave been reached

Have goals been reached? Have they been effective and efficiently reached? Were goals appropriate?

Quantitative.Experimental design.Before and after testing.

Decision-making

Furnishing information

Is the programme effective?Should the programme be continued, ended, or modified? How?

Estimating value based on hypothesis related to Context, Input, Process, and Product.

Participation of decision makers concerned with value of the programme.

Goal-freeEstimating efforts developed by the programme

What are the intentional and not intentional programme results?

How much value have the programme results?

Independent estimation from needs and judging criteria.

Quantitative and qualitative techniques.

ExpertExperts as judges and evaluation tools

How an external Professional classifies the programme?

Critical revision based on experience and collection of specific information and subjective opinions.

NaturalisticUnderstanding process underneath programme

What is happening to programme?

What are the different perspectives?

How is programming responding to the different interests?

Inductive discover based on qualitative methods.

Open interviews, participant observation and case studies.

Font: Pietro, D.S. (1983). Evaluation Sourcebook for Private and Voluntary Organizations. ACVAFS (American Council of Voluntary Agencies for Foreign Service), New York.

A Framework for Evaluating Effectiveness of e-Learning strategies

The Proposed Global ApproachThe global approach proposed to evaluate effectiveness of e-learning strategies can be considered as a mix of the decision-making, goal-free, and expert evaluation models. More specifically, the evaluation approach is framed by Bennett (1979)’s system of criteria for measuring programme impacts (Fig. 1). According to Bennett’s systems of criteria, to one hierarchy of impacts correspond one hierarchy of evidences. That hierarchy of evidences comprehend the following aspects: (1) Human and financing resources; (2) Developed activities; (3) Participation; (4) Reactions; (5) Change of knowledge, attitudes, abilities, skills and aspirations; (6) Change of practices; and (7) Final results.

98

Since this framework intends to be utilized for measuring effectiveness of e-learning strategies, it was considered appropriate to aggregate into three the 3,4, 5, 6 and 7 of Bennett’s hierarchy:

Participation (level 3);

Reactions (level 4);

Results (levels 5,6, and 7).

Figure 1: Bennett’s hierarchy of evidences 7. Final Results 6. Change in practice

5. Change in knowledge, attitudes, skills aspirations

4. Reactions

3. Participation

2. Activities

1. Resources

In addition to aggregation of the 3,4,5,6, and 7 Bennett’s levels, participation is conceptualized as a set of the stakeholders’ interventions in each phase, aspect, and activity of the e-learning programme. Similarly, reactions are taken as the target individuals’ responses to each phase, aspect and activity of the e-learning programme. Taking those changes into consideration, the Bennett’s hierarchy applied to evaluating effectiveness of e-learning strategies can be transformed in a 4-level hierarchy of evidences: (1) Resources, (2) Participation, (3), Reactions, and (4) Results (in terms of acquired knowledge, skills, and behaviour) (Fig.2).

Figure 2: Adapted from Bennett’s hierarchy of evidences

4. Results

3. Reactions

2. Participation

1. Resources

99

Criteria to evaluate effectiveness of e-learningIn addition to the global approach above presented which essentially serves as a conceptual matrix of reference to guide development of criteria and indicators, measuring effectiveness of e-learning needs a set of appropriated criteria. With that reference in mind, it appears that measuring effectiveness of e-learning should take into consideration nature, characteristics and objectives of the e-learning strategies. For this reason, approaching evaluation of e-learning effectiveness may be following e-learning dimensions: (1) Organisational & Management dimensions, (2) Pedagogical dimension, (3) Technological dimension, (4) Ethical considerations, (5) Learning assessment & certification, and (6) Evaluation Strategy.

a) Organisational & Management dimensionsThis dimension integrates aspects concerning organisational and management issues and how different types of affairs and services are organised and managed. It takes into consideration questions such as:

How administrative affairs are organised and managed? How learning related affairs are set up and functioning? How help services are organised and offered? How information is organised, managed and distributed? How learning environment is designed, set up and managed? How instructional and technical support services are organised and offered? How career counselling services are set up and functioning? Is there any other online and/or offline support services? How are they

functioning? How assessment of learning is set up and managed? How certification of learning is established and functioning?

b) Pedagogical dimensionThe focus of the pedagogical dimension is on aspects and issues concerning learning needs, objectives, strategies, and environments. Those aspects and issues can be illustrated through questions such as:

Have learning needs identification and analysis been done? How learning objectives are set up ? Are learning objectives in line with SMEs’ goals? How contents are developed, organised and offered? How learning objectives are related to contents? How learning strategies are designed and implemented? How learning environments are designed? How learning environments are linked to each other?

c) Technological dimensionThe technological dimension is related to aspects and issues dealing with design of the technological infrastructure and friendliness of navigation. The following questions illustrate what aspects are comprehended in this dimension:

How technological infrastructure is designed and implemented? How learning environments are designed and constructed? How software is designed and working? How site is designed and appealing to?

100

Is navigation ease and friendly?How usable and functional is the web-site?

d) Ethical considerationsEthical issues may be the first aspects that should be taken into consideration when one is measuring effectiveness of a e-learning programme. Within the frame of this dimension may be included questions such as:

How e-learning is related to social & cultural diversity? How e-learning is concerned with geographical diversity? How e-learning is approaching learners’ diversity? How e-learning is approaching social values? How e-learning is treating legal issues?

e) Learning assessment & certificationLearning assessment & certification integrates aspects concerning essentially how learning is assessed and certified. Those aspects assume particular importance once gives confidence not only to participants but also to who have to work with them. It takes into consideration questions such as:

How learning is assessed? How learning is certified? Is learning assessment & certification taking into consideration learners’

diversity? How? Is learning certification officially recognized?

f) Evaluation StrategySuch as assessing and certification of learning is fundamental for e-learners, global programme evaluation should be considered a relevant dimension given the need to know quality of the programming. This dimension includes questions such as:

How evaluation strategy is designed & planned? How evaluation of learning environments is designed and planned? How learning environment (s) is (are) evaluated? How useful is the learning content to learners? Are impact evaluation of learning strategies designed and implemented?

Final considerations The starting point for this paper was that one cannot know if it is worth using e-learning strategies if their efficacy is not known. Measuring the effectiveness of elearning constitutes a useful tool for taking decision about use of any e-learning strategy. In addition, knowing the effectiveness of e-learning strategies allows formative monitoring and improvement in implementation. Given the potential for change, evaluating the effectiveness of the e-learning strategies is integral to their implementation and improvement since it indicates the value for individuals and organisations of e-learning provision.In addition to the conceptual matrix of reference presented in the paper, which serves as a guide for developing criteria and indicators, measuring the effectiveness of e-learning should be based on a set of appropriate and specific criteria that take into consideration the nature, characteristics and objectives of the e-learning strategies.

101

References Bennett, C. (1977), Analyzing impacts of Extension Programs, Washington DC: USA Department of Agriculture.

Huba, M.E. & Freed, J. (1999). Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Needham Heights, MA: Allyn & Bacon.

Magli, R. (2003). E-learning Lost in Time and Space? The Conservative and Incomplete Emerging e-Learning Concept. (retrieved April 20, 2003, from http://www.elearningeuropa. info/docPrint.php?id=1289&1ng=1)

Pietro, D. S. (1983), Evaluation Sourcebook, for private and voluntary organizations, New York: ACVAFS (American Council of Volutary Agencies for Foreign Service).

Reeves, T.& Aggen, W. (2003). Enhancing E-Learning Assessment and Evaluation Strategies. (retrieved May 23, 2003, from www. learnwright.com/ ELearn% 202002% 20Reeves% 20Aggen% 20Paper.pdf)

Rosenberg, M. J. (2001). E-Learning: Strategies for delivering knowledge in the digital age. New York: MacGraw-Hill.

Stufflebeam, D. L. et al. (1971) Educational Evaluation and Decision-Making, USA: F. E. Peacock Publishers Inc.

102

Klaus Reich and Freidrich Scheuermann, ??? (Austria)

E-learning challenges in Austrian SMEs

IntroductionE-learning is changing the way enterprises gain competitive advantage through improved human performance. Especially small and medium-sized enterprises have to face the problem that e-learning technologies, methods and strategies have mostly been developed for the needs of large enterprises and cannot be exactly transferred to their needs. SME’s operate in almost every sector of the economy. As a consequence they vary widely in their learning and training needs. They have to deal with limited personnel, organisational and financial resources. The situation is furthermore stimulated by the difficulty to formulate detailed training strategies that will enable their employees to be better qualified to cope with increased competition. As they are more and more discovering the advantages of ICT-based learning it is necessary to provide a framework that takes into account these limitations and tries to find effective solutions. Based on a survey about e-learning in Austrian SME’s (Scheuermann & Reich, 2002) the current state of ICT based learning in Austria is analyzed and suggestions for better implementation and use of e-learning are made with a special focus on cooperative e-learning approaches.

E-learning in Austrian SME’sThe average annual growth of the e-learning market in Austria is currently estimated to be 102% compared with the European average of 96%. In 2003 this will result in a market of 116 million Euro growing to 196 million Euro in 2004. At present, by far the largest market share is e-learning for large enterprises, but small and medium sized enterprises (SMEs) are increasingly discovering the potential of e-learning (Zugmann, 2001). As the results of a survey carried out for CEDEFOP (European Centre for the Development of Vocational Training) (Attwell, forthcoming) show, the implementation of e-learning in SMEs is dependent on overcoming a series of barriers and problems. This articles gives an overview of the results of the CEDEFOP study on the use of e-learning in SMEs in different European regions. It focuses in particular on the problems in the implementation and use of e-learning in SMEs, on different pedagogical approaches, on partners and networks and on further needs in research and development. The Austrian case study for the Cedefop project was the Tyrol region. Most enterprises in this area currently make no use of e-learning and only a minority are considering the introduction of e-learning in the future (Scheuermann & Reich 2002). Hawke (2000) identifies multiple benefits in investment in the workforce, including increasing the employability and earnings of employees and increasing the productivity and profitability of businesses. The minority of Tyrolean SMEs that have introduced ICT based learning programmes have recognized these benefits and have attempted to provide their employees with up-to-date technical knowledge and skills. All those interviewed said that their employees need higher level and more specific skills for undertaking their jobs (Scheuermann & Reich, 2002).

103

Characteristics of implementation The following statements provide an overview of the main results of the case study in the use of e-learning in Austrian SMEs. The statements point to the issues and problems for SMEs in introducing e-learning. Learning needs were mostly identified through practical experience. Only a few

companies use programmes and questionnaires to identify skill and information deficiencies.

Learning is based in multiple locations and contexts including ‘on the job’, on the shop floor, away from the job and in training centres depending on needs and possibilities. E-learning takes place predominantly at the work-place and in specially equipped rooms in the enterprise. Learning takes place during and after working time.

Companies do not always have a choice in the selection of ICT-based learning materials and programmes. Many SMEs depend on larger enterprises to provide learning materials, e.g. simulations and technical information. Out of the SMEs that are able to select their learning materials most do not analyse the market before selecting the programme but ‘spontaneously’ choose a product that seems to fit to their needs. In most cases they do not develop e-learning materials themselves. These are usually provided by suppliers or external manufacturers.

Enterprises have very different approaches in selecting their students for training programmes. Often this is based on the skills and interest employees have in e-learning. In other cases groups of employees (from a particular office or workgroup) are selected to participate in an e-learning programme. In a third approach new employees or employees taking on new jobs are enrolled on an ICT based learning course The background of the learners is therefore very diverse, as is their prior experience of learning.

The main pedagogy deployed in learning in the enterprises is ‘learning by doing’ combined with face-to-face meetings. Employees are expected to become autodidactic learners through e-learning.

When trainers are involved in ICT supported learning they play the roles of tutors and coaches. Typically, no additional qualifications are required for this role. The skills of the trainers are based on their experience in traditional training.

SMEs rarely evaluate their ICT based learning provision. If they do, they use one of three methods (in order of usage): the evaluation of learning through practical application, evaluation by the department in charge of e-learning or evaluation by an external evaluator.

In general, those companies providing e-learning are satisfied with the outcomes but most have no plans to expand e-learning provision. Out of the enterprises that make no use of e-learning, only a few plan to introduce ICT based learning in the future.

Problems in introducing e-learning in SMEsEnterprises have to rethink their business and learning practices in order to facilitate the implementation of e-learning. The introduction of e-learning is often accompanied (or triggered) by problems in the management of the enterprise, in the learning culture within the SME or in infrastructure of the enterprise – to name but a few. These problems have to be addressed in advance, otherwise e-learning will not be effective at an enterprise level.

104

Management problemsShortage of human resources is a major problem for Austrian SMEs. Often decision-makers think - independent of size and sector - that their company does not fulfil the prerequisites for e-learning, e.g. people do not have sufficient time or the learning environment is unsatisfactory.Decisions to introduce e-learning often do not depend on objective decision criteria but are related to the attitude of the person(s) responsible for training. If ICT-based learning is introduced the lack of knowledge and skills often results in a narrow concept of the benefits and use of ICT based learning, with, for example, too strong a focus costs Owner managers of SMEs have their own learning characteristics and are often reluctant to devote time to formal learning.Decision-makers in enterprises have to develop new models and methods for learning. They have to learn to benefit from new emerging technologies and applications that are playing a more and more important part in education (learning software, network-technology, e-mail, application-sharing, etc.). However, many small and medium sized businesses only have limited financial and time resources allocated to staff training and the prospect of loosing staff for one or more days a week is not a popular option. The use of ICT based learning could be a solution rather than the problem it is presently viewed as. It allows staff to study at the workplace (therefore avoiding travel time and costs), use specific learning materials tailored to user needs and to acquire know-how when needed. It is understandable that many employers are reluctant to provide training that may provide individuals with accreditation and the opportunity to move to higher positions in other companies, but many manager owners of SMEs fail to recognise the added value of the further qualification of employees to their company (Scheuermann & Reich, 2002). One problem encountered in Austrian SMEs is described in detail by Goolnik (2002). He points to the importance of the business experience of the sector in which employers operate. Leading figures and organisations in a particular sector have more impact than those outside it (e.g. universities, colleges and other training advisers and providers). Their example can be either positive or negative. These leading persons and organisations have to be a primary target group if the issue of e-learning for SMEs in a particular region are to be addressed. The Austrian study showed no support for e-learning from such influential actors. One major step forward would be to persuade key individuals in SMEs of the potentials of e-learning. Many personnel managers and training directors said they would like to know more about examples of good practice. The study showed the need for more information and training courses on e-learning (Scheuermann & Reich, 2002).

Understanding the potential of e-learning The learning culture in enterprises is a major influence on the quality of ICT based learning. Hierarchies, power balances, controls, or the general esteem of training in SMEs can be major obstacles to the successful application of ICT based learning. Enterprises have to develop a learning culture where e-learning does not mean the same as playing games on the computer but is honoured as self development and as a means to strengthening the company. The learning culture also embraces the longer term planning of learning activities and the embedding of ICT based learning in broader training concepts. In contrast to this concept, employees are often forced to update their competencies by themselves. This is a very difficult task for many. As a consequence, Austrian enterprises often try dismiss those employees who are not able to adapt their know-how to deal with new emerging trends and technologies. Learning culture also implies an understanding of qualification that is connected to real needs in working life.

105

Certificates and formal qualifications only form a part of qualification. It is becoming ever more important to acquire skills and knowledge in a short time for special purposes (Scheuermann & Reich, 2002). ICT based learning directly addresses the continuous improvement of individual qualification, which is difficult to assess externally. Decision makers and managers lack insight into the learning needs of their employees and have to change their understanding of qualification. Employees that are engaged and interested in learning often feel that they do not get enough support and encouragement from their superiors. ICT based learning can be made more attractive through financial and other forms of recognition. Qualification, learning and further training should be recognized as essential components of culture and development in enterprises (Hipwell, 2000).Such a new orientation requires more strategic thinking. At the moment many problems are caused by the informal and unsystematic approach of SMEs to ICT based learning which often results in poor implementation and use of learning opportunities. Research by the DELOS project (Developing Learning Organisation models in SME clusters) into organisational learning in SMEs shows that most SMEs can be described as 'crisis-driven' with little evidence of organisational learning. Information gathering practices, knowledge acquisition strategies and competence development appear to be either absent or rudimentary, and the enterprise typically responds to challenges and opportunities rather than pursues an active policy of human resource development and strategic management. The evidence suggests that this type of firm constitutes the largest category of SMEs (http://www.pjb.co.uk/npl/bp19.pdf).

Inadequate infrastructuresLack of organisational and spatial prerequisitesAustrian SMEs often lack the organisational and spatial prerequisites for ICT based learning. For small enterprises it is often too expensive to adapt rooms or buy equipment for e-learning on-the-job. Employees working in industrial and technical jobs lack suitable workplaces and have to contend with unsuitable conditions for e-learning. Therefore it is necessary to develop new environments for learning, e.g. the construction of learning corners, learning islands or learning centres. It is important that employees have immediate access to information in order to obtain the qualifications needed for their work. Hence it is also necessary to analyse what help SMEs need to establish environments and structures that support ICT-based learning.Some spatial and organisational limitations could be avoided by combining e-learning at home with work (see Kräutler, 1999) and to allow open, flexible access. In the 2nd quarter of 2002, 62% of all Austrians over the age of 14 had access to a PC at home and 45% had internet access (http://mediaresearch.orf.at/internet.htm). Although these statistics suggest that some enterprises could try to outsource learning to their employees, none of the enterprises included in the survey had considered the potential of home based e-learning.

Learning materialsAlthough the market for ICT based training is growing fast, there is a lack of adequate learning materials for small enterprises. This is due to the following reasons. SMEs are often unable to articulate and scope their learning needs. There are difficulties in assessing the merit and value of available programmes and learning materials, which are often perceived as failing to meet firm-specific needs. Finding appropriate training is also made more difficult by a culture clash with external training providers, especially in the

106

public sector, who are seen as unable to understand business processes. Micro SMEs have particularly problems with the cost of training programmes (and associated travel and subsistence) and problems in releasing staff (Pye et al., 2002).SMEs lack the time and financial resources to undertake in depth research into training programmes that fit to their needs and the market for learning materials and course providers lacks transparency. As an example, in Germany 35,000 training centres, Chambers, and labour market organisations offer approximately 400,000 courses and seminars (Siebold, 2002)!Many firms are highly specialised. That makes the development of learning materials expensive and commercially unattractive. One company included in the study said that although learning materials were provided by a supplier they failed to meet the needs of the company in a geographical and economic context (Scheuermann & Reich, 2002). If enterprises find suitable learning materials they are didactically and methodically of poor quality in many cases. Most criticism was of the complexity and lack of user friendliness of products and also of the costs and time required. Quality certification of products would probably foster development of better learning materials.The availability and use of information and communications technologies and associated services to facilitate the entire range of communication, interaction and transaction (‘connectedness’) is poorly developed. E-learning is still seen as state-of-the-art technologies.From a pedagogical point of view e-learning can be problematic as it ‘facilitates’ isolated learning. Learning in an enterprise is a cooperative process taking place through the work process of the learners. New learning media have to integrate tools that support a more cooperate approach: through the learning media it should be possible to communicate with experts in and outside the enterprise, with tutors and coaches and other learners.For a real integration of learning in work processes it is necessary to learn with real data and real projects emerging from working life itself instead of examples and constructed case studies. Learning systems have to be implemented within the work flow connected to the enterprise databases (D’Atri & Pauselli, 2000).

EquipmentWhilst small enterprise in particular are concerned that they lack modern technologies for e-learning, in many cases technology is less the problem than the human resources (users and trainers). They lack the knowledge an skills to apply ICT based learning.The use of technology to enhance or broaden the learning experience has until now been looked at in too narrow a way. What is needed are ‘good models of learning' which are enhanced by technology. Whilst reviews of research show that there are excellent pilots, they lack critical mass, secure funding and therefore the likelihood of transferability.Nevertheless, there is still demand for the development of simple and cheap solutions that facilitate learning on the job. Laptops, Tablet PCs, handhelds etc. encourage flexible, mobile learning but often lack linkage to other information and knowledge resources. Although wireless networks can already be found in several Austrian SMEs, most enterprises still have fixed wire networks and lack the flexible infrastructure for learning on the job (Scheuermann & Reich, 2002).

Missing organisational perspective

Cooperation of different actorsAt present most Austrian SMEs act alone in facing problems of learning. For future development, it is necessary to strengthen cooperation between SMEs and between

107

SMEs, larger enterprises and public institutions (e.g. Chambers of Commerce). Development in this field has to aim at two goals:

SMEs cannot afford the infrastructure to provide ICT based learning environments or the production of tailored learning materials. If SMEs pool their resources they can afford high quality products. Cooperation in that sense could help SMEs to reduce costs and share know-how.

SMEs often need specific know-how or information that could easily be obtained through information and communication technologies. In many cases there is even no need for structured learning materials if video-conferencing or other communication tools are available. Cooperation could lead to the sharing of information and know-how and lead to collaborative forms of learning.

Austrian SMEs cooperate with other enterprises in e-learning if they are suppliers. Some SME managers said they had profited greatly from this cooperation. On the other hand, learning materials provided by larger enterprises are often standardized and do not take into account local needs. Furthermore, the supplier tries to pack as much information as possible into the learning materials making it difficult for a small enterprise to filter necessary information (Scheuermann & Reich, 2002).One solution may be to develop clusters or networks of SMEs. Co-operation between organisations within markets has long been identified as a factor in economic success and networking between organisations can contribute to stability and reduce uncertainty. These networks can evolve over time as ‘natural’ clusterings of enterprises, or can be ‘induced’ artificially as a result of interventions such as the development of business or science parks.As a large proportion of SMEs are in ‘crisis management’ rather than pro-active learning situations, they need to be encouraged to adopt a more participative style of collective learning. Support services need to be provided and resources shared (Hawke, 2000). No evidence could be found that Austrian SMEs build up such co-operation or are engaged in regional learning networks as described by Stahl (2002). He suggests the learning region can promote local change, empower SMEs through networks and partnerships and foster innovation. This concept could be extended to learning, especially considering that SMEs are consumers of education and training with particular requirements and constraints.Partnerships and networks at local and regional levels could certainly stimulate new experiments, actions and directions for learning. Clusters of SMEs, larger enterprises and public institutions that already exist in most European countries could act as starting points for the construction of learning networks. The government in Austria and the regional government in Tyrol have launced initiatives aimed at creating regional clusters. These clusters are built up by a group of enterprises in a specific branch or sector; e.g. Alpine technologies clusters in Tyrol (http://www.alpine-technologies.com). Informal knowledge transfer is already taking place in such networks but could be expanded into other areas.

Training provider SMEs are by nature of their structure and competence often unable to provide training themselves. In many cases training provided by an external education enterprise is a more effective solution for SMEs than internal training (Scheuermann & Reich, 2002).However the provision of training and consultancy has to take into account the specific characteristics of SMEs. These include the involvement of the management at the technical and operative levels, close personal relationships between managers and

108

employees patriarchal style of running the enterprise, a low degree of formality and organisation and the lack of hierarchies. This leads to specific problems, including a ‘dominance of daily business’ (and thereby neglecting co-operation with the consultant) and a lack of information about the economic and treading environment. The demand for consultancy is often reactive and depends strongly on the attitude of the SME owners and managers towards consultants (Kailer, 2000).As Stahl points out, modern enterprises require learning as an integrated activity. That means that external training institutions have to act in a customer-oriented way in order to foster these activities (Stahl, 2002). As we have discussed above, there is still considerable confusion in SMEs over the role of learning and human resource development in management policy. This requires capacity and competence in providing consultancy. Because of material restrictions there are also limitations in relation to human resource development. Most of them are financial and organisational (it is never easy for enterprises to send their staff to external, long-term seminars). Training institutions have to deal with these problems creatively. They should view these restrictions as challenges to foster innovative solutions. Training institutions need new concepts to meet these challenges. Close co-operation with SMEs is needed to deal with these problems and to integrate the activities of the external institution into the enterprise.

ConclusionThe present EU definition of an SME is too general to be of great use in research terms. It fails to recognise the qualitative criteria that can be seen as typical for SMEs, e.g. the majority of SMEs are family enterprises. Small and medium sized enterprises embrace very different forms of organisational structures and work in very different branches and sectors – from a traditional handicraft enterprise with less than ten employees to an innovative enterprise with 250 employees and several branches. It is obvious that these different kinds of enterprises have different demands for knowledge and knowledge management (OECD, 2000). If better e-learning solutions are to be found, groups of SMEs have to be identified based on quantitative and qualitative criteria. This is necessary as it is too expensive to develop solutions for individual SMEs.SMEs have to develop new organisational structures for the implementation of cooperative and collaborative forms of learning. At the moment most SMEs implement isolated learning solutions that fail to meet the needs of cooperative and collaborative forms of working and learning. A further step in e-learning would require investment in technology that supports synchronous as well as asynchronous communication at the workplace. This is probably too expensive for most SMEs but costs could be reduced by cooperation in investment in technology and the sharing of information.New approaches are needed in terms of understanding the broad context of learning. Pedagogical approaches in Austrian SMEs are often inadequate and do not meet the needs of e-learning. There is also a lack of flexible learning solutions in SMEs. Furthermore, there is still much to be discovered about how people learn using different technologies, particularly in relation to interactivity, and how materials can be developed and structured to enable all learners to make effective use of them.Learning at the workplace should partly replace “old” teaching in classrooms and face-to-face courses away from the enterprise, but this is difficult to implement. Learning should not take place besides work but with and through work. Learning at the workplace is different from learning at school and in a classroom. It is not important to go through a subject systematically but instead to solve problems arising from practice. Therefore flexible learning media are needed.

109

The term “e-learning” itself is somewhat ambiguous in a world that is rapidly changing. Where does e-learning start and where does it end? Where is the divide between e-learning and informal learning and is it necessary to differentiate between them? As technologies develop there will be far greater access to learning opportunities. However, our study has shown that considerable work is needed in order to ensure that the new e-learning opportunities are developed and implemented within SMEs.

ReferencesD’Atri, A., Pauselli, E.: Distance learning for SME Managers. 2000. Retrieved: 19.12.2002. http://cersi.luiss.it/articoli/ITHET2000.pdf

DELOS (Developing Learning Organisation models in SME clusters): Final report. http://www.pjb.co.uk/npl/bp19.pdf. Retrieved: 16.12.2002.

Goolnik, G.: E-Learning for Smaller Rurally Based Businesses: A Demand-Led Challenge for Scottish Educational Institutions. Online Journal of Distance Learning Administration, Volume V, NumberII, Summer 2002. Retrieved 14.12.2002. http://www.westga.edu/~distance/ojdla/summer52/goolnik52.html

Hawke, G.: Factors Influencing Active Learning in Small Enterprises. Paper presented at the Adult Education Research Conference (AERC) 2000. Retrieved 14.12.2002. http://www.edst.educ.ubc.ca/aerc/2000/hawke1-final.PDF

Hipwell, W.: Promoting your e-learning investment. 2000. Retrieved: 19.12.2002.http://www.learningcircuits.org/sep2000/hipwell.html

Kailer, N.: Co-operation between SME and consultants: Analysis of deficits and starting-points for improvements. 2000. Retrieved: 18.12.2002. http://www.ifgh.ac.at/kmuforum/20001/20001_06.asp

Kräutler, W.: Information-Society-Technologies: Chancen für Tirol. In: Tiroler Zukunftsstiftung (ed.): Nicht ohne Netz. Tiroler Unternehmen auf dem Weg ins Internet. Innsbruck, 1999.

Oberholzner, T., Sheikh, S.: Innovative Small and Medium Sized Enterprises and the Creation of Employment. Vienna, 2001. Retrieved: 12.12.2002.http://www.ifgh.ac.at/de/lot9.htm

Organisation for Economic Cooperation and Development (OECD): Small and Medium-sized Enterprises: Local Strength, Global Reach. 2000. Retrieved: 28.11.2002. www.oecd.org/pdf/M00005000/M00005918.pdf

Pye, Jo et al.: Promoting Workplace Learning with ICT: Modes and Models for Organisational Change. Edinburgh, 2000. Retrieved: 14.12.2002. www.leeds.ac.uk/educol/documents/00001611.htm

Scheuermann, F., Reich, K.: E-learning in Austrian SMEs. Innsbruck, 2002. In press.

Siebold, Heinz: Der vierten Säule des Bildungswesens gehört die Zukunft. In: Das Parlament. Nr.: 31-32. 2002. Retrieved 28.11.2002. http://www.das-parlament.de/2002/31_32/Thema/039.html

Stahl, Thomas: The learning region and its potential roles in lifelong learning. Retrieved: 14.12.2002. http://www2.trainingvillage.gr/download/publication/panorama/lllEN.pdf

Zugmann, J.: Die Zukunft virtueller Klassenzimmer. In: Der Standard, 13.10.2001. Retrieved: 13.12.2002. http://derstandard.at

110

John Munro and Philip Crompton, University of Stirling (UK)

Assessing the application of online learning in work-based setting

IntroductionThe UK Government has championed the concept of lifelong learning for organisations and their employees as a key strategy for developing a competitive economy to meet the demands of the global market. The importance attached to learning, by the UK Government, as a strategy to develop competitive businesses has been mapped out in its Green Paper 'The Learning Age' (DfEE, 1998). The Green Paper promotes the view that a) learning assists organisational development by increasing the skills base of the workforce, b) through the process of learning itself, it enables the organisation to manage and respond to change, and c) it increases the knowledge base of the organisation, which is seen as the key to discovery and innovation. As a consequence, developments in work-based learning are seen as particularly appropriate to increasing the development of workplace skills and knowledge, whilst at the same time widening access to learning (Seagraves et al, 1996).In tandem with promoting lifelong learning, the UK Government emphasises the role of Information Communication Technologies (ICTs) in contributing to competitive gains by small and medium sized firms (Southern and Tilles, 2000). According to The Scottish Office policy document ‘Opportunity Scotland: A Paper on Lifelong Learning’ (1998) the exploitation of ICT will be key to the activities of the recently established Scottish University for Industry (SUfI) in seeking to 'boost competitiveness and individual employability'. However, compared to the United States of America, the introduction of online learning using ICTs is present only within a small minority of United Kingdom and European companies (Pye, 2000).Seagraves et al (1996) relate how the daily operational needs and pressures within SMEs can adversely affect the opportunities and time that is required for effective learning to be developed at work. Other commentators have similarly reported how problematic structured learning in SMEs is due to the constant workload pressures that exist (Ram, 2000; Bridge, O'Neil and Comrie, 1998; Gibb, 1993). As Lange, Ottens and Taylor (2000) argue, it is not the issue of learning itself that is problematic for SMEs, but rather the 'luxury of allowing staff already at stretched capacity to take part in formal training'. Therefore, much of the learning that is done in SMEs is more likely to be unplanned, informal, and tailored more to matching and overcoming the daily operational demands and crises (Westhead and Storey, 1999; Gibb, 1999; Gray, 1999).At the same time, studies by Seagraves and Osborne (1995), Seagraves et al (1996) and Loots, Osborne and Seagraves (1998) highlight how the assessment of learner needs must address the issue of learner motivation and commitment to learning in the work-based setting. Indeed, as extensive studies by Boshier have shown, there is a link between learner motivations to study and the attendant drop out rates (Loots, Osborne and Seagraves, 1998).

111

Information TechnologyThe proliferation of the use of the worldwide web as an educational medium has seen the adaptation of the computer as a means of communication between learners. The web has matured from merely a provider of pages of information to an interactive environment through the provision of email, chat rooms, and web-boards. Many courses are now being developed directly onto the web and range from simple course notes to complete online courses in virtual classrooms. Online tutorials are not just limited to distance education courses but are also finding a place in regular university courses. The use of computer-driven interaction through email etc. is one of the fastest growing forms of communication (Palloff and Pratt 1999).According to recent research computer conferencing is said to offer many benefits that aid student learning and improved social relationships (Crook, 1994; Mercer and Wegerif, 1999; Ryan et al, 2000). The use of computer conferencing allows learners to present issues, clarify misunderstandings, prepare learners for new topics, engage in ongoing debate and discussion and deliver time-consuming administrative information (Mowker, 1996). This approach has lead to the increased adoption of co-operative and collaborative learning through the use of new technology. Instead of merely asking learners to write individual essays it is now possible to allow students to produce presentations including graphics, audio, video and other forms of multimedia; resources; data files etc. However, the amount of work required is too much to expect from an individual but the technology allows the students to work co-operatively in order to produce such work. Co-operative learning refers to learners working in small groups to achieve a mutually understood task through a process of interaction. Underwood and Underwood (1999) see co-operative learning as a mechanism that

“emphasises cognitive processes such as conflict resolution, hypothesis testing, cognitive scaffolding, reciprocal peer tutoring and overt execution of cognitive and meat-cognitive processes and modelling.”(p. 12)

McKendree et al. (1998) refer to the use of students acting as a resource for each other and that their online conversations can be captured for use by future learners. Mayes terms this use of preserving digital conversations for others to read as ‘vicarious learning’ (Mayes, 1997). The efficiency of these group interactions can be due to the use of inbuilt conflict set into the activity in order stimulate the learners to engage with each other as detailed by Piaget. The alternative mechanism is through the use of Vygotskyian co-constructive processes. Underwood and Underwood (1999) suggested that interaction between learners in the form of computer conferencing might be more effective in achieving learning outcomes. There is evidence to suggest that peer tutoring and peer collaboration can effect learner development. If a group is effective then there will be an exchange of knowledge and learners will accept information from each other as well as requesting information and assistance. There are very few studies on the effectiveness of computer-mediated conferencing (CMC) and detailed analyses of the nature of learners’ postings in an online instructional work-based setting.The hope that e-learning might revolutionise learning in the workplace has, so far, not been realised. Oberski et al (2000) also concluded that the continued workload pressures and the juggling of priorities made e-learner participation difficult to maintain. At the same, time lack of familiarity with the use of computers and the Internet are issues for many would-be learners. Information Technology brings its own array of technological issues and barriers that need to be faced by online learners (Berge, 1998). For example, problems with connectivity, slow Internet response rates, inadequate software and

112

hardware for the downloading of documents, sound and video files, and inadequate or inaccessible technical support.A review of the literature on small and medium sized business learning shows that research data is sparse and limited (Chaston, Badger and Sadler-Smith, 1999). This is also reflected in the limited, but growing research on the application of Information Communication Technologies (ICTs) in the workplace and within SMEs in particular (Southern and Tilles, 2000; Gray, 1999; Teague, 1999). This particular research project was designed to provide a case study of a work-based learning experience, using ICT Web-based online learning access, within a medium sized enterprise in the engineering sector of the Scottish economy. The company’s manufacturing output includes: machining, presswork, sheet metalwork, metal finishing and electro-mechanical assembly; and supplies most of its products to major computer hardware manufacturers around the world. The company holds the Investors In People (IIP) award and believes in maintaining a competitive advantage through a policy of training and developing a highly skilled and flexible workforce. In this study, 51 employees out of a total of 226 staff participated on an online learning module on finance that was part of the post-graduate Certificate in Small and Medium Enterprise Management (CSMEM), provided by the University of Stirling.

The company’s aims For the employees to develop awareness and understanding of the

company's financial performance.

Promote development of basic Internet/Web skills by the employees.

Encourage a culture of lifelong learning within the workforce.

Encourage employees to enhance their personal financial management skills.

The objectives of the investigationThe objectives for this particular investigation were:

To investigate if employees within the company could learn to develop a quality questioning approach, as an indicator of learning, in order to understand and interpret the financial information presented to them by the company, through participating in an online learning course on finance.

To evaluate the work-based learning experience of staff using a web-based e-learning experience.

To gain further knowledge and understanding of the issues that surrounds the process of learning within a medium sized enterprise.

The case study ran from October 2000 to February 2001 inclusive.

MethodologyA qualitative case study was designed and implemented over two phases using a before and after approach. Participants received their initial questionnaire, Questionnaire 1 prior

113

to starting the CSMEM online Finance Module. Throughout the duration of the course the number of recorded ‘hits’ by participants helped to identify the level of participation. This demonstrated that most employees were not active over the period of the study. The outcome being that 79% of the sample never logged-on and only 29% actively participated. Two separate focus groups were issued with the second questionnaire, Questionnaire 2, and these groups took part in semi-structured interviews at the end of the course. The Active Focus Group was representative of those who had had made some progress in the course, whilst the Non-Active Focus Group consisted of those who had not actively participated. The purpose of these Focus Groups was to draw out issues from both groups, which highlighted the experience of those actively participating, and issues that arose out of lack of participation. It was the intention that this investigation would identify any links existing between learning finance and the quality of questions that employees would subsequently ask their employers. The final stage was to hold a semi-structured interview with the Human Resource Manager to outline the views and perspective of the management following the particular learning experience.

Pre-course investigationA representative sample of 51 people received Questionnaire 1. This was the total number of participating employees who were registered on the WebCT CSMEM Finance Module. At the time of sampling, the company’s total workforce was comprised of 226 employees of which the sample size was 23%. The overall response rate was 53% resulting from the final sample size of 27 returned questionnaires. The questionnaire indicated that all the respondents recognised the value of computers and how the Internet could help with their learning. Of these, 96% had prior experience of using a computer and were at ease with this mode of learning, in fact 63% of the sample had their own personal computers at home.Thirty percent of the sample anticipated that undertaking a course on finance would help them to understand the company’s financial performance. The largest response of the sample (63%) felt that they needed to understand the company’s financial performance in order to determine the company’s profitability, and perceived their own job security as being linked to this. After the questionnaire had been returned a semi-structured interview was conducted with a Focus Group to explore employee pre-course thinking. At the time of sampling and interview, the management perceived that employee questions on finance were always related to aspirations for improved pay rates and more opportunities for overtime. It was the management’s hope that a course of finance might bring some revelation to the thinking of the employees, and hopefully the company’s figures would produce more awareness as to why employee’s aspirations could not be fulfilled. The results from the questionnaires did not support the management’s view of employee thinking, nor did the Focus Group interview. Rather it was the view of the employees that understanding finance would help them determine if the company was profitable, and if so, then their jobs would be secure. At the time of the study, Autumn 2000, the company was starting to experience a downturn in demand for its products from major manufacturers in the USA. It is likely that at the time of sampling this attributed to the workforce’s pre-occupation with job security and how they were seeing it impact on the production line.

114

Post-course investigationThirty-eight people remained registered as participants for the WebCT CSMEM Finance Module. Questionnaire 2 was distributed to this remaining group who were representative of 18% of the company’s workforce, which had reduced to 210 employees. The reduction in the total workforce resulted from a redundancy programme ushered in by the downturn in the information technology sector reducing demand for the metal casings made by the company, and partly due to some participants leaving to take up new employment. The final sample size of 25 completed questionnaires provided a response rate of 65% from the participants. The relevance of the subject (finance) was a primary concern for the majority of the participants, motivating only 20% of the sample to participate in the course of learning. The fact that is was not viewed as job-related learning explained the low participation. It was not seen as a means of helping them to perform better in their jobs, nor was it likely to help them manage their personal finances. If they had been given a greater choice of job-related subjects they would have been more motivated in engaging learning at work. Given adequate time, encouragement to learn, and the right subject, work-based learning was considered as a potentially worthwhile experience. However, in this particular leaning exercise the employees considered learning about finance to be something the company wanted them to do. They viewed it more as an imposition rather than an opportunity for learning. Furthermore, the second questionnaire and the follow-up Active and Non-Active Focus Group interviews clearly indicate that the majority of employees do not accept it is their responsibility to learn how to understand the figures. They expect the company to provide clear and unambiguous figures, and also to present them in an easily understandable manner. Very few of the employees used the figures as a bargaining tool for increased pay, overtime and more bonuses, which was actually contrary to the expectations of the management. As with the pre-course findings, the employees anticipated that a greater understanding of the company’s figures would have given them evidence of company profitability relating to their own job security.The small number of employees actively participating in the course on finance meant that it was impossible to determine if a link existed between the type of questions asked before and after the course of finance. Indeed as the Active Focus Group members reported, their own perceptions of the company's figures had not altered in any way as a consequence of participating on the Finance Module. Participants appreciated the relevance of Information Technology and were more interested in how to use computers in the workplace, develop programming language skills, and production machinery programming. Central to their thinking was the need to keep abreast of technology developments for improving their productivity and for ensuring the company maintained a competitive edge in the marketplace. This very much supported the management’s view of the importance of training and development in ensuring future business success. Previous studies by Seagraves et al (1996) and MacLaren and Marshall (1998) have shown that the motivation to study by work-based learners is more often driven by learning that is applicable to their work and how it enables them to do their jobs more effectively.e-learning was stimulating and increased interest in learning technologies and learning by the Internet. Forty eight percent of the sample considered this mode of learning to be appropriate and staff would be able to keep abreast of work-related technology developments by having access to learning technologies. Clearly, an interested in learning has been created by the implementation of online learning. Indeed, one technical supervisor enthusiastically indicated it brought the learning to him by providing an

115

environment and opportunity to learn without having to travel off-site which was difficult, as he did not have a driving licence and public transport was poor. In this employee’s case, transport was a real barrier to pursuing learning. Therefore the availability of online learning made it possible for him to access opportunities for learning that were previously impossible. The Human Resources Manager considered this to be a good example of how an interest in learning can be fostered given the right support and access to learning.One drawback of the learning was the slowness of the Internet. When participants had to wait sometimes for online questionnaires to be completed or for documents to be printed off they felt de-motivated. Although e-learning was useful, many preferred peer learning support, rather than the isolation of learning on their own in front of a PC. Another source of de-motivation was the isolation involved. The company had provided a small room with two PCs for participants to book time on for their learning to take place. It was their view that had they been given the opportunity to study together as a group in a computer classroom they would have been able to share problems and together identify suitable solutions. For example, when they experienced difficulties getting to grips with some of the financial exercises or when they encountered problems with the online technology, had they been able to engage in problem sharing and problem solving with colleagues face-to-face then this might have reduced personal frustrations with the learning experience and alleviated levels of de-motivation. This may indicate a need for a workplace champion to actively promote e-learning within the organisation, and who will encourage staff to pursue a course of learning, by providing support and helping them to overcome difficulties.The course would have been better designed had it made use of a central online discussion area that would have enabled the learners to leave messages for each other thereby enabling them to communicate with each other. Thereby allowing them to share their problems and solutions. As it was shared communication was less structured and patchy through conversations in the corridor or at the coffee machine. Failure to make use of this technical feature in the course was down to poor course design rather than a technological deficiency.It was also not surprising, as Seagraves et al (1996) and Oberski et al (2000) found in previous studies, that time and workload pressures made the greatest impact on participation and motivation. Sixty percent of the sample attributed time and workload pressures as the main reason for either not participating in the course or for restricted participation. This was further confirmed in both focus group interviews. Most staff taking dedicated time off to learn were hindered by the culture of the workplace, which tended to act against their desire to participate in the course. Pressures of work and daily operational demands took priority over learning despite the company offering an hour off work each week (to be matched by the employees giving an hour of their own time). There is evidence from the questionnaire responses to suggest that there is a genuine interest in lifelong learning given the right conditions and the right subjects to foster it. Yet in order for this to be encouraged in the workplace, operational demands need to be addressed before there can be any hope of the employees embarking on effective work-based learning. With reflection, it is was agreed by both the employees and the Human Resources Manager that it might have been more beneficial to have staggered the course rather than having so many people starting at the same time. The learning could have been offered to a smaller pilot group, which would have given an opportunity for any problems to be sorted early on in the programme, providing a more positive experience for those joining later on. The size of the initial group (51) made it difficult to provide effective support

116

within a busy operating environment. Clearly, more thought needs to go into how and when people learn. Although the company had requested that employees would complete the course in six weeks during the period October to December 2000, this proved to be unrealistic and so the period of learning required extension to February 2001. The management had not anticipated the effect of the approaching Christmas season as well as the operational demands on the employees. It proved a busy and stressful time, resulting in much reluctance to give an hour of their own time after work to pursue their studies.

SummaryFrom the outset, the results generated from the case study were not considered to representative of all SME experiences of online learning in a work-based setting. Nevertheless, this study did highlight the experiences specific to a company and its employees as a result of implementing e-learning within a work-based setting with the hope that the findings will enhance knowledge in the areas of both work-based and online learning. It is hoped that if all employees eventually completed the course, a longitudinal study would reveal if there was any link between completing the course and the nature of the questions asked at the end of the course.As mentioned earlier, work-based learning and online learning are considered to be important ways of boosting learning for the individual and enhancing company performance. However, this study reflects the common problems that can arise for small to medium sized enterprises particularly in the areas of time and workload pressures.The investigation has shown that when designing courses for learning it is important that they have relevancy in helping people to do their jobs more effectively. Furthermore greater consideration needs to be given as to how courses are converted for online delivery. What had been considered a satisfactory course when delivered face-to-face had problems when merely converted to a web format without due consideration being given to the design and delivery of the course from an e-learning point of view. Allowing students access to their fellow students both to discuss course matters and also for social exchanges we consider a crucial facet of learning online. This feature allows for the promotion of a Community of Learning that acts as a support network for its members.It was also seen from the study that the working environment is an important factor in influencing the motivation and interests of the employees. Even though it is laudable to encourage learning, the issue of relevance cannot be ignored. However, on the positive side Web-based learning was seen to be a good way of accessing learning opportunities related to how well they could perform in their jobs.

ReferencesBerge, Z.L. (1998) Barriers to Online Teaching in Post-Secondary Institutions: Can Policy Changes Fix It? Online Journal of Distance Learning Administration Vol 1 (2) pp 1-12. Found at: http://www.westga.edu/~distance/Berge12.html [Accessed 4 April 2000]

Bridge, S., O’Neil, K., and Cromie, S. (1998) Understanding Enterprise, Entrepreneurship and Small Business. Basingstoke: Macmillan Press.

Chaston, I., Badger, B., Sadler-Smith, E. (1999) Small Firm Organisational Learning: Comparing the Perceptions of Need and Style Among UK Support Service Advisors and Small Firm Managers Journal of European Industrial Training Vol 23 (1) pp 36-43.

117

Crook, C. (1994) Computers and the Collaborative Experience of Learning. London: Routledge

DfEE (1998) The Learning Age. Sheffield: Department for Employment and Education.

Gibb, A. A. (1993) Small Firms Training and Competitiveness. Building Upon the Small Business as a Learning Organisation International Journal of Small Business Vol 15 (3) pp 13-29.

Gibb, A (1999) SME Policy, Academic Research and the Growth of Ignorance, Mythical Concepts, Myths, Assumptions, Rituals and Confusions International Small Business Journal Vol 18 (3) pp 13-35.

Gray, D (1999) Work-based Learning, Action Learning and the Virtual Paradigm. Paper presented at the European Conference on Educational Research, Lahti, Finland 22-25 September 1999.

Found at http://www.leeds.ac.uk/educol/documents/00001260.htm

[Accessed 1 August 2000]

Lange, T., Ottens, M., Taylor, A. (2000) SMEs and Barriers to Skills Development: A Scottish Perspective Journal of European Industrial Training Vol 24 (1) pp 5-11.

Loots, C., Osborne, M., and Seagraves, L. (1998) Learning at Work - Work-based Access to Higher Education The Journal of Continuing Higher Education Vol 46 (1) pp 16-30.

MacLaren, P., Marshall, S. (1998) Who is the Learner? An Examination of the Learner Perspectives in Work-based Learning Journal of Vocational Education and Training Vol 50 (3) pp 327-337.

Mayes, T., (in press), Groundhog Day, Centre for Learning and Teaching Innovation, Glasgow: Glasgow Caledonian University.McKendree, J.; Stenning, K.; Mayes, T.; Lee, J.; Cox, R.. Why Observing a Dialogue May Benefit Learning. Journal of Computer Assisted Learning; v14 n2 p110-19 Jun 1998. 1998

Mercer, N. and Wegerif, R. (1999) Is ‘exploratory talk’ productive talk? ’ in Littleton, K. and Light, P. (eds) Learning with Computers: Analysing productive interaction. London: Routledge

Oberski, I., Palomar, A., Noya, C., Ruggiero, E., Herrera, F., Korhonen, K., Osborne, M., and Davies, P (2000) Evaluating Online Work-based Education for Managers in SMEs. Some Initial Observations. Industry & Higher Education Vol 14 (3) pp 200-203

Palloff, R, and Pratt, K., (1999) Building Learning Communities in Cyberspace: effective strategies for the online classroom. San Francisco: Jossey-Bass Pye, J (2000) Promoting Workplace Learning with ICT: Modes and Models for Organisational Change. Paper presented at the European Conference on Educational Research, Edinburgh, 20-23 September 2000. Author contact: Exeter University.

Found at: http://www.leeds.ac.uk/educol/documents/00001611.doc [Accessed 4 March 2001]

Ram, M. (2000) Investors in People in Small Firms: Case Study Evidence from the Business Services Sector Personnel Review Vol 29 (1) pp 69-91.

Ryan, S, Scott, B., Freeman, H, and Patel, D. (2000) The Virtual University: The Internet and Resource-based Learning. London: Kogan-Page

118

Seagraves, L., Osborne, M., Neal, P., Dockerell, R., Hartshorn, C., and Boyd, A (1996) Learning in Smaller Companies Final Report. Stirling: Educational Policy and Development University of Stirling.

Seagraves, L., and Osborne, M. (1995) Learner Motivation in Part-time Higher Education including Work-based Learning Paper Presented at Towards A Learning Workforce Conference, University of Lancaster, 12th to 13th September 1995.

Southern, A., and Tilles, F (2000) Small firms and information and communication technologies (ICTs): toward a typology of ICTs usage New Technology, Work and Employment Vol 15 (2) pp 138-154.

Teague, J. S. (1999) Computer Mediated Communication in Distance Post-Graduate Teacher Education: Students’ and Tutors’ Perceptions of Different Types of Computer Mediated Communication. Paper Presented at The British Educational Research Association Conference, University of Sussex, Brighton, 2-5 September 1999. Found at: http://www.leeds.ac.uk/educol/documents/000001100.doc [Accessed 12 November 2000]

The Scottish Office (1998) Opportunity Scotland: A Paper on Lifelong Learning. Edinburgh: Stationery Office.

Underwood and Underwood (1999) ‘Task effects on co-operative and collaborative learning with computers.’ in Littleton, K. and Light, P. (eds) Learning with Computers: Analysing productive interaction. London: Routledge

Westhead, P., and Storey, D. J. (1999) Training Provision and the Development of Small and Medium Sized Enterprises: A Critical Review. Scottish Journal of Adult and Continuing Education Lifelong Learning. Vol 5 (1) pp 35-41.

119

120

Kees Schuur, ??? (Netherlands)

A holistic vision of the future of e-learning

IntroductionDuring the last 15 years there has been considerable research and development into the use of Information and Communication Technologies (ICT) for learning. e-learning environments have become readily available during the last 5 years. If we believe the claimed advantages of e-learning, for example cost savings, Just-In-Time learning, any time, at the learners’ own pace, at any place, flexible learning styles, etc, the future looks bright. Despite this, only a small percentage of learners, especially in industry, are using e-learning environments.This article explores several factors, which influence the use of e-learning environments from the point of view of developments in different disciplines and in society, and provides a vision of the future for ICT based learning. It outlines for development of e-learning environments in the future. It is not intended to be an exhaustive list , but rather to show the direction of development and to be a starting point for further discussion. Examples11 of different systems and platforms are included to illustrate the discourse. What kind of impact will new technologies have on society? Where are we heading? How can we shape the development of technology? And what will the impact be on our learning and the instruments we use for learning, such as e-learning?One way to create a picture of the future is to identify present technologies and based on this to describe possibilities for the near future. A second way is to look back at historical developments and to extrapolate from these into the future. A third way is to combine elements from different disciplines and view different combinations as pictures of the future. This paper tries to combine these three ways

The increasing speed of change, development and the future

Changes: in the past, present and futureCreate the New World,Because we don’t understand it. Xerox, 1971

Life is dynamic and continuously changing. We would like to predict the future, but we know we never can be sure. There are too many uncertainties, too many variables and actors, which influence the developments taking place. The only thing we can be sure of is that the future will be different and dynamic.The speed of change is accelerating. In the past centuries, a change could be incorporated into society over a time-span of several generations. In this century, changes have a time-span of one generation; now changes are taking place within a generation. Changes in the past could be managed, now they only can be facilitated and barely incorporated, in the

11 Sometimes brand names are used. These are only meant as an example and are not meant as an evaluation of that product / brand. It only describes one small element of the instrument / environment, certainly not enough for positive or negative evaluation of that product / brand.

121

future there will barely be time enough to initiate them. It seems that change just takes place.

Time-scale in societal developmentIn the historical development of society a logarithmic scale can be seen. The developments reviewed here have different time-scales. Some are measured in thousand of years; others in hundred of years and still others have a time-span of only decades. Future time-spans will most likely be years, months, weeks, or even days. It is possible that future developments will have no longer have a direct impact on lifestyles. Change has become part of everyday life. We will be living in a second or third mode.Around 5000 years ago, people began to record their knowledge in writing on stones, parchment and later on paper. Before that period knowledge was preserved in the (collective) mind of individuals and groups. Knowledge was passed on from parents to children, from elderly to young. Security and the physical survival of the species were the most important issues in the agricultural age. Knowledge was expended on securing food and housing. With this security in place, people could devote some of their time to other activities such as art and technology.A new phase was entered when society had to address more complex issues. Urban centres emerged. The development of cities led to commerce and trade and manufacturing. Knowledge, now made explicit in writing, was centred on libraries like Nineveh and Alexandria, where scholars could come together to explore and combine explicit knowledge and create new knowledge. These centres of learning were the basis of the establishment of the universities.Structuring information and knowledge became necessary as activities like warfare and construction of buildings such as castles and churches became complex. Knowledge was distributed first by hand-copied manuscripts and later by printed books and documents. Explicit knowledge became available to more people.The industrial age brought mass, low cost production. Society required a trained, efficient and skilled workforce. Education and training systems were developed with mass school and classroom based education. It was the age of social control. But knowledge was becoming still more complex, with increasingly rapid change. What was learned at school was already outdated by the time it could be applied in the workplace. In the working process, knowledge was changing continuously. People were expected to participate in lifelong learning. Possibly more important was the un-learning process, where people had to forget what they had learned.The focus of education moved from the classroom to group based constructivist learning. Knowledge was broken down to small units and the timeframe for learning (especially in higher education) changed from courses, which lasted sometimes for years, towards day long seminars or workshops of less than an hour. The use of computers allowed storage and access to large quantities of information and explicit knowledge.In the last decade, access to information has become readily accessible. While writing this article, Altavista gave more than 500.000 hits for ‘e-learning’ and more than 600.000 for ‘e-learning’. The Google search engine already scans over one billion URLs. Around 600 million people have Internet connections of which 190 million are in Europe. Because it is impossible to read all this information or to contact all those people, the future approach to knowledge will be through networking, in communities of practice and on learning systems and processes within groups.

122

Personal developmentThe philosopher Cornelis describes the societal developments outlined above in individual terms. In the first part of someone’s life, a person lives in a protected environment, where he or she learns basic skills and social behaviour. In the second part of their life they live in a socially controlled environment reaching its highest level at the age when people get a job and become part of social groups (parents, yuppies, dinkies, etc). Cornelis sees the third phase of life as when one has experienced a lot, the kids have left the house, their income is assured. That phase he describes as the phase of communicative self-control.A problem is that the driving forces to use e-learning environments often come from highly educated people in the social control system and from more elderly people in the communicative self-control. These individuals and groups expect others to comply with their criteria for education and technology. What does all this mean for e-learning and the use of e-learning in SMEs. At this stage we can advance a number of ‘Learning- discussion points’.

Learning- / discussion points:

e-learning in SMEs should offer possibilities for ordering and classifying the most important information and knowledge and making it accessible both technically and through mental schemas.

In the age of networking, networks are crucial. As a network consists out of nodes and connections, e-learning hould focus on strengthening the competences of the individual and on developing connections between individuals (communication).

(e)Learning is becoming less important in content terms. The focus should be more on the development of processes in learning systems and on complexity management of the system.

New paradigms should be defined and adopted, where learning systems are not restricted to existing learning philosophies.

Economic changes and developmentIf we look at the economic development of e-learning environments we should expect these environments to be highly valued. The old-economy based advantages of efficiency, cost-effectiveness, the possibility of Just-In-Time learning, at the learners’ own pace and at any place should provide enough positive benefits to have a high value to the users.But after many experiments and five years of mainstream e-learning environments on the market, there is still limited value for the users. Despite the launch of many projects and pilots, there is relatively limited use.Economic value is often expressed in share prices. Shares of in companies producing learning environments - such as Smartforce, Digital Think, Saba and Docent - have dropped to sometimes less than 1% of their highest value (see http://finance.yahoo.com).It seems that economic principles are changing. Davies and Meyer (1998) describe, in their book ‘The speed of change in a connected economy’, the way in which forces of speed, intangibles and connectivity are challenging business behaviour. The table below illustrates new business behaviour or “Management Mind Set”. The last column of the table illustrates some of the implications for e-learning in SMEs.

123

Management Mind-set (Davis & Meyer, 1998)Product Service Offer For e-learningin SMEs

Time Horizon

Time of sale Period of contract Life of consumer need

Hour, day, (ir-)regularShort learning processes life long

Buyer concerns

Price, delivery, convenience

Ongoing support Upgrade ability

Upgrading horizontal and vertical is, more important than the content

Cost focus Direct Period Design Both ways in continuously designing

Source of value

Manufacturing Training, maintenance

Platform Community / social / collective learning

Design Fixed, uniform Customised Learning Ever developing, often old includingConstructivistic

RevenueModel

List price Subscription period

Subscription + user fees

Participative role

Marketing objective

Brand loyalty Relationshipbuilding

Community building

Learning together more important than supplier, the electronic learning environment and the available content

In the new economy the roles of the producer /supplier and consumer changes continuously and often both parties play at the same time both roles. As an example: a producer of packaging machines sells a packaging machine to a company. He or she sells the machine but implicitly buys information, such as the need for the machine or the problems that occur in implementation. He also buys the networks (direct or indirect) the company has with other potentially interested companies.As soon as a problem occurs the consumer offers information to the supplier that can be used for improving the machine and for providing advance improvements in services to other users.e-Learning takes places in such an ‘offer-environment’. Information and knowledge are exchanged in an iterative manner between SMEs and e-learningcompanies – both those providing learning environments and those providing learning materials and facilitation.Support and evaluation are particularly important for SMEs. Evaluation will often take place as an iterative process. Evaluation can allow the prediction of future scenarios and allow steering of technology implementation and learning processes.Another issue is that in the transition (buying/selling) process from producer to consumer both sides use different criteria in evaluating a product. Whereas the producer looks at stability and functionality, design, communication concepts, image and distribution, the consumer evaluates the ‘benefit’ it has for him or her (see table below).

Producer

Mar

ket

Val

ue Consumer

Distribution

Pri

ce

Eva

luat

ion

of th

e of

fer Social identity

Image Relation between buyer and seller

Communication concept AvailabilityDesign Desire

124

Functionality Benefit

Consumers of an e-learningenvironment have their own way of looking at an environment. Some examples of the way in which perceptions are shaped are given below.

Social identity:

A child does not evaluate the Microsoft Network (MSN) on the basis of its functionality, but on its potential for identifying with others in his or her peer group. It is difficult to convince consumers to participate in e-learningwhen there is no social or cultural acceptance and practice of learning in their peer group. One of the most important factors in determining participation in e-learningwill be the type of work and the community the individual is in.

Relation between buyer and seller:

Users of Lotus Notes will find it easy to use Lotus Learning Space. They are used to the environment and can access the learning environment from within Lotus.

If there is already a good and effective relationship with a training organisation, it will be difficult to persuade people to use an e-learning environment, other than e-mail for contact with the trainer, unless that training organisation itself promotes e-learning.

Availability:

If a trainer does not show up at a training session, learners will complain. Similarly if a trainer is not available when needed. The same applies for e-learning. An e-learning environment should do what it promises: Just-In-Time, Just-Enough, at any place, pace.

Often, e-learning courses do not match with the rhythms of the technology they utilise. Many courses are too long and do not correspond with the learning style of the individual or group.

Desire:

Why do so many people enjoy playing a simple game like Tetris or Pacman? What is driving people to use these programmes instead of beautifully designed learning environments? It seems that many people like to absorb themselves in the environment without having to learn new ideas.

The benefit of e-learning is different for each actor: For the learner: easy to learn, solving problems is easier, higher payment, status of

e-learning, etc. For the company: cheap, effective, efficient, not loosing working time, etc. For a trainer: new opportunity to provide training For the developer: money, promotion, etc. For the director of a training institute: break even or make a profit.

Learning / discussion points:

125

An e-learning environment should fulfil the needs of the user. Often a simple item in a programme can be more effective than a beautiful, complete and complex learning environment.

Evaluation of e-learning environments will need to be a continuous iterative process.

Evaluation of e-learning environments will need to focus more on the criteria of the user, than on design, functionality or cost benefit.

New paradigms should be developed applicable to a networked society.

The context: SME environmentSMEs are like spiders in a web. To survive and to make a profit they have to optimise all strands in the web. These include connections with suppliers, customers, the bank, the local and regional authorities, trade associations, links to other SMEs, links between employees.The job of managing a SME is demanding. Often employers and employees have to fulfil several work roles, have to work hard, be flexible, be innovative (products, production process, marketing, selling, cost reductions, sustainability, etc.) in order to stay in the market. In such an environment employers and employees are in a continuous learning process. In a natural, constructivist way they incorporate directly explicit and tacit knowledge or are making new combinations of knowledge. They are used to acquiring and improving skills and competences through informal learning. There is a conflict between cultures such as technology, learning, enterprise and within this complex situation SMEs have to strive to remain learning organisations. Generalisation will be difficult because cultures can differ per country, region or even locality.It is difficult to make time for formal learning during working hours. The increasing speed of change does not allow long courses. Any spare time is needed for reflection and developing social relations.It is clear that for SMEs and also for larger companies the focus has changed from products to clients. In his book “Mass-individualisation”, M. van Asseldonk (2000) says the approach is changing from an instruction and information (product) directed environment towards an interactive environment. The learning environment changes accordingly and should focus more on improving competences to deal with relationships between different actors, with innovation and with working in a changing environment. Van Asseldonk describes the changes within mass-customisation organisations in the new networked economy:

From homogeneity to heterogeneity From prediction to responsiveness From efficiency to responsiveness From Taylorism to empowerment Working with imbalance From knowledge to motivation From contradictions to coalitions From certainty to perspectives

The e-learning environment should prepare and support people working in such environments in order for them to cope better continuous change and to create new opportunities for the business. According to van Asseldonk an e-learningenvironment

126

should offer heterogeneity, responsiveness, empowerment, coalitions, perspectives and should motivate.

But the usefulness of (or interest in) e-learning environment by enterprises appears to be rather limited. According to a recent survey in the Netherlands, only 4% of company sponsored training courses are on-line and according to another survey only one out of twenty companies with an Internet connection is using e-learning. 2% say they are considering using e-learning.Most employers say that e-learning is not personal enough and it is more difficult to monitor online learning. A Canadian survey (1998) showed an increase in informal learning by adults. On average individuals learn 4 hour per week through formal courses and learn informally 15 hours a week. Companies that invest in formal training and e-learning environments focus more on formal training. In SMEs informal learning plays a crucial role. Another aspect is that 50% of participants do not complete online training programmes - 10-20% higher than ‘traditional’ training courses as an American survey showed. It seems that online training requires self-discipline and/or learning discipline within an organisation.These figures have to be viewed within the context of the overall training provided by enterprises. Another Canadian survey showed that, in 1998, 74 percent of all establishments sponsored or provided formal or informal training for employees - 55% sponsoring formal training. The figures for small enterprises (<20 employees) were significantly lower, 67% and 46% respectively. Given these figures there must be questions over the direct return on investment in the development, implementation and use of e-learning environments for small (and medium) enterprises. e-Learning is claimed to increase work effectiveness, but at the moment e-learning is only decreasing the time available for work or is making the working day longer.

Learning / discussion points:

Using e-learning in downtime or after work decreases the time people have to exchange information and ideas, socialise and reflect.

e-learning environments should support more informal learning processes in SMEs.

e-learning works best where ICT is already integrated in the working environment.

Organisation and learning within chaos and complexityThe choreographer, Tom Simons, has created a dance performance called “The idea of order”. He describes his dance performance like this:

“You cannot escape from the necessity to order, to organise.Yet the manner in which you order, organise, needs to be revised continuously.I have made fragments, which vary:

partly or as a whole, either or not combined with other fragments, either or not related to other elements, different in space and time.

The many ways in which you can arrange, determines the performance.”

127

The learning performance in SMEs is like his dance performance. In a chaotic and complex working environment and societal context, a person takes fragments of tacit and explicit knowledge, competences, situations and changes in the organisation around them and organises these in a special order, partly or as a whole, either combined or not with other fragments, either related or not to other elements, different in space and time, and so create an optimal learning performance and work output. This continuous process leads to an optimal, individual work-performance and, by interaction, as a group of individuals and, as a circle, feeds again the learning process. In e-learning we need a choreographer in each person, organising his or her own learning performance.Secondly, we need to make easily accessible a variety of fragments or elements, which can be used by and are useful for the learner. Explicit knowledge (e.g. information), learning materials, a virtual teacher and the ICT environment are only small parts of the elements and fragments the learner needs.Thirdly, in a complex, ever faster changing environment it is virtually impossible to control data and facts. In a second order process of e-learning the data / information / knowledge / competences and even the learning process are of secondary importance. It is like preparing for an earthquake: it is impossible to create an escape route for everybody. You never will know when it will happen and you have to take action, where you are, according to the situation, time and place. But it is important to build self-awareness of possibilities, of potential and alternative solutions and opportunities, for each individual to act at that moment and place and in that context. e-learning today is like a shelter or an escape route. We are creating or building this instrument, but we only use it if we must and if it is the best option in a given situation or context.In a chaotic and complex situation, like in SMEs, it is more important to rely on collective competence in dealing with new situations. In new forms of organisations people rely on their networks. As a second order process they create a latent reservoir of competences and expertise that can be used when needed.

Learning / discussion points:

New e-learning environments should focus on supporting second order processes and network learning processes.

e-Learning should offer a reservoir of knowledge and information to employees, communication between learners, and initiate and inspire people to learn how to quickly learn and act in changing environments.

E-portfolio and competencesThe e-portfolio is a perfect example of an instrument in a second order process. Learning is subordinated to the process of awareness raising, to making tacit knowledge explicit and using explicit knowledge as tacit knowledge, reflecting, initiating, inspiring, and navigating as an instrument to account for individual competence and development.The e-portfolio usually includes:

an open part where (proof of) products and reflections on competences can be stored

a closed part where (proof of) contracts, feedback, information and private information can be stored

Information can be ordered and presented in different ways, depending on the reason and context in which it will be used. Data also can be used in a collective environment. It can

128

be used in networks to identify the competences required for a job or to identify the need for new competences. At company level it can lead to a awareness of the supply and demand of competences and if stored at regional or national level it offers an insight into the status and development of existing competences and the need for stimulating (second order process) learning processes.

Learning / discussion points:

The e-learning environment should be closely linked to portfolio development. The focus must be on the competence development of a person, network,

company and / or society and should be supported by new technologies.

Learning, technology and performance in Communities of PracticeThe use of technology is not only dependent on functionality, efficiency, cost-effectiveness and design. New approaches to the use of ICT embrace social learning and constructivistic learning. The starting point for social learning theory is that humans are social beings, who have competences (implicitly having knowledge) and who know (participation, engagement). The driving force behind social theory is the act of meaning. In a network organisation, a SME, an institute or a social gathering, a community of practice can exist. Wenger (1998) describes social learning in his theory of Communities of Practice (CoP)He found that four components are essential within a CoP:

1. Meaning (within the experience)2. Practice (by doing)3. Community (by participating)4. Identity (by becoming)

Failing in one of these components means less effective (a combination of affective and effective) overall learning. Let us examine these components in an e-learning environment.

a. MeaningThere is a gap between e-learning and meaning. What is the meaning for an artisan or an operator to sit behind a computer to learn? What does it contribute to artisan competences? What is the meaning for a secretary to follow an online course in Word if he or she uses only some of the features of what is taught or when many things learned on such courses are only useful or needed much later?

b. PracticeWhat is the direct relationship between the e-learning environment and the work environment? Does it follow the principle of learning by doing? Can it directly be put in practice?

c. CommunityMost people belong to a group. Learning in a community is more effective (a combination of affective and effective) because of the social drivers behind it and the possibility to be part of the group. Why should children in gangs or in a club be so eager to learn and why can they learn so fast?

129

d. Identity What does an e-learning environment contribute to the becoming of a person?

Wenger (http://www.ewenger.com) describes the thirteen fundamental elements of successful communities of practice, which can be affected by technology.e-learning users follow this list unconsciously. If, for instance, the user cannot match the ‘rhythm’ of the e-learning environment to his or her personal rhythm and the rhythm of work, he or she will stop the programme.Another example is the learning process of a machine operator. The e-learning environment should give the machine operator short term value (the machine should work better, with as little as possible downtime), should involve the operator in the total process or provide learning when it is needed, and allow operators to share experiences.This list is useful for evaluating e-learning environments, their functioning and value.Wenger (2001) has evaluated technologies supporting CoPs. He puts forward eight different categories and proposes the optimum for a community of practice is towards the mid point of this list.The eight categories are:

1. Knowledge bases, where documents are stored2. Knowledge workers’ desktop, where an ongoing integration of work and

knowledge takes place3. Project places, where work is done4. Website communities, where social structures are formed 5. Discussion groups, where the conversation takes place6. A place for fleeting synchronous interactions7. e-learning spaces, where instruction takes place8. Access to expertise, where knowledge can be exchanged.

Whilst Wenger sees e-learning as important in supporting CoP, it is most effective when combined with other systems and approaches to the use of ICT. This is a critical issue for those seeking to develop e-learning in SMEs.

Learning / discussion points:

An e-learning environment should facilitate meaning, offer a place for meeting people with common and / or complementary practices, offer a place for a community and strengthen the personal identity.

130

TIME AND SPACE1. Presence and visibility2. Rhythm

PARTICIPATION3. Variety of interactions4. Efficiency of involvement

VALUE CREATION5. Short-term value6. Long-term value

CONNECTIONS7. Connection to the world

IDENTITY8. Personal identity9. Communal identity

COMMUNITY MEMBERSHIP10. Belonging and relationships11. Complex boundaries

COMMUNITY DEVELOPMENT12. Evolution: maturation and integration13. Active community-building

Wenger E: Supporting communities of practice: a survey of community-oriented

An e-learning environment should offer a community of practice support in developing a knowledge base, desktop, project places, website communities, discussion groups, synchronous interactions, instruction places and access to expertise.

Learning psychologyTheories are built on paradigms. But new paradigms arise, existing paradigms evolve and old paradigms reappear. Complexity management has to prepare people to use and combine fragments and elements to create new learning environments.Gardner has described the following ‘seven intelligences’:

1. Logical-Mathematical Intelligence 2. Kinaesthetic Intelligence3. Visual-Spatial Intelligence4. Musical Intelligence5. Interpersonal Intelligence6. Intra-personal Intelligence

Each person has his or her own combination and level of these intelligences leading to different learning styles. This creates a new variable in the already complex environment of learning. Each intelligence / learning style needs its own kind of technology application.Interpersonal intelligence is developed in pairs or small groups, where the e-learning environment is enhancing, sharing, discussing, reflecting, supporting and working collaboratively. Such an approach is underdeveloped for learning in SMEs. Computers are useful for visualising complex patterns and machines help people with a high visual-spatial intelligence.Gardner had to add two ‘new’ intelligences:

1. Naturalist Intelligence2. Existentialist intelligence

Contexts are often so complex and dynamic that it is difficult to follow a linear learning process. People combine different knowledge and thus create new knowledge. An e-learning environment should enable learners to learn using all their intelligences and leaning styles.

Learning / discussion points:

A broad based e-learning environment makes it possible to learn in different learning styles and change learning styles depending on the subject, the situation and the organisation.

(Youth) behaviourTimes are changing, so is behaviour. Thirty years ago – when many of the teachers of today were educated - there was a limited use of television and even more limited use of ICT in the classroom. The pace of change was slower. Linear instruction was normal. At home you played outside, were a member of a club, read a book or studied.Now things have changed slightly. In a recent survey in the U.S. (2001) on young people and the impact of ICT, a boy of 17 years wrote:

“I multitask every single second I am online.At this very moment:

131

– I am watching TV,– Checking my e-mail every two minutes,– Reading a newsgroup about who shot JFK,– Burning some music to a CD,– And writing this message.”

He is multitasking through several different processes and is learning while multitasking.In this survey children between 12 and 17 years old describe what they use the Internet for. The table below shows use of the internet by those who use it daily and those who are less frequent users.

Activity daily less

Send or receive mail 99 87

Send instant messages 89 64

Research products online 74 60

Download music 73 40

Listen to music online 70 52

Visit a chat room 62 50

Buy products online 39 26

Create a web page 34 16

Although not completely clear, it would appear that the internet is used more for social purposes than for learning.But whilst older people may have difficulties in using ICT for distance (informal) learning, young people have fewer problems. As a place for meeting people, it keeps an emotional distance. They also can be more their ‘true self’ and, by meeting people, explore who they are. 56% of children using the Internet have multiple identities. One of these identities is very much themselves, while others are more general (e.g. for school, relatives), or completely fake identities. Children are even able to enter a chat room with two or more identities and steer the discussion.They meet strangers (60%) and hold discussions (63% of the 60%). 50% of the children online have participated in Instant Messaging with people they have never metIn this environment young people live, learn and work.What does the older generation think about this? A 15-year-old girl said:

“I wouldn’t talk about it with my parents, they’d flip out and probably restrict my access to Internet.”

In the survey, answers from parents and children are compared.

Parents say: Kids say: I know where my kid is going online They don't know They often talk to kids about Internet use They don't Educational benefit of Internet Socialising and communication Kids use Internet for homework (65%) Music (57%), e-mail (56%), surfing for

fun (50%), games (48%), IM (40%), chat (39%), homework (38%)

38% of kids have e-mail account 71% often have more than one 28%: kids use Instant Messaging 56% do

132

Our one-to-one, often linear (although often named ‘interactive’) learning approach through present e-learning environments does not match with the behaviour ofthe new generation.

Learning – discussion points:

The future of the e-learning environments lies more in offering communication, possibilities for socialisation, multitasking, and emotions / fun.

Different phases of development of an individual need different approaches.

Technology that changes the way we learn and workComputers are incredibly fast accurate and stupid.People are incredibly slow, inaccurate and brilliant.Together they are powerful beyond imagination.Albert Einstein (1879-1955)

When looking at the development of e-learning we also have to look at the development of ICT and the impact on working and learning. In this chapter two arbitrary examples are described to trigger the imagination of the reader about e-learning in the future.The first example comes from the agricultural sector. This sector is known as a knowledge intensive sector. The first innovations took place in agriculture ten thousand years ago and food remains one of the basics for life on earth.After many experiments into the behaviour of animals and optimisation of the agricultural production process, the next step is the speaking animal. In experiments it is already possible to ‘understand’ and translate several cow-calls and distinguish between different cows (http://www.tb.fal.de/staff/jahns/animal.htm). It seems that in the future we need less (e)learning to learn about the behaviour of cows (if we would presume it is possible to learn it at all). Instead we hear from the cow directly what she needs, and could even develop a system to automatically meet her needs. Are the technology solutions overtaking the need for e-learning?A second example is wearable technology. Massachusetts Institute of Technology has developed ‘intelligent clothing’ (http://www.media.mit.edu/wearables/mithril/). A person will be able to get access to information and can communicate with other persons online every moment and at any place through their clothes. Explicit knowledge will be readily available. New technologies will arise. Systems are being developed where the fabric of the clothing itself will be the computer, and sensors and displays and other output devices will be integrated in glasses, lenses or earphones.

Learning / discussion points:

These learning points are really questions. There are no concrete answers for these questions yet.

What and how will people need to learn in technology-integrated environments in the future?

Which development schemes need to be followed in order to stimulate and steer the development of future oriented environments?

Will e-learning have a function and meaning in such an environment?

133

SummaryE-learning have been discussed from the point of view of nine different disciplines. The e-learning approach up to now follows the present learning approaches in schools and at courses. ICT only added the technology to replace certain parts of the learning system (library database, conversation online conversation, group discussion chat, etc.). By using the technology it is easier en more efficient to deliver, but in this article it is questioned if this approach is leading to improving learning and converting knowledge and activities into practice and finally in an increased sustainability and profitability (in that order) of SMEs. It is suggested to focus more on informal learning, supporting communities of practice and to create a continuous development process through the e-learning environment.Up to now e-learning environment only support or substitute existing learning environments. Technology has improved over the last decade and is able to support a transition from traditional learning to continuous competence development and to support the sustainability and innovativeness of SMEsThis article doesn’t give any conclusions. It offers per chapter several learning points, which can be useful for further discussion and for further development of effective e-learning environments.

ReferencesAsseldonk M van, Massa-individualisering: Maatwerk zonder meerkosten. Samson, Deventer, the Netherlands, 2000.

Cornelis Prof Dr A, De vertraagde tijd. Essence, Middelburg, The Netherlands, 1999.

Cross J, Sources of e-learningInformation. Internet Time Group, March 2000. (http://www.internettime.com/Learning/Information.htm#what)

Davis SM, Meyer C, BLUR: the speed of changing in the connected economy. Ernst & Young LLP, 1998.

Dugas T, Green L, Leckie N, The impact of Technologies on Learning in the Workplace: Final Report, Quebec Canada, March 1999.

Gibson R, Covey SR, Goldraff EM, Rethinking the future: rethinking business, principles, competition, control, leadership, markets and the world, Nicholas Brealey Publishers, 1998.

Keefe D, Dickinson D, How Technology Enhances Howard Gardner’s Eight Intelligences. (http://www.america-tomorrow.com/ati/nhl80402.htm)

Lenhart A, Rainie L, Lewis O, Teenage life online: The rise of the instant-message generation and the Internet’s impact on friendships and family relationships. Pew Internet & American Life Project, Washington, June 2001. (http://www.pewinternet.org/reports/pdfs/PIP_Teens_Report.pdf)

Linn CE, The Linn general Theory of Marketing. Meta Management AB, Stockholm, 1999, rev 2003. (http://www.metamgt.se/sidor/gtm_theory30302.pdf)

Livingstone DW, Exploring the icebergs of adult learning: Findings of the First Canadian Survey of Informal Learning Practices. University of Toronto, Canada, 1998. (http://www.oise.utoronto.ca/depts/sese/csew/nall/res/cjsaem.pdf)

Wenger E, Supporting communities of practice: a survey of community-oriented technologies. North San Juan, USA, March 2001

134

Wenger E, Mcdermott R, Snyder WM, Cultivating communities of Practice. Harvard Business School Press, Boston,

135

136

137