a proposal for skill evaluation via complex tasks in virtual learning environments
DESCRIPTION
Jaione Cubero, Mª Soledad Ibarra Sáiz and Gregorio Rodríguez Gómez. EVALfor Research Group University of CádizTRANSCRIPT
A proposal for skill evaluation via complex task in
virtual learning environments
A proposal for skill evaluation via complex task in
virtual learning environments
Jaione Cubero IbáñezMaría Soledad Ibarra Sáiz
Gregorio Rodríguez Gómez Universidad de Cádiz (Spain)
Jaione Cubero IbáñezMaría Soledad Ibarra Sáiz
Gregorio Rodríguez Gómez Universidad de Cádiz (Spain)
EE ALforALfor EVALfor Research Group
Presenting author: Gregorio Rodríguez Gómez
A proposal for skill evaluation via complex task in
virtual learning environments
A proposal for skill evaluation via complex task in
virtual learning environments
EE ALforALfor EVALfor Research Group
1. Introduction2. Context3. EDECOM program4. Objective5. Methodology6. Results7. Conclusions
1. Introduction2. Context3. EDECOM program4. Objective5. Methodology6. Results7. Conclusions
A proposal for skill evaluation via complex task in virtual learning environments
1. CONTEXTUALIZACIÓN
EE ALforALfor EVALfor Research Group
1. INTRODUCTION
The growing relevance of Information and Communication Technologies (ICT) in all aspects of society means that university education must adapt to new learning environments which are conditioned by virtualisation and digitalisation.
The use of e- assessment in higher education is a relatively new education practice that has been more frequently studied in recent years (Stödberg, 2012). It is, however, already recognised as a fundamental part of e-learning.(Whitelock, 2009).
A proposal for skill evaluation via complex task in virtual learning environments
1. CONTEXTUALIZACIÓN
EE ALforALfor EVALfor Research Group
1. INTRODUCTION
Skills, competences"a latent attribute, knowledge, attitude, ability, proficiency or faculty for the practice of a profession, a job or an academic task, carrying out the required labour or academic tasks and activities properly and correctly” (Rodríguez, G. e Ibarra, M.S. 2011: 70)
e- Assessment“electronic evaluation process in which ICTs are used for the presentation and implementation of activities and assessment tasks and record the answers”. JISC (citado en Rodríguez, G. e Ibarra, M.S. 2011: 29)
e-Assessment of skills New strategy for the skills e-assessment in virtual learning
environments.
A proposal for skill evaluation via complex task in virtual learning environments
1. CONTEXTUALIZACIÓN
EE ALforALfor EVALfor Research Group
2. CONTEXT: DevalSimWeb PROJECT
Skills development through participatory assessment Skills development through participatory assessment and simulation using web tools and simulation using web tools
ALFA III (2011)-10 ALFA III (2011)-10
A proposal for skill evaluation via complex task in virtual learning environments
1. CONTEXTUALIZACIÓN
EE ALforALfor EVALfor Research Group
3. EDECOM training program
“Evaluation and Development of Professional Skills in Higher Education”
- Blended-learning Program- Virtual learning environment- Destined to final year university students. - Aims to develop professional skills (which are transversal to evaluation) via their participation in e-assessment processes.
- analytical and critical thought- problem solving- sense of ethics- decision making - teamwork
A proposal for skill evaluation via complex task in virtual learning environments
1. CONTEXTUALIZACIÓN
EE ALforALfor EVALfor Research Group
3. EDECOM training program
The students will actively participate in e-assessment processes via the use of EvalCOMIX web service and electronic tools, such as forums, wikis and simulation games
It is based on the idea that training students in assessment skills enhances the self-regulation of their learning and therefore their academic success and employability.
A proposal for skill evaluation via complex task in virtual learning environments
1. CONTEXTUALIZACIÓN
EE ALforALfor EVALfor Research Group
4. AIM
Develop a methodological design for the e-assessment of skill level
A proposal for skill evaluation via complex task in virtual learning environments
EE ALforALfor EVALfor Research Group
WHICH ARE THE PROFESSIONAL SKILLS TRANSVERSAL TO EVALUATION?
A catalogue of work-related and professional skills was created.
To write this catalogue a study was developed in order to analyse the importance given to different soft skills by stakeholders (companies, teachers and students).
The results showed a need to promote five competences which are considered transversal to evaluation :
- analytical and critical thought- problem solving- sense of ethics- decision making - teamwork
5. METHODOLOGY
A proposal for skill evaluation via complex task in virtual learning environments
EE ALforALfor EVALfor Research Group
Skills Definition
Analytical and critical thoughtActs systematically, establishes hierarchies and uses tools for analysis; questions the
situations which arise and discusses the pertinence of the judgements in order to make decisions within his/her role.
Problem solvingIdentifies, analyses, builds criteria and applies knowledge when facing specific
situations, makes decisions and evaluates possible risks.
Sense of ethicsActs responsibly according to universal axiological principles and respects diverse
cultural contexts.
Decision making
Acts opportunely, efficiently and efficiently, agilely and proactively; uses support from relevant information which facilitates the identification, understanding, evaluation
and acceptance of risks in situations of uncertainty; chooses the best alternative via consultation with the most appropriate sources, comparing and contrasting them and
integrating them.
Teamwork
Makes responsible commitments and collaborates with other people in the achievement of common objectives, shares resources and knowledge, harmonises
various interests, evaluates the opportunities and therefore contributes to a working environment which facilitates the implementation and development of tasks.
Figure 2. Definition of the skills, according to the DevalsimWeb Catalogue of Skills.(Ibarra, Rodríguez y García, 2013)
A proposal for skill evaluation via complex task in virtual learning environments
EE ALforALfor EVALfor Research Group
Level Description
Everyday
Shows an intuitive knowledge and basic grasp of assessment, allowing the development of elementary
and complex activities concerning the assessment actions to be carried out.
Thoughtful (technical)
Establishes analysis and connections between the actions, using procedures, strategies and resources
which allow sensible explanations based on the obtained results.
Comprehensive (inclusive and
creative)
There is integration, application, creativity and critical thought in the decisions made concerning the
actions and results of the evaluation; they are discussed and information is provided which is objective, reliable,
useful and oriented towards improvement.
Figure 3. Levels of skill development (Ibarra, Rodríguez y García, 2013)
A proposal for skill evaluation via complex task in virtual learning environments
EE ALforALfor EVALfor Research Group
HOW TO ASSESS STUDENT SKILLS LEVEL? COMPLEX TASK
5. METHODOLOGY
An assessment system “based on the active participation of the students, allowing them to use their knowledge creatively to resolve real problems” (Villardón, 2006). Complex activities or problematic situations (Fernández, 2011) must involve:
•The assessment requires the observation of dimensions and situations
•Application to more than one situation to facilitate transference
•Being close to reality (authentic)
•Mobilisation of various resources, both internal (knowing, knowing how and knowing how to be) and external (requested autonomously by the student)
•Development indicators (levels of command)
•Involvement of various agents in the assessment process (concordance)
A complex activity was designed for each of the skills and the student must address and respond to the established questions.
A proposal for skill evaluation via complex task in virtual learning environments
EE ALforALfor EVALfor Research Group
This is a space through which the students will be able to give a response to different situations/problems (complex tasks) which are related to the selected skills.
6. RESULTS
The “EDECOM-DevalSimWeb Skill Level” virtual campus was designed.
Their responses will be evaluated by external evaluators using e-rubrics so as to determine their initial skill level.
The proposal to evaluate the initial skill level of students participating in the EDECOM-DevalSimWeb Training Program
The proposal to evaluate the initial skill level of students participating in the EDECOM-DevalSimWeb Training Program
A proposal for skill evaluation via complex task in virtual learning environments
EE ALforALfor EVALfor Research Group
A proposal for skill evaluation via complex task in virtual learning environments
EE ALforALfor EVALfor Research Group
6. RESULTS
All participants in the Training Program must provide a response to the five complex tasks as an obligatory requirement to be able to access the EDECOM-DevalSimWeb Training Program.
Each complex task presents three unavoidable elements to assist the students’ comprehension:
•A short introduction
•The material which must be watched or read (videos, articles, etc.)
•The submission area, which details the specifics of how to respond to the activity.
example
A proposal for skill evaluation via complex task in virtual learning environments
EE ALforALfor EVALfor Research Group
A proposal for skill evaluation via complex task in virtual learning environments
EE ALforALfor EVALfor Research Group
6. RESULTS
Skill Name of complex task Resource
Decision making It’s time to decide Video
Sense of ethics What would you do? Moral dilemma
Analysis and critical thought
Robot invasion! Opinion article
TeamworkWhat if we set up a
restaurant?Video
Problem solving What’s the problem? Written problem
Figure 4. Complex task and resource used for each skill.
A proposal for skill evaluation via complex task in virtual learning environments
EE ALforALfor EVALfor Research Group
Figure 5. e-rubric to evaluate the initial skill level of the Sense of ethics skill.
Authentic assessment and student participation: is it all too much?
EE ALforALfor EVALfor Research Group
7. CONCLUSIONS
• This proposal was shared in order to provide new strategies to address skill assessment via virtual learning environments, which are appearing ever more frequently in a university setting.
• The evaluation questions and tasks used in e-assessment must evolve into sophisticated tasks (Boyle & Hutchison, 2009), which are understood to be: media-rich stimulus materials which are interactive and complex - aspects which feature in the tasks designed in our proposal.
• The final goal is to be able to discover and compare the students’ final skill development level once they have completed their training in the EDECOM-DevalSimWeb Training Program.
http://devalsimweb.eu
EE ALforALfor EVALfor Research Group
THANK YOU FOR YOUR ATTENTIONTHANK YOU FOR YOUR ATTENTION
EE ALforALfor
EVALfor Research Group
PARTICIPANTS
Jaione Cubero Ibáñez. Universidad de Cádiz (Spain)[email protected]
María Soledad Ibarra Sáiz. Universidad de Cádiz (Spain)[email protected]
Gregorio Rodríguez Gómez. Universidad de Cádiz(Spain) [email protected]
e-mail: [email protected]ágina Web: http://www.uca.es/evalfor