dialogplus asap seminar 8/03/2006 dialogplus innovative distance-based teaching: deliverables from...

DialogPLUS ASAP Seminar 8/03/2006

DialogPLUS

Innovative distance-based teaching: deliverables from the DIALOGPLUS

project

University of California

Santa Barbara

University of Southampton

Penn State University

Leeds University

ASAP Seminar – 8 March, 2006

Helen Durham, Louise Mackay, Oliver Duke-Williams, Phil Rees, Katherine Arrell and Riz Nawaz

Chair: Dan Vickers


Outline of presentation

Phil Rees Project aims, partners, deliverables - nuggets

Louise Mackay Teaching and using e-learning materials

Helen Durham AIG, Online Atlas and GPS – resources available across whole school

Louise Mackay JORUM national repository

Oliver Duke-Williams

What is Shibboleth and how will it help?

[Katherine Arrell]

User of Penn State materials and co-developer of GPS materials

[Riz Nawaz] User of Upland Catchment Management materials, which he has enhanced and developed

Phil Rees The value added by international collaboration


Project aims

• To develop e-learning materials for UG courses (and Masters)

• To do this for four Geography Exemplar topics: Human Geography, Geomorphology/Environmental Management, GIS and Earth Observation

• To experiment with collaborations with other universities within the UK and in the US

• Exchanging topic materials within modules• Exchanging generic materials• Developing materials collaboratively• Exchanging whole modules• Exchanging students

• To link these materials with digital libraries of resources (particularly those funded by JISC)

• To preserve these teaching materials in a suitable repository (originally Alexandra Digital Library, now JISC’s JORUM repository)


Access Grid meetings cemented the partnership

In the past three months we have switched to Horizon Wimba

Soton: Hugh Davies

Dave Martin

Penn State:

Steve Weaver

David Dibiase

Leeds:

John Maber

Andrew Booth

Phil Rees

Helen Durham


Nuggets and their sharing

• I proposed the concept of nugget in the project proposal as a set of exchangeable e-learning materials

• We spent time at first in trying to understand how this was different from a learning object

• Learning object is simply too restrictive and closed a concept

• A nugget is elastic: part of a lecture, a practical or a whole module

• David Dibiase (Penn State) proposed the following definition


Learning Activity

Characteristics of DialogPLUS “nuggets”


Learning Activity

Supporting material

Characteristics of geography “nuggets”


Learning Activity

Supporting material

Self assessment

Characteristics of geography “nuggets”


Developing and teaching e-learning materials

The DialogPlus (D+) project materials:

Full courses from e-learning materials

Individual learning activities for embedding - census nuggets

Generic learning materials – academic integrity nugget

Learning materials developed via Collaborative Learning Activity Design (CLAD)

…some of these with our project collaborators (UK and US)

What I’ll be looking at:Full courses delivered at SOG using developed e-learning materialsDelivery using a mix of face-to-face/distance-based learning styles

Why?Provide insight to the introduction and teaching of e-learning courses in the undergraduate curriculum…


What courses have we taught?

The courses:Geog2750 - Earth observation (Earth observation and GIS of the Physical Environment)Geog3870 - Upland Catchment Management

EO:Introduce the physical principles of Earth observation and its role in physical Geography applications

UCM:To provide students with appropriate knowledge & skills to enable effective management of UK upland catchments

Note: EO and Environmental Management core Geography sub-areas of the D+ project, material useful for Leeds and collaborating partners


Courses breakdown…

EO

Level 2 module (1 semester) 2003 – delivered f2f - 10 lectures and 8

practicals 2004 – online lectures and distance taught

maintaining on-campus practicals Students: 2003 – 36, 2004 - 56

UCM Level 3 module (1 semester) Delivered for last three years Combination of face-to-face and online

teaching Students: 2004 - 24 , 2005 - 10, 2006 - 29


Course content and delivery

Content & Delivery:

EO 10 online lectures and self assessments

(Nathan Bodington each Monday) 8 on-campus practicals (demonstrator

support) End of term face-to-face sessions Email and discussion room with tutor support

UCM 7 online lectures (placed on Nathan Bodington

each Monday) 2 face-face sessions (extra optional face-face

sessions every 3 weeks) 1 field trip to Wharfedale Email and discussion room used by students

Assessment:For both - formative (MCQ’s, worksheets, practical assessments ) and summative (project reports, poster presentation, exam)


Delivering the on-line lectures through the Leeds VLE

The on-line lectures (plus additional resources) were delivered using the Leeds Bodington Common VLE.

Course “room” resources:

Course introduction;

Distance learning advice;

Weekly lecture;Practical material;Discussion and FAQ room;Lecture MCQ test;Additional Resources;Miscellaneous Resources


What were the drivers for the style and learning objectives?

Material driven VLE driven Project driven Availability of suitable staff

Lecture structure:On-line Lectures structured in asimilar manner as a traditional faceto-face lecture with an introduction,aims, signposted sectioned contentbut come with more detail…..

How did we structure the materials?


On-line Lectures - have a distinct “tone” to that of face-to-face lectures in order for students to work unaided.

The on-line versions have been adapted to add more to the learning experience by using graphical examples and for example in the EO course external links to image sources and tutorials and interactive tests… with the suggested readings these tools consolidate the introduced topics and direct self-study.

Lecture examples MCQ example

Examples of course materials…


Click on this video footage to the right and then identify the sources of errors in the stage-discharge method for river flow estimation

Materials may consist of individual learning objects…

Learning Activity 1:


Why undertake such an endeavour?

Delivery of knowledge and skills to the students on Earth Observation and Catchment Management;

Experimentation with a mixture of face-to-face and distance tutoring on an UG module;

Preservation of learning materials from year to year/preservation of a course in light of staff changes

Why discuss these two courses?Translating traditional courses to on-line courses(with the added challenge of staff changes)

Turning a Geography tutor into a Geography e-tutor

Is this an easy translation for the tutor, student and the managing department?

We shall see…..


Evaluating the e-learning courses…

The courses were both evaluated after 2004/2005 run (with the help of Karen Fill – our Southampton pedagogic expert)

A combination of: student questionnaire;group discussion;tutor interview.

and as we had the results from the complete f-2-f version of the EO module run the previous year from the on-line version…

statistical analysis of inter-year marks (assessments and examination)

What did we learn from this evaluation?...


Evaluating the e-learning courses

- student questionnaires

For both courses

None of the students had studied by distance learning before

For the questionnaire the students were given statements regarding the content and delivery of the course and asked to score according to their level of agreement (0 – no, 1 – somewhat, 2 - yes, and N/A).

A mean agreement score was calculated for each statement.

Example of questionnaire statements on style and content


Particularly positive with respect to:

- description of content and learning objectives- inclusion of required tools- accessibility of linked resources- appropriateness of the MCQs

Particularly negative feedback about:

- their own motivation as a result of course delivery; specifically the lack of face-to-face contact (with tutor & fellow

students).

After some discussion (in groups):

Majority Best: being able to work in their own time

Majority Worst: lack of face-to-face contact

Evaluating the distance-based EO course

- summarising student feedback


General questionnaire and discussion comments:

Note - The lack of motivation and lack of contact due to the delivery being face-to-face were the types of comments that were repeated by many students across many statements.

Distractions of

the Web.

Email answers

are needed

within a day.

Surprisingly good way of learning – practical sessions were vital however in keeping me focused/on time.

Distance learning – difficult for mathematical concepts.

Discussion board/emails - not always possible to explain oneself.

MCQs could stay up after lecture deadlines; helpful for revision.


Student feedback summary…

Specific Student requests:

UCM2004: Include Field Trip

2005: More optional face-to-face sessions

EO2004: More face-to-face support

For both…..more interactive activities!

Yet some conflicting student feedback…

Online discussion proved popular with the students in one class, but completely hated it in another - citing lack of anonymity and distrust of peer’s knowledge


What are some of the issues that have arisen?

Student Expectation:

Students disconcerted by the lack of on-campus support and tended to undermine their own ability, i.e., perceived that they hadn’t understood difficult technical topics

What could have given rise to this problem?

Students not familiar with distance-based learning Lack of suitable advertising (positive reinforcement) of the course and

its delivery style – especially for the inexperienced undergraduate!


Evaluating the distance-based EO course

- analysis of class marks

Following examination (February 2005)…

Module marks tested (t-test) by gender and by year (2004 Vs 2005)

Statistical analysis of overall marks obtained by students in 2003/04 and 2004/05 revealed:

no statistically significant differences year on year for all students; no statistically significant differences by gender within either year or across the

years;

And in some instances individual assessment marks were better for the distance-based year!

Note - there is no statistical anomaly for female students for individual practical marks or overall module marks.


What are the benefits & limitations for the e-tutor?

Overall distance-teaching aspects:

Main advantages

Materials can be delivered and learnt asynchronously Some working efficiency - students problems/queries can be

dealt with in “batches”

Main disadvantages

Number of student queries can be large and time consuming Lack of suitable training/administrative support for the course Labour intensive initial production (For the distance tutor) a feeling of isolation from the rest of

the faculty


What lessons can we give to the aspiring e-tutor?

1. Allay student fears with regards to delivery

How can we do this? - positive advertisement (e.g., student flexibility, one on one support with an expert via email) and at a time suitable for student’s module choices (especially for the inexperienced undergraduate)

2. Allay student fears with regards to their ability

How? – provide back-up on-campus “technical” support

3. Improve tutor support

How? – Faculty level training for on-line teaching; improve administrative/tutor communications – be clear on who does what specifically when distance-based.

4. Mix up delivery styles

How? – Have a blended approach – combining f-2-f and on-line delivery

To run successful e-learning courses…advertise, support, provide on-campus backup, blend delivery


Where are they now?

On Nathan Bodington

Other forms of storage:

– Outward facing project website

– Book with CD – E-learning in Geography

Long term storage:– Benefit to larger HE/FE community...talking about this a little

later…


But for today…

Developed courses can be viewed at our drop-in sessions:

See the courses and how to access them Pointers for aspiring e-tutors

The full courses - Earth Observation and Upland Catchment Management in

GeogEast G13 – 2pm to 4pm today!

and now…introducing our other materials: stand-alone learning activities, generic learning materials – some designed using Collaborative Learning Activity Design (and with international collaboration).


DialogPLUS - Generic resources

Resources available to staff and student communities throughout SoG:• Academic Integrity• Global Positioning Systems•Online Census Atlas


Academic Integrity Guidelines for Geographers

• Plagiarism: on the rise?• Learning material originated at Penn State

University• Adapted by Southampton and Leeds – joint

paper on Repurposing a learning activity on academic integrity: the experience of three universities submitted to JIME

• School resource available on Nathan Bodington as web-based material or PDF

• Includes context-based guidelines, in accordance with the University of Leeds rules and regulations

• MCQ available for assessment


Contents of AIG nugget


Examples of Multiple Choice Questions


Global Positioning Systems – Collaborative design case study

• Collaborators: David DiBiase (PSU), Katherine Arrell (Leeds) and Helen Durham (Leeds)

• Collaborative Learning Activity Design(CLAD)– Stages of design:

Audience

Learning outcomes

Learning activities

Nugget structure

Used ConceptVista to design 4 nuggets.


Visualising nuggets and collaboration


Understanding and using GPS – developed at Leeds


GPS nuggets developed by Penn State


Contents of PSU GPS nuggets


Online Census Atlas

• Simple Atlas to allow visualisation and exploration of Census variables from 1971, 1981, 1991 and 2001

• Discrete census years or inter-censal data• 88 variables, common across all Censuses• Data simultaneously displayed in conventional,

generalised boundaries and population-based cartogram (Universal Data Map) developed by Danny Dorling and Helen Durham

• Available on Manchester server at www.chcc.ac.uk/atlas (requires Athens authentication)

• Cartogram-only version available at www.ccg.leeds.ac.uk/teaching/chcc

• 3000-word Observation piece accepted by Area journal


Online Census Atlas – selection of map variables


Percentage change over time in age group 20-24 year olds by Local Authority, 1981-2001


Discussion points

• How to make more GPS nuggets accessible when out in the field? Suggestions?

• Should the Academic Integrity MCQ be compulsory?

Demonstrations of these three nuggets are available today in G.04 East Building

Time: 2-4pm


The storage and reuse of e-learning materials

What are we doing with these materials long term?

At the local level – Leeds VLE

At the project level – website and book with CD

At the national level –storage on JISC JORUM digital repository (http://www.jorum.ac.uk)

At the international level – possibly the US Alexandria Digital Library or other?

For the moment – introducing…JORUM

Local

Project

National

International


– what is it and who is it for?

What is JORUM?

JORUM is a new JISC funded digital repository allowing institutions and project teams to share learning and teaching materials with colleagues in the UK; to collect and share learning and teaching materials, allowing their reuse and repurposing, and standing as a national statement of the importance of creating interoperable, sustainable materials.

Who can use JORUM?

All HE/FE institution academic and support staff…

Where do the materials come from?

Centrally funded e-learning projects • JISC Projects (e.g. X4L Phase 1 & 2, Distributed e-Learning projects i.e., DIALOGPLUS)• HE Academy Centres of Excellence in Teaching and Learning (CETLs)

UK Further and Higher Education • Individual institutions• Institutions working as part of a Consortium

Working with other organisations/programmes • Other JISC services e.g. Digital Curation Centre• HE Academy Subject Centres

AND hopefully you!


What type of objects does JORUM support?

Whatever is contributed by the community:Learning Resources That teaching staff can use with learners, in blended learning,

classroom or online learning activities.

Teaching Resources Designed to support teaching staff e.g., tutor guides, lesson plans,

schemes of work and staff development materials.

Materials of varying size: Single assets (documents, images, video clips, diagrams);

Learning units made up of several elements (illustrations, articles, lectures, assessment exercises, etc).

Quality control – produced by teaching staff only – but can be targeted to tutor or student


- how do we use it?

The JORUM repository is a searchable online library of learning and teaching resources. The system contains resource metadata allowing users to search or browse for resources using specific educational classification systems and vocabularies.

Who is managing JORUM?JORUM is an online service offered by the JISC-supported national data centres, EDINA and MIMAS.

Download

Search by: title, keywords, contributor, resource type, user type & copyright

Preview metadata

Preview resource


- how do we use it?

Firstly – need to register with JORUM

All HE/FE institutions can register for JORUM contributor log-ins – only after institutional licence agreements are signed (thereby allowing your intellectual property to be used by others – for free!)

Once registered contributors can start to deposit materials…

Points to address prior to depositing:

Applicable – remove local context

Exchangeable – provide instructions

Exportable – remake into an interoperable format

Packaged – bundle multiple items together to make one resource

Use a content repackager such as RELOAD (http://www.reload.ac.uk)


How do we go about depositing?

Once logged into JORUM - on contributor service interface Provide metadata and subject classification data - to aid cataloguing Licence and publish to the service – catalogued by JORUM and added to user service

Deposit – upload, publish, catalogue – over to the user

JORUM - once deposited your learning resource becomes available to registered users

- how do we use it?

JorumContributor

“Putting content in”

JorumUser

“Getting content out”

Jorum


The upshot of using a repository like JORUM

What is the long term benefit of storage on such a repository? Use in wider FE/HE community

At the moment the service is in its infancy...there are further issues to address…

What are the current disadvantages of storage on JORUM? Issues of sustainability

Materials may become dated

Is this is an issue for JISC to address? Long term – of course

Lack of access for international collaborating partnersWe can receive but not give…JISC taking this point onboard for future JORUM development

BUT IN THE MEAN TIME – the repository is of benefit to us nowProject materials are not lost and are also released to the wider community

NOTE - The JORUM URL is: http://www.jorum.ac.uk


Providing access to material

• How it works in theory

• How it works in practice– Student at Leeds to take course at

PSU


StudentLeedsUser

WORKWORKWORK

WORKWORKWORK

WAYF

WAYF

WAYF

How it works: in theory

WAYF

1 Access request

4 User credentials

3 Login request

2a Origin discovery

8 Resource

2 Authentication request

6 Attribute request

7 Attribute response

5 Authentication response

Course module

PSU

Shibboleth Service Provider

User’s host Institution VLE

Bodington Common

ShibbolethIdentity Provider


Identity Provision and trust

• Members of Federations trust each other

• But…– Legal frameworks seem to dictate that

Federations are constrained by national borders

• Therefore…– Service Provider has to trust Identity

Provider


Service Provider fears…

• Infrastructure– How reliable is the service?– Who can we call if there are problems?– How can we test?

• Policy– Who has passwords?– What security policies are imposed?– Whose policies are relevant?


“Is everything ready?”

• Shibboleth an ‘emerging technology’– Software, terminology and methods are

subject to change

• Lots of metadata and configuration information to exchange

• Biggest hurdle is human aspect – finding the right people to talk to


Oct 21st

– “Clearly you [Leeds] guys trust your ID management system and are currently supporting your environment of higher stakes applications than the one that is represented by this activity for the WUN. … The real risk of malfeasance in this case is almost nonexistent. … I say we accept it.”

– “We … are also not aging now. However we happen to be working in a pilot project with our federal government, and they are insisting that we age passwords…”


Nov 4th

– “we have completed the final configuration of the Shibboleth Production Server and successfully tested with Leeds. We are ready for the Leeds student to participate in Penn State's GEOG 485 via shibboleth.”


Nov 5th : Security in the real world

– “A wrinkle has arisen, however…The Leeds student … recently reported that his computer was lost in a burglary.”


Lessons/Reflections: Students

• Students are comfortable in the use of on-line learning in a distance Masters (otherwise they would be on F-to-F programme).

• They want reliable, linear materials but with plenty of activity (I’m not sure they are ready for concept maps).

• Undergraduates are not so comfortable. “I paid for a face to face course not a distance course.” (This may simply be a natural reaction to an innovation.)

• David Dibiase: we should justify at least one on-line module to UGs as a preparation for life-long on-line learning after leaving university


Lessons/Reflections: Staff

• Very difficult to share nuggets within modules/courses

• Easier to share generic material (e.g. Academic Integrity)

• We need to learn how to share students (have our students do a course or two at another institution as part of their degree)


Sharing Programmes using SHIBBOLETH

• We are engaged at Leeds in experiments with this new software for linking users and resources at different institutions

• JISC is funding dozens of such experiments with the software (another Digital Libraries project has already used it to connect LSE students with a Columbia U course)


Conclusions: reflections on international collaboration

• US is substantially more advanced than the UK in developing online learning

• So we have probably learnt more from the US than the reverse

• However, we have delivered far more in terms of learning materials in the UK (Soton and Leeds) than our US partners

• We are working on an edited book that will point to the project resources and draw out lessons for the development of e-learning for Geographers

dialogplus asap seminar 8/03/2006 dialogplus innovative distance-based teaching: deliverables from...

Documents

dialogplus asap seminar

teaching elearning materials

teaching materials

generic materials

topic materials

dan vickers slide

following definition

jiscs jorum repository