TESTING OF AN ARCHIVAL DL PROTOTYPE

ASKING TO USERS:

THE CASE OF “UNA CITTÀ PER GLI

ARCHIVI”

Pierluigi FeliciatiUniversity of Macerata (Italy)

JCDL – Washington (USA) 2012.06.10

2

Felicia

ti - JCD

L 20

12

INTRODUCING THIS DISTANT PRESENTATION (1)

I am an ICT senior researcher in Macerata University (Italy), but my background covers archaeology, historical archives management and distance services development. I’m working especially on Web services quality

Rarely archives on line (archival DLs) are approached from the point of view of final users, and input and output activities and environments are often mixed in an unique layer

Users are probably not satisfied (but who knows it for sure?)

3

Felicia

ti - JCD

L 20

12

INTRODUCING THIS DISTANT PRESENTATION (2)

User studies in the prototypal stage of a DL life-cycle are rare but crucial, to finalise the quality of services.

The case study I will present is probably one of hugest projects in this sense in terms of methods and number of people involved

This video presentation will ask you 15-20 minutes of kind attention, then Milena and Nicci will be happy to ask to all of your questions instead of me (unabled to move to Washington)

4

Felicia

ti - JCD

L 20

12

MAIN TOPICS

1. Archives on line and Users: a museum of broken relationships

2. Testing a prototype: Why? What? How? Results implementation policy

3. The Città per gli Archivi (Italy) project

4. Asking to users 1: internal staff

5. Asking to users 2: target users / focus groups

6. Asking to users 3: expert evaluation

7. First conclusions and lessons learnt

Felicia

ti - JCD

L 20

12

5

ARCHIVES ON LINE AND USERS

Without the expert mediation of archivists what is based on the quality of access/use?

A lesson to be learnt: Input is not output!

Archival standards create encoded contents, not automathically good displays

Available very few (American) studies on Archives on line quality (from the final users satisfaction point of view)

Felicia

ti - JCD

L 20

12

6

ARCHIVES ON LINE AND USERS

Some typical issues: Terminology: use of jargon Archival hierarchical

structure: browsing is guided just by titles

Contents: too encoded Searching: no intuitive

tools with no subjects Coverage: what could I

find here?

7

Felicia

ti - JCD

L 20

12

TESTING A PROTOTYPE: WHY AND WHAT?

User analysis could be very useful in the early stages of a project (prototype)

Evaluation is called formative, when conducted while the DL is still under development

Part of the data gathered during this stage will focus on the content and the functionalities of the DL and how these correspond to the users’ needs

User needs could be checked asking directly to users, who can provide feedback that helps in the recognition of flaws in the interaction design.

8

Felicia

ti - JCD

L 20

12

TESTING A PROTOTYPE: WHY AND WHAT?

Morever, expert opinions can provide assistance when the DL prototype has been refined, giving their insights on the quality features of the system

9

Felicia

ti - JCD

L 20

12

TESTING A PROTOTYPE: HOW?

To directly involve users in the prototype step, we could launch a survey (e.g. using on-line questionnaires), if we have already a designed community to address

On the other hand, we have to organize as more focus groups as possible trying to address user panels who could be interested to the future DL we are working on

Finally, the expert study could involve both experts on content and functionality, to gather advanced qualitative data on performance, usability and expected success of our project

10

Felicia

ti - JCD

L 20

12

TESTING A PROTOTYPE: DEFINING A RESULTS IMPLEMENTATION POLICY

During formative evaluations, the researcher may return to previous stages of development and adopt their findings, thus altering the status of the DL.

11

Felicia

ti - JCD

L 20

12

THE CITTÀ PER GLI ARCHIVI PROJECT

Una città per gli archivi (A City for Archives CPA) is a project supported by two banks foundations, with the collaboration of other institutions (archival and city administration)

Its mission is to improve the preservation, valorization and full access for the most important Archives related to the contemporary history of the city of Bologna (Italy)

The project focused not only on institutional archival records but as well on audiovisual and photographic archives

This huge cultural project started in 2007

12

Felicia

ti - JCD

L 20

12

THE CITTÀ PER GLI ARCHIVI PROJECT

More then 200 Archives have been preserved and described, made of 300.000 records, paper documents, photographic and graphic materials, sound and audiovisual recordings

70 archivists were involved 200.000 digital objects created A Web portal (an archival DL…) was

developed to enable users to access the described archives and the digital reproductions

Export of metadata towards Italian National Archival System, Italian Cultural Portal and Europeana was scheduled

13

THE CITTÀ PER GLI ARCHIVI PROJECT

A wide program of restoration and sorting out of Archives at risk of loss,redaction of guidelines for descriptions and authority control

Felicia

ti - JCD

L 20

12

Thus, during the prototypal step of the project, a user and functionality study was launched

Felicia

ti - JCD

L 20

12

14

CPA portal prototypea snapshot

14

15

Felicia

ti - JCD

L 20

12

CPA USER STUDYASKING TO USERS 1: INTERNAL STAFF For the quality archival descriptions, the

competence of archivists is crucial Rarely those input content providers are

directly involved in the development of output environments

The CPA users and functionalites study planned 2 special focus groups, inviting together archival staff and DL developers (beginning and end of the study)

Those meetings were moderated according to a special protocol, mixing typical focus groups method with that of a moderated brainstorming (archivists vs engineers)

16

Felicia

ti - JCD

L 20

12

CPA USER STUDYASKING TO USERS 2: FOCUS GROUPS

5 focus groups were organized to collect precious data about potential target users needs and satisfaction against CPA portal1 with high school students (16 pople, aged

16-19)2 with university students (total 20 people,

aged 23-29)1 with general public (16 people, aged 39-63)1 with archivists and cultural heritage (LIS

and museums) professionals (16 people, aged 29-54)

17

Felicia

ti - JCD

L 20

12

CPA USER STUDYASKING TO USERS 2: FOCUS GROUPS

All the focus groups were based on the same detailed protocol, asking 100 minutes and including 6 sections: 1. Demographic and ICT skills

questionnaire2. CPA prototype presentation3. First impressions and discussion4. 5 tasks, whose results were recorded

asking 3-4 questions for each task5. Advanced impressions and discussion6. Final discussion

18

Felicia

ti - JCD

L 20

12

CPA USER STUDY ASKING TO USERS 3: EXPERT EVALUATION

Finally an expert study activity was scheduled for next September, when the prototype will be almost ready

The 7 experts involved are: 3 archivists (2 university professors and a City Archive director), 3 DL and metadata experts, 1 ICT university professor

The protocol adopted is focused on performance, functionality, usability issues, terminology, quality of content

19

Felicia

ti - JCD

L 20

12

FIRST CONCLUSIONS (AND LESSONS LEARNT)

Archives on line are special DLs: they open peculiar issues to face users expectations and guarantee satisfaction

Users study are crucial, to finalise the DLs and ensure a (really) good quality

The prototypal stage of the project, when possible, offers a good occasion for evaluation and assessment activitis based on users involvment

The Italian CPA project shows that such activities are not that expansive, they need an expert (and neutral) coordination, a good organizational staff and a network of people and institutions to be involved

THANK YOU FOR YOUR ATTENTION,

WISH YOU ALL A GREAT WORK

TOGETHER!

pierluigi.feliciati@unimc.it

This presentation is licensed under:

CC-BY-SA