d 12.6 final testing report - ariadne pluslegacy.ariadne-infrastructure.eu › ... ›...

D 12.6 – Final Testing Report

Version 1.0 (final)

January 26th, 2016

Grant Agreement number:

313193

Project acronym: ARIADNE

Project title: Advanced Research Infrastructure for Archaeological Dataset Networking in Europe

Funding Scheme: FP7-INFRASTRUCTURES-2012-1

Project co-ordinator name, Title and Organisation:

Prof Franco Niccolucci, PIN Scrl - Polo Universitario "Città di Prato"

Tel: +39 0574 602578

E-mail: [email protected]

Project website address: www.ariadne-infrastructure.eu

ARIADNED12.6FinalTestingReport

2

The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7-INFRASTRUCTURES-2012-1) under grant agreement n° 313193.

Authors: Anthony Corns

Evie Monaghan

Louise Kennedy

Marie Brohet

Contributing partners

Quality control review

The Discovery Programme

Holly Wright, UoY ADS

Paola Ronzino, PIN

Franco Niccolucci, PIN


3

Table of Contents ExecutiveSummary...........................................................................................................4

1 IntroductionandObjectives.........................................................................................5

2 UseRequirements,ServiceDesignandImplementation...............................................7

3 InfrastructureDesign...................................................................................................83.1 InfrastructureElements.............................................................................................................83.2 ContentProvisionWorkflow.....................................................................................................9

4 Usability&Testing.....................................................................................................11

4.1 ResultsofD12.4.......................................................................................................................114.2 TestingMethodologies............................................................................................................154.3 InternalTesting.......................................................................................................................17

4.3.1 Methodology..................................................................................................................................174.3.2 Testing............................................................................................................................................17

4.4 OnlineSurvey..........................................................................................................................234.4.1 SystemUsabilityScale(SUS)...........................................................................................................234.4.2 SurveyResults.................................................................................................................................26

5 Conclusions................................................................................................................355.1 Registrytooltesting................................................................................................................355.2 MOReaggregationtooltesting...............................................................................................365.3 Portaltesting...........................................................................................................................36

6 References.................................................................................................................37

7 Annex1:OnlineSurvey..............................................................................................39


4

ExecutiveSummary

This document is a deliverable (D12.6 Final Testing Report) of the ARIADNE project (“AdvancedResearch Infrastructure forArchaeologicalDatasetNetworking in Europe”),which is fundedunderthe European Community's Seventh Framework Programme. D12.6 is associated with Task 12.4Testing within WP12, which is titled Implementing Interoperability, and falls within the largerARIADNEconceptualframeworkfortheARIADNEe-Infrastructure.It isanextensionofD12.4Initialinfrastructure testing report, and reports the outcomes of the final acceptance testing, taking onboard the outcomes of the initial testing. This deliverable was also used to inform Tasks 12.3Implementing integration and 12.4 Testing, to facilitate fine tuning of the infrastructure beforecompletionoftheproject.

As outlined in D12.3 one of the goals of the ARIADNE infrastructure is to integrate data andmetadata from different providers across Europe into one common schema, and also to providesemantic integration along different axes (e.g. subject, space, time). This integration intends toprovideusefulanduser-friendly informationservices forarchaeology.Theservicesare intendedtobeavailablenotonlytoresearchersandrelatedstakeholders,butalsotoawiderrangeofpotentialusersrequiringaccesstocollectionsanddatasets.

It was this user-centred approach that was adopted for the testing of the infrastructure and isreported here. The testing comprised a survey of content-providing partners and formal testingmethods.EachstrandaimedtobalancetherequirementssetoutinD12.1withtheservicesdesignedandimplementedinD12.2andD12.3.FollowingonfromtherecommendationsandresultsofD12.4(which are addressed below) this document tests the infrastructure using broadermethodologiesincludingtheapplicationofusabilityheuristicsandtheSystemUsabilityScale(SUS).Thisdocumentreports on the results of an online survey of content-providing partners and indicates that whiletherewasagenerallevelofsatisfactionwiththeinfrastructurethereareanumberofareasinwhichimprovements would likely benefit the opening out of the infrastructure to an even broaderaudience,particularlyinrelationtouser-friendlinessandhelpdocumentation.

Results of D12.6 testing are presented according to the methods used with further subdivisionsaccording to the services tested. Comments and further recommendations are provided in eachsection.

It must be noted that some of the comments provided in the survey do not concern theimplementation level requiredby theprojectGrantAgreementor the requirements setout in theinfrastructureinitialdesign.Suchcommentsratherconcernfeaturesthatcouldimprovethesystemusability andeffectiveness, andwill be taken intoaccount in thenext version.Asdiscussed in theconclusions,thereisspaceforimprovementinfutureversions,notunexpectedlyforacomplexandnovelsystemastheARIADNEone.


5

1 IntroductionandObjectives

This document is a deliverable (D12.6 Final Testing Report) of the ARIADNE project(“Advanced Research Infrastructure for Archaeological Dataset Networking in Europe”),whichisfundedundertheEuropeanCommunity'sSeventhFrameworkProgramme.D12.6isassociatedwithTask12.4TestingwithinWP12,whichistitledImplementingInteroperability,andfallswithinthelargerARIADNEconceptualframeworkfortheARIADNEe-Infrastructure.ItisanextensionofD12.4Initialinfrastructuretestingreport,andreportstheoutcomesofthe final acceptance testing, taking on board the outcomes of the initial testing. ThisdeliverablewasalsousedtoinformTasks12.3Implementingintegrationand12.4Testing,tofacilitatefinetuningoftheinfrastructurebeforecompletionoftheproject.

Theobjectivesofworkpackage12are:

• ToadaptinfrastructuresprovidedtoARIADNEforintegration.• Todesignandsetup thenecessary tools (crosswalks,mappings)and resources for

interoperability.• Tosetuptheinternal(APIs)andexternal(human)interfacestoaccesstheintegrated

resource.

WorkPackage12iscomprisedofthefollowingtasks:

• 12.1Userequirements• 12.2DesignandSpecifications• 12.3ImplementingIntegration• 12.4Testing

Task12.4testedtheintegratedinfrastructurebuilt inTask12.3 ImplementingIntegration,based on the requirements and design specified in Task 12.1Use Requirements and12.2InfrastructureDesign.Infrastructuralelementswithuserinterfacesweretestedforthistask.

Documented in D12.2, the ARIADNE Registry is a key element of the interoperabilityframework and drives resource discovery within the public-facing ARIADNE portal. Inparticular, the deposit service allows registered users to deposit data and metadatadescriptionsfollowingtheARIADNEDatasetCatalogueModel(ACDM)schema.TheprovidedmetadataismanagedthroughaDigitalAssetsManagementservice,andispresentedtothepublic through theARIADNEportal. TheResourceDiscoveryServices (mainly indexingandretrieval) enables access to data resources and integrated viewing of data resourcedescriptions, through the portal. The vocabulary management service is responsible formaintaining a list of SKOSified vocabularies and thesauri. The metadata enhancement


6

service allows for automatic enhancement of metadata found in ACDM records. Theseenhancements include the mining of relations, and automatic linking with thesauri andvocabulariesetc.[D12.2,10]

Insummarythearchitectureprovides:

1.thefunctionalitiesoftheARIADNECatalogue;

2.interoperabilityandintegrationfunctionalitiesforarchaeologicalresources;

3.theplatformuponwhichtheARIADNEserviceswillrun.[D12.2,11]

Thisreport is focusedontestingeachcomponentoftheARIADNEinfrastructuretoensurethateachmoduleiscompatibleandcontributestotheintegratedsystem.Themethodsusedto test the ARIADNE integrated infrastructure were varied and included group meetingsrelating to portal testing, communication by e-mail/skype with individual partners aboutimplementing the recommendations of D12.4, an online survey of content-providingpartnersaswellasin-housetestingusingformalusabilitymethods.Thisreportalsooutlinesthe current workflows in place for providing content for integration into the ARIADNEinfrastructure.


7

2 UseRequirements,ServiceDesignandImplementation

TheworkofWP12 specifieduse requirements for theARIADNE infrastructure (noting theextensiveworkofWP2onuserneedsandWP3ontheRegistry),todesignandimplementaspecification for the infrastructure based on the use requirements. Use requirementsspecified in D12.1 focused on three areas: datasets, metadata standards, schemas andvocabularies and access and sharing policies. The use requirements reflected the dataavailableinthearchaeologicalsector,andprioritiesfortheintegrationactivitiesfordifferenttypes of data. Use requirements for the infrastructure have both practical and policyimplications,bothofwhichmustbereflectedintheimplementationasappropriate(D12.1,4-5).Thedesignspecificationfocusedonanintegrationstrategywhichaddressedmetadataintegrationanddataintegration,andinthefirstinstance,enabledcross-searchofresourcesthroughWhat,Where,When and Resource Type facets. Consistency and quality are keyrequirementsforcontentaddedtotheARIADNEinfrastructure(D12.2,16-17).

D12.4providedacomparisonoftheuserequirementsofD12.1,theservicedesignofD12.2anditsimplementationinD12.3.Italsoofferedpreliminarytestingofthevariousinterfacesof the infrastructure which were available at the time. This document provides furthertestingof the infrastructurevia severalmethodswhich incorporateaspectsof thegeneraluserequirementsofD12.1withadditionaltestingusingformalmethods.


8

3 InfrastructureDesign

3.1 InfrastructureElements

TheintegratedinfrastructureincorporatestheARIADNEregistry,aggregationandvalidationservices (MORe) as well as the ARIADNE portal. These combined elements enabledarchaeological and cultural heritage content in various forms to be made available toresearchers,chieflythroughtheirintegrationintheonlinewebportal.Althoughtheportaliscrucialtodiscovering,visualisingandaccessingarchaeologicaldata,evenmorecrucialisthequalityof themetadatasubmittedto the infrastructurebyARIADNEpartners.This isbornoutincommunityfeedbackandconcernsarticulatedinD2.1andacrossotherdisciplinesandinfrastructures (see for example metadata principles in the Framework of guidance forbuilding good digital collections of the National Information Standard Organisation). D2.1reported that 50% of archaeological researchers felt that metadata quality was ‘veryimportant’whileafurther34%felt itwas ‘rather important’. The infrastructuresoughttomitigatemetadata quality issues by validating deposited data against the ACDM schema,thus ensuring aminimum standard which could be used for integration. The enrichmentservices aim to further improvemetadata quality by allowing automatic enhancement ofACDM records. To continue to ensuremetadata quality the workflow for how the datareachestheportalmustbeclearandwell-definedforfuturecontentproviders.


9

Figure1.TheARIADNEdataaggregationworkflowfromD12.3[p.13]

3.2 ContentProvisionWorkflow

ARIADNE content-providing partners have different routes for sharing their data in theinfrastructure,theseareoutlinedbelow.

Stage1:Ingest

Themetadataregistryprovidesthreedifferentinterfacesformanaginginformation:

1. WebbasedUI.This isdesigned formanualeditingof informationbyhumanusers.Eachcontentproviderhastheirownuseraccountthroughwhichtheycanaddnew/edit existing informationper class and linkdifferent instances together. TheuseofthistoolwasextensivelydocumentedinD12.4,seepp.18-33.

2. Batch Import. The batch import is available to partners through contact withtechnicalpartnersandallowsthebatchingestofdatasetsandcollectionsthrough:a)OAI-PMH target, b) XML file, c) Excel file. Templates for Excel and XML files areavailable through the Registry support portal andmust be correctly structured foringest.TheExcelfilecontainsallmandatoryclasses,aswellassomerecommendedclasses,andisthensenttotechnicalpartnersforinclusionintheregistry.Similarly,providerscancreateanXMLfilefortheirdata,thiscanbevalidatedagainstan.xsdfilealsoavailableintheregistrywebtoolandthensenttotechnicalpartners.


10

3. REST API. The REST API allows users to export all catalogue information (perprovider)inXMLformatandtomanipulateitthroughaRESTAPI.TheaccesstotheAPI is controlledbya sessionkey that is generated fromwithin theWeb interface(perprovider).RESTAPIcanbeused to return,edit,addanddelete records in theARIADNECatalogue.Theauthenticationrequiredtoperformthesetasksisprovidedthrough the ARIADNE Catalogue, using API keys for the machine interface anduser/passwordchallengeonthehumaninterface(D12.2,34).

Stage2:Validation,enrichmentandpublication

TheMOReaggregationworkflowhasbeenproventhroughexperienceinLoCloud,CARAREand other European projects.MORe provides a flexible architecture, and is used here toaggregatecontent invarious formats,andfromavarietyofsources, totheRDFstore, theregistry and Elasticsearch. MORe incorporates a micro-service oriented architecturesupportingthefollowingcoreservices(D12.3,13):

• Input• Validation• Transformation• Enrichment• Publication

The toolhasauser-friendly interface formetadata ingest, validationandenrichment.Theworkflowforthetooliswelldesigned,astheuserisguidedthroughthestepsnecessaryforcompleting the publication ofmetadata to the portal by a virtual assistant. An illustratedworkflowisgiveninAnnexII.

After validation and enrichment, data is then published to theweb portal for search anddiscoveryaswellasintegratedviewingofdataresourcedescriptions.


11

4 Usability&Testing

4.1 ResultsofD12.4

TheinitialtestingoftheinfrastructurewasreportedinD12.4,theresultofwhichwasalistof specific recommendationswhichwere circulated to relevant partners.Upon consultingwithtechnicalpartnersitwasagreedthatthewebtoolcomponentoftheregistrywouldnotbedeveloped furtherowing to its limitedusebycontentproviders. Itwas recommended,however, that should the tool be used more frequently in the future by new contentproviders, the recommendationsofD12.4 relating to theserviceshouldbeaddressedandincorporatedintothewebinterface.Therecommendationsrelatedtootheraspectsoftheinfrastructurearelistedbelow,withtheadditionofacommentsfieldindicatingthecurrentstatus of the recommendation. These comments were informed by recent testing of theinfrastructureaswellaspartnerfeedbackfromthesurveyconductedforthisdeliverable.Itmustbeobservedthatmanyoftheissuesevidencedconcernfuturenewusers,andthuswillneedtobeconsideredwhenplanninganewreleaseofthesystem,wherecontentprovisionwouldbeenlargedtoamuchwideraudience.

Recommendations Comments

3 12.1Requirements

R31 3.1.1

Resolvetheuseoflanguagearoundtheterms‘Registry’and‘Catalogue’inanysupportdocumentationanduserinterfaces.

To be addressed in future versions fornewcontentproviders.Currentusersareclear about the equivalence of the twoterms.

R32 3.1.1

ClarificationoftheuserequirementsforthePreviewService.

Clarified:Previewservicewillnotcontainapreviewofdatabutwillprovideapreviewofmetadataintheformofsearchresultsintheportal.


12


R33 3.1.2

Appropriate,targetedguidancefortheuserintheuseoftheRegistrytoadddataresourcesisrequired.

Tobeaddressedinfutureversions.Althoughthereisuserdocumentationavailable,thereshouldbeonesinglesectionontheportalinterfacewhichallowsuserstodeterminehowtheymightdepositdata,includingbasicworkflow,ingestformatsandcontactdetailsforpartnersresponsibleforthis.

R34 3.1.2 Abusinessandsustainabilitymodelisrequired.

Ongoing:Thisisbeingexaminedaspartofthesustainabilityplan.

R35 3.1.3 Guidanceforusersonthenatureoftheerrorsencounteredduringthedepositandingestprocesswouldbehelpful.

Partiallyresolved,andtobecompletelyaddressedinfutureversions.TherearenofurtherdevelopmentsconceivedfortheRegistrytool.Errorsmanifestedthroughotherdeposit/ingestprocesses(e.g.Excel,XML)arecommunicatedtoprovidersdirectlybytechnicalpartners.TheMORetoolhasrobusterrornotifications.

R36 3.1.3 Awordcloudofthe‘derivedsubject’facetorGettyAATterms,eitherasafilterorasecondwordcloudinthe‘What’browsingarea.

Resolved:ThishasbeenresolvedbyWP13andthewordcloudisnowpopulatedwithGettyAATterms.However,thisleadstoweightinginfavourofthosedatasetswhichhavebeensuccessfullyenrichedbyGettyAATterms,andisperhapsnotfullyrepresentativeoftheentiretyofthedatainthecatalogue.

R37 3.1.4 Theprojectshouldcommunicateclearlytousersitsapproachtodataandmetadata.

Resolved:Whilesomeuserswereuncertainastowhetherdatawastobeingestedatitemorcollectionlevel,partnersareclearthatthecataloguecontainsmetadatawithexternallinkstodata.Experimentsfordataintegrationatitemlevelareongoingandaretakingplacewithinalternativemetadatarepositories(CNR-ISTI).Thesewillbeimplementedinaseparateexperimentalsectionoftheportalinterface.


13


R38 3.1.5 ThesourcevocabulariesavailablethroughtheVocabularymatchingtoolcouldbeexpandedtoallowbroadermappingtoAAT.

Resolved:CurrentlythevocabularymatchingtoolcontainsvocabulariesfromBritishproviders.Asthetoolisofferedasapartnerservice(asopposedtoonedevelopedspecificallywithinARIADNE)itisnotpracticabletoexpandthesourcevocabularies,inadditionmostpartnershavemappednativevocabulariestotheGettyAATaspartofWP15.

R39 3.1.5 Guidancedocumentationonhowtopreparevocabulariesforinclusioninthetoolandformappingwouldberequired.

Resolved:PartnerscommunicateddirectlywithWP15partnersandmappedvocabulariesareavailablethroughMOReforenrichmentofproviderdata.

R40 3.2.1 DocumentationisrequiredforthePeriodOenrichmentservice.

Resolved:ThePeriodOenrichmentisinbeingimplementedinMORe.

R41 3.2.5 Clarify,fromD12.1andD12.2,whatkindofintegrationandinteroperabilityissoughtforscientificdatasets.

Resolved:ThisaspectisbeingaddressedbyWP16wheredataminingisongoingfordendrochronologicaldata.

5 Aggregationservices

R42 DocumentationfortheuseofMOReshouldbeavailablewithintheSupportPortalandaccessiblefromwithinMORe.

Postponed.SomepartnersexpressedconfusionabouttheroleofMOReintheworkflow.InfutureversionstherewillneedtobeclearguidelinesonpartneraccesstoMORe.

R43 IncludeworkedexamplesofprovisionoftheCollectionclassinuserguidance,includingmetadataprovisionfortheitemsordataresourcesofwhichthecollectioniscomposed,wouldillustrateforusershowtheircontentwillberepresentedintheportal.

Postponed.Severalsurveyrespondentsproposedthatfurtherexamplesbeprovidedintheuserguidanceinfutureversions.


14


R44 FurthertestingshouldbecarriedoutwithapanelofuserswhodonothavepreviousexperienceofMOReandassociatedtools.

Resolved:Seeinternaltestingsectionofthisdocument.

R45 ManualentryworkflowintheRegistryshouldbeharmonisedwithuserguidanceandthelatestversionoftheACDM.

Resolved:Nofurtherdevelopmentofthetoolinthenearfuture.Thismaychangeifitisusedmorefrequently.Awalk-throughvideoofthetoolmaybeusefulforfuturefirsttimeusers.

6 SupportPortal

R46 Userguidanceshouldincludeexamplesofalldataresourceclassesworkedwiththeirlanguageresourceassociations.

Postponed to future versions. See D12.4pp38-9.

R47 ExpandtheRegistryFAQ. Postponedtofutureversions.QuestionsintheRegistryFAQarequitespecific,moregeneralquestionsshouldbeadded,pageneedstobere-structuredtoincludesectionsforease-of-use.

R48 Changenavigationaroundaddinganinstanceoffoaf:agentfromwithinadataresource.(Seerecommendationunder‘AddDataset’4.1.3above).

Postponedtofutureversions.Eventhoughthistoolhasceasedactivedevelopment,thisfeatureshouldbeaddressedinthefutureasapotentiallysignificantusabilityissuefornewusers.

R49 Comprehensivestep-bystepworkflowandmappingguidancecouldbeprovidedbothinoneresourceandbrokenintopartswhichreflectvarioususerinteractionswiththeinfrastructure.

Postponed to future versions. This wasremarked on by only one respondent ofthe online survey. They noted that thesupport portal needs ‘to be edited forusers not familiar with the ARIADNEproject, and with their ease of use inmind’,suchaspotentialnewcontributorsthe ARIADNE registry envisaged in thefuture.

Table1ListofRecommendationsfromD12.4Initialinfrastructuretestingreport.


15

4.2 TestingMethodologies

There are several methodologies for testing infrastructural systems like the ARIADNEregistry and portal. D12.4 outlined further methods for testing, including theimplementationoftheSystemUsabilityScale(SUS)inthetestingoftheregistry(D12.4,40).Thisisoneofthemostcommonmethodsforusabilityandisdiscussedindetailbelow.

Other usability methods include inspection and test methods (see table 2). Inspectionmethodsarecarriedoutbysystemcreatorsanddonotuseendusersaspartofthetestingprocess. The methods included under this type are heuristic evaluation, cognitivewalkthroughandactionanalysis.Thesecondtype;knownastestmethods,involvetheuseofendusersinthetestingphaseandalsoencompassesthreestrands,thinkingaloud,fieldobservationandquestionnaires(Holzinger,2005).

Inspection methods seek to improve interface usability by checking design againstestablished standards (Holzinger, 72). Studies have shown that inspectionmethods oftenfindmany issues that are overlookedby testingwith real users, and likewise user testingillustratedproblemsthatweremissedininspectionmethods,thereforethemostbeneficialcourseistotestusingacombinationofinspectionandusertestmethods(Nielsen(1994a),413).


16

Inspectionmethods Testmethods

Heuristicevaluation ThinkingAloud

CognitiveWalkthrough FieldObservation

ActionAnalysis Questionnaires

Table2UsabilityEvaluationTechniques

It is commonly agreed that there are five essential usability characteristics: learnability,efficiency, memorability, low error rate and satisfaction. The performance of a systemrelative to these characteristics ensures the systemwill be successful to endusers. JakobNielsenextendedusabilityprinciplesandoutlinedten“heuristics”orprincipleswhichuserinterfacesshouldcomplywith,seebelow(Nielsen,1994b,25-62).

For the purposes of testing the ARIADNE integrated infrastructure, a combination ofmethodswasused.Fromtheinspectionmethodstier,heuristicevaluationwasusedasthemosttime-efficientmethod,whilefromthetestmethodstier,aquestionnaireintheformofanonlinesurveywasconducted.

Heuristicevaluation

“Heuristicevaluationisausabilityengineeringmethodforfindingtheusabilityproblemsinauserinterfacedesignsothattheycanbeattendedtoaspartofaniterativedesignprocess”(Nielsen,1994b,25-6).

Inthis inspectionmethod,ausabilitytestercarriesoutseveralrun-throughsofthesysteminterfaceand tests themagainsta setofdefinedprinciples.Themostcommonlyusedsetare the ‘heuristics’ of Jakob Nielsen which set out ten standards against which systemsshould be tested.Nielsen’s heuristicswere used for testing the interface of theARIADNEintegratedinfrastructure.Heuristicevaluationisacostandtimeeffectivemethodfortestingand has been noted to have a low “intimidation barrier” to its implementation (Nielsen,1994b,25).


17

4.3 InternalTesting

4.3.1 Methodology

Each individual evaluator inspects the interface alone. After evaluations have beencompleted,findingscanbeaggregated,butpriortothis,thereisnocollaborationbetweenevaluators inordertoensure independentconclusions(Nielsen,1994b26).Astheregistrywasdeemedtohavetoosteepalearningcurveinthistestingphase,onlyusersfamiliarwiththetoolassessedthisaspect.Abetterindicationofuserexperiencewiththeregistrycanbegaugedfromthesurveyquestionnairecirculatedtocontent-providingpartners(seebelow).

4.3.2 Testing

Agroupofinternaltesterstestedtheinfrastructureindividually,astheywerefamiliarwiththe overall aims of the ARIADNE project and experienced in both content provision andutilisation of the tools. Each user was given a form outlining the usability principles andaskedtogivethemarating(•Poor/••Moderate/•••Good).Thetesterswerealsoaskedtoprovide comments on aspects of the infrastructure that addressed, or didn’t address, theusabilityprincipleinquestion.

Feedbackfromtesterswasamalgamatedandrelevantcommentsprovidedbelow.Asusual,theyconcernimprovementstobeimplentedinfutureversions.Table3showstheusabilityprinciplewithcorrespondingdescription.

Visibilityofsystemstatus

Thesystemshouldalwayskeepusersinformedaboutwhatisgoingon,throughappropriatefeedbackwithinreasonabletime

Matchbetweensystemandtherealworld

Thesystemshouldspeaktheuser’slanguage,withwords,phrases,andconceptsfamiliartotheuser,ratherthansystem-orientedterms.Followreal-worldconventions,makinginformationappearinanaturalandlogicalorder.

Usercontrolandfreedom

Usersoftenchoosesystemfunctionsbymistakeandwillneedaclearlymarked“emergencyexit”toleavetheunwantedstatewithouthavingtogothroughanextendeddialogue.Supportundoandredo.

Consistencyandstandards

Usersshouldnothavetowonderwhetherdifferentwords,situations,oractionsmeanthesamething.Followplatformconventions.


18

Errorprevention Evenbetterthangooderrormessagesisacarefuldesignwhichpreventsaproblemfromoccurringinthefirstplace.

Recognitionratherthanrecall

Makeobjects,actions,andoptionsvisible.Theusershouldnothavetorememberinformationfromonepartofthedialoguetoanother.Instructionsforuseofthesystemshouldbevisibleoreasilyretrievablewheneverappropriate.

Flexibilityandefficiencyofuse

Accelerators-unseenbythenoviceuser-mayoftenspeeduptheinteractionfortheexpertusertosuchanextentthatthesystemcancatertobothinexperiencedandexperiencedusers.Allowuserstotailorfrequentactions.

Aestheticandminimalistdesign

Dialoguesshouldnotcontaininformationwhichisirrelevantorrarelyneeded.Everyextraunitofinformationinadialoguecompeteswiththerelevantunitsofinformationanddiminishestheirrelativevisibility.

Helpusersrecognize,diagnose,andrecoverfromerrors

Errormessagesshouldbeexpressedinplainlanguage(nocodes),preciselyindicatetheproblem,andconstructivelysuggestasolution.

Helpanddocumentation Eventhoughitisbetterifthesystemcanbeusedwithoutdocumentation,itmaybenecessarytoprovidehelpanddocumentation.Anysuchinformationshouldbeeasytosearch,focusedontheuser’stask,listconcretestepstobecarriedout,andnotbetoolarge.

Table3UsabilityheuristicsasoutlinedbyNielsen.[Source:J.Nielsen,

'HeuristicEvaluation'Usabilityinspectionmethods17.1(1994)p.30.

ARIADNERegistryTool

Ratings:•Needingimprovement/••Satisfactory/•••Good

Usabilityfeature Rating Comments

Visibilityofsystemstatus • Itwassometimesunclearwhendatahadbeensuccessfullyingestedintotheregistry.Itisrecommendedthatinfutureversionsusersaregivenfeedbackonsuccessfulorunsuccessfulactionsinthetool.

Itisdifficulttodeterminewhereintheprocessoneis,whetherthesystemissavingprogressandthestatusfordatainputatorganisationlevel(multipleseparateaccountsforindividualsinoneorganisation,withoutflagsofothercontentorvisibilityacrossaccounts.)


19


•• Somehelptextscouldbeimprovedtoallowinformation‘appearinanaturalandlogicalorder’.

Usercontrolandfreedom •• Thereareadequateexitpointsforusers,howeverthereisscopetosupportundoandredo.

Consistencyandstandards •• ForthemostpartthetoolfollowedthewordingandstructureoftheACDMwhichledtoastandardmethodofexpression,however,itisnecessarytopointuserstotheACDMspecificationshouldtheyneedtochecktheirmetadataagainstthemodel.ThiscouldbeintheformofalinktothedescriptionoftheACDMclass,propertyorattributeinthespecification.

Errorprevention •• Therewerefewexperiencesofdirecterrorsinthetool,forspecificexamplesseeD12.4,p.22.

Requiringvaluesincertainfieldsreducesthelikelihoodoferrorsoccurringinlaterstagesofthedataingestionprocess.

Recognitionratherthanrecall •• Optionsandactionsarevisibletotheuserinthetoolwiththesameinterfaceforeachresourcetobeaddedwhichfacilitatesuserrecognition.

Flexibilityandefficiencyofuse • Bothexpertandnoviceusersinteractinthesameway,thereisonlyoneinterfaceforallusers.Thereisnopossibilitytotailorfrequentactions.

Aestheticandminimalistdesign •• Althoughtheuserinterfaceisminimalist,itmaybenefitfromaclearervisualstructure.Sevenoftheavailableresourcetypesarenotvisibleonthispage,includingthemandatoryfoaf:agent.Keydataresourceclassescouldbegroupedtogether,andlanguageresourcesgroupedseparately,reflectingtheresourcetypes.Abasicvisualisationmayhelptheusertobetterunderstandtherelationshipsbetweendataresourcesandlanguageresources,andillustratewhichlanguageresourcesaremandatoryorrecommendedforparticulardataresources.


•• Therewerenoexperiencesofcodederrormessages,however,thelackoffeedbackoncedatasetshavebeenaddedandsavedcreatesuncertaintyaboutwhetherresourceshavebeensuccessfullyadded.Apop-upmessageindicatedthatresourcehasbeensuccessfullysaved/ingestedwouldbeuseful.

Helpanddocumentation •• ThereisahelpsectionontheRegistrybutthiscouldbeexpanded.Itisimportanttoincludeawayofcontactingpartnersfortechnicalsupport.However,whentechnicalsupportwasrequiredresponsewasswiftandefficient.


20

AggregationService,ValidationServices,MetadataEnrichmentServices:

theMOReaggregationtool

Ratings:•Needingimprovement/••Satisfactory/•••Good


Visibilityofsystemstatus ••• Thetoolhasverygoodfeedbackmechanisms.Usersreceivealertseachtimetherehasbeenachangeinthestatusofapackage(ingestion,validation,enrichmentetc.)


••• Thetoolissimpletouse.Avirtualguidetakesusersthroughthesystemwithsimplepop-upinstructionsonceeachstagehasbeencompleted.Languageandconceptsareplainandeasilyunderstandabletonon-technicalusers.

Usercontrolandfreedom ••• Supportsundo:Packagescanbeeasilyrejectedordeletedatanystagepriortoandafterpublication.Thereareclearexitpathsforusersatallstageswhenusingthetool.

Consistencyandstandards •• Languageusedinthetoolisuniformthroughout.However,thereissomeneedtoupdatethesupportdocumentationinthemicro-servicessectionwhichcurrentlypointstoLoCloudsupport.

Errorprevention ••• Therewasnoexperienceofdirecterrorsinthetool.

Recognitionratherthanrecall ••• Thetoolhasaveryclearvisualstructurewitheachcomponentaccessiblefromthemaininterface.Objects,actionsandoptionsarevisibleatalltimesandremainsofromonepartofthedialoguetoanother.

Flexibilityandefficiencyofuse ••• Frequentactionscanbetailoredbytheabilitytobuildanenrichmentplanfordata,thiscanbere-usedfornewlyingesteddata.

Aestheticandminimalistdesign ••• Thetoolhasaveryclearinterfacedesign.


••• Messagesareeasytounderstandfornon-technicalusers.Validationerrorsareclearlyindicated,withusersabletogetdetailsfortherecordsandelementsthatarecausingtheerrors.Anotificationlogfrompreviousactionscanalsobeaccessedinthetool.

Helpanddocumentation •• WhilethereissupportdocumentationavailablewithintheMORetool,thisisfortheLoCloudproject.ThisdocumentationshouldtailoredforandbemadeavailablefortheARIADNEinstanceofthetool,oratleastbedescribedontheARIADNEwebsite.


21

Portal1


Visibilityofsystemstatus ••• Itiseasytoseewhereyouareinasearch,asthefacetschosenwhichleadtocurrentviewsaredisplayedonscreenbesidethesearchresults.


••• Therearenosystem-orientatedtermsandtheportalfollowsreal-worldconventions.Itmaybenecessarytore-structureandre-writethedescriptionsonthesupportpagefornon-technicalusers.ForexampleitisnotimmediatelyclearwhytheACDMisdescribedhere.Itwouldbemoreusefultohaveaguidetotheportal,ratherthaninformationabouttheregistry.

Archaeologists’severalpathstodataandinformationarewellreflectedintheportal.Extensiveeffortsinintegratingarchaeologicalvocabulariesandexistinginformationstructuresensureagoodmatchbetweenrealworldarchaeologyandthesystem.

Usercontrolandfreedom ••• Itiseasyforuserstorefinetheirsearchparametersusingthefilters,thesefacetscanbeaddedorremovedwithease,thussupportingundoandredo.

Alinktothesearchpagetobeginagainisalwaysavailable.

Consistencyandstandards •• Informationdisplayisclearandconsistent.WhileARIADNEhasastrongdatastandard,ensuringadherencetothestandardacrosspartnersfromdifferentcontextsiskeytoconsistencyacrossarchaeologicaldataintheportal.

Theremaybesomeconfusionamongusersinthenumberoffacetsrelatedtothematicconcepts-inparticularthepresenceofthreesimilarrefinements-subject,keywordandoriginalsubject.Itshouldbemadeclearthat

1Notetheportalhasalsobeentestedtogetherwiththeservices inWP13,readersareurgedtoalsoconsultWP13deliverables.


22

‘subject’relatestotheGettyAATterm,whiletheotherfacets(fromthenativemetadata)maybenefitfromahelptexttopointuserstowherethesetermsoriginate(i.e.withdepositors).

Errorprevention ••• Theportalalwaysreturnssearchresultsratherthanproducingerrormessages.Itisalsoclearwhentherearenoresultstodisplayforagivensearch.

Recognitionratherthanrecall ••• Theresourcelandingpagesandsearchpagesdisplaybuttonsforrelevantoptionsinauniformway.However,nogeneral‘help’buttonisvisiblewithintheportal.

Flexibilityandefficiencyofuse •• Useoffacetedsearchingmayincreaseefficiencyofuseforexperiencedusers.Optionstosaveshortcutsforfrequentactionsarenotavailable.

Itwouldbeworthexploringtheadditionofaloginsothatuserscancreateanaccountintheportalandsaveorexporttheirsearchresults.

Aestheticandminimalistdesign ••• Theportalverysuccessfullysharessufficientrelevantinformationandoptionswithoutcrowdingthesearchorlandingpages.

Someoftherollingimagesonthelandingpageoftheportalmayneedtobereviewedasthereneedstobeahighcontrastbetweenthesearchandbrowseboxesandthebackgroundimages.Someimagesaretoo‘busy’andneedtobeahigherresolution,filesizesshouldbereviewedasoccasionallycertainimagestakelongertoloadthanothers.

Help users recognize, diagnose, andrecoverfromerrors

•• Therewereveryfewexperiencesoferrorsintheportal.Thefunctionforviewingarecord’smetadatainACDMXMLisnotcurrentlyfunctioning.

Helpanddocumentation •• HelpanddocumentationisnoteasilyvisibleoraccessiblewithinthePortal.TheARIADNEPortalGuidePDFdocumentisaclearandeffectiveguidewhichshouldbeaccessiblefromtheportal‘about’section,ratherthantheprojectwebsite.


23

4.4 OnlineSurvey

Asurveywasconductedamongcontent-providingpartnerstotheARIADNE infrastructure.Thiswasdonethroughanonlinesurveytool (SurveyMonkey)andthe linkcirculatedtoallpartners through basecamp. There was a good response rate from content-providingpartners,withabout65%ofpartnerscompletingthesurvey.ThequestionnaireimplementsausabilitytechniquefromthesecondtierofTable2,balancingtheheuristicevaluationusedabove.

Themain impetusbehindthesurveywastotestthe integratedfeaturesandworkflowsoftheARIADNEinfrastructure.Assuchitposedquestionsontheregistry,enrichmentservicesandtheportal,withafocusoncontentpreparation,ingestandpublication.Therewasalsosignificant examination of the help/support documentation available to users. HowintegratedtheoverallarchitectureiswillbeessentialforthefutureintegrityoftheARIADNEproject.Thesurveyprovidedmultipleopportunitiesforpartnerstoexpandontheiranswersor comment on aspects of the infrastructurewhich they felt were not highlighted in thesurveyitself.

The survey results arepresentedherebasedon the various sectionsof the infrastructurewhich were evaluated. The results of the survey were communicated to the technicalpartnersforreviewandanalysis,andforeventualintegrationintotheinfrastructure.Thisisanongoingprocessasmorecontentisingestedandtheinfrastructureisrefined.

A copy of the survey questions is available in Annex I. Within the survey there was anembeddedusabilitytestingmechanismcalledtheSystemUsabilityScale(SUS).

4.4.1 SystemUsabilityScale(SUS)

The SystemUsability Scale (SUS)was created by John Brooke in 1986 and is designed toprovidethefollowingbenefits:

• Isaveryeasyscaletoadministertoparticipants• Itcanbeusedonsmallsamplesizeswithreliableresults• Isvalid-itcaneffectivelydifferentiatebetweenusableandunusablesystems

TheSUSmustbeunderstoodintermsofthelimitationsofusabilityasatangiblequalityandmustbeviewed‘intermsofthecontextinwhichitisused,andtheappropriatenesstothatcontext.’Itis,asBrookeoutlined,‘impossibletospecifytheusabilityofasystem…withoutfirstdefiningwhoaretheintendedusersofthesystem,thetasksthoseuserswillperformwith it, and the characteristics of the physical, organisational and social environment in


24

which itwill be used.’2 (Brooke, 1996) It has been found to be highly reliable and usefulacrossarangeofinterfacetypes(Bangor,KortumandMiller,2008).

TheSUSissimpleten-pointscalewhichaimstogiveaglobalviewofsubjectiveassessmentsof usability. Itwas born of the need to replace hitherto complicated evaluationmethodswhichwere time-consuming, expensiveand frustrating forusers. The ten statements thatcomprisetheSUSaredesignedtocoveravarietyofdifferentaspectsincludingtheneedforsupport, training and complexity and therefore ‘have a high level of face validity formeasuringusabilityofasystem.’

SUSstatements:1. IthinkthatIwouldliketousethissystemfrequently2. Ifoundthesystemunnecessarilycomplex3. Ithoughtthesystemwaseasytosue4. IthinkthatIwouldneedthesupportofthetechnicalpersontobeabletousethis

system5. Ifoundthevariousfunctionsinthissystemwerewellintegrated6. Ithoughttherewastoomuchinconsistencyinthissystem7. Iwouldimaginethatmostpeoplewouldlearntousethissystemveryquickly8. Ifoundthesystemverycumbersometouse9. Ifeltveryconfidentusingthesystem10. IneededtolearnalotofthingsbeforeIcouldgetgoingwiththissystem

Respondentsareaskedtorateeachof theabovestatementsona fivepointscalerangingfromstronglydisagree(1)tostronglyagree(5).

SUSScoring

SUS scores are calculated by totally the score contributions from each statement, eachstatement’s score ranges from0-4. For statements1,3,5,7and9 the score is the ratinggivenbytherespondent-1.Forstatements2,4,6,8and10thescoreis5–theratinggivenby the respondent. The total of all the scoresmultiplied by 2.5 gives the overall systemusabilityvalue.Thisrangesfrom0to100.

TheaverageSUS score is 68, scoresabove this are considered ‘aboveaverage’. The scoreranges from0 (negative) to100(positive).However, theSUSscore isnotapercentage. InadditiontothescoringBangor,KortumandMillerhavedevelopedanadjectiveratingscaletoprovidemoremeaningfulandhelpfulresults.Theseareshowninthechartbelow.

2TheseaspectshavebeenaddressedinthecourseoftheARIADNEprojectbyseveralstrands,inparticularbytheworkofWP2andtheresultingD2.1FirstReportonuser’sneeds.


25

Figure2.MeanSUSscoreratingscorrespondingtothesevenadjectiveratings(errorbars+/-onestandard

errorofthemean).FromBangor,KortumandMiller,‘DeterminingwhatindividualSUSscoresmean:addingan

adjectiveratingscale’inJournalofUsabilityStudies,4:3(2009)

SUStestingwascarriedoutthroughtheTask12.4surveyfortheregistryandportal;resultsaregivenbelow.

Registry

Results for the registry varied somewhat frompartner to partner (see comments below).The lowest SUS scorewas 37.5with thehighest being75.On average the SUS scorewas58.75,belowwhatisconsideredaverage.Itgivesanadjectiveratingof‘Ok’.

Portal

TheportalhadconsiderablyhigherscoringthantheRegistry.TheSUSscoresfortheportalranged from 42.5 (from one partner, the next lowest score being 65) to 90 indicating ahigher levelof satisfactionamongpartners.TheaverageSUSscorebeing77.5,wellabovetheaverage.Thisgivesanadjectiveratingof‘good’.

Comments: While the results indicate that there is general satisfaction with how theinfrastructure performs, therewas in each respondent’s case a lower level of satisfactionwith the registry than therewaswith theportal. This isperhapsa falsedichotomyas theregistrycontains thedatapowering thecontent thatappears in theportal. It is likely thatthe lower scoremore fully represents some difficulties experiencedwith adapting nativecontenttotheARIADNEinfrastructurewhichofteninvolvedvariedtechnicalconsiderationsonthebestmethodformapping,ingestingandexposingprovider’sdatabasestotheproject.Duringsystemdevelopment,uploadingcontenttotheregistrywassometimescumbersomeas it happens in earlydevelopment stages,while thesedifficulties substantiallydecreasedwhenthesystemwasfullyoperational.Possibly,thememoryoftheproblemsencountered


26

duringthisinitialdevelopmentstageinconsciouslyinfluencedsomeanswersinthenegative,especiallywithnon-technicalpartners.Theingestionactivitywascarriedoutatapartner-to-partner level and in the initial stages involved an iterative process whereby theinfrastructure was being developed as different (and sometimes unforeseen) types ofcontent was being offered by partners. This (largely undocumented) work is a keyconsideration for future collaborationwithinARIADNEand thepublicationof case studiesforhownativedatarepositorieswereexposedtotheinfrastructuremaybeusefulforfutureusers.

Animportantaspectoftheresponsesisthedifferenceofexperiencefrompartnerswhohadlarge infrastructural resourcesat theirdisposalandthosewhohadsmalleroperationsandfewer technical staff. This is a critical factor to consider when assessing the way theconsortium will operate in the future. Should larger partners share resources (training,workshops, technical services etc.) or should these partners act as conduits for contentprovisionbysmallerproviders?Alternatively,shouldtheinfrastructurebeadaptedtoensurethat smaller institutions with fewer resources are catered for by improving the overallaccessibilitytorelevantdocumentationandtechnicalsupport?

4.4.2 SurveyResults

The five areas defined as important for archaeological research data highlighted by D2.1were:

• Datatransparency:havingagoodoverviewofavailabledatasets• Dataaccessibility:therequireddatasetsareavailableinanuncomplicatedway• Metadataquality:theavailabledatasetsarewelldescribed• Dataquality:theavailabledatasetsarecompleteandwellorganised• Internationaldimension:havingeasyaccesstointernationaldatasets

TheseareaswereassessedinthecourseoftheonlinesurveyconductedforWP12testing.Respondents were asked to rate how they felt different aspects of the infrastructureaddressedthekeyfiveareas.


27

TheRegistry

Figure3.ChartshowingthelevelofsatisfactionwiththeregistryinrelationtothegeneraluserequirementsoftheARIADNEinfrastructure

ThePortal

Figure4.ChartshowingthelevelofsatisfactionwiththeportalinrelationtothegeneraluserequirementsoftheARIADNEinfrastructure

0

10

20

30

40

50

60

70

80

Excellent Good Average Poor VeryPoor

Datatransparency DataAccessibility Metadataquality

DataQuality InternaronalDimension

0102030405060708090

Excellent Good Average Poor VeryPoor

Datatransparency DataAccessibility Metadataquality

DataQuality InternaronalDimension


28

Comments:Thefeedbackfromrespondents inrelationtothegeneraluserequirementsofthe ARIADNE infrastructure reflects the results of the SUS scores. A higher level ofsatisfaction with the portal indicates that there are needs to be a more user-friendly,documented workflow for data ingest to the registry. Some responses indicate thatdepositorsdesiredahigherlevelofusercontrolovertheingestprocesssothatdatawouldappear according to provider’s needs in the portal. It is important to recognize that theportalrepresentsthefinalstageofthedataingestionworkflowandassuchismorelikelytoreflect satisfaction compared with earlier stages of content provision, which may haveinvolvedtechnicalandinfrastructuralissuestobeovercome,andisingenerallessrewardingintermsof“seeingtheresultsofworkdone”.

Additionalsurveythemes

Theonlinesurveyalsosoughttounderstandthe levelofsatisfactionwithotheraspectsofthe infrastructure including how familiar partners were with other features of theinfrastructure, how their data was ingested into the infrastructure and their level ofsatisfactionwiththeprocessfromingesttopublication.


29

TheARIADNECatalogueDataModel(ACDM)

ARIADNECatalogueDataModel is anextensionof theDataCatalogVocabulary (DCAT), aquasi-recommendation of the W3C Consortium that “is well-suited to representinggovernmentdatacataloguessuchasData.govanddata.gov.uk”.ThereasonforadoptingtheDCAT Vocabulary (apart from re-use) is that DCAT is proposed as a tool for publishingdatasets as Open Data. Its adoption places therefore ARIADNE in an ideal position forpublishing datasets as Open Data as well (see http://portal.ariadne-infrastructure.eu/about).

Respondents were asked about their level of familiarity with the ACDM as well as hownecessarytheyfeltaknowledgeoftheACDMwasfortheprovisionofcontenttoARIADNE.

Figure5.Chartshowingresponsestothequestion'HowfamiliarareyouwiththeACDM?'

The majority of respondents were familiar with the ACDM, the largest number (50%)considered themselves ‘somewhat familiar’. Respondents were further asked about hownecessarytheyfeltaknowledgeoftheACDMwasfor ingestingdata,withall respondentsconsidering iteithernecessary (70%)orpartiallynecessary (30%).60%ofrespondents feltthat there was insufficient training and resources available for using the ACDM withinARIADNE, while the remainder felt that there were sufficient resources. Severalrecommendationsfrompartnersonhowthismightbeachievedweresuggested,thesearegroupedaccordingtosimilaritybelow:


30

• ‘…Publishing worked examples of application…. I think more examples arenecessary….Addingmore examples ofmapping.Using amapping tool that showsthepreviewofthemappingresults………Byaddingtutorialsandexamplefiles(e.g.spreadsheetorxmlfile)……AguidelistinganddescribingeachACDMconcept(asitissomearemissing)…….’

• ‘….makeiteasiertofindthemostcurrentinformation/help/tutorials....TheHelplinkin the registry should be linking to the support portal http://support.ariadne-infrastructure.eu/. Within the support portal, there is a heading titled 'MappingGuidelines' but what follows aren't Mapping Guidelines', but rather the manyiterations of the Specification of the ARIADNE Catalogue Data Model. The actualmappingguidanceisburiedattheendofthesectionandcalledtheDataResourcesuserguide.Thiswholepageandthedocumentswithinitneedwillneedtobeeditedfor users not familiar with the ARIADNE project, and with their ease of use inmind…..’

Comments:ThespecificationfortheACDMiscurrentlycontainedinthe‘About’sectionoftheportal,[http://portal.ariadne-infrastructure.eu/about]theuserguideshouldbeupdatedwithmore examples and added to this section. The help section of the registry includesexampleExcelandXMLfiles.Theseshouldbeexpandedtoreflectmorevariedcontentandin particular the Excel file should include several more rows of sample data. This isparticularlyimportantgiventhattheseformatsweremostcommonlyusedbypartners(seebelow). In addition to this, as highlighted by a user above, the help link in the registryshould be linked to the support portal at http://support.ariadne-infrastructure.eu/#intro.Theexamplefilesshouldalsobeincludedonthisinterface.Ingeneral,however,itmustbenoted that understanding the ACDM and the required mappings requires substantialtechnical skills thatwere not always available for the content provider partners, as thosecommentingabove.Thusitislikelythatjustaddingmoreexamplesmaynotsolvetheissue,unlessaccompaniedbytrainingofcontentproviders.

IngestingData-TheARIADNEregistry

Thissectionshouldbeusedinordertoassessthebestmethodsforfuturedevelopmentoftheingestworkflowasitreflectstheformatsbestsuitedformappingfromprovider’snativedata(andpresumablythearchaeologicalcommunityatlarge).


31

Figure6.Chartindicatingwhichmethodsusedbypartnerstoingestdataintotheinfrastructure.Notethatsomepartnersusedmorethanonemethod.


32

Figure7.ChartindicatingthetypeofdigitalresourcesingestedtotheARIADNEinfrastructure.

Comments:ItshouldbenotedthatnopartnersusedtheRESTAPItoingestrecords.Thereistechnical documentation for the API contained in the Registry help section as well as inD12.3. It may be useful to edit this to provide more suitable documentation for non-technicalusers,withstepbystep instructions.SmallerproviderswithoutOAI-PMHtargetsarelikelytousetheExcelandXMLfilestoingestdata,thereforethecommentsnotedabove(aboutmoredetailedexamplefiles)shouldbeaddressedfornewusers,possiblysettingupatrainingprogram.Itwasrecommendedthatawebpagewithcleardetailsonhowdatacanbeadded to the registry is added to theportal (as thiswill generallybe the firstpointofinteractionwith the infrastructure). This page should include contact details for technicalpartnersforfuturecollaboratorswhowishtosubmitcontenttotheARIADNEinfrastructure.


33

The type of content ingested is given in Figure 6. Although no respondents reportedingestingGISdata,40%indicatedthattheywouldbelikelytoingestGISdatainthefuture.

Partners were given the opportunity to comment on any issues which they encounteredwhen ingesting data to the registry. Some reported that there were difficulties withaligning/normalising their native data with the requirements of the registry. Tworespondentscommentedontheneedforguidanceonwhetherdatashouldbeingestedatitemorcollection level.Thesedifficultiesweremainly resolvedbycontacting thepartnersresponsiblefortechnicalsupport.Respondentsindicatedagenerallevelofsatisfactionwithindividual technical support, how this supportwill bemaintained in the future should beconsideredbytheconsortiumatlarge.

MOReaggregationtool

OnlyasmallnumberofrespondentsreportedusingtheMOReaggregation,validationandenrichment tool. Users are directed to read section 4.3.2 which details internal testingcarriedoutonthisfeature.Ofthoserespondentswhousedthetool,allweresatisfiedwiththe enrichment of their data with only two enrichment services recorded as being used,LoGeoandThesaurimappings.Therefore,itisrecommendedinthefirstinstancethatthesetwo services, as well as PeriodO, be comprehensively documented in the tool and linksupdated to refer to ARIADNE documentation rather than LoCloud documentation. Someenrichmentmicro-servicesmaybe imrpovedwithmorecomprehensivedescriptions.ForadetailedworkflowofthistoolseeAnnexII.

Support/Helpdocumentation

There was an overall level of satisfaction with the support documentation with 70% ofrespondents finding the documentation either helpful or sometimes helpful. The chartbelowgivesdetailsonhowrespondentsfeltthedocumentationcouldbeimproved.


34

Figure8.Chartindicatinghowthesupportandhelpcouldbeimproved

Reiteratingwhatwasoutlined for the ingestprocess (seeabove) respondentscommentedthatexampledatasetsandmoremappingexampleswouldimproveuserexperience.


35

5 Conclusions

5.1 Registrytooltesting

Basedupon the requestsof theuser testing inD12.4 the resolutionofoutstanding issueshadbeencompletedinseveralareaswith50%ofissuesreachingsolutionnowand50%ofissues postponed to the next version of the system, where future new users will find afriendlierenvironment.This isparticularly thecase in the supportportal thatwillneed totakeintoaccountthecharacteristicsofanewwaveofcontentproviders.

Overall the user feedback from the new round of internal testing of the Registry Toolresultedinasatisfactoryscorewithspecificimprovementstobeplannedintoseveralareastoimprovetheusability,including:

• AdditionofUndo/Redofunction• Ensureallhelptextsarepresentincludinghelplinkswhichdescribethedifferent

propertiesoftheACDM• Employaclearervisualstructurefortheinterfacewhichreinforcesgroupingofkey

resourcingclasses

Areaswheretheregistrationtoolcouldbeimprovedincludealso:• Developmentofa“basic”and“expert/advanced”setting• Provisiontheuserfeedbackontheresultsoftheiractionanderrorreporting

Based upon the SUS testing average score result of 58.75 the Registry tool would beconsidered below average so some level of improvement is required in futuredevelopments.Thealreadynotedfactthatthecurrentproviders/testers/respondershadtopay a price for contributing to a system being developed and tested, with the relatedfrustration and dissatisfaction of availing, at least in the beginning, of a still incompletesystem,somehowmitigatesthisresult.

BasedupontheonlineusertestingtheRegistryreceivedpredominantlygoodratingacrossall the areas of testing including: data transparency, data accessibility, metadata quality,dataqualityandinternationaldimension.

Based upon the online testing users’ feedback effort should be made to improve theunderstanding and comprehension of the ACDM as familiaritywas relatively low and theprovisionoftrainingresourcesshouldbeconsideredasveryimportant.


36

5.2 MOReaggregationtooltesting

OveralltheuserfeedbackfromtheinternaltestingoftheMOReaggregationtoolresultedinamoderatetogoodscorewithgoodmarksgiventoallthetestingcriteria.Wherethereisthe potential for a slight improvement would be to update support documentation tocoordinatetooltoreferencetheARIADNEproject.BasedupontheonlineuserfeedbacktheMOReaggregationtoolwasnotbeingutilizedbyseveralprojecttestersandthebenefitsofitsuseshouldbepromotedwidely,specificallyasregardstheuseofthePeriodO,LoGeoandThesaurimappingfunctions.

5.3 Portaltesting

OveralltheuserfeedbackfromtheinternaltestingofthePortalresultedingoodscorewithgoodmarks given to all the testing criteria. The areas highlighted for slight improvementinclude:

• Abilityforuserstosaveanduseprevioussearchqueries• Improveerrorreportingfunctionsandvisibilityofhelpandguides

The first feature was not included in the initial specifications of the portal, and will beconsidered inARIADNE2.0.There isalwaysspace to improvehelp,andusers’ feedback isthebestway tounderstandwhichaspectsneed furtherdetail, as they look simple to thedeveloperbut are insteaddifficult for theuser,whileothers treatedwith greatdetail aremaybeoflittlerelevancefortheuser.

Based upon the SUS testing average score result of 77.5 the portal the Portal would beconsideredwellaboveaveragewithanadjectiveratingofgood.

Basedupontheonlineusertestingtheportalreceivedpredominantlygoodratingacrosstheareasof testing including:metadataquality,dataqualityand internationaldimension,andexcellenttogoodratingsfordatatransparencyanddataaccessibility


37

6 References

Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In P. W. Jordan, B. Thomas, B. A.Weerdmeester,&A.L.McClelland(Eds.),UsabilityEvaluationinIndustryBangor, A., Kortum, P., & Miller, J.A. (2008) ‘The System Usability Scale (SUS): An EmpiricalEvaluation’,InternationalJournalofHuman-ComputerInteraction24(6).

Gavrillis, D., Papatheodorou, C., Canstantopoulos, P.,D12.2: Infrastructure Design, February 2015,http://www.ariadne-infrastructure.eu/Resources/D12.2-Infrastructure-Design

Gavrillis, D., Papatheodorou, C., Afiontzi, E., Makri, D.N., Dallas, C., D12.3: Initial InfrastructureImplementation Report, June 2015, http://www.ariadne-infrastructure.eu/Resources/D12.3-Initial-Infrastructure-Implementation-ReportHolzinger,A.,‘UsabilityEngineeringMethodsforSoftwareDevelopers’inCommunicationsoftheACMJan2005,Vol.48,No.1.Sauro,J.10ThingsToKnowAboutTheSystemUsabilityScale(SUS)availableathttp://www.measuringu.com/blog/10-things-SUS.php]

Selhofer, H., Geser, G., D2.1: First Report on Users’ Needs, April 2014, http://www.ariadne-infrastructure.eu/Resources/D2.1-First-report-on-users-needs

Selhofer,H.,Geser,G.,D2.2: SecondReport onUsers’Needs, February2015, http://www.ariadne-infrastructure.eu/content/view/full/1188Aframeworkofguidanceforbuildinggooddigitalcollections(2007)[http://www.niso.org/publications/rp/framework3.pdf]


38


39

7 Annex1:OnlineSurvey

Q1.AreyouaffiliatedwithanARIADNEparticipatingpartner?

AnswerChoices Responses

Yes 92.3%

No 7.7%

Total 13

Q2.Whatisthenameofyourorganisation?

Responses

SvenskNationellDatatjänst

AustrianAcademyofSciences

StuartEveConsulting

NationalInstituteofArchaeologywithMuseumattheBulgarianAcademyofSciences(NIAM-BAS)

ArheoVestAssociation

ArchaeologyDataService

GermanArchaeologicalInstitute

HungarianNatioanlMuseum

CentralInstitutefortheUnionCatalogueofItalianLibrariesandBibliographicInformation

ZRCSAZU

DigitalCurationUnit,AthenaResearchCentre

ARUP_CAS_17

Inrap


40

Q3.Whatbestdescribesyourrole?

AnswerChoices Responses No.

Archaeologist 76.9% 10

Researcher 69.2% 9

ComputerScientist 0% 0

Softwaredeveloper 0% 0

DigitalArchivist/RecordsManager/Informationscientist

38.5% 5

Other(pleasespecify)“DataManager”

7.7% 1

Q4.HowfamiliarareyouwiththeAriadneCatalogueDatasetModel(ACDM)?


Veryfamiliar 10% 1

Quitefamiliar 30% 3

Somewhatfamiliar 50% 5

Notfamiliar 10% 1

Total 10

Q5.HownecessarydoyouthinkaknowledgeoftheACDMiswhenusingtheMetadataRegistry?


Necessary 70% 7

Partiallynecessary 30% 3

Notnecessary 0% 0

Total 10


41

Q6.DoyoufeelthatuserswhoarenotfamiliarwiththeACDMwouldbeabletoingestmaterialintotheregistry,asitiscurrentlydesigned?


Yes 20% 2

No 40% 4

Perhaps 40% 4

Total 10

Q7. Do you think there is sufficient training and resources (e.g. information/help/tutorials) onusingtheACDMwithinARIADNE?


Yes 40% 4

No 60% 6

Total 10


42

Q8.HowcouldthetrainingandunderstandingoftheACDMbeimprovedinARIADNE?

Responses

Makeiteasiertofindthemostcurrentinformation/help/tutorials.

Publishingworkedexamplesofapplication.

Ithinkmoreexamplesarenecessary

TheHelplinkintheregistryshouldbelinkingtothesupportportalhttp://support.ariadne-infrastructure.eu/.Withinthesupportportal,thereisaheadingtitled'MappingGuidelines'but

whatfollowsaren'tMappingGuidelines',butratherthemanyiterationsoftheSpecificationoftheARIADNECatalogueDataModel.TheactualmappingguidanceisburiedattheendofthesectionandcalledtheDataResourcesuserguide.ThiswholepageandthedocumentswithinitneedwillneedtobeeditedforusersnotfamiliarwiththeARIADNEproject,andwiththeireaseofusein

mind.

Forarchaeologists,whoneverdonesuchthingitisdifficulttounderstandandperphapsastep-by-stepexplanationisneeded.

AddingmoreexamplesofmappingUsingamappingtoolthatshowsthepreviewofthemappingresults

AguidelistinganddescribingeachACDMconcept(asitissomearemissing)

Byaddingtutorialsandexamplefiles(e.g.spreadsheetorxmlfile)

Q9.Whichmethoddidyouusetoingestdata?Morethanoneoptionispossible.


ARIADNEregistrywebtool(manually) 33.3% 3

Excelfile 33.3% 3

XMLfile 44.4% 4

RESTAPI 0% 0

OAI-PMH 33.3% 3

Other(pleasespecify) 0% 0

Total 9


43

Q10.Whattypeofdatadidyouingestintheregistry?Morethanoneoptionispossible.


Collection 55.6% 5

Dataset 66.7% 6

Database 55.6% 5

GIS 0% 0

MetadataSchema 0% 0

Service 11.1% 1

Vocabulary 22.2% 2

TextualDocument 11.1% 1

License 0% 0

Total 9


44

Q11.Whattypeofdataareyoulikelytoingestinthefuture?


Collection 66.7% 6

Dataset 66.7% 6

Database 55.6% 5

GIS 44.4% 4

MetadataSchema 11.1% 1

Service 0% 0

Vocabulary 22.2% 2

TextualDocument 11.1% 1

License 0% 0

Total 9

Q12.Didyouingestdataatitemorcollectionlevel?

AnswerChoices Responses

Itemlevel 22.2% 2

Collectionlevel 33.4% 3

Both 22.2% 2

Notapplicable 22.2% 2

Total 9


45

Q13.Didyouencounteranyproblemswheningestingdata?Pleasedetailconcisely.

Responses

None,ingestingtotheregistry.ThoughencounteredseveralproblemsingestingtoMoRE.Therewassomeconfusionastowhetherweshouldingestatcollectionoritemlevel,whichwefeel

shouldhavehadmoreguidanceon.

Asourdatawasthetestdata,therewerealotofproblems,butthiswastobeexpected.Weencounteredproblemswiththesizeofourfiles(harvestedfromtheOAI-PMHtarget)andwith

mappings(astheyhadtobedonemanuallybyATHENA).I'msuretheseissueswerelikelyresolvedbythetimeotherpartnerswereingestingtheirdata.

Wearestillworkingoningestingdata.

InourdatatheACDM:subjectcorrespondstodc:title,inwhichthevaluescanbeoneofthefollowingones:type(eg.Vagoconcapocchiaconombrellino)orobjectdefinition(eg.Fibula)orsubjecttitle(eg.ToroFarnese)orthenameoftheobject(eg.VasodiBaratti).Tonormalizethesevalueswehadtomapdc.titlevaluetothevocalularyAATandwecreatedaTAGnative:subjectwith

thenormalizedvalues.

Ingestingatitemlevelneedsanon-manualmethod.Excelisalwaysmentionedasapossibilitybutitturnedoutthatitwasnotso.Weneededexternalhelp(providedwithinARIADNE,manythankstoAchille!)tocreateXML'sfromExcel.Theprocesswasnottransparentandtherewasalotofback

andforth(duetothemistakesinourDB,butonesthatarecommoninmostDB's).AmoretransparentandaboveallsystemicwaytoingestExcelisneededsincethisistheonlymeansthat

90%ofarchaeologistsarefamiliarwith.IfthereisnoExceltheARIADNEportalwillbelimitedtothebiginstitutionswithinternalcomputertechnicians/scientistsupport.

Distributionofmultiplesubjectsandkeywords;placenameshavenotbeenusefulforgeocoding


46

Q14.PleaseratehowyoufeeltheregistryaddressesthegeneraluserequirementsoftheARIADNEinfrastructure:

AnswerChoices Excellent Good Average Poor Verypoor Total

Datatransparency[i.e.doestheregistryfacilitatetheaimofhavingabetteroverviewof

availabledata?]

0%

0

77.8%

7

22.2%

2

0%

0

0%

0

9

DataAccessibility[i.e.doestheregistryfacilitatedatabeingavailabletotheuserinanuncomplicatedway?]

11.1%

1

55.6%

5

22.2%

2

0%

0

11.1%

1

9

MetadataQuality[i.e.Doestheregistryensurethatingesteddataiswell-describedandimplementsstandardsfor

interoperability?]

11.1%

1

55.6%

5

33.3%

3

0%

0

0%

0

9

DataQuality(i.e.Doestheregistryensurethatingesteddataiscompleteandwell

organised?)

11.1%

1

77.8%

7

11.1%

1

0%

0

0%

0

9

InternationalDimension(Doestheregistryallowforeasy

accesstointernationaldata,e.g.throughdataintegration)

11.1%

1

66.7%

6

11.1%

1

11.1%

1

0%

0

9


47

Q15.HowwouldyourateyouroverallexperienceoftheARIADNEmetadataregistry?

AnswerChoices Stronglydisagree

Disagree Neutral Agree Stronglyagree

Total

IthinkthatIwouldliketousetheregistryfrequently.

11.1%

1

11.1%

1

22.2%

2

55.6%

5

0%

0

9

Ifoundtheregistryunnecessarilycomplex.

22.2%

2

33.3%

3

44.4%

4

0%

0

0%

0

9

Ithoughttheregistrywaseasytouse.

0%

0

11.1%

1

55.6%

5

33.3%

3

0%

0

9

IthinkthatIwouldneedthesupportofatechnicalpersontobeabletousetheregistry.

11.1%

1

44.4%

4

33.3%

3

11.1%

1

0%

0

9

Ifoundthevariousfunctionsintheregistrywerewell

integrated.

0%

0

0%

0

55.6%

5

44.4%

4

0%

0

9

Ithoughttherewastoomuchinconsistencyintheregistry.

0%

0

44.4%

4

33.3%

3

22.2%

2

0%

0

9

Iwouldimaginethatmostpeoplewouldlearntousethe

registryveryquickly.

0%

0

22.2%

2

33.3%

3

33.3%

3

11.1%

1

9

Ifoundtheregistryverycumbersometouse.

11.1%

1

44.4%

4

22.2%

2

22.2%

2

0%

0

9

Ifeltveryconfidentusingtheregistry.

11.1%

1

11.1%

1

44.4%

4

33.3%

3

0%

0

9

IneededtolearnalotofthingsbeforeIcouldgetgoing

withtheregistry.

11.1%

1

11.1%

1

44.4%

4

33.3%

3

0%

0

9


48

Q16.DidyouusetheARIADNEmetadataregistrywebtool?


Yes 55.6% 5

No 44.4% 4

Total 9

Q17.Whattypeofaccountdidyouregisterwithinthetool?


Individual 33.3% 2

Institution 66.7% 4


Total 6

Q18.Didotherindividualsfromyourinstitutionregisterseparately?


Yes 16.7% 1

No 83.3% 5

Idon'tknow 0% 0

Total 6

Q19.Ifyes,didthisresultindifficultieswheningestingdatatotheregistry?


Yes 0% 0

No 100% 4

Total 4


49

Q20.Doyoufeeltheregistrationprocedurecouldbeimproved?


Yes 20% 1

No 80% 4

Total 5

Q21.Ifyes,pleaseselectfrombelow


Tightercontrolsoverdataproviderse.g.registrationofinstitutionswhocontrollocalaccess

100% 1

Implementationofahierarchyofusers,e.g.institutionalmanager/editor

100% 1

Onfirstregistrationusersarepresentedwithawalkthroughvideo/navigationpop-ups/furtherguidanceetc

100% 1

Visibilityofsupportdocumentationimprovedonregistrylandingpage

100% 1


Q22.Howwouldyouratetheuserinterface/designofthemetadataregistrywebtool?


VeryGood 0.00% 0

Good 60.00% 3

Average 40.00% 2

Poor 0.00% 0

Verypoor 0.00% 0

Total 5


50

Q23.Whatimprovementswouldyouconsidernecessarytothemetadataregistrywebtool?


Betterlayout&design 20% 1

Improvedintroductoryand/orexplanatorydocumentation

60% 3

Improvedhelp/supportdocumentation 60% 3

Improvedtechnicalsupport 20% 1

Other(pleasespecify)“Thepossibilitytoloaddatadirectlyfrom

spreadsheetsorXMLfiles”

20% 1

Total 5

Q24.Pleaseratehowimportantitistoimprovethefollowingintheregistrywebtool:

AnswerChoices VeryImportant

Important Lessimportant

NotImportant

No.

Design/UserInterface 0%

0

0%

0

60%

3

40%

2

5

Supportdocumentation 40%

2

60%

3

0%

0

0%

0

5

Facilityforpreviewingdata 40%

2

20%

1

40%

2

0%

0

5

Helptexts/pop-ups 20%

1

40%

2

20%

1

20%

1

5

Validation/Qualitycontrols 60%

3

40%

2

0%

0

0%

0

5


51

Q25.DidyouusetheMOReaggregationtoolaspartofyouringestprocess?


Yes 33.3% 3

No 33.3% 3

Idon'tknowwhatthisis 33.3% 3

Total 9

Q26.Wereyou…


familiarwiththeMORetoolfromotherEUprojects,e.g.Locloud,Carare

33.3% 3

giventrainingintheMORetoolaspartoftheARIADNEproject 0% 0

learnedhowtousetheMORetoolthroughonlinesupport/documentation

33.3% 3

Other(pleasespecify)“Idon'tknowwhetherthiswasunusualtoARIADNEpartners,butourMORetoolwassetup,andcompletedinthefirstinstancebytheDCU.

Itwasonlymuchlaterthatweweretoldabouttheaccount”

33.3% 3

Total 9


52

Q27.DidyouuseanyenrichmentservicesinMORe?


Yes 66.7% 2

No 33.3% 1

Total 3

Q28.Ifyes,whichenrichmentservicesdidyouuse?


Language 66.7% 2

DaiGazeteer 0% 0

LoGeo 50% 1

Geocode 0% 0

Rev.Geocode 0% 0

GeoInvertion 0% 0

Pelagios 0% 0

Vocabulary 0% 0

BackLink 0% 0

Thesaurimappings 100% 2

PleiadesPlus 0% 0

Perio.domappings 0% 0


BackLink 0% 0


53

Q29.Wereyousatisfiedwithyourenricheddata?


Yes 100% 2

No 0% 0

Total 2

Q30.Didyouuseanyofthehelp/supportdocumentationfortheregistry?


Yes 75.00% 6

No 25.00% 2

Total 8

Q31.Didyoufindthisdocumentationhelpful?


Yes 14.3% 1

Sometimes 57.1% 4

No(Pleasegivedetails)“Wedidn'thavetoconsultthisdocumentastheingestionhadbeen

doneforusinthefirstinstancebytheDCU.”“It'stoobriefandincomplete”

28.6% 2


54

Q32.Howdoyouthinksupport/helpdocumentationcouldbeimproved?


Morevisibleand/oraccessible 66.7% 4

Moredetailed 50% 3

Lessdetailed 0% 0

Needstobeupdated 50% 3

Needsmoreoptionse.g.livetechsupport,walkthroughvideosetc. 50% 3

Other(pleasespecify)“Exampledatasetstogetanideaofwhatshouldbeenteredwhere.”

“moremappingexamples”

33.3% 2

Q33.PleaseusethisopportunitytocommentonanyotheraspectsoftheRegistrywhichyoufeelneedimprovement:

AnswerChoices

BettercommunicationofwhatourresponsibilitieswerewithregardtoMORe.Aworkshopperhaps,aswasdonewiththeRegistry.PoordesignwithregardstomappingtotheGettyAAT.AIAChaddevelopedourOAI-PMHtodelivertheAATURIs,butMORerequiredustolookuptheAATpreferredterms,whichwebelieveresultedinunnecessarydoublehandlingofthedatato

reachthesameendresult.

Thereweresomeproblemswiththeacdm:temporal

Ichoseaverageforquestions14-15wedidn'tusetheregistry,andtherewasnowaytoskipthequestions.


55

Q34.WereyousatisfiedwithhowyourdataappearedintheARIADNEportalafteringestion?


Yes 55.6% 5

No(pleasespecify)“Ithasnotbeeningestedintotheportalyet.”

“Wehadseveralproblemstobeginwith.Firstourdatespansweren'tworking,second,weweren'tadvisedontheneedtohavelandingpagesastheywereinitiallyindicatedasoptional.We'redissatisfiedwiththefactthattheuserisunabletoselecttwopublishersinthefilteroptions,sothatour

datacanbecomparedwithothers.”“Wearestillworkingoningestingourdataintheportal”

“Placenameshavenotbeenusefulforgeocodinganditstilldoesn'tappearsthemappingofthesauri(enrichment)”

44.4% 4

Total 9

Q35.PleaseratehowyoufeeltheARIADNEPortaladdressesthegeneraluserequirementsoftheARIADNEinfrastructure:

AnswerChoices Excellent Good Average Poor Verypoor Total

Datatransparency[i.edoestheportalfacilitatetheaimofhavingabetter

overviewofavailabledata?]

33.3%

3

66.7%

6

0%

0

0%

0

0%

0

9

DataAccessibility[i.e.doestheportalfacilitatedatabeingavailabletotheuserinanuncomplicatedway?]

33.3%

3

66.7%

6

0%

0

0%

0

0%

0

9

MetadataQuality[i.e.Doestheportalensurethatdataiswell-describedand

implementsstandardsforinteroperability?]

0%

0

77.8%

7

22.2%

2

0%

0

0%

0

9

DataQuality[i.e.Doestheportalensurethatdataiscompleteandwell

organised?]

0%

0

88.9%

8

11.1%

1

0%

0

0%

0

9

InternationalDimension[i.e.Doestheportalallowforeasyaccessto

internationaldata,e.g.throughdataintegration]

55.6%

5

44.4%

4

0%

0

0%

0

0%

0

9


56

Q36.HowwouldyourateyouroverallexperienceoftheARIADNEportal?

AnswerChoices Stronglydisagree

Disagree Neutral Agree Stronglyagree

Total

IthinkthatIwouldliketousetheportalfrequently

0%

0

0%

0

22.2%

2

55.6%

5

22.2%

2

9

Ifoundtheportalunnecessarilycomplex.

44.4%

4

44.4%

4

0%

0

11.1%

1

0%

0

9

Ithoughttheportalwaseasytouse. 0%

0

0%

0

22.2%

2

55.6%

5

22.2%

2

9

IthinkthatIwouldneedthesupportofatechnicalpersontobe

abletousetheportal.

66.7%

6

22.2%

2

0%

0

11.1%

1

0%

0

9

Ifoundthevariousfunctionsintheportalwerewellintegrated.

0%

0

0%

0

33.3%

3

55.6%

5

11.1%

1

9

Ithoughttherewastoomuchinconsistencyintheportal.

33.3%

3

55.6%

5

0%

0

11.1%

1

0%

0

9

Iwouldimaginethatmostpeoplewouldlearntousetheportalvery

quickly.

0%

0

0%

0

11.1%

1

55.6%

5

33.3%

3

9

Ifoundtheportalverycumbersometouse.

44.4%

4

33.3%

3

0%

0

22.2%

2

0%

0

9

Ifeltveryconfidentusingtheportal. 0%

0

0%

0

22.2%

2

55.6%

5

22.2%

2

9

IneededtolearnalotofthingsbeforeIcouldgetgoingwiththe

portal.

44.4%

4

33.3%

3

22.2%

2

0%

0

0%

0

9


57

Q37.Areyousatisfiedwiththedesign/layout/userinterfaceoftheARIADNEportal?


Yes 88.9% 8

No(pleasespecify)“WhenIsearchforsomethingitisnotclearwhetherthereisdocumentationtobedownloadedornot.InmanycaseswhenIfoundinterestingthingsIrealizedthatthereisnofurtherdocumentationthatIcandownloadonlyalittleinformationonthesite.Sincetherearelotsofsitesinthedatabasetoomuchtimeisneededtofinddocumentationssinceyouhavetoclickthrough

dozensofthemtofinddocumentations.”

11.1% 1

Total 9


58

Q38.Whichofthefollowingfacets/filtersdoyoufeelareimportant?

AnswerChoices VeryImportant

Important SlightlyImportant

Neutral NotImportant

Total

Where 100%

9

0%

0

0%

0

0%

0

0%

0

9

When 77.8%

7

22.2%

2

0%

0

0%

0

0%

0

9

ResourceType 55.6%

5

33.3%

3

11.1%

1

0%

0

0%

0

9

NativeSubject 33.3%

3

33.3%

3

33.3%

3

0%

0

0%

0

9

DerivedSubject 33.3%

3

33.3%

3

33.3%

3

0%

0

0%

0

9

Keyword 55.6%

5

22.2%

2

22.2%

2

0%

0

0%

0

9

Contributor 22.2%

2

44.4%

4

22.2%

2

11.1%

1

0%

0

9

Publisher 22.2%

2

33.3%

3

22.2%

2

11.1%

1

11.1%

1

9

Place 77.8%

7

0%

0

22.2%

2

0%

0

0%

0

9

Dating 88.9%

8

0%

0

11.1%

1

0%

0

0%

0

9

Rights 44.4%

4

22.2%

2

33.3%

3

0%

0

0%

0

9

Language 22.2%

2

44.4%

4

33.3%

3

0%

0

0%

0

9


59

Q39.Doyoufeelthatthemetadataintheportalis..

AnswerChoices Yes Partially No Total WeightedAverage

ofasufficientstandard 77.8%

7

22.2%

2

0%

0

9 1.22

isdetailedenough 66.7%

6

33.3%

3

0%

0

9 1.33

displayscorrectly 66.7%

6

22.2%

2

11.1%

1

9 1.44

iseasilyunderstandable 44.4%

4

55.6%

5

0%

0

9 1.56

Q40.Doyouthinkanyadditionalmetadatafieldsshouldbedisplayedintheportal?

AnswerChoices Yes Total

No 88.9% 8

Yes(pleasespecify)“Itwouldbeusefultoshowwherearedocumentationsthatcanbe

downloaded”

11.1% 1

Q41.Areyousatisfiedwiththebrowsefunctions(Where,When,What)oftheportal?


Yes 66.7% 6

No(Pleasegivedetails)“Largelyyes,butIbelievethe'DisplayasSearch'buttonthatfeaturesontheWherebrowser,needstobemademoreprominentontheWhenbrowser.”“IftheARIADNEsubjectisthesameasAATsubject,itwouldbebettertosay

thatandhavethesubjectsbetransparent.”“themapjumpswhenIzoomonit(zoomsout),itcannotbeusedproperly”

33.3% 3


60

Q42.Areyousatisfiedwiththesearchfunctionoftheportal?


Yes 88.9% 8

No(Pleasegivedetails)“WedidnotyetexactlytestitbutcurrentlyIamstrugglingtofindthesites

thatIknowarethere(e.g.IronAgeburialmoundcemeteryinStična,Slovenia).”

11.1 1

Q43.Didyouhaveanyissuesaccessingdatathroughtheportal?


No 100.00% 9

Yes(Pleasespecify) 0.00% 0

d 12.6 final testing report - ariadne pluslegacy.ariadne-infrastructure.eu › ... ›...

Documents