LEA

RN

ING

NO

TE

Data standards: how to increase their use in data aiming for impact

GODAN Action supports data users, producers and intermediaries to effectively engage with open data and maximise its potential for impact in the agriculture and nutrition sectors. In particular, we work to strengthen capacity, to promote common standards and best practice, and to improve how we measure impact.

IntroductionOne of the three focal areas for GODAN Action is data standards. The idea is to enable producers and users of agri-food data to work more effectively by helping them to adopt more coherent, more inter-linked and agreed-upon standards, and deliver services that facilitate their use.

The work of the focal area to date has been to map existing agri-food data standards, conduct a gap analysis and make recommendations for improvements. To make this a more manageable task, we have narrowed our focus to look at particular use cases, with the first one being weather data.

The underlying assumption in this work is that such improvements increase the usability of data standards and therefore their actual use in datasets, thus enhancing the ability to create links between datasets, aggregate them together, compare them and rapidly develop new tools and services using data.

LESSONS FOR

STANDARDS DEVELOPERS

DATA PUBLISHERS

DATA USERS

MARCH 2018

GODAN ACTION LEARNING NOTE •••• DATA STANDARDS: HOW TO INCREASE THEIR USE IN DATA AIMING FOR IMPACT

2

LEA

RN

ING

NO

TEApproachThree GODAN Action partners - the Food and Agriculture Organisation of the UN, Agroknow and the Open Data Institute - led the work on this area and we adopted the following four-step approach:

1. Mapping currently available agri-food data standards in an online catalogue (https://vest.agrisemantics.org). In conducting the mapping we were careful to build on what existed by ingesting or linking to pre-existing catalogues and repositories. We also adopted a strong user focus making sure that descriptions, categorisations and search functionalities privilege aspects of usability.

2. Identifying gaps that inhibit the effective use of data standards, based on experts’ feedback. For this we applied and combined two existing frameworks:

• the assessment process of the UK Government’s Open Standards Board: https://standards.data.gov.uk/assessing-standards-proposals;

• the ODI Open Data Certificates criteria: https://certificates.theodi.org/en/.This exercise also entailed different engagement methodologies to involve experts, including email questionnaires and face-to-face interviews.

3. Producing recommendations for filling identified gaps. For technical openness and usability gaps, we applied ODI’s experience as developers of data-driven applications, and of standards, to suggest approaches. The developer experience review helped understand the gaps from the perspective of someone working in the topic area. We also emphasised identifying interventions that could be done by a variety of organisations.

4. Ensuring collaboration and uptake. Throughout the process we tried to cultivate synergies between different communities and initiatives to ensure the endorsement and uptake of the results.

Lessons learnt: • Identifying and weighing existing gaps and the major needs in terms of uptake of data standards is

difficult, especially considering the variety of actors using standards and the different needs they have. Developers have technical requirements regarding formats and APIs, data publishers need guidance in the selection, scientists need standards compliant with scientific models, big-data industries and global networks need more efficient and compressed formats.

• Assessing the overall quality of a data standard, especially at content/fitness level, requires detailed feedback from different types of knowledgeable users: scientists, practitioners, domain authorities. Doing an evaluation of the overall fitness of a standard with limited input from these categories of existing and potential users risks yielding only partial results. Therefore, recommendations regarding fitness and authoritativeness are very difficult to formulate.

Practical lessons for key stakeholder groupsThe key stakeholders identified through this piece of work are data publishers, data users and standards developers. All three stakeholder groups would benefit from improved and more usable data standards.

In turn, each of the three groups can do a lot to help the other two benefit from data standards, addressing the major obstacles to usability identified in the gap analysis.

Standards developers could help standards users by: • publishing their standards in new interoperable, semantic and linked ways;

• understanding the changing needs of developers so that standards and related tools can be improved;

• providing additional services on top of data standards, like tools and APIs (e.g. parsers, validators, converters, lookup services) that facilitate their use;

• engaging with potential users of their standards to help them understand how the standards could benefit them, e.g. through a help desk (either a dedicated one for their specific standard or, better still, by contributing to a general data standards help desk where users can navigate through different standards); in the case of new standards, engaging with potential users from the outset;

• creating mappings between their data standards and other widely used ones, to increase usability and adoption.

LESSONS FOR STANDARDS DEVELOPERS

3

LEA

RN

ING

NO

TE Data publishers who use standards when publishing their data could help data users by:• ensuring their data uses widely adopted and interoperable standards, considering both the content and

reputation of the standard (fitness to purpose and actual adoption/endorsement by relevant bodies) and technical openness and usability features (format, services); consulting the map of agri-food data standards can help in the selection process;

• helping users understand how datasets use the standards, e.g. by mentioning the data standards used in the dataset metadata;

• providing feedback to standards publishers on usability challenges.

Data and standards users who create tools and services that consume data are mostly the beneficiary of improvements of data standards, but they could also further facilitate the reuse of data by:

• repackaging data in a standardised and reusable way;

• using clear licensing and provenance metadata in their prosumer services;

• providing feedback to data and standards publishers on usability challenges.There is also clearly a role for other types of intermediaries to provide more integrated and added-value products and services, for instance:

• For a specific domain or type of data, a detailed evaluation and comparison of the existing standards with guidance on when and where to use them to publish data, and which conventions are useful. This could be done by domain authorities (e.g. WMO or MET Offices for weather data, or AgGateway for farm data) perhaps with support from an open data initiative (GODAN or ODI). The GODAN Action Map partially does that, but not at the domain-specific level.

• A general community/helpdesk dedicated to agri-food data standards, helping users identify relevant standards and understand how to apply standards to support publication of their data. The GODAN Action project is launching such a platform on the AIMS (Agricultural Information Management Standards) website of FAO.

Implications for the sectorWe identified some key broader indications for different actors in the sector, beyond the individual recommendations reported above:

• The gap analysis showed that in the agri-food sector there are big differences among domains (plant science, agronomy, nutrition, soil and natural resources, farm management, meteorology, etc.) and even more between different types of stakeholders (research, government, big industry, service providers) in the way standards are created, agreed upon and used, mainly because of different perspectives and priorities. The gap analysis on weather data standards (Pesce, Dodds, Tennison and Zervas, 2017) illustrates this in more detail for that domain.

GODAN ACTION LEARNING NOTE •••• DATA STANDARDS: HOW TO INCREASE THEIR USE IN DATA AIMING FOR IMPACT

LESSONS FOR DATA PUBLISHERS

LESSONS FOR DATA USERS

4

LEA

RN

ING

NO

TEAn interesting lesson to learn is that, even in the same domain, researchers may tend to use different standards than government or industry, thus hindering data exchange and aggregation across the same value chain. Since this responds to specific needs, differences cannot be completely overcome, but there is room for service providers to create bridging services and for communities of practice across the various value chains.

• Beyond the technical soundness of a data standard, a key element for its actual use is its reputation and the level of recognition in the reference communities. Authoritative bodies and scientific communities have an important role in this: the endorsement of a specific standard, the harmonisation of one’s standard with others and the avoidance of competition and duplication can greatly improve the standards landscape.

• Advocacy and capacity building are needed to promote good practices amongst standards developers.

• There should be more collaboration between standards developers, data publishers and data users in order to make standards and data both usable and actually used.

• Third party contributions should be encouraged to create supplementary materials that fill some of the gaps in provision, such as documentation or validators.

• The discovery issues relating to certain types of data (like weather data) could be addressed by improving use of metadata standards to help catalogue and describe data published by existing organisations, ideally registering them in discoverable open data catalogues.

In general, as the findings above illustrate, actual and improved use of data standards, better interoperability of data and better services are the result of the engagement of different actors in the data value chain. This is why demonstrating causality between better data standards > better data > better services is difficult without considering the variable of institutional and sectoral engagement.

This finding has implications on the evaluation of the impact of open data in the sector as it means that we cannot assume cause and effect relationships between the availability of better and more usable standards and the impact the project wants to make. At best, we can foresee plausible causal effects. For this reason, we find it more appropriate to scale down the Theory of Change for this focal area to an “outcome evaluation” rather than an “impact evaluation”. This is in line with the findings of the project impact evaluation focal area: “Examining outputs and outcomes may be in some cases more appropriate to study than longer-term impacts”.

What we can say is that, based on the typical data value chain data capture → data quality and integration → data enrichment → data analysis → decision making services → impact

work on data standards definitely contributes to the identified outcomes and impact, because the use of standards is key in all intermediary steps. In conclusion, better data standards are necessary but not sufficient.

References Pesce, V.; Tennison, J.; Mey, L.; Jonquet, C.; Toulet, A.; Aubin, S. and Zervas, P. (2016) A map of agri-food data standards GODAN Action. bit.ly/GA-map

Pesce, V.; Kayumbi, G.; Tennison, J.; Mey, L.; and Zervas, P. (2016) Agri-food data standards: a gap exploration report GODAN Action. bit.ly/GA-gap-exploration

Tennison, J.; Dodds, L.; Kayumbi, G.; Pesce, V,; Zervas, P. (2016) Recommendations for filling identified gaps GODAN Action. bit.ly/GA-recommendations

Pesce, V.; Dodds, L.; Tennison, J. and Zervas, P. (2017) Gap exploration report - weather data standards GODAN Action. bit.ly/GA-gap-weather

Tennison, J.; Dodds, L.; Kayumbi, G.; Pesce, V,; Zervas, P. (2017) Recommendations for filling identified gaps - weather data GODAN Action. bit.ly/GA-rec-weather

AuthorsValeria Pesce https://orcid.org/0000-0003-3860-4304Global Forum on Agricultural Research and Innovation, valeria.pesce@fao.org

Pauline L’HénaffProject Manager, Open Data Institute, pauline@theodi.org

GODAN Action is supported by the UK Department for International Development (DFID), led by Wageningen Environmental Research with international partners AgroKnow, AidData, CTA, FAO, GFAR, IDS, Land Portal, and the ODI.For more information visit the GODAN website www.godan.info/godan-action This GODAN Action publication is licensed under a Creative Commons Attribution 3.0 Unported License.