sc2 workshop 1: big data challenges and solutions in agricultural and environmental research

Big Data challenges and solutions in agricultural and environmental researchIGAD / RDA Big Data Workshop, 22 September 2015

Rob Lokers, Sander JanssenAlterra, Wageningen URThe Netherlands

Outline

Historic perspective (agricultural & environmental modelling)

Expectations for the (near) future Some Big Data examples from the agri-food domain Big Data technologies in modelling and remote sensing Expectations versus reality in 2015

2

3

1960-1980

Crop science

Animal science

Food Science Economics

Institutional data collection




1980-2000

2000-2010

2010-2015

First computer models

Institutional applications

Integrated modelling frameworks







Open data across sectors

IT improvements (meta data, semantics)




4

Crop science

Animal science

Food Science

Economics

2010-2015

Open data across sectors





2015

-

2020

BIG DATA: one massive linked data pool across disciplines and strong computational capabilities

Computational capabilities:• Amazon• Microsoft

Azure• Google Earth

Engine• EC research

infrastructures

New data sources:• Remote sensing• Crowd sourcing• Rapid

phenotyping/Omics

• Social media

Potential to solve problems on agriculture, nutrition, food security, climate change?

Data analysis and integration, Models, Artificial Intelligence, Linked Open Data, Semantic web technologies, ...

Policy options, Products, Services, Costs, Benefits, Scenarios, Impact Assessments, Decision Support Systems, Integrated models, .....

Decision domain (policy/industry)

Process of data based value creation and roles involved

Policy makers/industry/societal stakeholders

Wisdom

Knowledgeinfo +

application

Informationdata + added meaning

(Big) Data raw material

Knowledge domain (science / consultants)

Interests (economic, social, environmental), values, preferences, trade-offs, risks, intangibles, ethics, ....

Databases, Satellites, Sensor networks, Social media, Citizen Observatories, ... O

pen

(dat

a) S

tand

ards

, (m

eta)

data

rep

osit

orie

s,

Busi

ness

dev

elop

men

t, V

isua

lizat

ion

tool

s an

d m

etho

ds, C

onte

xtua

lizat

ion,

Kno

wle

dge

Brok

erag

e, ..

.

Food Security example: Monitoring Agricultural ReSources (MARS)

Wisdom

Knowledge

Information

Data

Owned and operated by EC-JRC Crop forecasts at EU level needed to take

rapid decisions on Common Agricultural Policy instruments during the year

Provide information on vulnerability in specific food insecure areas

In support of:● European Common Agricultural

Policy on commodities & subsidies (focus on Europe, Asia)

● Food aid (focus on Africa) Monitoring weather and crop conditions of

current growing season (early warning, extreme events)

Example: Monitoring Agricultural ReSources (MARS)

Wisdom

Knowledge

Information

Dataweather archives live data streams

crop, soil databases Models

Food production example: Smart Farming: Monitoring, planning & control

11

cloud-basedevent

management

smart sensing & monitoring

smart analysis & planning

smart control

Genome sequencesFeed uptakePerformanceManureTemperatureActivityHeart ratepHAntibodiesBiomarkersMedicine use................

SizeLocationPerformanceManureWaterEnergyNutritionHealth management. . . . . .. . . . . .

Distance to . .Public healthLiving environmentMineral cyclesHealthy productsDisease risksEconomic figuresEnvironmental issues. . . . . . .. . . . . . .

Crop or Animal level Farm level

Environmental level

Supporting sustainable food production and contributing to the realization of

(inter)national policy agenda’s.

Market pricesLogisticsRegulations. . . . . . .. . . . . . .

Market level

Big Data technologiesTechnologies used (agricultural research): RDBMS, geo-databases Various “old & proven” programming languages (esp. for

modelling, data processing) Remote sensing: dedicated tools & environments for

processing and analysis, ENVI, R, GDAL etc. Harmonized information / data models (but still per discipline) High Performance clusters / grids

Experimental (ICT research for agriculture): RDF databases Vocabularies and ontologies (no alignments) NLP algorithms etc

Expectations versus reality in 2015...

New technological solutions (RDF databases, ontology alignment, NLP)

Successful initiatives use hybrid solutions, often build on “proven” technologies

“Magical” semantic (and linguistic) query processing “Technical” query processing (e.g. through SPARQL) Transparent access to big, distributed, heterogeneous datasets Mainly successful on metadata level and bibliographic sources,

cumbersome first attempts to harmonize big heterogeneous data streams

Custom-build data collection and processing chains still remain dominant

Source: Gartner (August 2015)

14

Thank you for your attention

15

sc2 workshop 1: big data challenges and solutions in agricultural and environmental research

Data & Analytics