SWAT GRIDIFICATIONSHORT SUMMARY

Łukasz Kokoszkiewicz

CONTENTS

EnviroGRIDS project overview SWAT explanation

SWAT gridification assumptionsParallelization results SWAT model parallelization LH-OAT uncertainty analysis parallelization SWAT-CUP sufi2 parallelization

EnviroGRIDS VOMonitoring Tools

DHMO

o Project timeframe: April 2009 – March 2013o Total budget of €8.1 mio. (€6.2 mio. EC contribution) and 1367

man-months.o The EnviroGRIDS Project Team includes 27 partners from 15

countries, representing also several European (CERN, EEA) and United Nations organizations (UNEP, UNESCO).

o Among these partners, 22 belong partially or entirely to the Black Sea Catchment.

o Eight partners belong to an International Cooperation Partner Countries (Ukraine, Georgia and Russian Federation) and five to Associated Countries (Switzerland and Turkey).

o Five partners can be defined as Small and Medium Enterprises (SME).

TECHNICAL OBJECTIVES

data gap analysis modeling of large scale, high resolution distributed hydrologic

processes developing a grid-enabled Spatial Data Infrastructure (SDI) spatially explicit regional scenarios of development develop access to real time data from sensors and satellites develop early warning and decision support tools at regional,

national and local levels Build capacities in the Black Sea region to enhance data

interoperability, by promoting on OGC and open standards (ISO 19115/19119/19139; web services WMS, WFS, WCS)

SWAT

SWAT (Soil and Water Assessment Tool) is a river basin, or watershed, scale model.It predicts the impact of land management practices on water, sediment and agricultural chemical yields in large complex watersheds with varying soils, land use and management conditions over long periods of time.

Models

SWAT

Hydrological modeling

GRIDIFICATION ASSUMPTIONS

SWAT Gridification

Creating EnviroGRIDS VO

Research which use cases are

possible to utilize on the

GRID

Creating easy to use monitoring

tools for the community

Envirigrids VO exists for few months now

Easy to use client application called hBrowse framework was created and

based on it Diane and Ganga monitoring tools

were created

3 use cases, for the SWAT gridification, were found

SWAT MODEL GRIDIFICATION

Sub-basin based model splitting followed by routingThe main idea is to split model itself based on sub-basins.Pre and post processing of the model is required to complete splitting and merging steps.After splitting, each sub-model can be run separately (in parallel) on the grid.Subbasin based parallelization is very interesting for large models.

DANUBE RIVER 15K HRU'S MODELGRID RUNS

As was observed SWAT model sub-basin based grid executions show some promises but to be fully utilized it requires much bigger jobs. Ideal situation would be when single sub-basin execution time takes more than half an hour, in this kind of situations the grid overhead would be negligible.

Tim

e [s

]

Simulation length [years]

LH-OAT SENSITIVITY ANALYSISALGHORITM PARALLELIZATION

LH-OAT is an uncertainty analysis method.In uncertainty analysis we are sorting out parameters sensitive to selected real world physical process (eg. nitrogen transport).Then an effective model calibration can be applied.The mechanism behind LH-OAT method parallel run is very similar to SWAT-CUP parallelization.We simply run many SWAT runs with a different parameters.

SWAT Model

LH-OAT Splitter

SWAT Model, params set 1

SWAT Model, params set 2

SWAT Model, params set n

swat swat swat

Output Output Output

LH-OAT Merger

GRID

SWAT-CUP SUFI2 ALGORITHM PARALLELIZATION

SWAT-CUP is an application for calibration of SWAT models. Calibration of SWAT models requires running SWAT simulations hundreds of times with different parameters setThe idea here is to run them separately or in small independent groups and resolve the best parameter configuration that gives simulation effects most corresponding to realityWe have “ready to go” example script set, to run swat sufi2 calibration efficiently from command line

Parameters

New SWAT Inputs

SWAT Outputs

swEdit.exe

swExtract.exe

swat

Output

SUFI-2

MCMC

ParaSol

GLUE

GRID

SWAT-CUP algorithms

ENVIROGRIDS VO

EnviroGRIDS VO is up and running since Autumn 2010Resources from Romania were added shortly after the VO were started and are being used since CE - ce01.mosigrid.utcluj.ro:8443/cream-pbs-envir – 960CPUs SE - se01.mosigrid.utcluj.ro - 11.739489 terabytes

Registration https://lcg-voms.cern.ch:8443/vo/envirogrids.vo.eu-egee.org/vomrs Approval is automatic for a listed members of the EG project People that are not members should be “recommended” by a listed member

VO configuration https://voms.cern.ch:8443/voms/envirogrids.vo.eu-egee.org/

Configuration.do voms113.cern.ch needs to be replaced by voms.cern.ch inside the VOMSES

string CIC VO Identity card: https://cic.egi.eu/index.php?

section=vo&vo=envirogrids.vo.eu-egee.org

MONITORING

hBrowse framework hBrowse.net It’s a common, easy to configure, client side, tool for creating job

monitoring systems It can visualise any hierarchical data up to 4 levels deep in a neat,

table based environment Google charts and highcharts can serve as renderer in creating

visually attractive charts

Based on hBrowse framewrok many monitoring tools was created including: http://dashb-atlas-job.cern.ch/templates/client/index.html http://dashb-atlas-job.cern.ch/templates/prodsys/index.html http://gangamon.cern.ch/ganga/ http://dianemon.cern.ch/diane/

ActiveMQ messages

MONITORING

The main system component is a JavaScript client which serves as a presentation layer.Different server layers can be applied.Server layer serves only as a data source.Client and server communicates using JSON data transfer format.JSON in this case is a perfect solution because of its syntax: in most cases it is the same as in

JavaScript and Python

HBROWSE SYSTEM ARCHITECTURE

CLIENT

JavaScriptAjaxjQuery

DATABASEWEB SERVER

SERVER

JSON

HTTP REQUEST

FINAL THOUGHTS

SWAT Gridification state The state of the SWAT gridification is that we can

run swat tasks on the grid With monitoring tools we can easily monitor our

simulations/calibrations

SWAT Gridifications final requirements Support of the linux executables (preferably both

32-bit and 64-bit) from SWAT developers is critically required

THANK YOU