scripting geoserver with geoscript
DESCRIPTION
GeoServer is a solid and mature implementation of a variety of OGC services including Web Feature Service, Web Map Service, Web Coverage Service, and Web Processing Service. Add to this a KML engine, integrated security framework, powerful styling language with SLD and this rich feature set makes GeoServer very appealing to the user. However it has always been somewhat lacking when it comes to the developer. Developing with GeoServer has a steep learning curve and requires expert knowledge to do simple tasks like writing new output formats, implementing new WPS processes, and adding custom filter functions. GeoScript to the rescue!GeoScript adds spatial capabilities to popular languages such as Python, JavaScript, Scala, and Groovy. Scripting languages are the perfect tool for developers who want to do simple coding tasks quickly in a lightweight development environment. GeoScript builds on top of the very powerful GeoTools library to provide an interface to its capabilities though concise and easy to use API's. Recent extensions to GeoServer now allow developers to write components and plug-ins in the scripting language of their choice, using GeoScript as the engine for spatial functionality.This presentation is geared toward developers who are interested in developing with GeoServer but not necessarily ready to get their hands dirty with low level Java. The talk will detail the various scripting hooks available and provide examples, complete with code, of how to write some simple plug-ins. Check out this presentation and you'll be developing with GeoServer in no time.TRANSCRIPT
Scripting GeoServer with GeoScript
Justin Deoliveira and Tim Schaub
GeoScript?
• Spatial capabilities for scripting languageso Groovyo JavaScripto Pythono Scala
• Higher level api for GeoToolso Convenient, concise, completeo Make easy things easy
• Faster development turnaroundo No recompilation/rebuilding
GeoScript
GeoScript
GeoScript...import org.geotools.data.DataStore;import org.geotools.data.postgis.PostgisNGDataStoreFactory;import org.geotools.feature.FeatureCollection;import org.geotools.feature.FeatureIterator;import org.geotools.jdbc.JDBCDataStoreFactory;import org.opengis.feature.Feature;...
Map<String,Serializable> params = new HashMap<String, Serializable>();params.put(JDBCDataStoreFactory.HOST.key, "localhost");params.put(JDBCDataStoreFactory.PORT.key, 5432);params.put(JDBCDataStoreFactory.DATABASE.key, "geoscript");params.put(JDBCDataStoreFactory.DBTYPE.key, "postgis");params.put(JDBCDataStoreFactory.USER.key, "jdeolive");PostgisNGDataStoreFactory factory = new PostgisNGDataStoreFactory();DataStore pg = factory.createDataStore(params); FeatureCollection features = pg.getFeatureSource("states").getFeatures();FeatureIterator it = features.features();try { while(it.hasNext()) { Feature f = it.next(); System.out.println(f.getProperty("STATE_NAME").getValue()); }}finally { it.close();}
Java
GeoScript
from geoscript.workspace import PostGIS
pg = PostGIS('geoscript')for f in pg['states'].features(): print f['STATE_NAME']
Python
JavaScript
var PostGIS = require("geoscript/workspace").PostGIS;
var pg = PostGIS("geoscript");pg.get("states").features.forEach(function(f) { print(f.get("STATE_NAME"));});
GeoServer
GeoServer
Script Hooks - Data Formats
Data Formats• Drivers for spatial formats• Map to internal data model
Script Hooks - Data Formats
from geoserver import datastorefrom geoscript.layer import readJSON
class GeoJSON(object):
@datastore('GeoJSON', 'GeoJSON', file=('GeoJSON file', str)) def __init__(self, file): self.json = readJSON(file)
def layers(self): return [self.json.name]
def get(self, layer): return self.json
GeoJSON DataStore
Script Hooks - Data Formats
from geoserver import datastorefrom geoscript.layer import readJSON
class GeoJSON(object):
@datastore('GeoJSON', 'GeoJSON', file=('GeoJSON file', str)) def __init__(self, file): self.json = readJSON(file)
def layers(self): return [self.json.name]
def get(self, layer): return self.json
GeoJSON DataStore
Script Hooks - Output Formats
Output Formats• Drivers for exchange formats• Map from internal data model
Script Hooks - Output Formats
from geoserver.format import vector_format
@vector_format('property', 'text/plain')def write(data, out): for f in data.features: values = [str(val) for val in f.values()] out.write('%s=%s\n' % (f.id, '|'.join(values))
Property File Format
states.1=MULTIPOLYGON (((37.51 -88.07, ... 37.51 -88.07)))|Illinoisstates.2=MULTIPOLYGON (((38.97 -77.00, ... 38.97 -77.01)))|District of Columbiastates.3=MULTIPOLYGON (((38.56 -75.71, ... 38.56 -75.71)))|Delawarestates.4=MULTIPOLYGON (((38.48 -79.23, ... 38.48 -79.23)))|West Virginia . . .
.../wfs?request=GetFeature&typename=topp:states&outputformat=property
Script Hooks - Process
Processes• Smarts of WPS• Simplicity of scripting
Script Hooks - Processvar Process = require("geoscript/process").Process;exports.process = new Process({ title: "JavaScript Buffer Process", description: "Process that buffers a geometry.", inputs: { geom: { type: "Geometry", title: "Input Geometry" }, distance: { type: "Double", title: "Buffer Distance" } }, outputs: { result: { type: "Geometry", title: "Result" } }, run: function(inputs) { return {result: inputs.geom.buffer(inputs.distance)}; }});
JavaScript
Script Hooks - Filter Functions
Filter Functions• Filtering for WFS and WMS• Callable via SLD
from geosever.filter import functionfrom geoscript.geom import Polygon@functiondef areaGreaterThan(feature, area): return feature.geom.area > area
Area Function
Script Hooks - Transactions
exports.beforeCommit = function(details, request) { LOGGER.info("beforeCommit"); var records = details["PreInsert"] || []; records.forEach(function(record) { var feature = record.feature; feature.geometry = feature.geometry.simplify(10); });
};
JavaScript
Intercept WFS transactions with a wfs.js script in your data directory.
Script Hooks - Web/HTTP
"apps"• Simple WGSI-like environment• Access to catalog/data
def app(environ, start_response): start_response('200 OK', [('Content-type','text/plain')]) return 'Hello world!'
Hello World App
Data Summary Appfrom geoserver.catalog import Layerfrom StringIO import StringIO
def app(env, start_response): kvp = dict([tuple(kv.split('=')) for kv in env['QUERY_STRING'].split('&')]) layer = kvp['layer'] l = Layer(layer, store=None)
buf = StringIO() buf.write('Layer: %s\n' % l.name)
data = l.data buf.write(' Format: %s\n' % data.format) buf.write(' Feature count: %d\n' % data.count()) buf.write(' CRS/Projection: %s\n' % data.proj.wkt)
b = data.bounds() buf.write(' Bounds: (%f,%f,%f,%f)\n' % (b.west, b.south, b.east, b.north))
buf.write(' Fields:\n') buf.write('\n'.join([' %s' % repr(fld) for fld in data.schema.fields])) buf.write('\n')
start_response('200 OK', [('Content-type','text/plain')]) return buf.getvalue()
Data Summary Appfrom geoserver.catalog import Layerfrom StringIO import StringIO
def app(env, start_response): kvp = dict([tuple(kv.split('=')) for kv in env['QUERY_STRING'].split('&')]) layer = kvp['layer'] l = Layer(layer, store=None)
buf = StringIO() buf.write('Layer: %s\n' % l.name)
data = l.data buf.write(' Format: %s\n' % data.format) buf.write(' Feature count: %d\n' % data.count()) buf.write(' CRS/Projection: %s\n' % data.proj.wkt)
b = data.bounds() buf.write(' Bounds: (%f,%f,%f,%f)\n' % (b.west, b.south, b.east, b.north))
buf.write(' Fields:\n') buf.write('\n'.join([' %s' % repr(fld) for fld in data.schema.fields])) buf.write('\n')
start_response('200 OK', [('Content-type','text/plain')]) return buf.getvalue()
Data Summary Appfrom geoserver.catalog import Layerfrom StringIO import StringIO
def app(env, start_response): kvp = dict([tuple(kv.split('=')) for kv in env['QUERY_STRING'].split('&')]) layer = kvp['layer'] l = Layer(layer, store=None)
buf = StringIO() buf.write('Layer: %s\n' % l.name)
data = l.data buf.write(' Format: %s\n' % data.format) buf.write(' Feature count: %d\n' % data.count()) buf.write(' CRS/Projection: %s\n' % data.proj.wkt)
b = data.bounds() buf.write(' Bounds: (%f,%f,%f,%f)\n' % (b.west, b.south, b.east, b.north))
buf.write(' Fields:\n') buf.write('\n'.join([' %s' % repr(fld) for fld in data.schema.fields])) buf.write('\n')
start_response('200 OK', [('Content-type','text/plain')]) return buf.getvalue()
Data Summary Appfrom geoserver.catalog import Layerfrom StringIO import StringIO
def app(env, start_response): kvp = dict([tuple(kv.split('=')) for kv in env['QUERY_STRING'].split('&')]) layer = kvp['layer'] l = Layer(layer, store=None)
buf = StringIO() buf.write('Layer: %s\n' % l.name)
data = l.data buf.write(' Format: %s\n' % data.format) buf.write(' Feature count: %d\n' % data.count()) buf.write(' CRS/Projection: %s\n' % data.proj.wkt)
b = data.bounds() buf.write(' Bounds: (%f,%f,%f,%f)\n' % (b.west, b.south, b.east, b.north))
buf.write(' Fields:\n') buf.write('\n'.join([' %s' % repr(fld) for fld in data.schema.fields])) buf.write('\n')
start_response('200 OK', [('Content-type','text/plain')]) return buf.getvalue()
Data Summary App
Demo
Fusion Tables DataStore
class GFT(object):
@datastore('GFT', 'Google Fusion Tables', user=('User email', str), passwd=('Password', str)) def __init__(self, user, passwd): token = ClientLogin().authorize(user, passwd) self.ft = ftclient.ClientLoginFTClient(token)
def layers(self): return [tbl.name for tbl in self.tables()] def get(self, layer): try: return Layer(filter(lambda t: t.name == layer, self.tables())[0]) except IndexError: raise Exception('No table named %s' % layer)
def tables(self): tables = self.ft.query(SQL().showTables()) return [Table(self,*row.split(',')) for row in tables.split('\n')[1:-1]]
Fusion Tables DataStore
class GFT(object):
@datastore('GFT', 'Google Fusion Tables', user=('User email', str), passwd=('Password', str)) def __init__(self, user, passwd): token = ClientLogin().authorize(user, passwd) self.ft = ftclient.ClientLoginFTClient(token)
def layers(self): return [tbl.name for tbl in self.tables()] def get(self, layer): try: return Layer(filter(lambda t: t.name == layer, self.tables())[0]) except IndexError: raise Exception('No table named %s' % layer)
def tables(self): tables = self.ft.query(SQL().showTables()) return [Table(self,*row.split(',')) for row in tables.split('\n')[1:-1]]
Fusion Tables DataStore
class GFT(object):
@datastore('GFT', 'Google Fusion Tables', user=('User email', str), passwd=('Password', str)) def __init__(self, user, passwd): token = ClientLogin().authorize(user, passwd) self.ft = ftclient.ClientLoginFTClient(token)
def layers(self): return [tbl.name for tbl in self.tables()] def get(self, layer): try: return Layer(filter(lambda t: t.name == layer, self.tables())[0]) except IndexError: raise Exception('No table named %s' % layer)
def tables(self): tables = self.ft.query(SQL().showTables()) return [Table(self,*row.split(',')) for row in tables.split('\n')[1:-1]]
Fusion Tables DataStore
class GFT(object):
@datastore('GFT', 'Google Fusion Tables', user=('User email', str), passwd=('Password', str)) def __init__(self, user, passwd): token = ClientLogin().authorize(user, passwd) self.ft = ftclient.ClientLoginFTClient(token)
def layers(self): return [tbl.name for tbl in self.tables()] def get(self, layer): try: return Layer(filter(lambda t: t.name == layer, self.tables())[0]) except IndexError: raise Exception('No table named %s' % layer)
def tables(self): tables = self.ft.query(SQL().showTables()) return [Table(self,*row.split(',')) for row in tables.split('\n')[1:-1]]
Fusion Tables DataStore
__types = {'string':str, 'number': float, 'location':Geometry}
class Layer(object):
def __init__(self, tbl): self.tbl = tbl self.name = tbl.name self.workspace = tbl.gft
self.proj = Projection('epsg:4326') self.schema = Schema(tbl.name, [(col[0], __types[col[1]]) for col in tbl.schema()])
def bounds(self): return reduce(lambda x,y: x.expand(y.bounds), self.features(), Bounds())
def features(self): ...
Fusion Tables DataStore
__types = {'string':str, 'number': float, 'location':Geometry}
class Layer(object):
def __init__(self, tbl): self.tbl = tbl self.name = tbl.name self.workspace = tbl.gft
self.proj = Projection('epsg:4326') self.schema = Schema(tbl.name, [(col[0], __types[col[1]]) for col in tbl.schema()])
def bounds(self): return reduce(lambda x,y: x.expand(y.bounds), self.features(), Bounds())
def features(self): ...
Fusion Tables DataStore
__types = {'string':str, 'number': float, 'location':Geometry}
class Layer(object):
def __init__(self, tbl): self.tbl = tbl self.name = tbl.name self.workspace = tbl.gft
self.proj = Projection('epsg:4326') self.schema = Schema(tbl.name, [(col[0], __types[col[1]]) for col in tbl.schema()])
def bounds(self): return reduce(lambda x,y: x.expand(y.bounds), self.features(), Bounds())
def features(self): ...
Fusion Tables DataStore
class Layer(object):
...
def features(self): rows = self.tbl.gft.ft.query(SQL().select(self.tbl.id)) rows = rows.split('\n')[1:-1]
for row in rows: vals = csv.reader([row]).next() atts = [] for i in range(0, len(vals)): val = vals[i] fld = self.schema.get(i) if issubclass(fld.typ, Geometry): val = readKML(val) atts.append(val) yield Feature(atts, schema=self.schema)
Fusion Tables DataStore
Demo
H2 Output Format@vector_format('h2', 'application/zip')def write(data, out): dir = tempfile.mkdtemp()
# create the database and copy the features into it db = H2(data.schema.name, dir=dir) layer = db.create(schema=data.schema) for f in data.features: layer.add(f) db.close()
# zip and ship file = tempfile.mktemp() zip = zipfile.ZipFile(file, 'w') for root, dirs, files in os.walk(dir): name = abspath(root)[len(abspath(dir)):] for f in files: zip.write(join(root,f), join(name,f), zipfile.ZIP_DEFLATED) zip.close()
shutil.copyfileobj(open(file, 'r'), out)
# clean up os.remove(file) shutil.rmtree(dir)
H2 Output Format@vector_format('h2', 'application/zip')def write(data, out): dir = tempfile.mkdtemp()
# create the database and copy the features into it db = H2(data.schema.name, dir=dir) layer = db.create(schema=data.schema) for f in data.features: layer.add(f) db.close()
# zip and ship file = tempfile.mktemp() zip = zipfile.ZipFile(file, 'w') for root, dirs, files in os.walk(dir): name = abspath(root)[len(abspath(dir)):] for f in files: zip.write(join(root,f), join(name,f), zipfile.ZIP_DEFLATED) zip.close()
shutil.copyfileobj(open(file, 'r'), out)
# clean up os.remove(file) shutil.rmtree(dir)
H2 Output Format@vector_format('h2', 'application/zip')def write(data, out): dir = tempfile.mkdtemp()
# create the database and copy the features into it db = H2(data.schema.name, dir=dir) layer = db.create(schema=data.schema) for f in data.features: layer.add(f) db.close()
# zip and ship file = tempfile.mktemp() zip = zipfile.ZipFile(file, 'w') for root, dirs, files in os.walk(dir): name = abspath(root)[len(abspath(dir)):] for f in files: zip.write(join(root,f), join(name,f), zipfile.ZIP_DEFLATED) zip.close()
shutil.copyfileobj(open(file, 'r'), out)
# clean up os.remove(file) shutil.rmtree(dir)
H2 Output Format@vector_format('h2', 'application/zip')def write(data, out): dir = tempfile.mkdtemp()
# create the database and copy the features into it db = H2(data.schema.name, dir=dir) layer = db.create(schema=data.schema) for f in data.features: layer.add(f) db.close()
# zip and ship file = tempfile.mktemp() zip = zipfile.ZipFile(file, 'w') for root, dirs, files in os.walk(dir): name = abspath(root)[len(abspath(dir)):] for f in files: zip.write(join(root,f), join(name,f), zipfile.ZIP_DEFLATED) zip.close()
shutil.copyfileobj(open(file, 'r'), out)
# clean up os.remove(file) shutil.rmtree(dir)
H2 Output Format
Demo
Scripted WPS and WFS Transaction Hooks
Demo
Thanks!
http://geoscript.orghttp://geoserver.org
Questions?