big data for a big country - un-ggimggim.un.org/meetings/2016-4th_hlf/documents/1-dr minchin... ·...
TRANSCRIPT
![Page 1: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/1.jpg)
Big Data for a Big Country: The Australian Geoscience Data Cube
Dr. Stuart Minchin, Geoscience Australia
![Page 2: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/2.jpg)
Insert document footer here
![Page 3: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/3.jpg)
Traditional remote sensing process
![Page 4: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/4.jpg)
Developing the Australian Geoscience Data Cube
• Earth Observation data are
calibrated to surface reflectance
observations
• Calibration of data to a single
uniform standard allows the
comparison of data through time
• In the AGDC every unique
observation is kept and included
for analysis creating very dense
time-series
![Page 5: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/5.jpg)
![Page 6: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/6.jpg)
Surface Water Menindee lakes
40 billion observations
15 minutes processing time
![Page 7: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/7.jpg)
• 27 Years of data
• 25m N ominal Pixel Resolution
• Approx. 300 000 individual scenes
in approx. 20 000 passes
• Entire 27 years of 1 312 087
ARG25 tiles => 93x1012 pixels
visited
• 0.75 PB of data
• 3 hrs at NCI
Continental Scale Water Observations from Space
![Page 8: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/8.jpg)
Cubbie Station: floodplain water storages
![Page 9: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/9.jpg)
Cubbie Station: floodplain water storages
![Page 10: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/10.jpg)
Cubbie Station: floodplain water storages
![Page 11: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/11.jpg)
Cubbie Station: floodplain water storages
![Page 12: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/12.jpg)
Coastal Change Detection
Aquatic Applications on the AGDC
1988 Landsat 5 First Water Observation Anomaly 2013 Landsat 7 Last Water Observation Anomaly
![Page 13: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/13.jpg)
Coastal Change Detection
Aquatic Applications on the AGDC
1988 Landsat 5 First Water Observation Anomaly 2013 Landsat 7 Last Water Observation Anomaly
![Page 14: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/14.jpg)
Coastal Change Detection
Aquatic Applications on the AGDC
1988 Landsat 5 First Water Observation Anomaly 2013 Landsat 7 Last Water Observation Anomaly
![Page 15: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/15.jpg)
Coastal Change Detection
Aquatic Applications on the AGDC
1988 Landsat 5 First Water Observation Anomaly 2013 Landsat 7 Last Water Observation Anomaly
![Page 16: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/16.jpg)
Mangrove change detection:
Timing and
strength of
change
for any (data
cube) variable
National &
regional shoreline
erosion studies,
mangrove
change mapping
A B
![Page 17: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/17.jpg)
Multiple Algorithms: Time series data extraction
![Page 18: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/18.jpg)
Tracking agricultural change
![Page 19: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/19.jpg)
Water quality monitoring: Lake Burley Griffin
![Page 20: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/20.jpg)
Tracking development conditions
![Page 21: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/21.jpg)
Some prototyped applications for the data cube:
• Vegetation change, agricultural production
• Flood inundation mapping, farm dam development
• Wetland management and characterisation
• Carbon accounting
• Seagrass and substrate mapping
• Coastal change and water quality
• Shallow water bathymetry
• Mining footprint and urban development
• Bushfire scar mapping and forestry inventory
![Page 22: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/22.jpg)
Big Data for a Big Planet: a global network of regional data
cubes?
Data Cubes for: Africa, Antarctica, South
America, Europe, North
America, …
Connecting the EO, Spatial and Statistical
world to support global SDGs?
![Page 23: Big Data for a Big Country - UN-GGIMggim.un.org/meetings/2016-4th_HLF/documents/1-Dr Minchin... · 2017-10-16 · •27 Years of data • 25m N ominal Pixel Resolution • Approx](https://reader033.vdocuments.net/reader033/viewer/2022042202/5ea361be5f7ffe47fe1e1553/html5/thumbnails/23.jpg)
Thankyou
Open source code
Built with open data
Built with open standards