AI4EO WILDFIRES SUMMARY TECHNICAL NOTE

MAY 2021

- 1 -

Public

Introduction

The Artificial Intelligence for Earth Observation (AI4EO) Wildfires project began in December 2020 and will run for 8 months, focussing on developing and demonstrating a burned area (BA) mapping service that combines EO data, specifically Sentinel-2 optical data, with an AI-enabled algorithm. The use of AI within a data processing service is expected to provide improvements on the more traditional approach and therefore help target specific user requirements.

The project consortium is led by CGI UK, utilising their legacy of developing cloud-based EO-data processing portals, with project partner University of Leicester (UK) who has been involved in a number of projects focused on wildfire mapping including the European Space Agency (ESA) CCI Fire project.

The resulting demonstration service will be showcase on the EO4SD Lab, an online portal that provides a range of tools and processing services along within extensive EO data archives. The aim of the EO4SD Lab is to provide easy to use tools and services to facilitate and support greater use of EO by the sustainable development user community. Within the context of AI4EO users will be able to run the demonstration burned area mapping service on data of interest along with the full processing tools and services provided by the EO4SD Lab.

Importance of a robust wildfire monitoring

Apart from Antarctica, fire is a presence in most countries of the world, from the densest tropical rainforests (such as the fires recently reported in Brazil) to the low biomass tundra in northern Russia. The global distribution of fires was first reported by Tansey et al. i ii when SPOT-VGT was used to map global fires at a resolution of 1km. Since then global fire products have continued to be developed by the MODIS Science Teams and through the Copernicus Global Land Service and are now available on a near daily basis at resolutions of around 300miii. In the natural environment, the impact of fire is dictated by weather conditions, the amount of fuel, fuel moisture, the spatial distribution of the fuel on the surface, the ignition source and location. The resulting severity of the fire on the landscape can be quite different. Furthermore, the recovery of the landscape after the fire can be of the order of weeks for a tropical savannah environment to several years in the case of a boreal forest. The assessment of the frequency, severity and impact of fires in forested landscapes is of high importance for biodiversity considerationsiv and carbon loss assessmentv.

Project Objectives & activities

The aim of this project is to showcase how EO data can support wildfire management through the development and demonstration of a pre-operational wildfire burn scar detection service that uses a machine learning (ML) algorithm. This will provide two new aspects of functionality, firstly enabling more effective use of current operational platforms (particularly Copernicus’s Sentinel-2), which provides improved data availability. Secondly the use of AI has the potential to improve the service’s accuracy in distinguishing between burned and non-burned areas. The AI algorithm will be deployed alongside extensive EO data and scalable processing capabilities to allow service demonstrations relevant for the user community.

The ultimate goal is to provide a service that uses machine learning to help meet the performance and operational needs of the fire monitoring community.

- 2 -

Public

Project timeline

The project will run from December 2020 until September 2021. Key activities will include:

Task 1: User engagement & requirements consolidation. Engagement with users took place in early 2021 to record and consolidate their requirements.

Task 2: Service Specification. The overall service needed to meet the key user-defined requirements

took place alongside Task 1 in early 2021.

Task 3: Service Implementation. Technical activities to develop and test the algorithm and subsequent deployment of the service onto the EO4SD Lab. This will take place in between May and June 2021.

Task 4: Service demonstration. To demonstrate and make available to users, the pre-operational service on the EO4SD Lab. This will take place in between July and September 2021.

Task 5: Conclusions and next steps. To focus on summarising how the service helps meet user needs and define the next steps needed to support use uptake. This will place in August and September 2021.

Key service requirements

Based on user discussions and experience in previous wildfire projects, the AI4EO team have determined the key service requirements, as listed below:

Provision of a value-added (VA) product, specially a map that correctly identifies burned regions within a user-defined AOI.

As well as burned area extent, a burn severity layer is also highly important.

Service should process appropriate input EO data that provides good temporal and geographic coverage along with the most appropriate spatial, spectral and temporal frequency.

The service should utilise open and freely available data and software when possible.

The initial demonstration service should be freely accessible to users, although it would be expected that a subsequent service may be offered commercially.

The service should be available to user’s on-demand with users selecting the location and time-period of interest.

Users should also be able to setup the service for routine / scheduled processing by pre-defining locations and specific time periods in advance.

The service should be able to process a suitably large level of data, both in terms of geographic extent and temporal range.

Direct access to a wide archive of input EO data, which should be up-to-date.

Resulting products should be easily accessible to users.

- 3 -

Public

BA products should be in a data format that allows for easy ingestion into existing user systems. They also should have suitable metadata, including details for each burned area polygon identified such as burned area and unique reference identifier. Provision of information related to provenance and overall quality are also important.

The validity and relative importance of these goals will be explored within the user engagement activities of the project.

Use of AI within Earth Observation

This project’s focus is to use AI, in the form of machine learning (ML), to create the AI-Burned Area Mapping (AIBAM) service that helps meet user requirements. This is based on an underlying assumption that AI provides advantages in the development of BA products. This section will provide an overview of recent work that justifies this assumption. The advancement in the use of ML within this project is the ability of the algorithm to map burned area across different vegetation types, burning conditions and size of fire. Importantly, it is not always possible to view the surface in its state that is unburned or burned because of cloud cover, fire smoke, haze or missing data. It is intended that this project will use ML to make every attempt to determine if an area is burned or unburned using all available imagery.

What are the benefits of using AI within EO?

The monitoring of wildfires is a globally important activity, particularly with the increased relevance due to climate change. Whilst considerable effort has been put into improving global burned area algorithms, accuracy levels are still only around 60%vi. This is because many fires are not visible to optical satellites due to cloud cover, low severity and, importantly, the size of the burned area. There is also large uncertainty of using 300m resolution products to determine the actual burned area that are required by many user organisations to determine compensation funds, remediation and recovery operations. It is therefore necessary to generate products at higher resolution with dedicated properties that are relevant for a range of users. However, to manage the variable nature of fire occurrence at a number of regional locations across the globe, multiple stand-alone satellite detection algorithm solutions would be needed. Such an approach would be impractical and expensive. However, ML approaches might offer a generic solution, but only if training data characterising the breadth of fire activity across the globe. The training data will contain examples of different types of fire, severities of fire and will remove the need for additional on the ground, immediate training data which may be difficult or dangerous to collect.

As part of ESA’s Climate Change Initiative (CCI), the University of Leicester, generated a vast reference data set from Landsat TM image pairsvii. Over a 1000 images were processed to derive a validation data set for global burned map products. The reference data was used to characterise the latest global burned area products developed through the ESA CCIviii. The reference data set will be used as a training data for ML algorithms because of its unprecedented spatial and temporal

distribution that characterise the nature of vegetation fires globally.

- 4 -

Public

By training results on multiple previous scenes, the AIBAM algorithm will be able to identify patterns of spatial and spectral characteristics that can indicate a burned area. This will be more sophisticated than other services, which simply apply a threshold on change images between two scenes. An extensive training data set will help us to understand the nature of the impact of fire on vegetated landscaped in Australia and France.

What AI algorithms are suitable?

The use of ML approaches to classify burned area is quite widespread. Most of the research has been at regional and landscape scales. This reflects the challenge and difficulty in generating high quality training data. There is a further difficulty in acquiring data to support the validation and accuracy assessment methods. For example, Mithal et alix developed a novel ML framework to look at tropical forest burning in SE Asia and South America using MODIS data. They used a method referred to as RAPT (Rare Class Prediction in the Absence of True Labels). It is true that a burned area class is relatively rare, but it can also be quite common in extensively burned area within a Sentinel-2 or Landsat frame. They evaluated using Landsat image reference frames. Knopp et alx develop a deep-learning approach using a convolutional neural network to perform segmentation tasks of burned area using Sentinel-2 data and a new training and validation data set to undertake the study. They test the method on three small fires located in Europe with good outcomes. A further deep-learning approach was built and tested by Pinto et alxi using VIIRS data. Their approach was favourable towards satellite data that were available on a daily basis which was important for the training component of the approach. In this paper, identifying the date of burning was also important and justified the use of a sensor providing daily imagery.

The University of Leicester have been working on a number of approaches to data fusion methods and algorithm training and development using ML approaches. These include:

IPSO-BP: Back Propagation (BP) neural network and the Improved Particle Swarm Optimization

algorithm (IPSO)-BP neural network)xii This work was undertaken in the context of agricultural yield

assessment.

ESTARFM: Enhanced Spatial and Temporal Adaptive Reflectance Fusion Modelxiii. This work focused

on data fusion techniques between MODIS scaled imagery and Sentinel-2 scale imagery. Data fusion

techniques may be relevant as we explore differences between Landsat and Sentinel-2 data in this

project.

RF: Random Forest in addition to the specific reference data processingxiv. This work focused on the use

of Random Forest ML methods to estimates burned area with both Sentinel-1 and Sentinel-2 data.

SVD: Spatiotemporal data fusion and singular vector decompositionxv. This work focused on data fusion

techniques to improve crop yield estimates with MODIS, Sentinel-3 and Sentinel-2 data.

Preferred AI algorithms

Based on the above analysis it is proposed to use the RF and SVM algorithms. These are chosen as the development risk is known and deemed to be low and hence enabling the creation of a service within the project timescale. The RF approach is also a tried and tested method for burned area mappingvii xiv. It is the intention of this project to make the code implementation sufficiently generic to allow the swap in-swap out of different ML approaches that potentially allows for optimisation of an existing method or the testing of a new one.

The AI4EO Wildfires project team are currently evaluating the ML problems, particularly the implication of working with Sentinel-2, and then evaluate the performance characteristics and requirements and evaluate the most appropriate method to test. We can then test against RF, Support Vector Machines or a different ML approach to determine if any improvements are made due to the specificities of the problem to be solved. All of this will be done as part of the algorithm development process with the best performing algorithm(s) being provided in the demonstration service.

- 5 -

Public

AI algorithm development and testing

Once the specific AI algorithms have been selected these need to be developed and tested prior to deployment as a service on the EO4SD Lab. this will involve a number of steps:

1. Preparing the training dataset: Previously UoL have developed an extensive training dataset as part of the ESA CCI fire project. This dataset was primarily made up of Landsat-7 data, and as this project is focusing on using Sentinel-2 data this dataset will need to be amended to include Sentinel-2 data. We have developed a tool where image pairs from high resolution data can easily be imported and training data can be collected. Over 200 pairs of Landsat images were investigated over the continent of Australia because of the propensity of vegetation to burn. To minimise variations between scenes, standard pre-processing steps are applied.

2. Updating existing algorithms: As well as updating the training data, the AI algorithms will need to be updated. Currently UoL have an existing RF tool developed to work on Landsat 5, 7 and 8 at a global scale. This will need to be updated to use Sentinel-2 data either using a data fusion model or looking at existing fusion data sets such as the one described by Wang et alxvi. Training data has / is being collected from over 100 Sentinel-2 images covering a wide range of fire-affected vegetated regions in Australia and France.

3. Algorithm testing: validation activities to ensure that the developed AI algorithm is performing as expected will be done in two steps:

a) Firstly, a selection of the ML parameterisation data will be applied to mono-temporal and bi-temporal images from Sentinel-2 to detect changes where the algorithms have not been previously trained. We will evaluate the performance against manual inspection using the tool we have developed. As well as different ML algorithms within a traditional bi-temporal (comparing pre- and post-event images), within the project a novel approach of a mono-temporal (single post-event image) will be evaluated.

b) Secondly, user-provided fire locations will be used for additional validation. This is important as it will provide an assessment of the quality of the algorithm over scenarios of interest to the users. Within the project key users are from Australia (GeoScience Australia) and France (ONF), which will enable a range of scenarios to be explored. User will be asked for details (namely a set of fire polygons with information on the date and period of burning) related to significant and representative, fires during the user engagement activities. Below is an example of a 2019 wildfire in Stirling Range National Park, Australia mapped with the RF classifier applied to pre- and post-fire Sentinel-2 images.

- 6 -

Public

Hosting the AIBAM demonstration service

Users will be able to access the AIBAM service through the EO4SD Lab1, which is an online-portal that provides access to an extensive archive of EO data along with tools and services used to derive useful information from such data. The AIBAM service will be deployed both as an on-demand service (in which the user selected the pre- and post-fire images to be processed to generated a single products) and a systematic service (in which the user selected a region of interest and a start and end date and the service will generated multiple products from all available data in that region). A user will normally select an on-demand service to investigate known fires , whereas the systematic approach allows for continuous monitoring of a specified region and provides users with information of the evolution of burn scars over a given time period.

To supplement the AIBAM service, the EO4SD Lab currently has a range of other services relevant to fire monitoring. These services are described below. The figure below shows a burned area map of the Chernobyl, Ukraine from April 2020. This map was generated by creating a difference Normalised Burn Ratio (dNBR) image from two Sentinel-2 image and applying the USGS colour scale for burn severity.

Service Description & relevance for wildfire monitoring Service Offering

AIBAM Uses ML to create a burned area map using Sentinel-2 images. A secondary layers of burn severity will also be provided. This will allow an understanding of

both Fire severity and fire extent.

On-demand and systematic

Burned Area

Creates a difference Normalized Burn Ratio (ΔNBR) from two Sentinel-2 images taken from before and after a fire event. The NBR uses a combination of NIR and SWIR bands to monitor vegetation canopy structure, with a strong decrease indicating significant removal of canopy (such as in a fire event). Standard thresholds as defined by the United States Geological Service (USGS), are applied to indicate burn severity.

On-demand and systematic

Active Fire

Combine Sentinel-2 visible, near infrared and short-wave infrared spectral bands to emphasize regions of active fire. This is not a classification, rather an image

can be used to help visually identify regions of active fire.

On-demand and systematic

1 https://eo4sd-lab.net/app/

- 7 -

Public

Service Description & relevance for wildfire monitoring Service Offering

RASTER Generates full resolution RGB (optical) and single-channel (SAR) GeoTIFF images from EO data, which can be visually interpreted.

On-demand and systematic

COMBI This service allows users to create new 3-band images that combine data from

multiple EO data products or acquisitions. This can be helpful to visually identify particular features or to highlight specific changes over time.

On-demand

COIN This service provides geocoded composites of coherence and amplitude images from a pair of Sentinel-1 TOPSAR IW data pairs. These change images can be

used to explore the surface chances caused by a fire event.

On-demand and systematic

S2 NDVI This service calculates Normalised Difference Vegetation Index (NDVI) from a Sentinel-2 image. NDVI is a highly common indices that highlights the presence

of green vegetation, using red and near infrared bands. .Additional radiometric indices, including SAVI, RVI and NDWI are also available on the EO4SD Lab.

On-demand and systematic

How to access the AIBAM service?

The AIBAM service will be deployed on EO4SD Lab. This gives the user a simple interface to define the input data / region of interest / time period along with a few other key parameters to execute the service and generate (and visualise) the results. Users can request free EO4SD Lab accounts, which can be done via email2 or via the contact page3.

Next steps?

The next steps for the AI4EO team will include:

Analysis and review of potential AI algorithms with preliminary selection of algorithms to be evaluated with test data

Deployment of the demonstration AIBAM service

Contact Details

It you would like to know more please contact the AI4EO Wildfires project team, whose details are provided below:

Project Manager: Clive Farquhar (CGI) clive.farquhar@cgi.com

Science Lead: Kevin Tansey (CGI) kjt7@leicester.ac.uk

Technical Lead: Susana Baena (CGI) susana.baena@cgi.com

2 support@eo4sd-lab.net 3 https://eo4sd-lab.net/contact

- 8 -

Public

Bibliography

i Tansey, K., Grégoire, J-M., Stroppiana, D., Sousa, A., Silva, J.M.N., Pereira, J.M.C., Boschetti, L., Maggi, M., Brivio, P.A., Fraser, R., Flasse, S., Ershov, D., Binaghi, E., Graetz, D. and Peduzzi, P., 2004, Vegetation burning in the year 2000: Global burned area estimates from SPOT VEGETATION data. Journal of Geophysical Research - Atmospheres, 109, D14S03, doi:10.1029/2003JD003598

ii Tansey, K., Grégoire, J.-M., Defourny, P., Leigh, R., Pekel, J.-F., van Bogaert, E., and Bartholomé, E., 2008, A new, global, multi-annual (2000–2007) burned area product at 1 km resolution. Geophysical Research Letters, 35, L01401, doi:10.1029/2007GL031567.

iii Chuvieco, E., Yue, C., Heil, A., Mouillot, F., Alonso-Canas, I., Padilla, M., Pereira, J. M., Oom, D., and Tansey, K., 2016, A new global burned area product for climate assessment of fire impacts. Global Ecology and Biogeography, 25, 619–629, doi: 10.1111/geb.12440 iv Hoscilo, A., Tansey, K., and Page,S.E., 2013, Post-fire vegetation response as a proxy to quantify the magnitude of burn severity in tropical peatland. International Journal of Remote Sensing, 34, 412-433, doi:10.1080/01431161.2012.709328

v Konecny, K., Ballhorn, U., Navratil, P., Jubanski, J., Page, S.E., Tansey, K., Hooijer, A., Vernimmen, R., and Siegert, F., 2016, Variable carbon losses from recurrent fires in drained tropical peatlands. Global Change Biology, 22, 1469-1480, doi: 10.1111/gcb.13186

vi Padilla, M., Stehman, S.V., Ramo, R., Corti, D., Hantson, S., Oliva, P., Alonso-Canas, I., Bradley, A.V., Tansey, K., Mota, B., Pereira, J.M., and Chuvieco, E., 2015, Comparing the accuracies of remote sensing global burned area products using stratified random sampling and estimation. Remote Sensing of Environment, 160, 114-121, doi: 10.1016/j.rse.2015.01.005 vii Padilla, M., Olofsson, P., Stehman, S.V., Tansey, K., and Chuvieco, E., 2017, Stratification and sample allocation for reference burned area data. Remote Sensing of Environment, 203, 240-255. doi:10.1016/j.rse.2017.06.041

viii Chuvieco, E., Lizundia-Loiola, J., Lucrecia Pettinari, M., Ramo, R., Padilla, M., Tansey, K., Mouillot, F., Laurent, P., Storm, T., Heil, A., and Plummer, S., 2018, Generation and analysis of a new global burned area product based on MODIS 250m reflectance bands and thermal anomalies. Earth Syst. Sci. Data, 10, 2015–2031, doi: 10.5194/essd-10-2015-2018

ix Mithal, V., Nayak, G., Khandelwal, A., Kumar, V., Nemani, R., Oza, N.C., 2018, Mapping burned areas in tropical forests using a novel machine learning framework. Remote Sensing, 2018, 10(1), 69. x Knopp, L., Wieland, M., Rättich, M., Martinis, S, 2020, A deep learning approach for burned area segmentation with Sentinel-2 data. Remote Sensing, 2020, 12, 2422 xi Pinto, M.M., Libonati, R., Trigo, R.M., Trigo, I.F., and DaCamara, C.C., 2020, A deep learning approach for mapping and dating burned areas using temporal sequences of satellite images. Isprs J. Photogramm. Remote Sens. 2020, 160, 260–274 xii Tian, H., Wang, P., Tansey, K., Zhang, S., Zhang, J., and Li, H., 2020, An IPSO-BP neural network for estimating wheat yield using two remotely sensed variables in the Guanzhong Plain, PR China. Computers and Electronics in Agriculture, 169, 105180, doi.org/10.1016/j.compag.2019.105180

xiii Zhou, X., Wang, P., Tansey, K., Zhang, S., Li, H., and Wang, L., 2020a, Developing a fused vegetation temperature condition index for drought monitoring at field scales using Sentinel-2 and MODIS imagery. Computers and Electronics in Agriculture, 168, 105144, doi.org/10.1016/j.compag.2019.105144.

xiv Tanase, M.A., Belenguer-Plomer, M.A., Roteta, E., Bastarrika, A., Wheeler, J., Fernández-Carrillo, Á., Tansey, K., Wiedemann, W., Navratil, P., Lohberger, S., Siegert, F., and Chuvieco, E., 2020, Burned area

- 9 -

Public

detection and mapping: Intercomparison of Sentinel-1 and Sentinel-2 based algorithms over tropical Africa. Remote Sensing, 12, 334, doi.org/10.3390/rs12020334 xv Zhou, X., Wang, P., Tansey, K., Zhang, S., Li, H., Tian, H., 2020b, Reconstruction of time series leaf area index for improving wheat yield estimates at field scales by fusion of Sentinel-2, -3 and MODIS imagery. Computers and Electronics in Agriculture, 177, 105692, https://doi.org/10.1016/j.compag.2020.105692 xvi Wang, Q., Blackburn, G.A., Onojeghuo, A.O., Dash, J., Zhou, L., Zhang, Y., and Atkinson, P.M., 2017, Fusion of Landsat 8 OLI and Sentinel-2 MSI data. IEEE Trans. Geosci. Remote Sens., 55, 3885–3899