adapting multicomponent predictive systems using hybrid adaptation strategies with auto-weka in...

Adapting Multicomponent Predictive Systems using Hybrid Adaptation with Auto-WEKA

in Process Industry

Manuel Martín Salvador, Marcin Budka and Bogdan Gabrys{msalvador,mbudka,bgabrys}@bournemouth.ac.uk

Data Science Institute, Bournemouth University

AutoML @ ICML 2016New York, USAJune 24th, 2016

Example of MCPS with parallel paths

dummy dummy

i o

Random Feature Selection

RandomSubspace

Decision Tree

Mean

Maintaining an MCPS● Data distribution can change over time and affect predictions

○ External factors (e.g. weather conditions, new regulations)○ Internal factors (e.g. quality of materials, equipment deterioration)

Source: INFER project

Training and testing process

1. Training data is provided

2. Best MCPS found is selected

3. New batch of unlabelled data requires prediction

4. MCPS generates predictions

5. True labels are provided

6. Predictive accuracy is reported

7. MCPS is adapted using the last batch of labelled data

Evaluated strategies

Datasets from chemical production processes

Average classification error (%)

Average classification error per batch (%)

BaselineBatchBatch+SMACCumulativeCumulative+SMAC

drierthermalox

Batch adaptation doesn’t help! :(

Batch adaptation does help! :)

MCPS similarity analysis

Batch+SMAC Cumulative+SMAC

catalyst catalyst

Same components, only hyperparameters are adapted

Large difference between batches

Thanks!Paper: http://bit.ly/adapting-mcps-paper

Manuel Martín Salvadormsalvador@bournemouth.ac.uk

@draxus