decision support for complex planning challenges · 2014-11-12 · chalmers university of...

Chalmers University of Technology

Decision Support for Complex Planning Challenges:

Combining Expert Systems, Engineering-oriented Modeling, Machine Learning, Information Theory, and

Optimization Technology

Larry M. Deschaine Physical Resource Theory

Department of Energy and Environment Chalmers University of Technology

Dissertation Seminar 27 February, 2014 Faculty Opponent: Peter Dittrich

Friedrich-Schiller-University, Jena, Germany

Supervisor: Peter Nordin Examiner: Kristian Lindgren


Decision Approaches from Ancient times

“The Academy” Plato points upwards to “The Ideals” whilst Aristotle points downward, grounded in observable reality 2


Thesis Objective

• Develop, and evaluate a generalized framework for selecting the most appropriate mix of modeling paradigms for a given problem challenge with focus on integrated modeling

• Evaluate framework utilizing industrial problems concerning energy and environment

– Utilize individual methods of analyses

– Integrate methods using machine learning

3


Generalized Framework

This integrated structured process (Thomistic) combines deduction (Platonic) and induction (Aristotelian) 4

Genetic Programming

Expertise Observations Natural Laws

Deduction moves from the general to the specific

whilst induction typically moves from specific

examples to the general


Thesis Approach: Concepts to Process

Problem–Solution Space

Parse Problem by Type Model Building

Framework Process

5 Approach exploits each category to maximize the information practically obtained from specific analysis.

Integration provides a coherent and comprehensive model


Specification of Analyses Methods

Analysis methods are problem specific

– Subject Matter Expertise (SME): High level analysis, assembles management goals/constraints, stakeholder input, and regulatory requirements

– Engineering Oriented Models: Captures important natural laws

– Data-driven: Provides equation from observed data

– Integration: Combines the disparate information sources into one comprehensive model

Approach leverages all key decision information

6 Integrated approach provides an accuracy floor equal to the most accurate single method


Pre-existing Methods

Analysis Component Category Technique

Subject Matter Expert AHP

Machine Learning CGPS, WEKA, PCA

Numerical Models PTC, BioFT3D, MODHMS, UTCHEM

Machine Learning / Data Fusion CGPS, Kalman Filter w/GSLIB, WEKA

Optimization LP, SLP, SLA, SQP, GA, OA, LGO, MINLP

Information Theory Eckschlager, K., Stepanek, V., (1979), mRMR

For acronym definitions, see thesis: Table 3 (Page 28) 7


Extension of Methods

Knowledge Quadrant Contributions

Subject Matter Expert (Environmental)

Extended Analytical Hierarchy Process (AHP) to use in stochastic optimization. Includes probability distribution functions of expert opinions, constraints and resources. Optimization implemented using genetic algorithm.

Engineering-oriented (Environmental)

Enhanced the Kalman Filtering approach to better integrate with groundwater flow and transport models for optimization of groundwater monitoring programs. Utilizes volumetric instead of point estimates for sample data worth.

8


Development of Methods

Knowledge Quadrant Accomplishments

Integrated Modeling (Environmental)

Developed and validated a geophysical predictive model with increased accuracy thereby reducing the level of effort in characterizing the subsurface

Integrated Modeling (Energetic)

Developed and validated high accuracy munitions of explosive concern identification algorithm

9


Applicability of Research: Problems and Magnitudes

• Environmental: – Groundwater contamination restoration cost estimates

(US Department of Defense) at $22.8 Billion

– Unexploded ordnance impacts approximately 15 million acres, remediation cost $8-35 Billion

• Energy: – The cost and availability of global energy production is

affected by: • Optimal technology selection for power plants

• Cost and location of resources versus point of use

• Business models such as regulated versus economic markets

10

Even small improvements represent large dollar savings; a problem worth tackling


Research Issues: Maximize Utility of Each Category, Optimally Integrate

Issue A: Capturing knowledge from subject matter experts

Issue B: Increasing information obtainable from simulation models and observed data

Issue C: Replacing data with equations

Issue D: Developing and demonstrating integrated modeling

Papers I-III document individual methods testing whilst Paper IV provides integrated method 11


Paper I

Application of the Analytic Hierarchy Process to Compare Alternatives for

the Long-Term Management of Surplus Mercury

Randall PJ, Brown LA, Deschaine LM, DiMarzio J, Kaiser G, and Vierow J Journal of Environmental Management, 71 (2004) 35-43, Elsevier Press

12

Issue A: Capturing Knowledge from Subject Matter Experts


EPA Mercury Retirement Analysis

Challenge & Results

• Culmination of several decision support projects since 1985

• Develop a systematic method for comparing long-term mercury management options

• When all criteria considered, land-filling is the preferred option – Analysis also supports storage

for a few decades until technologies for permanent retirement mature

EPA Technology Transfer Document

Issue A: Capturing Knowledge from Subject Matter Experts

A majority of the analysis value was in deciding on the decision method, developing the approach, and constructing

the framework for the stakeholders to use

AHP Calculations provided in

Appendix A

13


Paper II

Simulation and Optimization of Large Scale Subsurface Environmental

Impacts; Investigations, Remedial Design and Long Term Monitoring

Deschaine LM.

Journal of Mathematical Machines and Systems, Kiev

Number 3, 4. (2003) pages 201-218

Thesis contains an updated and extended version of this paper

14 Paper II Simulation and Optimization of Large Scale Subsurface Environmental Impacts; Investigations, Remedial

Design and Long Term Monitoring

Issue B: Increasing information obtainable from simulation models


Plume Manager (Site wide

Integration)

Plume Investigator

Find Plume

Monitor Plume

Kalman Filter

Plume Simulator

Multiphase

Multicomponent

Flow and Transport Numerical

Codes

Plume Cleaner

Optimal Solutions

Lipschitz Global

Optimization and

Outer Approximation Algorithms

Research Methodology and Results

Results

• Groundwater contaminate

plume monitoring to protect

public drinking water supplies

– 14 000 000 gallons per day

(Amarillo, Texas)

– 32 000 000 gallons per day

(Anniston, Alabama)

• Optimal dewatering design

– Iron Ore Mine, Australia

• $9 100 000 savings when

compared to original design

Site-wide Optimization

Framework

15 Paper II Simulation and Optimization of Large Scale Subsurface Environmental Impacts; Investigations, Remedial

Design and Long Term Monitoring



Technology Transfer Guidance Documents Developed and Distributed

Department of Energy Technical Advisory Group

Interstate Technology and Regulatory Council


Technology transfer documents provide the technical information to implement these approaches

Chapter 9

16


Paper III

Extending the Boundaries of Design Optimization by Integrating Fast

Optimization Techniques with Machine-Code-Based Linear Genetic

Programming

Francone FD and Deschaine LM

Information Sciences Journal, Elsevier Press, Vol. 161(3-4), pages 99–120,

(2004), Amsterdam, the Netherlands

17 Paper III: Extending the Boundaries of Design Optimization by Integrating Fast Optimization Techniques with

Machine-Code-Based Linear Genetic Programming



Genetic Programming: Ability to Induce Engineering and Scientific Natural Laws

From Data (Darcy’s Law)

Genetic Programming evolved model matched Darcy’s Law

Q=K*I*A

where: Q = flow [L3/T]

K =hydraulic conductivity[L/T]

I=gradient [L/L]

A = area [L2]

Acceleration of Kodak’s Production Model (from hrs to milliseconds)

Data Set Data Set Size

Training 2506

Testing 2520

Blind Validation 2521

Fitness [R2]

Single Solution

Team Solution

Training 0.9934 0.9975

Validation 0.9893 0.9939

Applied 0.9783 0.9889

Paper III: Extending the Boundaries of Design Optimization by Integrating Fast Optimization Techniques with

Machine-Code-Based Linear Genetic Programming (Compiling Genetic Programming System, Nordin, 1995)

18


R2 = Coefficient of Determination, the

percent of the explainable variation

between data and model


Is Evolutionary Computation Alone Sufficient?

• Soils Hydraulic Determination: Cone Penetrometer Study (Department of Energy)

– Subject matter expertise and symbolic equations

• R2 = 0.24-0.40

– Genetic Programming using observed data

• R2 = 0.60

– Integrated Genetic Programming model utilizing subject matter expertise, data and scientific inputs met performance specifications

• R2 = 0.72

19

Sometimes yes, but not always

Issue D: Integrated Modeling


Paper IV

A Computational Geometric / Information Theoretic Method to Invert Physics-Based MEC Models Attributes

for MEC Discrimination

Deschaine LM, Nordin JP, and Pinter JD Journal of Mathematical Machines and Systems,

Kiev. Number 2, (2011), (pages 50-61) Thesis contains an updated and extended version of this paper

20 Paper IV: A Computational Geometric / Information Theoretic Method to Invert Physics-Based MEC Models Attributes




Research Questions and Methodology

Research Question

– Is high accuracy MEC discrimination possible using non-destructive field testing methods and an integrated decision support approach?

– If so, will the approach provide measurable benefit?

• Acceptability

• Performance

Munitions and Explosives of Concern (MEC)





Underground Explosives Identification


for MEC Discrimination (From: Deschaine LM, et. al (2002)

Desired accuracy

Demonstration Test Site: U.S. Army Jefferson Proving Ground, Indiana

Best

result




for MEC Discrimination (From: Deschaine LM, et. al (2002)

Demonstration Test Site: U.S. Army Jefferson Proving Ground, Indiana

Underground Explosives Identification Desired accuracy

Genetic

Programming

Solution


Fully Integrated Approach Camp Sibert, Alabama, USA

Sources of Discrimination Information

Case SME-Based Data-Driven Physically

Based

Solution

MEC

(yes/no)

e1 SME1(i=1…a) DD1(i=1…b) PB1(i=1…c) L1



. . . . .

. . . . .

. . . . .

en SMEn(i=1…a) DDn(i=1…b) PBn(i=1…c) Ln

Paper IV: A Computational Geometric / Information Theoretic Method to Invert Physics-Based MEC Models Attributes


24

SME-Based

Physically Based Data-Driven

Information Components

Underground Explosives Identification Discrimination: Repeatability Assessment

SME= Locate Targets of Interest: 174 •67 “live” and 107 “inert” objects

Data Driven features: EMI = 551 Inverse Physics features = 7


Physics Inversion and Computational Geometric Approach: Camp Sibert, Alabama, USA

Physics Model Inversion Computational Geometry

25

Partitioned ellipses are the foundation for the 551 discrimination features

Paper IV: A Computational Geometric / Information Theoretic Method to Invert Physics-Based MEC Models Attributes




Comparison: Physics Inversion vs. Computational Geometry (EMI)

ROC = Receiver Operator Characteristics curve •Most Accurate results when curve follows the upper left hand border •For explosives removal, most important is when last live item removed

26

Final unexploded item removed at 30% false positive

Detailed examination: Almost all information content in the inverse physics model subsists in the geometric method




Comparison: Physics Inversion vs. Computational Geometry (MAG)

This time, inverse physics outperformed data-only Features Physics =6 Data = 82

27 Detailed examination: Information content utilized from both the inverse physics geometric methods





Combined Physics Inversion and Computational Geometry (MAG)

Both examples, the integrated modeling performed best •Efficiency gain •Redundancy of techniques guards from errors

28



Detailed examination: information content utilized from both the inverse physics geometric methods


Candidate’s Algorithm Contributions

• Acceptability: – Subsurface unexploded items identification process

was certified by the U.S. Office of the Secretary of Defense in 2009

• Performance: – Discovered Genetic Programming as a viable method

for high accuracy discrimination of unexploded items from clutter (2002)

– Extended data-driven discrimination method using Computational Geometric approach (2011)

– Integrated method performs as well or better than individual methods




Summary

Decision approach assists one to

• Understand system interactions between different information components

• Quantitatively assess value of various solution approaches

• Produces well informed solutions for use in decision making

30


Conclusions • The principle objective of this thesis was to

provide a framework for selecting the most appropriate mix of modeling paradigms for a given problem challenge

– Integrated approach demonstrated is general and validated on a geotechnical, and underground explosive identification problems

– Realized benefits include cost savings, design efficiency, stakeholder confidence, gains in worker safety, and published guidance documents

31 Thesis provides representative solutions, additional examples found in Candidate’s over 100 published works


Thank you for your attention!

30


Future Directions Electrical Power Grid Challenge

PJM Interconnection: • Regional Power Grid Operator

• 61 million customers – (20% of U.S.)

• 185,600 MWs capacity

• ~5,000 power generators

• PJM delivered 682

terawatt-hours of electricity

in 2009

A1 Paper III: Extending the Boundaries of Design Optimization by Integrating Fast Optimization Techniques with

Machine-Code-Based Linear Genetic Programming (extended example: Ott, 2010)

Issue C&D: Large Scale Machine Learning Development and Deployment


Predictive Power Generation Model

A2

AGM Model Real-time Generator MW Supply Prediction (20-min)

CONSUMER DEMAND

PJM Optimal Dispatch Decisions

Historical Response from

Power Resource

Generators’ Produce

Megawatts

• Probabilistic response and Adaptive Generation Modeling (AGM) with machine learning. On-the-fly adaptive learning capability to accurately predict generators’ response to economic dispatch signals

Paper III: Extending the Boundaries of Design Optimization by Integrating Fast Optimization Techniques with

Machine-Code-Based Linear Genetic Programming (extended example: Ott, 2010)


Feedback loop: Dispatch signal, generator response, predicted and actual power consumption, weather, data, grid status, pricing. Data retention (1-year) Frequency: 1-minute intervals


PJM Blind Test – Predict MW (t+20min) PJM Interconnection

GPM Model Candidate’s

AGM Model

Identifies limits of single method approach and need for integrated approach (graphic adapted from Ott, 2010) A3

Meg

awat

t O

utp

ut

Actual Time of Day Actual Time of Day

Meg

awat

t O

utp

ut

Elapsed Time (Hrs)

8 0 4

Elapsed Time (Hrs)

8 0 4

Machine

Learning (data

only)

Integrated

models

(incorporates

plant physical

performance

knowledge)
