kriging tutorial
DESCRIPTION
Kriging tutorialTRANSCRIPT
2005 NA ADV Committee Initiatives #4: “Complete analysis of reasons
for design change”* .
Don Jones
*
Gain insight
Quantify tradeoffs
Facilitates multi-disciplinary balancing since each discipline has “bottled up” version of analysis and can react real-time to change proposals
Robust design
Adjust control parameters to desensitize design to variation, shift mean
Explore benefits of tolerance reductions
*
“Standard” response-surface methods fit model by least squares
Polynomial regression
Neural networks (back propagation = steepest descent on squared error)
Two reasons to use kriging for computer experiments
Standard methods were developed for physical experiments, which have very different characteristics and challenges than computer experiments.
Least-squares methods have a little-discussed theoretical assumption that is patently false in the case of computer experiments.
*
Physical
(usually 2 or 3)
Can sometimes sample all or nearly all combinations.
Key challenge is to separate the effect of the control variables from noise variables
Computer
Variables usually are continuous
Input-output relationships are nonlinear, a few levels won’t do
No way to cover the entire design space.
*
Regression:
Try to capture as much as possible with many polynomial terms, selected via stepwise procedure from large set.
Kriging:
Don’t spend much time guessing functional form. Use either constant or linear model.
ERROR TERM:
unpredictable. Can think of as measurement error.
Kriging
Errors are treated as left out terms in x that cannot be precisely predicted but can be roughly estimated
Kriging and regression are actually sub-cases of the same general statistical model.
*
Captures nonlinearity without the need to assume functional form
If simulation is very nonlinear — and if a sufficient number of points are sampled — kriging will be more accurate than regression
Interpolates sampled points
At the sampled points, predicts the sampled value, as it should
Provides confidence intervals with desirable properties
intervals are bigger in the gaps between sampled points
intervals shrink to zero at the sampled points
Provides way to infer if we have enough points
All this will become clear as we go along…
*
When do polynomials fit poorly?
Polynomial response surfaces may not fit well if the surface has sweet spot, asymptotes, or similar nonlinear shapes.
Kriging can “pick up” these shapes from data if one has enough points because it is not constrained by an assumed form.
*
Constructing a kriging surface
*
Constructing a kriging surface
*
good approximation should
reasonably smooth.
*
Constructing a kriging surface
Kriging can be interpreted as a statistically-based way to generate many such interpolators.
Here we show 30.
*
Constructing a kriging surface
The value of the kriging surface at a given point is the mean of the values for all the possible interpolators.
*
Kriging confidence intervals
At any point, we can not only compute the mean of the possible interpolators, but also compute the standard deviation. From this we can compute upper and lower confidence intervals.
*
Cross-validating the surface
To check if the confidence intervals can be trusted, we can refit the surface with one of the points left out…here the fourth point.
If the surface is working, the value of the left-out sampled point should lie within the confidence interval.
*
Effect of adding more points
Because kriging interpolates, as you add points the surface will begin to “snap” to the function.
If the surface “doesn’t change much” when more points are added, you can assume that you have “converged.”
Idea due to Jian Tu, GM R&D
*
Sequenced sampling
Checking for convergence requires a “sequenced sample,” that is, the points should gradually fill in the space as they are added.
The available Kriging Wizard software provides such sequenced DOEs.
First 50 points
First 75 points
All 100 points.
First 25 points
ANOVA functional decomposition
A function of n variables is decomposed into the sum of
a constant (overall mean)
Interpreting the results
Main effects provide way to visualize effect of individual variables
Can assess importance of a main effect or two-way interaction by computing how much squared error goes down when add that term
If no interactions are important, can optimize one variable at a time and essentially read the best design from the main effect plots
If a two-way interaction exists between a control and noise variable, you may be able to adjust control to desensitize design to the noise factor.
Computing decomposition requires performing multidimensional integrals, but for a kriging surface these reduce to manageable one-dimensional integrals!
*
Sample data from a recent N&V study.
*
*
*
The available Kriging Wizard software computes the ANOVA percent contributions.
*
Testimonial
Don,
As you know I was involved for some in the GMT 900 pick-up frame development. The issue designing frame and isolator tuning package to reduce shake.
Task was formulated as optimization problem, with auxiliary model used to modify frame structure and. I ended up with problem with 52 structural and 15 isolator tuning variables and 6 system model runs to evaluate objective function and constraints. To solve it using iSIGHT for optimization would've required about 2 days just for one set of constraints, without any trade-off studies. So I decided to use DOE in combination with kriging response surface and DIRECT optimization tool in Excel.
A DOE of 1200 points was executed and data used to fit response surface in Excel. Spreadsheet for optimization based on kriging surface was prepared and combined with another one to calculate frame section as combination of auxiliary model and underlying baseline (produced by Joe Wong and Marv Zurek). As a result, almost instant design optimization became possible.
Various combinations of packaging, manufacturing and strength constraints were evaluated during meeting between supplier, DRE and Development Engineer. At the end, I didn't even have to be present at those meetings! Your vision and tools you developed made it possible to solve complex trade-off problems in real time by people without in-depth background in optimization. Even though work still continues, I think impact on the program was quite significant and proof-of-concept frame will be built by mid July to confirm analytical studies.
Mikhail
* .
In the presentation, we will do an interactive demo.
Here, in the notes, we will do a quick tutorial based on screen shots.
*
Installation
You can find the Kriging Wizard site by searching on “Kriging Wizard” in Socrates. You will get links to Kriging Wizard the on the homepages of the Robust Synthesis and Analysis group and the Optimization group.
*
Switch to the DOE tab in the spreadsheet.
*
Switch to the DOE tab of the Wizard.
Fill in the fields as shown and click “Write Design to Worksheet.”
The tool is pretty self explanatory and explains what data is expected in each field.
*
Make an experimental design (DOE)
*
Fitting a surface
Switch to worksheet “Small Data”. Select the data as shown on the left.
The first row indicates what data is in the column. SKIP means ignore the column. X means the column has an independent (x) variable. Y(interp) means the column has a output (y) variable that you want to interpolate.
The next row has the names of the variables (header row).
The remain rows have the data.
*
Click Tools, Kriging Beta…” to launch the Wizard.
The tool should auto recognize the XY data row with header and the Variable Type row.
Enter “Small Data” in the field for “Enter prefix for output worksheets.”
*
Fitting a surface
When fitting is complete, two new worksheets will have been made:
Small Data SURF
Small Data CV
*
Switch to the worksheet “New Points and Predictions.”
Launch the Kriging Wizard and switch to the Predict tab.
Fill in the Predict tab as shown and click Predict.
*
Switch to the worksheet “New Points and Predictions.”
Launch the Kriging Wizard, switch to the Insert Function tab, fill in the information as shown, and click Insert Function.
A function will be inserted into your worksheet that does the prediction.
*
Performing ANOVA
*
Switch to the Monte Carlo worksheet.
Launch the Kriging Wizard, switch to the Monte Carlo tab, fill in the information as shown, and click Create Cloud, and then exit the wizard.
A cloud of points will be created. If you change the mean or standard deviation of a variable, the cloud will adjust.
Predict outputs at the cloud points using “Insert Function” to see how this input variation translates into output variation.
Global
Sampled Points
99% Confidence Interval = Mean +/- 2.58 StdDev
Sampled Points
Lower 99%
300
350
400
450
500
550
600
300
350
400
450
500
550
600
300
350
400
450
500
550
600
300
350
400
450
500
550
600
Don Jones
*
Gain insight
Quantify tradeoffs
Facilitates multi-disciplinary balancing since each discipline has “bottled up” version of analysis and can react real-time to change proposals
Robust design
Adjust control parameters to desensitize design to variation, shift mean
Explore benefits of tolerance reductions
*
“Standard” response-surface methods fit model by least squares
Polynomial regression
Neural networks (back propagation = steepest descent on squared error)
Two reasons to use kriging for computer experiments
Standard methods were developed for physical experiments, which have very different characteristics and challenges than computer experiments.
Least-squares methods have a little-discussed theoretical assumption that is patently false in the case of computer experiments.
*
Physical
(usually 2 or 3)
Can sometimes sample all or nearly all combinations.
Key challenge is to separate the effect of the control variables from noise variables
Computer
Variables usually are continuous
Input-output relationships are nonlinear, a few levels won’t do
No way to cover the entire design space.
*
Regression:
Try to capture as much as possible with many polynomial terms, selected via stepwise procedure from large set.
Kriging:
Don’t spend much time guessing functional form. Use either constant or linear model.
ERROR TERM:
unpredictable. Can think of as measurement error.
Kriging
Errors are treated as left out terms in x that cannot be precisely predicted but can be roughly estimated
Kriging and regression are actually sub-cases of the same general statistical model.
*
Captures nonlinearity without the need to assume functional form
If simulation is very nonlinear — and if a sufficient number of points are sampled — kriging will be more accurate than regression
Interpolates sampled points
At the sampled points, predicts the sampled value, as it should
Provides confidence intervals with desirable properties
intervals are bigger in the gaps between sampled points
intervals shrink to zero at the sampled points
Provides way to infer if we have enough points
All this will become clear as we go along…
*
When do polynomials fit poorly?
Polynomial response surfaces may not fit well if the surface has sweet spot, asymptotes, or similar nonlinear shapes.
Kriging can “pick up” these shapes from data if one has enough points because it is not constrained by an assumed form.
*
Constructing a kriging surface
*
Constructing a kriging surface
*
good approximation should
reasonably smooth.
*
Constructing a kriging surface
Kriging can be interpreted as a statistically-based way to generate many such interpolators.
Here we show 30.
*
Constructing a kriging surface
The value of the kriging surface at a given point is the mean of the values for all the possible interpolators.
*
Kriging confidence intervals
At any point, we can not only compute the mean of the possible interpolators, but also compute the standard deviation. From this we can compute upper and lower confidence intervals.
*
Cross-validating the surface
To check if the confidence intervals can be trusted, we can refit the surface with one of the points left out…here the fourth point.
If the surface is working, the value of the left-out sampled point should lie within the confidence interval.
*
Effect of adding more points
Because kriging interpolates, as you add points the surface will begin to “snap” to the function.
If the surface “doesn’t change much” when more points are added, you can assume that you have “converged.”
Idea due to Jian Tu, GM R&D
*
Sequenced sampling
Checking for convergence requires a “sequenced sample,” that is, the points should gradually fill in the space as they are added.
The available Kriging Wizard software provides such sequenced DOEs.
First 50 points
First 75 points
All 100 points.
First 25 points
ANOVA functional decomposition
A function of n variables is decomposed into the sum of
a constant (overall mean)
Interpreting the results
Main effects provide way to visualize effect of individual variables
Can assess importance of a main effect or two-way interaction by computing how much squared error goes down when add that term
If no interactions are important, can optimize one variable at a time and essentially read the best design from the main effect plots
If a two-way interaction exists between a control and noise variable, you may be able to adjust control to desensitize design to the noise factor.
Computing decomposition requires performing multidimensional integrals, but for a kriging surface these reduce to manageable one-dimensional integrals!
*
Sample data from a recent N&V study.
*
*
*
The available Kriging Wizard software computes the ANOVA percent contributions.
*
Testimonial
Don,
As you know I was involved for some in the GMT 900 pick-up frame development. The issue designing frame and isolator tuning package to reduce shake.
Task was formulated as optimization problem, with auxiliary model used to modify frame structure and. I ended up with problem with 52 structural and 15 isolator tuning variables and 6 system model runs to evaluate objective function and constraints. To solve it using iSIGHT for optimization would've required about 2 days just for one set of constraints, without any trade-off studies. So I decided to use DOE in combination with kriging response surface and DIRECT optimization tool in Excel.
A DOE of 1200 points was executed and data used to fit response surface in Excel. Spreadsheet for optimization based on kriging surface was prepared and combined with another one to calculate frame section as combination of auxiliary model and underlying baseline (produced by Joe Wong and Marv Zurek). As a result, almost instant design optimization became possible.
Various combinations of packaging, manufacturing and strength constraints were evaluated during meeting between supplier, DRE and Development Engineer. At the end, I didn't even have to be present at those meetings! Your vision and tools you developed made it possible to solve complex trade-off problems in real time by people without in-depth background in optimization. Even though work still continues, I think impact on the program was quite significant and proof-of-concept frame will be built by mid July to confirm analytical studies.
Mikhail
* .
In the presentation, we will do an interactive demo.
Here, in the notes, we will do a quick tutorial based on screen shots.
*
Installation
You can find the Kriging Wizard site by searching on “Kriging Wizard” in Socrates. You will get links to Kriging Wizard the on the homepages of the Robust Synthesis and Analysis group and the Optimization group.
*
Switch to the DOE tab in the spreadsheet.
*
Switch to the DOE tab of the Wizard.
Fill in the fields as shown and click “Write Design to Worksheet.”
The tool is pretty self explanatory and explains what data is expected in each field.
*
Make an experimental design (DOE)
*
Fitting a surface
Switch to worksheet “Small Data”. Select the data as shown on the left.
The first row indicates what data is in the column. SKIP means ignore the column. X means the column has an independent (x) variable. Y(interp) means the column has a output (y) variable that you want to interpolate.
The next row has the names of the variables (header row).
The remain rows have the data.
*
Click Tools, Kriging Beta…” to launch the Wizard.
The tool should auto recognize the XY data row with header and the Variable Type row.
Enter “Small Data” in the field for “Enter prefix for output worksheets.”
*
Fitting a surface
When fitting is complete, two new worksheets will have been made:
Small Data SURF
Small Data CV
*
Switch to the worksheet “New Points and Predictions.”
Launch the Kriging Wizard and switch to the Predict tab.
Fill in the Predict tab as shown and click Predict.
*
Switch to the worksheet “New Points and Predictions.”
Launch the Kriging Wizard, switch to the Insert Function tab, fill in the information as shown, and click Insert Function.
A function will be inserted into your worksheet that does the prediction.
*
Performing ANOVA
*
Switch to the Monte Carlo worksheet.
Launch the Kriging Wizard, switch to the Monte Carlo tab, fill in the information as shown, and click Create Cloud, and then exit the wizard.
A cloud of points will be created. If you change the mean or standard deviation of a variable, the cloud will adjust.
Predict outputs at the cloud points using “Insert Function” to see how this input variation translates into output variation.
Global
Sampled Points
99% Confidence Interval = Mean +/- 2.58 StdDev
Sampled Points
Lower 99%
300
350
400
450
500
550
600
300
350
400
450
500
550
600
300
350
400
450
500
550
600
300
350
400
450
500
550
600