large timestep issues

Large Timestep Issues

Lecture 12

Alessandra Nardi

Thanks to Prof. Sangiovanni, Prof. Newton, Prof. White, Deepak Ramaswamy, Michal Rewienski, and Karen Veroy

Last lecture review

• Transient Analysis of dynamical circuits– i.e., circuits containing C and/or L

• Examples

• Solution of ODEs (IVP)– Forward Euler (FE), Backward Euler (BE) and

Trapezoidal Rule (TR)– Multistep methods– Convergence

• Consistency

Outline

• Convergence for multistep methods– Stability

• Region of Absolute Stability• Dahlquist’s Stability Barriers

• Stiff Stability (Large timestep issues)– Examples– Analysis of FE, BE– Gear’s Method– Variable step size

• More on Implicit Methods– Solution with NR

• Application of multistep to circuit equations

Multistep Equation:

1 1 1,l l l lx t x t t f x t u t

FE Discrete Equation: 1 11ˆ ˆ ˆ ,l l l

lx x t f x u t

0 1 0 11, 1, 1, 0, 1k

Forward-Euler Approximation:

Multistep Coefficients:

Multistep Coefficients:

BE Discrete Equation:

0 1 0 11, 1, 1, 1, 0k 1ˆ ˆ ˆ ,l l l

lx x t f x u t

Trap Discrete Equation: 1 11ˆ ˆ ˆ ˆ, ,

2l l l l

l l

tx x f x u t f x u t

0 1 0 1

1 11, 1, 1, ,

2 2k Multistep Coefficients:

0 0

ˆ ˆ ,k k

l j l jj j l j

j j

x t f x u t

Multistep Methods – Common AlgorithmsTR, BE, FE are one-step methods

1) Local Condition: One step errors are small (consistency)

2) Global Condition: The single step errors do not grow too quickly (stability)

Typically verified using Taylor Series

All one-step methods are stable in this sense.

Multistep Methods – Convergence Analysis

Two conditions for Convergence

We made the LTE so small, how come the Global error is so large?

10 0 1 1

l l l k lk kt E t E t E e

Multistep Method Difference Equation

Why did the “best” 2-step explicit method fail to Converge?

ˆlv l t v Global Error

LTE

Multistep Methods – StabilityDifference Equation

Three important observations

iIf for all<1 , the m xn al jji x C u

where does not depend on C l

i >1 , If for any then there existsia bounded such th atj lu x

i i iIf for all and 1 , =1,if is d t is incti

then maxl jjx Cl u

An Aside on Solving Difference Equations

Consider a general kth order difference equation1

0 1l l l k l

ka x a x a x u

Multistep Method Difference Equation

Definition: A multistep method is stable if and only if

Theorem: A multistep method is stable if and only if1

0 1The roots of 0 are either k kkz z

Less than one in magnitude or equal to one and distinct

10 0 1 1

l l l k lk kt E t E t E e

0, 0,max max for any l l l

T Tl l

t t

TE C e e

t

Multistep Methods – StabilityDifference Equation

Given the Multistep Method Difference Equation

If the roots of

10 0 1 1

l l l k lk kt E t E t E e

• less than one in magnitude• equal to one in magnitude but distinct

are either

Then from the aside on difference equations

maxl llE Cl e

From which stability easily follows.

0

0k

k jj j

j

t z

Multistep Methods – StabilityStability Theorem Proof

1-1Re

Im

0

roots of 0 for a nonzero k

k jj j

j

t z t

As 0, roots t move inward to

match polynomial 0

roots of 0k

k jj

j

z

Multistep Methods – StabilityStability Theorem Proof

Multistep Methods – StabilityA more formal approach

)(xqxtk

j

jljj

k

j

jljj 1 0ˆ0ˆ

00

• Def: A method is stable if all the solutions of the associated difference equation obtained from (1) setting q=0 remain bounded if l

• The region of absolute stability of a method is the set of q such that all the solutions of (1) remain bounded if l

• Note that a method is stable if its region of absolute stability contains the origin (q=0)

Multistep Methods – StabilityA more formal approach

1.ty multiplici of are modulusunit of roots that theand

roots,distinct ofnumber theis where11that

such are 0 of roots theallsuch that q ofset theis

method a ofstability absolute ofregion theshown that becan It

11

0

ppi

k

j

jkjj

k,...,k,iz

zq

Def: A method is A-stable if the region of absolute stability contains the entire left hand plane (in the space)

Re(z)

Im(z)

-1 1

Re()

Im()

-1

• Each method is associated with two polynomials and : : associated with function past values : associated with derivative past values

• Stability: roots of must stay in |z|1 and be simple on |z|=1

• Absolute stability: roots of (q must stay in |z|1 and be simple on |z|=1 when Re(q)<0.

Multistep Methods – StabilityA more formal approach

• First: For a stable, explicit k-step multistep method, the maximum number of exactness constraints that can be satisfied is less than or equal to k (note there are 2k coefficients). For implicit methods, the number of constraints that can be satisfied is either k+2 if k is even or k+1 if k is odd.

• Second: There are no A-stable methods of convergence order greater than 2, and the trapezoidal rule is the most accurate.

Multistep Methods – StabilityDahlquist’s Stability Barriers

TR very popular (SPICE)

1) Local Condition: One step errors are small (consistency)

2) Global Condition: One step errors grow slowly (stability)

Exactness Constraints up to p0 (p0 must be > 0)

0

roots of 0 must be inside the unit circle k

k jj

j

z

2

0, 0,max maxl l

T Tl l

t t

TE C e

t

0 1

1 00,

max for pl

Tl

t

e C t t t

0

0,max

plT

lt

E CT t

Convergence Result:

Multistep Methods – Convergence AnalysisConditions for convergence – Consistency & Stability

Difference EqnStability region 1-1

1z t

Im(z)

Re(z)

Im

Re

Forward Euler

ODE stability region

2

t

Region ofAbsolute Stability

Multistep MethodsFE region of absolute stability

Difference EqnStability region 1-1

Im(z)

Re(z)

Im Backward Euler 1

1z t

Region ofAbsolute Stability

Multistep MethodsBE region of absolute stability

Summary

• Convergence for one-step methods– Consistency for FE– Stability for FE

• Convergence for multistep methods– Consistency (Exactness Constraints)

• Selecting coefficients

– Stability• Region of Absolute Stability• Dahlquist’s Stability Barriers

Stiff Problems (Large Timestep Issues)Example

t-λt-λ

t-λ

-eexx(t)

,λλxx

-es(t)dt

tdstsx

dt

tdx

21

2

1 :solutionExact

110 )0(

1 where)(

))(()(

0

26

10

1

Interval of interest is [0,5]Uniform step size (for accuracy)

t 10-6

5x106 steps !!!

11 5For

0 105For 2

10

6

t-λ

t-λ-

-et

ext

Strategy (for previous example): Take 5 steps of size 10-6 for accuracy during initial phase and then 5 steps ofsize 1.

Stiff problem:1. Natural time constants2. Input time constants3. Interval of interest

If these are widely separated, then the problem is stiff

Stiff Problems (Large Timestep Issues)Example

C1

R2

R1 R3 C2

1Ri1Ci

2Ri

2Ci3Ri

1v 2v

Application ProblemsSignal Transmission in an IC – 2x2 example

1 2 1 3 2Let 1, 10, 1C C R R R

1.1 1.0

1.0 1.1

A

dxx

dt

Eigenvectors

11 1 0.1 0 1 1

1 1 0 2.1 1 1A

Eigenvalues

Forward-Euler Computed Solution

The Forward-Euler is accurate for small timesteps, but goes unstable when the timestep is enlarged

Stiff Problems (Large Timestep Issues)FE on two time-constant circuit

Backward-Euler Computed Solution

With Backward-Euler it is easy to use small timesteps for the fast dynamics and then switch to large

timesteps for the slow decay

small t

large t

( ) ( )d

x t Ax tdt

( ) 2.1, 0.1eig A

Circuit Example

Stiff Problems (Large Timestep Issues)BE on two time-constant circuit

0( ), 0d

v t v t v vdt

Scalar ODE:

Forward-Euler:

Backward-Euler: 1 1 1 1

ˆ ˆ ˆ ˆ ˆ1

l l l l lv v t v v vt

If 1 1 the solution grows even if <0t

1If 1 the solution decays even if 0

1 t

Trap Rule:

1ˆ ˆ ˆ ˆ1l l l lv v t v t v

1 1 1 1 0.5ˆ ˆ ˆ ˆ ˆ ˆ0.5

1 0.5

ll l l l ltv v t v v v v

t

Multistep Methods (Large Timestep Issues)BE, FE, TR on the scalar ODE problem

Im

Re

ODE stability region

2

t

Region ofAbsolute Stability

Stiff Problems (Large Timestep Issues)FE on two time-constant circuit

1.2 and 1.0

1.0

21

t

Im

Re

ODE stability region

2

t

Region ofAbsolute Stability

1.2 and 1.0

1

21

t

Im

Region ofAbsolute Stability

Stiff Problems (Large Timestep Issues)BE on two time-constant circuit

1.2 and 1.0

1.0

21

t

1.2 and 1.0

1

21

t

Im

Region ofAbsolute Stability

Stiff Problems

• We showed that:– The analysis of stiff circuits requires the use of variable

step sizes– Not all the linear multistep methods can be efficiently used

to integrate stiff equations

• To be able to choose t based only on accuracy considerations, the region of absolute stability should allow a large t for large time constants, without being constrained by the small time constants

• Clearly A-stable methods satisfy this requirement

Backward Differentiation Formula - BDF (Gear Methods)

0 re whe)ˆ(ˆ 000

lk

j

jlj xftx

• Note that Gear’s first order method is BE

• It can be shown that: – Gear’s methods up to order 6 are stiffly stable and are

well-suited for stiff ODEs– Gear’s methods of order higher than 6 are not stiffly

stable

• Less stringent than A-stable

Gear’s Method region of absolute stability(outside the closed curve)

k=1 k=2

Gear’s Method region of absolute stability(outside the closed curve)

k=3 k=4

Variable step size

• When the step size is changed during the integration, the coefficients of the method need to be recomputed at each iteration

• Example: Gear’s method of order 2

1

002

21

1

where

0 re whe)ˆ(ˆˆˆ

lll

ll

lll

ttΔt

xftxxx

Variable step size

)]2([

)]2([

)]2([

)(

:)constraint exactness the(usingobtain We

:Setting

:polynomial 2-orderan consider now usLet

11

10

11

2

211

21

1

12

1

2210

lll

ll

lll

l

lll

ll

llll

lll

ΔtΔtΔt

ΔtΔt

ΔtΔtΔt

Δt

ΔtΔtΔt

ΔtΔt

ΔtΔttt

Δttt

tctccx(t)

More observations

• To minimize the computation time needed to integrate differential equations, the t must be chosen as large as possible provided that the desired accuracy is achieved– Several approximation are available. SPICE2 uses Divided

Differences

• At a certain time point, different integration methods would allow different step size– Advantageous to implement a strategy which allows a

change of method as well as of t

Summary on Stiff Stability

• FE: timestep is limited by stability and not by accuracy

• BE: A-stable, any timestep could be used

• TR: most accurate A-stable multistep method

• Gear: stiffly stable method (up to order 6)

• The analysis of stiff circuits requires the use of variable timestep

11 ˆ( ) (0) 0 , 0x t x x t f x u

2 1 12 1ˆ ˆ ˆ( ) ,x t x x t f x u t

1 11ˆ ˆ ˆ( ) ,L L L

L Lx t x x t f x u t

Forward-Euler

Requires just functionEvaluations

Backward-Euler

1 11 1ˆ ˆ( ) (0) ,x t x x t f x u t

2 1 22 2ˆ ˆ ˆ( ) ,x t x x t f x u t

1ˆ ˆ ˆ( ) ,L L LL Lx t x x t f x u t

Nonlinear equationsolution at each step

0Stepwise Nonlinear equation solution needed whenever 0

Multistep MethodsMore on Implicit Methods

1 1

0 0 ˆ ˆ ,ˆ ˆ , 0l lk k

l j l jj j j

j jl lxx t f x u t f x ut t

Rewrite the multistep Equation

b ˆIndependent of lxSolve with Newton

,

, 1 , , ,0 0 0 0

ˆ ,ˆ ˆ ˆ ˆ ,

l jl l j l j l j l j

l

f x u tI t x x x t f x u t b

x

Here j is the Newton iteration index

Jacobian ,l jF x

Multistep Implicit MethodsSolution with Newton

Newton Iteration:

,

, 1 , ,0 0

ˆ ,ˆ ˆ ( )

l jl l j l j l j

f x u tI t x x F x

x

Solution with Newton is very efficient

lt1lt 2lt 3lt l kt

Easy to generate a good initial guess using polynomial fitting

,0ˆ lx

Polynomial Predictor

ˆ lxConverged

Solution

0

,

0 0

ˆ , as 0

l jlf x u

It

I t tx

Jacobian become easy to factor for small timesteps

Multistep Implicit MethodsSolution with Newton

Application of linear multistep methods to circuit equations

0 :unknowns as )(with

equationsnonlinear algebraic ofset aobtain we

)()(

:equation method sGear' Using

0 :form in the

equations algebraic-aldifferenti mixed ofset a have We

0

0

0

0

),t,xΔtβ

)x(tα

F(tx

t

tx

dt

tdx

,x,t)dt

dxF(

lll

k

jklj

l

l

k

jklj

l

Transient Analysis Flow Diagram Predict values of variables at tl

Replace C and L with resistive elements via integration formula

Replace nonlinear elements with G and indep. sources via NR

Assemble linear circuit equations

Solve linear circuit equations

Did NR converge?

Test solution accuracy

Save solution if acceptable

Select new t and compute new integration formula coeff.

Done?

YES

NO

NO

Summary• Transient Analysis of dynamical circuits• Solution of ODEs (IVP)

– FE, BE and TR– Multistep methods– Convergence

• Consistency• Stability

• Stiff Stability (Large timestep issues)– Gear’s method

• Application of multistep to circuit equations• Did not talk about:

– Runge-Kutta– Predictor-Corrector Methods

Summary on circuit simulation

• Circuit Equation Formulation – STA, MNA

• DC Analysis of Nonlinear Circuits– Solution of Linear Equations (direct and iterative

methods)– Solution of Nonlinear Equations (Newton’s method)

• Transient Analysis of Nonlinear Circuits– Solution of Ordinary Differential Equations- IVP

(multistep methods)

Appendix to circuit simulation Preconditioners

• The convergence rate of iterative methods depends on spectral properties of the coefficient matrix. Hence one may attempt to transform the linear system into one that is equivalent, but that has more favorable spectral properties.

• A preconditioner is a matrix that effects such a transformation:

Mx=b A-1Mx=A-1b

• The choice of a preconditioner is largely application specific