the evolution of planning...

27
Imagination at work The Evolution of Planning Software “Get it done… Faster.” July 29, 2015

Upload: others

Post on 18-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Imagination at work

The Evolution of Planning Software “Get it done… Faster.” July 29, 2015

Page 2: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Software Products Concorda Software Suite

Proven tools used extensively in the industry

MAPS

• Evaluates power system economics and impact of congestion

• Provides locational prices, unit schedules, line flows, emissions

• Used in high wind penetration and standard market design studies

• Supports GE’s multi-billion dollar energy investment portfolio

PSLF

• Simulates physical behavior of the grid and connected equipment

• Provides voltages and line flows, system dynamic behavior

• Extensively used to study 2003 Northeast US blackout

• Mechanism for ensuring that equipment is properly modeled

MARS

• Assesses reliability of supply in meeting energy demand

• Provides loss of load expectation (LOLE) and other reliability indices

• Applied for major regional reliability and reserve margin studies

• In use at most ISO’s in the US

The Evolution of Planning Software | 29 July 2015 2

Page 3: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Our HPC Experience

Internal HPC in production since 2006

Used for consulting studies with MAPS and MARS

MAPS HPC, MARS HPC, and PSLF HPC commercial

products

Custom HPC application development

The Evolution of Planning Software | 29 July 2015 3

Page 4: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

MAPS – The Early Days

Initial release in the early 1970s

Targeted the IBM System/360s, eventually GE and Honeywell machines

Batch job submission…

• Input case sent to computer center as a deck of cards by courier

• Results returned by courier as a paper printout

• Model dimensions (circa 1970): 11 areas, 225 generators, bubble model

• Runtime: ~ a day

The Evolution of Planning Software | 29 July 2015 4

Page 5: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

MAPS – Intervening years

MAPS transitioned through several generations of

machines:

• GE/Honeywell mainframes; Digital VAX microcomputers; PRIME microcomputers; Sun desktop workstations

Finally ending up on Windows/Intel through the late

1990s and early 2000s

Model dimensions (2006): 130 areas, 4000 generators,

nodal model, bihourly

Runtime: ~ 48 hours

The Evolution of Planning Software | 29 July 2015 5

Page 6: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Problems

Limited sensitivity analysis

• Computer charges and runtimes prohibitive

Seems issues

• Eastern Interconnect cut into 4 regions

• Caused issues modeling details of border regions

Turnaround times

• Client had time sensitive needs, could not tolerate 48 hour turnaround

The Evolution of Planning Software | 29 July 2015 6

Page 7: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Introduction of MAPS HPC

Started in 2006

First cluster: ~100 cores, about 20% of the current generation’s capability per core

Runtimes: 48 hours -> 6 hours

Current cluster: ~8,500 cores (Xeon E5, 3GB RAM/core)

Model dimensions: 150 Areas, 8000 generators, nodal model, bihourly

Job turnaround limited by the slowest time slice, currently ~90 minutes

The Evolution of Planning Software | 29 July 2015 7

Page 8: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

PSLF HPC

Low hanging fruit: contingency analysis

Many small, independent tasks (1 contingency)

Spawn PSLF instances to receive contingency

descriptions and evaluate

The Evolution of Planning Software | 29 July 2015 8

Page 9: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

PSLF HPC Runtimes

WECC Model, 30,000 N-1-1 contingencies

• Conventional desktop: ~2 days

• 2 x 8 core machines: 90 minutes

Custom contingency processor for a utility (used intraday to support ops):

• 3.2 hours -> 18 minutes

The Evolution of Planning Software | 29 July 2015 9

Page 10: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

MARS

Model created in the late 80s to support New York

Power Pool

Sequential Monte Carlo model – state in one hour

depends on the state in the previous hour

The Evolution of Planning Software | 29 July 2015 10

Customer 1: • Workstations: 11 hours; 100

core HPC: 25 minutes Customer 2: • Server grade AMD: 100

hours; 12 core HPC: 5 hours

Our internal use: • Model dimensions: 64

areas, 9 pools, 4200 units • 120 cores: 2 hours

Page 11: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

MARS Runtimes

The Evolution of Planning Software | 29 July 2015 11

Page 12: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

How?

Page 13: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

MAPS

Split into 1 week time slices

Heuristics used for approximating the previous and next week

The Evolution of Planning Software | 29 July 2015 13

Page 14: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

MARS

Program can “skip” replications

Advances random number generator with a known pattern

Provides identical results when run in parallel or

sequential

The Evolution of Planning Software | 29 July 2015 14

Page 15: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

PSLF

Two pronged approach:

• Parallelization of contingency processing across HPC hardware

• Take advantage of SMP hardware

– Ordering algorithms

– Parallel simulation of dynamic models

The Evolution of Planning Software | 29 July 2015 15

Page 16: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Speed-up of 2.21x on 12 threads using PNNL’s PIC computer and EI case

The Evolution of Planning Software | 29 July 2015 16

0

50

100

150

200

250

300

350

400

450

0 5 10 15 20

Tim

e(s

)

Number of threads

20s simulation with fault - results from PNNL PIC computer

Runtime

Page 17: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

My jobs are done faster… So?

Page 18: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Model Fidelity

Improved runtimes make running a more detailed model more palatable.

Run the entire Eastern Interconnect in the time it used

to take to run the North East region

The Evolution of Planning Software | 29 July 2015 18

Sensitivity Analysis

Less time computing base case

More time for sensitivity analysis

• Paint a more complete picture

Page 19: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Automation Benefits

The Evolution of Planning Software | 29 July 2015 19

Consistent process

• Build up a repeatable workflow for all cases to use

• Same versions of tools are used by all users

• Tracking of input files, assumptions, etc.

Page 20: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Our Consulting Group’s MAPS Workflow

Internal tool developed to assemble data files and

prepare jobs

As weeks complete on the HPC, post processing is

done to create final deliverable

Excel tool to read the final deliverable

The Evolution of Planning Software | 29 July 2015 20

Page 21: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Our Consulting Group’s MAPS Workflow

Everything is “hands off”… the user just clicks “Start”

Versions of all of the tools are kept the same from run to run and pulled from a central source

Allows for a repeatable process

The Evolution of Planning Software | 29 July 2015 21

Page 22: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Centralized Tool Versioning

Have the current versions of all tools in a known

place, grouped together in a folder

When ready to migrate to a new version of your tools,

create a new folder and use it

Allows you to re-run against older versions (benchmarking, rework)

Versions become immutable

The Evolution of Planning Software | 29 July 2015 22

Page 23: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Resource Utilization

One scheduler – multiple compute resources

Automatically starts next task, schedule backlog

Balance work among available resources

Retry failed tasks, automatically handle failed

hardware

The Evolution of Planning Software | 29 July 2015 23

Page 24: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Questions?

The Evolution of Planning Software | 29 July 2015 24

Page 25: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Visit, Join, Follow…

Visit our website http://wwww.geenergyconsulting.com

Join us on Linkedin https://www.linkedin.com/company/ge-energy-consulting

Follow us on Twitter @pwrsystexperts

The Evolution of Planning Software | 29 July 2015 25

Page 26: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually
Page 27: The Evolution of Planning Softwaresite.ieee.org/pes-hpcgrid/files/2016/08/2-15PESGM2796-The-Evolutio… · Initial release in the early 1970s Targeted the IBM System/360s, eventually

Images

URL Author License

https://flic.kr/p/7EJoqn Marcin Wichary CC-BY

https://flic.kr/p/4Ab3qP

Marcin Wichary CC-BY

https://flic.kr/p/HeLJo David Goehring CC-BY

https://flic.kr/p/5akwi9 gothopotam CC-BY

https://flic.kr/p/9L4DC Stewart Butterfield CC-BY

https://flic.kr/p/9Po3ro Polycart CC-BY

The Evolution of Planning Software | 29 July 2015 27