Optimal Electricity Supply Optimal Electricity Supply Bidding by Markov Decision Bidding by Markov Decision

ProcessProcessAuthors: Haili Song, Chen-Ching Liu, Jacques Lawarree, & Robert DahlgrenAuthors: Haili Song, Chen-Ching Liu, Jacques Lawarree, & Robert Dahlgren

Presentation Review By:

Feng Gao, Esteban Gil, & Kory Hedman

IE 513 Analysis of Stochastic Systems

Professor Sarah Ryan

March 28, 2005

OutlineOutline

Summary of the previous presentationSummary of the previous presentation Model ValidationModel Validation Implementation and case studyImplementation and case study Description of ExamplesDescription of Examples SummarySummary

Summary of previous presentationSummary of previous presentation IntroductionIntroduction

Electric Market is now CompetitiveElectric Market is now Competitive GenCos Bid on DemandGenCos Bid on Demand

PurposePurpose MDP Used to Determine Optimal Bidding StrategyMDP Used to Determine Optimal Bidding Strategy

Problem FormulationProblem Formulation Transition Probability Determined by Current State, Subsequent State, Transition Probability Determined by Current State, Subsequent State,

& Decision Made& Decision Made 7 Variables to Define a State7 Variables to Define a State Aggregation Used to Limit Dimensionality ProblemsAggregation Used to Limit Dimensionality Problems

Model OverviewModel Overview 7 Day Planning Horizon7 Day Planning Horizon Objective is to Maximize Summation of Expected RewardObjective is to Maximize Summation of Expected Reward Value IterationValue Iteration

Value Iteration DiscussionValue Iteration Discussion V (i, T+1): Total V (i, T+1): Total

Expected Reward in Expected Reward in T+1 Remaining Stages T+1 Remaining Stages from State Ifrom State I

At the last stage T = 0At the last stage T = 0 Value Iteration Value Iteration

(Backward Induction)(Backward Induction) Ignore discount factorIgnore discount factor The immediate reward The immediate reward

is dependent on the is dependent on the initial state, following initial state, following state and decision astate and decision a

Model Overview ClarificationModel Overview Clarification

Sum of all Scenarios S Sum of all Scenarios S that result in a given spot that result in a given spot price, cleared quantity, price, cleared quantity, and production limit. and production limit.

Prob to Move from State i Prob to Move from State i to j given decision a = to j given decision a = [Prob (that the spot price, [Prob (that the spot price, production level are production level are correct and load forecast correct and load forecast = demand)*prob(of having = demand)*prob(of having the proper load forecast)]the proper load forecast)]

Resulting Spot Price can Resulting Spot Price can be dependent on Decision be dependent on Decision a if the bidder has market a if the bidder has market powerpower

Model ValidationModel Validation

For model validation:For model validation: Accumulate actual data and observations from the Accumulate actual data and observations from the

market over a period of time (e.g. 1 year)market over a period of time (e.g. 1 year) Market data set provides the actual scenariosMarket data set provides the actual scenarios Relationship between estimated by the BIDS Relationship between estimated by the BIDS

representation r(i,j,a) and actual rewards w(i,j,a) representation r(i,j,a) and actual rewards w(i,j,a) can be analyzed by linear regression.can be analyzed by linear regression.

Case StudyCase Study 3 suppliers: GenCoA, GenCoB, and GenCoC, all 3 suppliers: GenCoA, GenCoB, and GenCoC, all

bidding in the spot marketbidding in the spot market GenCoA is the decision maker using the Markov GenCoA is the decision maker using the Markov

Decision Process techniqueDecision Process technique GenCoA: 1 generating unitGenCoA: 1 generating unit GenCoB: 2 generating unitsGenCoB: 2 generating units GenCoC: 2 generating unitsGenCoC: 2 generating units Planning Horizon: 7 days (bid decision for next day Planning Horizon: 7 days (bid decision for next day

considers the entire week aheadconsiders the entire week ahead

Case StudyCase Study GenCoA makes a decision from a set of pre-specified GenCoA makes a decision from a set of pre-specified

decision optionsdecision options GenCoA does not know exactly how GenCoB and GenCoA does not know exactly how GenCoB and

GenCoC are going to bidGenCoC are going to bid But their individual bidding behavior is modeled by But their individual bidding behavior is modeled by

bid prices, quantities and the associated probabilities bid prices, quantities and the associated probabilities based on GenCoA’s knowledge and informationbased on GenCoA’s knowledge and information

Transition probabilities and rewards are calculated Transition probabilities and rewards are calculated using algorithm described in previous presentationusing algorithm described in previous presentation

Two Basic Market SituationsTwo Basic Market Situations

EXAMPLE 1:EXAMPLE 1: Decision-maker has a production limit over the Decision-maker has a production limit over the

planning horizonplanning horizon Decision-maker does not have market power Decision-maker does not have market power

(perfect competition)(perfect competition) Optimal strategy is time dependent due to the Optimal strategy is time dependent due to the

production limitproduction limit In some states the optimal decision is not to sell, In some states the optimal decision is not to sell,

but to save the resources for more profitable daysbut to save the resources for more profitable days

Two Basic Market SituationsTwo Basic Market Situations EXAMPLE 2:EXAMPLE 2:

Decision-maker has market power: it can manipulate Decision-maker has market power: it can manipulate the bid to influence the spot pricethe bid to influence the spot price

Decision-maker has no production limitDecision-maker has no production limit Decision-maker makes the bidding decision to Decision-maker makes the bidding decision to

maximize the expected reward over the planning maximize the expected reward over the planning horizonhorizon

Daily maximum strategy is time independent: Daily maximum strategy is time independent: decision-maker makes the same decision as long as the decision-maker makes the same decision as long as the system is in the same statesystem is in the same state

BIDS value iteration is time dependent: it takes into BIDS value iteration is time dependent: it takes into account how current biddings affect future spot pricesaccount how current biddings affect future spot prices

Comparison of Two CasesComparison of Two Cases

Without market power, bidder is concerned with Without market power, bidder is concerned with saving resources for more expensive periodssaving resources for more expensive periods

With market power, bidder is concerned with With market power, bidder is concerned with properly influencing the future spot price to properly influencing the future spot price to maximize profitmaximize profit

Knowing whether the bidder has market power or Knowing whether the bidder has market power or not is crucial since the relationship between spot not is crucial since the relationship between spot prices and decisions would depend on each otherprices and decisions would depend on each other

SummarySummary Model OverviewModel Overview

7 Day Planning Horizon7 Day Planning Horizon Objective is to Maximize Summation of Expected RewardObjective is to Maximize Summation of Expected Reward Value IterationValue Iteration

Model ValidationModel Validation Comparison of Predicted and Actual Results (by linear Comparison of Predicted and Actual Results (by linear

regression)regression) Implementation and case studyImplementation and case study

Three GenCos, GenCo A is the Decision MakerThree GenCos, GenCo A is the Decision Maker 5 Generators among the 3 GenCos5 Generators among the 3 GenCos

Description of 2 Examples:Description of 2 Examples: Production Limit without Market PowerProduction Limit without Market Power Market Power without Production LimitMarket Power without Production Limit

Next Time: Presentation and Discussion of Results and ConclusionsNext Time: Presentation and Discussion of Results and Conclusions

Questions???Questions???