A Two-Level Optimization Problem for

Analysis of Market Bidding Strategies

J. D. Weber T. J . Overbye
overbye @ uiuc.edu jd-weber @ uiuc.edu
Department of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
Abstract
Electricity markets involve suppliers (generators) and
sometimes consumers (loads) bidding f or MWhr generation
and consumption. In this market model, a central operator
solves an Optimal Power Flow (OPF) based on the
maximization of social welfare to determine the generation
and load dispatch and system spot prices. In this structure,
market participants will choose their bids in order to
maximize their profits. This presents a two-level
optimization problem in which participants try to maximize
their profit under the constraint that their dispatch and
price are determined by the OPF. This paper presents an
efJici6nt numerical technique, using price and dispatch
sensitivity information available from the OPF solution, to
determine how a market participant should vary its bid
portfolio in order to maximize its overall profit. The paper
further demonstrates the determination of Nash equilibrium
when all participants are trying to maximize their profit in
this manner.
Keywords: Electricity market, simulation, bidding
strategy, optimal power flow, Nash equilibrium
1. Introduction
Electricity markets throughout the world continue to be
opened to competitive forces. The creation of mechanisms
for suppliers, and sometimes consumers, to openly trade
electricity is at the core of these changes. Most of the
mechanisms being used and developed are based on spot
pricing ideas. Good overviews and examples of spot pricing
issues are found in [1]-[2]. All these spot pricing-based
markets consist of suppliers submitting bids for electricity in
the form of MW outputs and associated prices. Some
markets, such as in England and the PJ M interconnection
[3], consist of only supply-side bidding. Other markets,
such as in New Zealand [4] and California [ 5] , also
incorporate demand-side bidding, allowing the consumers in
the market to react to pricing.
In [6], a market simulation was developed where
individuals submit supply andor demand bids to a central
operator. The operator then solves an optimal power flow
(OPF), with the objective of maximizing social welfare, to
determine the generation and load dispatch, as well as all the
spot prices. This paper will develop an algorithm allowing
an individual to maximize their personal welfare in such a
market. Possible equilibrium behavior can be determined
by simulating a market in which all players use this new
0-7803-5569-5/99/$10.00 0 1999 IEEE
algorithm. A simple example will be used to demonstrate
behavior that would be expected in the electricity market.
2. Notation
x : voltages, angles, and other variables in OPF problem
s : supply vector (generation vector)
d : demand vector (load vector)
p : price vector
Ci(s)=bj+c,sZ =supplier cost (function of supply)
Bi ( d) =bp+cj z =consumer benefit (function of demand)
Social Welfare = x Bi ( d ) - CC, ( s ) =B(d)-C(s)
h(x,s,d) =0 : equality constraints for OPF including the
power f l ow equations
g(x,s,d) I O : inequality constraints for OPF including
transmission line limits and voltage limits
f (s,d,p) : individual's welfare taking into account
the price paid or received for goods
VL : the gradient of the OPF Lagrangian
H : the Hessian of the OPF Lagrangian
z =[ Sr dT
all camurnen all suppliers
xr A. ' p' : vector of all OPF variables
3. Electricity Market Setup using OPF
Many electricity markets throughout the world include
some form of supplier bidding in which suppliers submit
MW outputs, along with associated prices. These bids
generally take one of the two forms shown in Figure 1.
Biock Bidding Cpntinuous Bid Curve
f Price =p [$MWhr]
P 3 T - - - - 1 - - Pz- ----- q
PI- !
Price =p [$MWhr]
v
Figure 1 Supplier Block Bid and Continuous Bid Curve
Additionally, some markets also include consumer
bidding. Consumers can be modeled in a manner analogous
to the suppliers. A demand function is used, which is
mathematically similar to the supply function, except the
demand function decreases as the price increases. In this
paper, we assume a continuous bid curve model for both
suppliers and consumers such as those seen in Figure 2.
682
Figure 2 Consumer and Supplier Bid Curves
The setup of the electricity market simulation is the
same as in [ 6] , and is briefly summarized here for
convenience. For the market simulation, bids are taken as
inputs to an OPF that solves the maximization of social
welfare problem to determines supplies, demands and
prices. These bids are limited to linear bids' as shown in
Figure 2. Incorporating these bids into the OPF formulation
involves converting the bid into a cost or benefit function,
e.g.
p( s , k) =k -s+ p,, 3 c ( s , ~ ) =k -s' +p,,,,,,~
r: 1 [2:, 1
[ 6] , [ 8] . The Lagrange multipliers of the OPF solution
determine the spot prices, p. The suppliers in the market are
paid the spot price at their node, and the consumers are
charged the spot price. A thorough treatment of the
mathematics involved in integrating supply and demand
bids into the OPF formulation is provided in [ 8] .
It should also be noted that bids are required to have a
minimumpower of zero: no minimum generator outputs are
allowed. This simplifying assumption is made to more
concisely explain the algorithm. The algorithm could be
augmented in the future to account for output limits by
including a unit commitment solution prior to the OPF.
Thus far we have restricted our market participants to
either a single generator or a single load. In reality, any
combination of several generators and several loads could
constitute an economic entity; therefore, the word individual
is used to describe a set of supplies and demands whose
bidding is controlled by a single economic entity.
4. Individual Welfare Maximization
The market setup fromthe previous section essentially
defines the market rules for our system; however the
variation in bidding will be limited to the variation of a
single parameter, k, for each consumer or supplier as shown
in Figure 3. This parameter will vary the bid around the true
marginal curve of the supplier or consumer. The supply
curve that reflects true marginal cost is defined as the linear
function p( s ) =-S +p, , , while for the consumer, a true
marginal curve is defined as p ( d ) =--d +p , .
1
m,
1
md
bPrice=p . 4 price =p
True Marginal
enefit Curve
Cost Curve
Bid
[MWI
Figure 3 Bidding Variation for Supply and Demand
While this limits market behavior, it will be shown in
Section 7 that from an individual's viewpoint, the shape of
this curve is not important for a single market solution.
Note that modifying a bid in this manner is the same as
multiplying the cost or benefit function used in the OPF by
k.
Here weassume that each individual seeks to maximizie
its personal welfare. A single consumer's welfare is defined
as the amount of benefit received from using the power,
minus the expenses incurred in purchasing the power.
Similarly, a single supplier's welfare is defined as the
amount of revenue received from selling the power, minus
the cost of supplying the power. An individual wants to
maximize the total welfare of all the consumers and
suppliers that it controls.
=(-dTCdd+B:d)-&)-(srC,s+B:s)+ , - (1.s)
Revenuer
P
BenefiIS Expenses CAIS;O-
where Cd and C, : are diagonal matrices of quadratic coefficients
for supply cost and demand benefit functions
BdandB,: are vectors of linear coefficients for supply
cost and demand benefit functions
Note that an individual's welfare fls,d,h) is not an explicit
function of its bid variable k. However f is an implied
function of k since s, d, and h are all determined by an OPF
solution which is a function of k. Thus s, d, and h are all
implicit functions of the bids k.
Assuming the individual has some estimate of what
other individuals in the market are going to bid, the
individual's goal is to maximize its welfare by choosing a
bid which is a best response to the other individuals' bids.
As a result, the maximization of an individual's welfare
forms a two-level optimization problem where the
individual maximizes its welfare subject to an OPF solution
which maximizes social welfare based on all
market. That is
Max f (s,cl k)
s.t. (s,d,k) are determined by
k
bids in the
(2)
' While only allowing single-segment linear bids may seem limiting, the
analysis of theCalifornia power market done in [7] shows that over time
supply bid curves appear to be two-segment linear curves. Two-segments
could be added to this development in the future, but [7] shows that linear
bids are reasonably accurate.
683
5. Solution of Individual Welfare
Maximization
One approach to solving (2) is to represent the OPF
maximization sub-problem by the Kuhn-Tucker necessary
conditions as done in [9]. Thus the two-level optimization
problem may be written as a standard optimization problem.
This paper presents an alternative technique that makes
minimal modifications to an existing OPF to solve (2).
Consider solving the OPF problem for a given set of
bids. Then from the information available at this OPF
solution, the individuals profit sensitivity to variations in its
bid can be used to determine a Newton-step that improves
profits. This Newton-step is defined the customary way as
shown in (3).
af af
Evaluation of (3) requires determination of and - ,
ak ak
( 5)
Evaluation of (4) and ( 5 ) requires determination of
ad as an a2d azk
--- - -,and 7. Because s, d, and h are
akakak ak2ak2 ak
variables of the OPF solution, these derivatives can be
determined directly from values available from the OPF
solution. In a Newton-based OPF, an iteration of the
mismatch equation Az =-H- (VL) is done until Az =0,
where z =ET dr X A T r . Therefore the derivatives
of s, d, and h can be found by taking derivatives of this
mismatch equation. These can be found to be
These equations can be simplified by recalling at an OPF
solution Az =0, and that because of the structure of our
problem, k shows up as a linear term in both the gradient
a2H a2vL
and the Hessian [IO], therefore?=O and = O .
ak ak
Thus (6) and (7) may be simplified to
(9)
aH az aZz dH -,aVL
ak ak ak ak ak
- =2H-I -H - =-2H-I __
and - aH are extremely sparse. The partial
Both -
ak ak
avL
derivative- is a matrix which has exactly one entry in
each column, while - is 3-dimensional tensor that has
exactly one diagonal element in each matrix as you move in
the dimension corresponding to k. Because these are so
sparse, the amount of arithmetic is much smaller than it
would seem.
Using (8) and (9) to substitute into (4) and ( 5) , use (3) to
perform a Newton-step which improves an individuals
welfare. The following algorithm is used to determine an
individuals best bidding strategy.
ak
aH
dk
Algorithm: Individual Welfare Maximization
1. Choose an initial guess for vector k.
2. Solve the OPF maximization of social welfare given the
individuals assumption of other individuals bids and
the individuals guess at its own vector k.
Use (3) to determine a step direction for vector k.
If Ilkncw -k,II is below some tolerance stop, else go
back to step 2.
3.
4.
This algorithm will be effective as long as the binding
inequalities of the OPF algorithm do not change. Changes
in binding inequalities result in discontinuities of - af
ak
which means that the function f becomes non-differentiable.
The non-differentiability can be overcome by recognizing
that a change in binding inequality can be detected from
other available information. From one side of the non-
differentiable point, the value limited by the inequality
approaches its limit. From the other side of the point, the
Lagrange multiplier associated with the inequality
approaches zero. This is shown in Figure 4.
Welfare
Limited value Lagrange multiplier
I *
kin, Increasing
Figure 4 Binding Inequality Change
Using this information, the Individual Welfare
Maximization algorithm is modified so a multiplier reduces
the step direction determined by (3) if this step direction
684
will step across a non-differentiable point. The multiplier
instead tries to bring the answer directly to this non-
differentiable point.
6. Market Simulation
With the ability to determine an individuals best
response to the market, a market simulation can now be
performed to learn more about potential market behavior.
For example, the determination of economic equilibrium
points such as Nash equilibria [ 121 is of interest.
Definition: Nash Eauilibrium
An individual looks at its opponents behaviors
The individual determines that its best response to its
opponents behaviors is to continue its present behavior
This is trueFOR ALL individuals in themarket
To determine a Nash equilibrium the individual welfare
maximization can be iteratively solved by all individuals
until a point is reach where each individuals best response
is to continue with the same vector of bids. A similar
iterative technique for finding Nash equilibria was used in
[ 1 11, although difference individual maximization was used.
The following algorithm describes this process.
Algorithm: Find Nash Eouilibrium
0
Start all individuals with a bid vector k =1.
Run the Individual Welfare Maximization algorithm for
each individual and update all bids
Continue running this until individuals stop changing
their bids
6.1. Market With No Constraints
To demonstrate the Find Nash Equilibrium algorithm,
consider the two-bus example with two suppliers and one
consumer shown in Figure 5.
B2(d2) =k,,(-0.04dz2 +30dzn
g +jb =-j20.6
C,(S,) =k,,(0.01s12 +I OS , ) C,(S,) =k,,(0.01s,2 +l OS, )
Figure 5 Two-bus System: Two Suppliers and a Consumer
Only consider the supplier bidding behavior for this
example, therefore assume the consumer in this market
always bids according to it true benefit function, i.e. l ~ 2 =
1.00. It is important to maintain the price-dependent
demand. Otherwise, when a transmission line limit is added
to the system, which will be done shortly, supplier #2 could
have part of the constant load to serve with no competition.
The solution for supplier #2 would be to bid k.2 equal to
infinity. This is an unreasonable result.
The Find Nash Equilibrium algorithm results in both
suppliers bidding Is, = k2 =1.1502. This is called a pure
strategy equilibrium [ 121 because the equilibrium is at a
point where each bidder always bids that same value of k.
Figure 6 shows the bid progression of the algorithm toward
the equilibrium point. Figure 7 shows a complete solution
to the problem with the optimal response of each supplier to
any possible bid by the other supplier. The point where the
two curves in Figure 7 meet is the Nash Equilibrium point.
Iterations
Figure 6 Bid Progression with no Line Limit
z 1.4- ;Optimal Response of
.-
I
Supplier #1 to Bids by #2
1 .o
I 1
1.0 1.1 1.2 1.3 1.4 1.5 1.6
Variation of Supplier #l Bid
Figure 7 Optimal Responses with no Line Limits
6.2. Market With Constraints
To further demonstrate the algorithm, consider the
system of Figure 5 again, but now add a constraint to the
system: limit the flow on the transmission line to 80 MVA.
The bid progression that results from this system case is
shown in Figure 8.
1.6 I I I
0 5 10 15 20 25
Iterations
Figure 8 Bid Progression with 80 MVA Line Limit
685
This does not result in a Nash equilibrium point, but
limit cycle - like behavior. To better explain why no
equilibrium point is reached, the optimal response curves
over all possible bids by each individual are determined.
These are shown in Figure 9.
1.1 1.15 1.2 1.25 1.3 1.35 1.4 1.45 1.5 1.55 1.6
Variation of Supplier ##1 Bid
Figure 9 Optimal Responses with 80 MVA Line Limit
Figure 9 shows that the optimal response curves for the
two suppliers never intersect because of a discontinuity in
the supplier #2 best response curve. In order to determine
what is causing this discontinuity, supplier #2 profit vs. bid
curves are created on either side of the discontinuity. These
are shown in Figure 10.
variation of Supplier Ws Bid whan k., =1.35
*e
f
-
-
a
z
Supplier #2 Bid
Variation of Supplier WE Bid when k., I 1.40
Supplier #2 Bid
Figure 10 Supplier #2 Profit vs. Bid on either side of the
Discontinuous Point
Figure 10 shows that the profit function for supplier #2
is non-convex, having two local maxima. When supplier #1
bids k1 = 1.3720, these local maxima have the same
magnitude. At this point, supplier #2 has no preference
between bidding either ks2 of 1.525 or 1.246. On either side
of this point the optimal response Ijumps to the other local
maximum. Ultimately, this is the reason that no pure Nash
equilibrium exists. Thus, we have shown that the
introduction of a transmission line constraint, even in a
trivial two-bus system, eliminates our pure strategy
equilibrium point.
This does not mean that no Nash equilibria exist
however. Only pure strategies have been considered.
Mi xed strategies [ 121 are also possible. A brief definition of
a mixed strategy in our application follows.
Definition: Mi xed Strutenv
An individual chooses several pure strategies and assigns a
probability to each. The individual then submits these pure
strategy bids according to their associated probabilities.
Allowing the possibility of mixed strategy equilibria,
one can easily generate a Nash equilibrium. For the
previous example, one mixed strategy equilibrium is:
Nash Equilibrium for Line Limited Case
Supplier #1: Bid k,, =1.372 with Probability 1.00
Supplier #2: Bid ks2=1.246 with Probability 0.56 and
ks2=1.525 with Probability 0.44
When supplier #1 bids k,l =1.372, supplier #2 has no
preference between either bidding ks2 =1.246 or ks2 =1.525,
thus bidding these two pure strategies with arbitrary
probabilities is one optimal response. When supplier #2
bids these two pure strategies with the probabilities shown,
the expected profit vs. bid curve for supplier #1 is as shown
in Figure 11. In Figure 11, the maximum is indeed at a bid
This shows that while pure strategy equilibria are
eliminated by the inclusion of transmission limits, mixed
strategy Nash equilibria still exist.
of =1.372.
G m
Sup$er #;Bid 9, I2 Sup$er#iBld
For IG2 E 1.246 with Prob 0.56 and ku =1.525 with Prob 0.44
I * $ 1 t YI 3 1 8, s 11 ( 25 ( 1
Supplier#l Bid
Figure 11 Optimal Bids for Supplier #1 in Response to a
Mixed-Strategy by Supplier #2
7. Generalizing the Bid Curves
To this point, bidding in our electricity market has been
limited to one dimension through changes in the multiplier,
k, for each consumer and supplier. This does not allow the
bidder to vary the slope and intercept of the bid curve
independently. It can actually be shown that varying both
slope and intercept will not result in increased personal
welfare. To demonstrate this, the example with no line
limits from Section 6.1 is considered. The optimal
intercepts for several fixed slopes are determined, and the
resulting optimal bid curves are shown in Figure 12.
686
Optimal Curves for several fixed slopes
300
250
s
2 100
150
%
a
-
0)
50
6 10 12 14 16 18
Price [$/MWhr]
Figure 12 Optimal Curves for Several Fixed Slopes
These optimal bid curves all intersect at the same point
in the supply-price space. Also, for all bids the market
solution is at this intersection point. This means that
independently varying both the slope and intercept will not
result in a higher increase in personal welfare. From this it
is also learned that when performing the individual
maximization, the individual is really trying to find a point
in the supply/demand-price space that maximizes its profit.
The individual is merely looking for a bid curve that crosses
this point. The multiplier k used throughout this paper, is a
very good variation parameter. Varying k will cover the
entire supplyldemand-price space with positive values of
slope and x-intercept, thus allowing the algorithm to find the
desired point.
While the bidding algorithm is only looking for a single
point in this development, a future extension of this work
could study the effect that uncertainty in the estimates of
other individuals bids plays in moving this optimal point
around. It is likely that the shape of the bid curve could be
chosen to maximize personal welfare for small perturbations
around the optimal point found in this paper.
8. Conclusions
The algorithm developed has been successful in
analyzing the small systems shown in the paper. It has been
shown that iteratively using the objective of maximizing
personal welfare can be an effective way of simulating
electricity markets and studying the equilibrium behavior of
the market. The recognition that an individual is looking for
a point in the supplyldemand-price space is also important.
A future improvement that must be achieved is allowing
for the non-convexity of the individual welfare function as
shown in Figure 10. The calculus-based algorithm
presented here worked for small systems, but will be unable
to effectively find global maxima for individual welfare on
larger systems due to the non-convexity. Some sort of
hybrid algorithm utilizing the work done here along with a
genetic algorithm or simulated annealing technique will be
investigated later.
9. Acknowledgements
The authors would like to acknowledge the support of
the Power Affiliates program of the University of Illinois at
Urbana-Champaign and the Grainger Foundation.
10. References
[l ] F.C. Schweppe, M.C. Caramanis, R.D. Tabors and R.E.
Bohn, Spot Pricing of Electricily, Kluwer Academic
Publishers, Boston, 1988.
[2] H. Chao, S . Peck, A Market Mechanism for Electric
Power Transmission, Journal of Regulatory
Economics, J uly 1996, pp. 25-59.
[3] The PJM website found at htta://www.pim.com. See
section on Training for detailed information.
[4] T. Alvey, D. Goodwin, X. Ma, D. Streiffert, D. Sun, A
Security-Constrained Bid-Clearing System for the New
Zealand Wholesale Electricity Market, IEEE
Transactions on Power Apparatus and Systems, Vol.
13, No. 2, May 1998, pp. 340-346.
[5] The California Power Exchange website found at
htta:llwww.cal~x.corn. See section on About the PX
for detailed information
[6] J.D. Weber, T.J . Overbye, P.W. Sauer, Simulation of
Electricity Markets with Player Bidding, Proceedings
of Bulk Power Systems Dynamics and Control - IV
Restructuring, Santorini, Greece, August 24-28, 1998,
[7] P. Skantze, J . Chapman, Price Dynamics in the
Deregulated California Energy Market, Proceedings of
IEEE PES Society 1999 Winter Meeting, New York,
New York, J an. 31 - Feb 4, 1999.
[8] J.D. Weber, T.J . Overbye, C. DeMarco, Modeling the
Consumer Benefit in the Optimal Power Flow,
Decision Support Systems, Vol. 24, No. 3-4, March
[9] J .B. Cardell, C.C. Hitt, W.W. Hogan, Market power
and strategic interaction in electricity networks,
Resource and Energy Economics, 1997, pp. 109-137.
[lo] J.D. Weber, Implementation of a Newton-based
Optimal Power Flow into a Power System Simulation
Environment, M.S. Thesis from the University of
Illinois, J anuary 1997.
[ll]C.A. Berry, B.F. Hobbs, W.A. Meroney, R.P. ONeill,
W.R. Steward, Analyzing Strategic Bidding Behavior
in Transmission Networks, Game Theory Applications
in Electric Power Markets, 99TP-136-0, IEEE PES
Winter Meeting 1999, pp. 7-32.
[12]D.M Kreps, A Course in Microeconomic Theory,
Princeton University Press, Copyright 1990, Chapters
11 and 12.
pp. 331-339.
1999, pp. 279-296.
687