development of a multi-agent management system for an ... · sim ulation results sho w the e ectiv...

Development of a multi-agent management

system for an intelligent charging network

of electric vehicles ?

Joao Miranda ∗ Jose Borges ∗ Mario J. G. C. Mendes ∗∗

Duarte Valerio ∗

∗ IDMEC/IST, TULisbon, Av. Rovisco Pais 1, 1049-001 Lisboa,Portugal (e-mail: [email protected], [email protected],

[email protected]).∗∗ IDMEC/ISEL-Instituto Superior de Engenharia de Lisboa, Rua

Conselheiro Emıdio Navarro 1, 1959-007 Lisboa, Portugal(e-mail: [email protected])

Abstract: This paper addresses the modelling and simulation of a battery charging infras-tructure for electric vehicles, with the objective of pro-actively scheduling the charging of upto fifty vehicles so as not to overcharge the electrical network. Benefits of having the chargingstations differ (as much as possible while satisfying end-user requirements) battery chargingfor those hours when electricity consumption is otherwise low include rendering electricityconsumption more uniform along the day. A multi-agent system was used to design a distributed,modular, coordinated and collaborative multi-agent management system for this infrastructure.Simulation results show the effectiveness of this approach under the conditions of four real-lifescenarios.

Keywords: Modeling and simulation of power systems, Intelligent control of power systems,Smart grids, Energy Management Systems, Multi-Agent Systems, Electric Vehicles

1. INTRODUCTION

Electric Vehicles (EV) provide an alternative technologyallowing diversity in energy sources for transportation,therefore reducing oil dependency, while providing mobil-ity at competitive costs. Independent market analysis byGartner and Wheelock (2010) forecasts a steep expansionof EV market in the coming 10 years. The charging in-frastructure for EV will increase stress on the power gridand, therefore, needs to be managed in order to complywith both charging needs and end-client satisfaction, whileavoiding disruptions in the electricity distribution.

The management and control of electricity distribution isa complex problem for both the utility grid and smallscale distribution networks. The dynamics within thesenetworks are the result of interactions between severalindividual components and objectives: on one hand wehave the grid infrastructure, which includes hardware andsoftware, on the other hand we have the objectives of allstakeholders associated with the distribution chain, whichinclude electricity producers, owners of transmission lines,retailers and end-clients. In order to comply with all these,sometimes conflicting, interests, a major research efforthas been put towards the development of smart gridsthat can provide management and control at the networkinfrastructure level, while delivering high quality servicesin the electricity distribution chain.

? This work was supported by FCT, through IDMEC, under

LAETA.

Therefore, it is imperative to implement autonomous anddecentralized management platforms that are capable ofcoordinating charging operations while ensuring safety,preserve efficiency and smartly manage the distributionof electricity for the EV batteries. Additionally, there is aneed of suitable management software for distribution thatcan deliver extended services and support new paradigmsof business models for the electricity retailers. The impor-tance of this problem will increase as the market share ofEV increases.

In order to implement an autonomous and decentralizedmanagement system to charge the batteries of EV, newstrategies are needed for this implementation and a goodoption is to use Distributed Artificial Intelligence (DAI)techniques. In particular, methodologies based on multi-agent systems (MAS) are a good choice to design a dis-tributed, modular, coordinated and collaborative multi-agent management system for an intelligent charging net-work of electric vehicles.

This paper is organised as follows: section 2 presents thelayout of the system, section 3 its modelling employing aMAS, section 4 its implementation in Matlab, and section5 the simulation results obtained. Conclusions are drawnin section 6.

2. SYSTEM LAYOUT

This paper addresses a battery charging station for elec-tric vehicles (Borges et al., 2009), assumed to have fiftycharging points for as many vehicles. Such stations will be

Preprints of the 18th IFAC World CongressMilano (Italy) August 28 - September 2, 2011

Copyright by theInternational Federation of Automatic Control (IFAC)

12267

implemented in parking lots, and so parked cars may ormay not be charging in each instant (because the battery isalready full, or because too many cars are trying to chargetheir batteries at the same time). Different neighbourhoods(e.g. residential, commercial or office areas) are expectedto lead to very different electricity demands. Users willpay electricity according to a previously chosen profile,that dictates the car’s priority in obtaining electricity:cheaper prices will correspond to lower priorities and henceto longer charging times; to ensure shorter charging timesand thus higher priorities the price paid for the electricitywill be higher.

The objective is to satisfy charging requests, respectingthe hierarchy of priorities defined by what each client ispaying, and all this without overcharging the electricalnetwork. Indeed, the idea is that charging stations will, tothe maximum extent possible, differ battery charging forthose hours when electricity consumption is otherwise low.In this manner, peak hours when electricity consumptionis higher (typically during the day) will not be furtherstrained, while those periods of the day (typically, theseare rather periods of the night) when electricity could becheaply available will be put to good use. The benefits ofthus rendering electricity consumption more uniform alongthe day could be enormous.

To achieve this a two level decision system was conceived:1) charging stations must communicate with each otherand decide, taking into account the expected number ofcars in the next few hours and their charging priorities,how the available electricity will be split among them; 2)each charging station must then distribute the electricityit has been allocated among the cars therein parked.Trying to sort things out taking into account every singleindividual car in all the charging stations of a largegeographical region would likely prove to be impossiblefrom the computational point of view. This would also bevery liable to fail because of communication failures. Inthis way, a high level distribution will first be performed,for some large time interval, and then each charging stationwill be left with some autonomy to deal with its localconditions, allocating electricity to particular vehicles,with the ability of changing the allocation in shorter timeintervals, and being thus flexible to deal with unforeseeablefluctuations in demand.

This paper only considers this second level, assumingthat the first has already been carried out, and that theelectricity available in each instant has been determined.Thus the layout corresponds to that of figure 1, wherethe Local Power Manager (LPM) is responsible for thecharging station under study. Further assumptions are asfollows:

• the base price of energy was taken from the regulatedmarket defined by legislation;

• vehicle-to-grid and vehicle-to-home concepts (wherebycars may, if that is found profitable, provide energyto the grid or to the owner’s home (Botsford andSzczepanek, 2009)) were not considered;

• no fast charge of batteries (whereby batteries canreach 80% of their maximum capacity in only 30minutes (Botsford and Szczepanek, 2009)) was con-sidered;

Fig. 1. System layout

• batteries were considered to allow interrupted charg-ing without damaging performance.

3. MULTI-AGENT MODELLING

In face of the above, the concept of agent (without accessto data from all the system, but only partial informationfrom a small part of the system) comes as a solutionto this problem. MAS are an “artificial social system”(Wooldridge, 2002), composed by individual agents ableto decide autonomously and with “social” capabilities(communicating and collaborating with each other), inorder to achieve their own design objectives. MAS are usedin several different research, operation and business areas.Macal and North (2006) suggest its use in areas as businessand organisation, economics, society and culture, militaryand biology. Applications based on MAS are also beingintroduced to fault tolerant control (Hines and Talukdar,2003; Mendes, 2008) and largely used in power networks(Negenborn, 2007) and its robustness is proved with theapplications in military services (Phillips et al., 2006).

In the particular plant addressed in this paper, MASprovide a very efficient tool, since we are dealing with acomplex system, with many different ramifications (thecars) that do not depend on each other. MAS allow usto develop a managing platform that would otherwisedeal with too much information, by decentralising it, andgaining advantage of DAI flexibility. A two layer MASwas elaborated, where two different design objectives areimplicit: one layer is responsible for managing the park,decide on which car to charge and ensure the power limitwill not be exceeded; the second layer will be composedby agents, each one of them responsible by a car withthe objective of charging it according to the costumerneeds. The communication between those two layers isestablished by an auction, described below in section 4.3.

3.1 Macro-level architecture

Due to the specific characteristics of a distributed electriccharging system, the MAS macro-level architecture pro-posed in this work is a federated architecture (shown infigure 2), since it allows the natural common relations andcommunications between the different charging agents,where the different charging point (CP) agents only needto communicate with a Facilitator, and not between them.The LPM agent takes on the role of Facilitator, whilethe CP agents have no direct communication betweenthem, being coordinated by the LPM Agent, which willconduct an auction process, by sending data to the CP


12268

Fig. 2. Multi-Agent System: Macro-level architecture

Fig. 3. Local Power Manager Agent Architecture

Agents, receiving back the bids and assigning the chargingconfirmations according to the Auction inference system(IS) rules.

3.2 Micro-level architectures

The design of micro-level agent architectures should rep-resent and define individual agents’ behaviour and inter-actions.In this system, both the LPM agent and the CPagents should be intelligent agents to provide the necessaryperformance in the charging process. To achieve some dif-ferent degree of intelligence and to correctly design the twodifferent behaviours of these agents, two different types ofmicro-level architectures have been designed.

The LPM agent will be responsible for managing the carparking lot and in particular the communications withthe CP agents, providing data for the auction from theelectricity stake-holders such as energy availability andprices. The architecture for the LPM agent is shown infigure 3. The communications layer is constituted by theexternal link (responsible for the communications with theexterior of the park) and the CP link (responsible for thecommunications with the CP agents). In the operationslayer, the data base conceptually represents an informationbuffer and connections. The decision making module willperform the negotiations with the other LPM agents (notaddressed in this paper). The auction IS will manage theauction process and assign the charging confirmations tothe respective CP agents. Both will be conditioned by thegoals module, that represents the overall objectives of theLPM Agent.

The CP agent will be responsible for ensuring maximumbattery charge, respecting the client profile. Decisions onbids offered during the auctions will be supported by datainformation provided by the car in real-time, by the LPMagent, and by a model of the battery. CP agent architec-ture is presented in figure 4. The communications layer isconstituted by the LPM link (responsible for exchanginginformation with the LPM agent), the sensor (to receiveinformation from the EV battery) and the actuator (thatsends the charge confirmation). The operations Layer con-

Fig. 4. Charging Point Agent Architecture

Fig. 5. Evolution of the battery’s SOC during the chargeprocess

tains a data base (with the same function as that in theLPM agent), the bid decider (to perform the auction bid),and the battery model evaluation (to evaluate batterycharging behaviour).

4. IMPLEMENTATION

4.1 Battery

The battery charging process is non-linear and was mod-elled as shown in figure 5, that gives the battery’s state ofcharge (SOC, the ratio between energy stored and maxi-mum capacity) as a function of time for an uninterruptedcharge. This behaviour was implemented with a Takagi-Sugeno fuzzy logic model (Michels et al., 2007), so thatin the future it will be possible to make use of past dataon the evolution of the SOC with time during the chargeof each particular battery, adopting different models fordifferent batteries. Thus it will be taken into accountthat not all batteries charge in the same way, because oftheir past history and different wear off. This adaptivemodelling has not been implemented yet.

4.2 Multi-agent platform

To implement the multi-agent system described in section3, the FTNCS-MAS Designer toolbox (Mendes et al.,2009) for Matlab was used. This toolbox allows config-uring macro-level architectures simply, automatically im-plementing the communications between agents allocatedin different computers with the User Datagram Protocol(UDP). It was chosen because, while designed for FaultTolerant Networked Control problems as the name standsfor, it is very versatile and suits very well the problemunder study. The agents were distributed over five personal


12269

Fig. 6. Agent distribution over the five computers

computers, each containing 10 CP Agents an as shown infigure 6.

Because the UDP (unlike the Internet Protocol Suite(TCP/IP), for instance) does not ensure that informationsent by a computer is effectively received by another(which means that the data packages send in every instantcould be lost), the following strategies were followed:

• a sampling time of 1 second was used, which isenough, and avoids heavy network traffic;

• flag variables were sent over 20 seconds (the proba-bility of loosing data every single second during these20 seconds is neglectably low);

• variables were fed back after being received, to ensureno communication failures.

Furthermore, the RT block toolbox (Daga, 2008) was usedto ensure that all agents run in a constant ratio betweenreal time and simulation time (otherwise some batteriesmight take so long to stop charging when ordered to, whileothers were already starting to charge, that there wouldbe very short but significant energy consumption peaks,for about 1 sampling time).

4.3 Auction

CP agents interact by an auction, as mentioned above insection 3, so as to sort out among them which vehicles arebeing charged in each instant. This auction is organised bythe LPM every 15 minutes (this period was chosen becauseit is the frequency with which available information isupdated by the Portuguese electric grid owner), and eachCP agent makes a bid B for the energy C15 that it cancharge during the next 15 minutes. These bids bear norelation to the price the client is actually paying, beingsolely values used internally by the system. The LPMsorts the bids by price as seen in figure 7, and honours asmany requests as possible, by order of bid, up to the limitimposed from grid conditions. (As mentioned in section 2,this limit will in the future be determined by a negotiationbetween the LPMs of different charging stations.)

Bids are reckoned as

B = PT

(

1 + αTfull + βT50 + γHpt

)

(1)

where

• PT is the electricity base price (given by the LPM)for the current time instant T ;

• Tfull = ffull

(

tleft

tfull

)

, where tleft is the time the car

is expected to remain plugged (according to the

Fig. 7. Sorting of bids and honoured requests

client’s profile), tfull is the time needed to reachSOC= 100% under continuous charge (as computedby the battery model), and ffull is a monotonouslydecreasing function;

• T50 = f50

(

tleft

t50

)

, where t50 is the time needed

to reach SOC= 50% under continuous charge (ascomputed by the battery model), and f50 is amonotonously decreasing function;

• Hpt = fpt

(∑

3

t=1PT+t−PT

PT

,150

∑

50

i=1ti

left

tleft

)

, where PT+t

is the electricity base price in the future time instantT + t (hence the first fraction reflects how the price ofelectricity will evolve with time), tileft is the time thecar in charging point i is expected to remain plugged(which information, according to the client’s profile,is given by the LPM, and hence the second fractionreflects the relation between the time left for each carand the average of the time left for all others), and fpt

is a function that monotonously increases with bothits independent variables;

• and α, β and γ are coefficients that depend on theclient’s profile.

Functions ffull, f50 and fpt are currently linear and wereimplemented with a Takagi-Sugeno fuzzy logic model, sothat in the future it will be easy to render them non-linear,optimising the relation according to what experience showsmore useful.

5. RESULTS

There being as of writing no market studies to substantiatewhat the best commercial strategy for a charging stationsmight be, three different profiles that appeared reasonablewere hypothesised:

(1) priority client, willing to pay much to fully charge thebattery as soon as possible at all times (such clientswill expectedly be a minority);

(2) client who cannot charge at home and must thuscharge during the day, when prices are higher, withthe objective of reaching SOC=50% paying as less aspossible;

(3) client who can charge at home and thus does not needto charge during the day, as long as the battery isabove SOC=50%.


12270

Table 1. Coefficients for (1) for each profile

Profile

1 2 3

α 1.0 0.3 0.4

β 0.0 0.3 0.6

γ 0.0 0.4 0.0

These profiles, corresponding to the parameters of (1)given in table 1, were used to carry out tests in fourdifferent scenarios:

(1) an office zone, where the affluence is significant from8 AM to 19 PM, and most clients chose profile 2;

(2) a residential zone, where the affluence is the oppositeof that of scenario 1, and most clients chose profile 3;

(3) a commercial zone, with high rotation of cars in the12 AM to 12 PM period, and mixed profiles;

(4) a car silo, with high affluence at all hours, and mixedprofiles, the mix depending on the hour (at nightthere will be more profile 3 clients, during day hoursprofile 2 will be more significant).

Car affluence is shown for the four scenarios in figure 8,which also shows how many cars are effectively able tocharge at each instant. Figure 9 shows power availabilityassumed for each scenario (after negotiation with otherLPMs). Figure 10 shows how many clients of each profilewere assumed for each scenario, how their batteries werecharged when arriving at the park, and how they werecharged when leaving. These results show that the objec-tives of each profile are being satisfied in each case; theease with which that is done depends, of course, on theconstraints of each scenario (e.g. in scenario 3, profile 1cars cannot often be fully charged because they seldomremain long enough). It seems thus reasonable to assumethat the modelled system satisfies its requirements. Whenactual data on car affluence, available energy, clients’ ex-pectations, etc. becomes available, the flexibility of theimplementation is expected to allow adapting the systempromptly.

6. CONCLUSIONS

Distributed charging systems for EV can be modelledusing MAS. We have discussed the main features of suchmodel, namely both its macro- and micro-level architec-tures, and implementation issues, such as an auction mech-anism. The MAS model that has resulted complies withload constraints imposed by the grid while attempting tomaximise the number and charge of plugged-in EV in thesystem. Three different profiles, which characterise groupsof typical EV owners, were defined for the CP Agents.The model was simulated under conditions of four real-life scenarios in order to understand both the distributionnetwork dynamics and the EV affluence in the parkinglot, therefore providing a customisable tool to support thedesign and dimensioning, as well as assessing the economicviability, of such type of charging infrastructures.

REFERENCES

Borges, J., Borges, A., Campos, J., and Nina, M. (2009).Sistema de distribuicao e disponibilizacao de energiaelectrica para carregamento de baterias de veıculos

Fig. 8. Car affluence and batteries charging for scenarios1 to 4(from to top to bottom)

electricos (in Portuguese). 104395. Portuguese Instituteof Industrial Property.

Botsford, C. and Szczepanek, A. (2009). Fast charging vsslow charging : Pros and cons for the new age of electricvehicles. EVS24 International Battery, Hybrid and Fuel


12271

Fig. 9. Power availability assumed for scenarios 1, 3 and 4(top) and for scenario 2 (bottom)

Cell Electric Vehicle Symposium, 1–9, Norway.Daga, L. (2008). URL http://leonardodaga.insyde.it/Simulink/RTBlockset.htm.

Gartner, J. and Wheelock, C. (2010). Electric vehiclecharging equipment. Technical report, Pike Research.

Hines, P. and Talukdar, S. (2003). Autonomous agentsand cooperation for the control of cascading failuresin electric grids. Proceedings. 2005 IEEE Networking,Sensing and Control, 2005., 273–278.

Macal, C. and North, M. (2006). Tutorial on agent-based modeling and simulation part 2: how to modelwith agents. Proceedings of the 2006 Winter SimulationConference, 73–83.

Mendes, M.J.G.C. (2008). Multi-Agent Approach to FaultTolerant Control Systems. Ph.D. thesis, Instituto Supe-rior Tecnico - Universidade Tecnica de Lisboa.

Mendes, M.J.G.C., Santos, B.M.S., and Sa da Costa,J. (2009). A Matlab/Simulink multi-agent toolkit fordistributed networked fault tolerant control systems.In SAFEPROCESS’2009, Preprints of the 7th IFACSymposium on Fault Detection, Supervision and Safetyof Technical Processes, 1073–1078, Barcelona, Spain.

Michels, K., Klawonn, F., Kruse, R., and Nurnberger,A. (2007). Fuzzy Control: Fundamentals, Stability andDesign of Fuzzy Controllers. Springer, Berlin.

Negenborn, R.R. (2007). Multi-Agent Model PredictiveControl with Applications to Power Networks. Ph.D.thesis, Technische Universiteit Delft.

Phillips, L., Link, H., Smith, R., and Weiland, L. (2006).Agent-based control of distributed infrastructure re-sources. Technical Report January, U.S. Departmentof Commerce: National Technical Information Service.

Wooldridge, M. (2002). An Introduction to Multi-AgentSystems. John Wiley & Sons, Ltd. Chichester, England.

atarrival

whendeparting

atarrival

whendeparting

atarrival

whendeparting

profile 1 (7%) profile 2 (62%) profile 3 (32%)

SO

C/

%

0

20

40

60

80

100

atarrival

whendeparting

atarrival

whendeparting

atarrival

whendeparting


SO

C/

%

0

20

40

60

80

100

atarrival

whendeparting

atarrival

whendeparting

atarrival

whendeparting


SO

C/

%

0

20

40

60

80

100

atarrival

whendeparting

atarrival

whendeparting

atarrival

whendeparting


SO

C/

%

0

20

40

60

80

100

Fig. 10. SOC of batteries when arriving and when leavingthe charging station for scenarios 1 to 4 (from to topto bottom)


12272

development of a multi-agent management system for an ... · sim ulation results sho w the e ectiv...

Documents