gooseing your paralleling scheme

GOOSEING YOUR PARALLELING SCHEME

PREPARED FOR

Power & Energy Automation Conference 31 JANUARY 2017

®

®

©Bl

ack

& V

eatc

h Ho

ldin

g Co

mpa

ny 2

011.

All

right

s res

erve

d.

©Bl

ack

& V

eatc

h Ho

ldin

g Co

mpa

ny 2

017.

All

right

s res

erve

d.

Power & Energy Automation Conference | GOOSEING YOUR PARALLELING SCHEME

BLACK & VEATCH | Introduction i

TABLE OF CONTENTS 1 Introduction ....................................................................................................................... 1

1.1 DTE Energy .............................................................................................................................. 1 1.2 Stone Pool Substation .......................................................................................................... 1

2 DTE Energy Functional Requirements ...................................................................... 3 2.1 Substation LAN Impacts ..................................................................................................... 3 2.2 Breaker Status for the Controllers ................................................................................. 5 2.3 Testing Method ...................................................................................................................... 5 2.4 Conceptual Architecture ..................................................................................................... 5

3 Detailed Design ................................................................................................................. 6 3.1 Remote I/O ............................................................................................................................... 6 3.2 Design Drawings .................................................................................................................... 7 3.3 Detailed Logic ......................................................................................................................... 7 3.4 Configurations ........................................................................................................................ 8 3.5 Lab Testing ............................................................................................................................... 8 3.6 Test Plan.................................................................................................................................... 9

4 Status ..................................................................................................................................... 9 5 Lessons Learned ................................................................................................................ 9 6 Bibliography ....................................................................................................................... 9

LIST OF FIGURES Figure 1 - LTC Controller with Traditional Circulating Current Equipment .................... 2 Figure 2 - P2P Default Configuration Settings .............................................................................. 3 Figure 3 - Original Substation LAN with Transformers and IEDs Required to

Provide Breaker Status ................................................................................................... 4 Figure 4 - Conceptual Architecture ................................................................................................... 6 Figure 5 - Logic Diagram ....................................................................................................................... 8

LIST OF FIGURES Table 1 - Remote I/O Requirements ................................................................................................. 6


BLACK & VEATCH | Introduction 1

1 Introduction In late 2012, DTE Energy embarked on deploying IEC 61850 at new substations in an effort to reduce costs. One area where DTE Energy sees the most cost benefit in deploying IEC 61850 is in GOOSE messaging. DTE Energy has been successful deploying two GOOSE schemes to date. Recently, DTE Energy decided to use GOOSE messaging to replace their transformer paralleling scheme. This paper discusses the new GOOSE application and its impact on the substation automation system architecture, the validation testing, the results of the installation, and any lessons learned from the implementation.

1.1 DTE ENERGY Founded in 1903, DTE Electric is the largest electric utility in Michigan and one of the largest in the nation. Based in Detroit, Michigan, DTE Energy is a diversified energy company that develops and manages energy-related businesses and services nationwide. Its operating units include DTE Electric, which generates, transmits and distributes electricity to 2.2 million customers in southeastern Michigan. With an 11,084 megawatt system capacity, the company uses coal, nuclear fuel, natural gas, hydroelectric pumped storage and renewable sources to generate its electrical output.A

1.2 STONE POOL SUBSTATION The Stone Pool Substation is a new Class “S” type substation located in Detroit, MI and built to serve new load in Midtown Detroit. The substation consists of three 120 kV-13.2 kV, 40MVA transformers and two, 12-position 13.2 kV metalclad switchgear/power distribution center (PDC) line-ups, fed in a 4-bus ring configuration. ITC Transmission Company owns and operates the 120 kV equipment up to the high-side disconnects. One future PDC lineup is planned. The low-side also includes four capacitor banks located in their own enclosure.

Each transformer was originally ordered with a load tap changer (LTC) on the low side of the transformer and a corresponding LTC controller and transformer monitor. The substation operation could operate all three transformers in parallel or in any combination.

The original design of the substation automation system specified two isolated Ethernet local area networks (LANs) for each PDC, each connecting the following types of intelligent electronic devices (IEDs):

1. Substation data concentrator 2. Protection relays 3. Transformer IEDs (originally a transformer monitor and LTC controller) 4. LTC controllers 5. Capacitor bank controllers 6. Substation computers

Today, one PDC includes 36 IEDs and the other contains 42 IEDs. Each PDC’s substation LAN includes Ethernet switches and routers that are substation rated and compliant with IEEE 1613 and IEC 61850-3. The LAN architecture is straightforward:

LAN A connects primary IEDs.

GOOSEing Your Paralleling Scheme | Power & Energy Automation Conference

2 FEBRUARY 2017

LAN B connects most backup IEDs. The data concentrator and substation computer are dual ported and connected to both LAN A

and LAN B. Other IEDs that are not redundant are connected to root switches.

This substation is the fourth substation to implement IEC 61850 and included the lessons learned from the previous projects:

1. DNP3 TCP/IP is the primary SCADA data collection protocol 2. IEC 61850 GOOSE is used for two schemes:

a. Fast bus trip scheme b. Capacitor bank control scheme

3. SNTP is the primary time synchronization protocol

The traditional transformer paralleling scheme was implemented on previous substations up until Stone Pool Substation. The traditional scheme uses a circulating current method that is typical to the industry and requires several devices in addition to the LTC controller (refer to Figure 1). The LTC controllers installed as part of the design support several different transformer paralleling methods1:

1. Traditional circulating current as described in [B, C].

2. Master/Follower. 3. ΔVAR® as described in [D]. 4. ΔVAR® method 1. 5. ΔVAR®2 (KeepTrack™). 6. ΔVAR® Peer to Peer (P2P), which

uses IEC 61850 messaging for paralleling up to sixteen transformers and requires the use of status inputs to each controller for knowledge of the status for each transformer’s load-side breaker, left breaker, and right breaker. Refer to Figure 2 for all P2P settings in the LTC controller.

1 Refer to the Beckwith Electric manual for the M2001D for a detailed technical description of all available paralleling methods.

Figure 1 - LTC Controller with Traditional Circulating Current Equipment


BLACK & VEATCH | DTE Energy Functional Requirements 3

Figure 2 - P2P Default Configuration Settings

2 DTE Energy Functional Requirements DTE Energy wanted to implement the P2P scheme instead of the traditional circulating current method, but the following had to be evaluated:

The substation LAN architecture Provision of breaker status to each controller Testing method

2.1 SUBSTATION LAN IMPACTS The original design provided complete separation of each PDC, with two transformers “belonging” to one PDC. The P2P requires the controllers to be able to “hear” the GOOSE messages from the other controllers involved in the P2P. Refer to Figure 3.


4 FEBRUARY 2017

Figure 3 - Original Substation LAN with Transformers and IEDs Required to Provide Breaker Status

There were several options available to ensure that the controllers could “hear” each other:

1. Encrypted VPN tunnel between the PDCs. a. The performance of the encrypted VPN tunnel was not known and the performance

requirements for the P2P were not published. These requirements are necessary to confirm proper function of the P2PE,F.

b. The routers would be in the critical path for the P2P GOOSE messages and they were not redundantF.

2. Move one of the transformers to the other PDC. a. There was limited availability of fiber optic ports, so a new Ethernet switch would

be required. b. The data concentrator configurations would require modification to move the data

collection to the other PDC. 3. Consolidate the LANS from two independent LANs into one flat LAN with a ring of switches

and a backup router to provide some improved performance. a. A new IP address scheme would be required as the old IP address scheme had

subnets that were too small for all of the IEDs. b. The data concentrator configurations would require updating to change the IP

addresses.

Substation

PDC 1

XFMR 1

G4TIE_G_87P

A1CBL_TIE_A_B87C

F1XFMR1_F_C67

H4CBL_TIE_H_50BF

M2001D

XFMR 2

M2001D

XFMR 3

M2001D

PDC 2

S5TIE_S_87P

M2CBL_TIE_M_B87

C

R2XFMR2_R_C67

T5XFMR3_T_C67

1LINE BREAKER

LEFT BREAKER

RIGHT BREAKER

LINE BREAKER

LEFT BREAKER

RIGHT BREAKER

LINE BREAKER

LEFT BREAKER

RIGHT BREAKER


BLACK & VEATCH | DTE Energy Functional Requirements 5

2.2 BREAKER STATUS FOR THE CONTROLLERS Impacting the decision on network architecture was how to provide the required breaker status to each controller (refer to Figure 4). While the concept sounds simple, the substation’s bus configuration:

1. Did not necessarily have one breaker status input for the load side breaker. 2. Did not necessarily have one breaker status input for the left breaker. 3. Did not necessarily have one breaker status input for the right breaker. 4. Some breaker statuses had to be sent to multiple controllers.

These breaker statuses could have been hardwired from each PDC to each controller as required, which was a lot of wiring that was not planned. Plus there was already a LAN connection to the transformer cabinets to support the transformer IEDs. A way was needed to minimize wiring changes while maximizing the communications network that was already installed. Another GOOSE application was born!

An additional set of GOOSE messages was added between each IED owning a breaker status required by the controller. A new IED was needed that could create three calculated outputs from the required breaker statuses, do so in a way that would cause the P2P to “fail safe” in the event that the new IED failed or had GOOSE communication problems, and fit into the available space in the control cabinet.

2.3 TESTING METHOD DTE wanted a test plan so that the P2P was completely tested. The vendor does provide a testing method in its standard manual, but it only provided a test procedure for testing the paralleling scheme application. There was no test plan for ensuring all of the IEDs involved were configured properly and communicating. There was no test plan for addressing communication failure modes and whether the application “failed-safe”.

2.4 CONCEPTUAL ARCHITECTURE The functional requirements were reviewed by a team comprised of the vendor, the substation contractor, and DTE Energy resources from protection, SCADA, IT, and transformers. The group decided to (refer to Figure 4):

1. Combine the substation LANs into one network. 2. Install a new remote I/O device in the transformer cabinets to calculate the required

breaker status points from another set of GOOSE messages. 3. Create additions to the test plan to ensure the proper operation of the IEDs and network

and to test different failure modes.


6 FEBRUARY 2017

Figure 4 - Conceptual Architecture

3 Detailed Design The detail design involved the following major elements:

1. Finding a remote I/O device that could meet a minimum set of requirements. 2. Updating the design drawings to show the design changes and provide a bill of material for

all new equipment. 3. Creating detailed logic to convert GOOSE messages to outputs that the LTC controller inputs

could use for breaker status. 4. Modifying the configurations of all impacted IEDs. 5. Lab testing to confirm proper operation. 6. Creating a test plan.

3.1 REMOTE I/O A set of minimum requirements was developed for the remote I/O to ensure proper support of the P2P application. The following requirements were developed and compared against several products available on the marketG. Refer to Table 1.

Table 1 - Remote I/O Requirements

DESCRIPTION REQUIREMENT

Substation

PDC 1

XFMR 1

G4TIE_G_87P

Remote I/O

A1CBL_TIE_A_B87C

F1XFMR1_F_C67

H4CBL_TIE_H_50BF

M2001D(3 or 4)

XFMR 2

Remote I/OM2001D(3 or 4)

XFMR 3

Remote I/OM2001D(3 or 4)

PDC 2

S5TIE_S_87P

M2CBL_TIE_M_B87

C

R2XFMR2_R_C67

T5XFMR3_T_C67

1LINE BREAKER

LEFT BREAKER

RIGHT BREAKER

LINE BREAKER

LEFT BREAKER

RIGHT BREAKER

LINE BREAKER

LEFT BREAKER

RIGHT BREAKER


BLACK & VEATCH | Detailed Design 7

DESCRIPTION REQUIREMENT

Size 8-1/2" h x 6-1/8" w x d 5.88" h x 5.88" w x d

GOOSE Subscription Limit (messages) Minimum 3

Logic Required

Inputs Minimum 1

Outputs Minimum 4

Output Voltage 120 VAC

Power Supply 120 VAC

Operating Temperature Outdoor-cabinet rated

Fiber optic Ethernet Required

The market for remote I/O devices has remarkably expanded since 2009. Several products were compared and one model recommended for implementation. DTE Energy approved the selection and detailed design began.

3.2 DESIGN DRAWINGS The following types of design drawings were modified:

Communication Block Diagram (2) Network Closet drawing Fiber Optic Network drawing Ethernet Network drawing (2) Transformer drawings (3) Networking Diagrams (2) Bill of materials (2)

3.3 DETAILED LOGIC Once the detailed design was completed, detailed logic was created based upon the conceptual design shown in Figure 4.

The final logic is shown in Figure 5. Highlights of the logic are to calculate the following for each LTC controller:

1. Calculate breaker status based upon received GOOSE messages and “Device OK”. 2. Calculate an internal logical point called “Device OK” to indicate either GOOSE communication

failure or IED failure. 3. Use the auto/manual switch from the LTC control panel and GOOSE message quality to calculate

a new auto/manual status that would place the LTC controller in manual if the auto/manual switch was in manual or there was any GOOSE communication failure.


8 FEBRUARY 2017

Figure 5 - Logic Diagram

3.4 CONFIGURATIONS The following configurations were modified or created:

1. Substation data concentrator configuration was modified to update the IP addresses and to poll the remote I/O device for data related to its status.

2. Remote I/O device configurations were created. 3. Eight relay configurations were modified to broadcast a new GOOSE message for the remote I/O

device to subscribe. 4. All other IED configurations were modified for a change in IP addresses.

3.5 LAB TESTING The configurations were loaded into a small test system that included three relays and three LTC controllers plus a complete mock-up of all of the LAN equipment. The goals of the lab testing were to confirm:

1. Network communications are as expected under network failure modes. 2. The modified configurations would correctly download to the IEDs without generating any

errors. 3. The configurations modified for the P2P scheme generated the expected network traffic. 4. The P2P failure modes performed as expected. 5. Data was reported to the data concentrator as expected.

The majority of testing time was spent on the first three items because of problems with the network switch configurations and properly modifying the LTC controller’s default GOOSE configurations.

CBL_TIE_A_B87C

IN1 GOOSE OUTPUT61850-INDGGIO01_IND01

TIE_G_87P

CBL_TIE_H_50BF

CBL_TIE_M_B87C

XFMR2_R_C67

TIE_S_87P

XFMR3_T_C67

XFMR1_F_C67

XFMR1_SEL-2411

XFMR1_F_C67.GCB1.PRO.INDGG01_IND01

XFMR2_SEL-2411

XFMR2_R_C67.GCB1.PRO.INDGG01_IND01

XFMR3_SEL-2411

DEVICE_OKSV05

CBL_TIE_A_B87C.GCB1.PRO.INDGG01_IND01

SV03

RIGHT_BKR_1VB002

SV05DEVICE_OK

OUT302

OUT303

SV05

OUT301

HALARM

VB008

VB009

VB010

VB011

CBL_TIE_A_B87C.GCB1.MessageQuality

TIE_G_87P.GCB1.MessageQuality

CBL_TIE_H_50BF.GCB1.MessageQuality

CBL_TIE_M_B87C.GCB1.MessageQuality

NOTES1. IN101 INPUT FOR 43TCS AUTO/MANUAL SWITCH. MANUAL POSITION IS HIGH INPUT. 2. SV05 USED TO FORCE ALL BREAKERS TO SHOW OPEN, AND LTCCONTROLLER TO MANUAL, UPON SEL-2411FAILURE INDICATION OR LOSS

OF ANY GOOSE INPUT. 3. MESSAGE QUALITY IS THE GOOSE MESSAGE QUALITY: A 1 INDICATES BAD QUALITY AND IS LOGICAL OR CONDITION OF ALL OF THE

GOOSE MESSAGE CHECKES FROM EACH DEVICE. SEL ARCHITECT SETTINGS DEFINING QUALITY ARE SET TO DEFAULTS. ALL SEL DEVICES BLOCK THE GOOSE MESSAGE ON BAD QUALITY.

4. THE NUMBER OF GOOSE ALLOWED IN THE SEL-2411 IS: 6 INCOMING AND 8 OUTGOING5. SEE SCHEMATIC DIAGRAM PTD01031L FOR THE OPERATION OF SEL- 2411 OUTPUTS6. THE GoID SETTING IS “LTCParallel”7. THE AppID SETTING IS 0004 HEX8. THE BROADCAST MAC ADDRESS IS 01-0C-CD-01-00-039. THE MAX TIME ALLOWED TO LIVE SETTING FOR THE RELAYS IS 1000 MS.

XFMR2_R_C67.GCB1.MessageQuality

TIE_G_87P.GCB1.MessageQuality

CBL_TIE_H_50BF.GCB1.MessageQuality

CBL_TIE_M_B87C.GCB1.MessageQuality

TIE_S_87P.GCB1.MessageQuality

XFMR3_T_C67.GCB1.MessageQuality

TIE_S_87P.GCB1.MessageQuality

XFMR3_T_C67.GCB1.MessageQuality

CBL_TIE_A_B87C.GCB1.MessageQuality

IN101

SV05

HALARM

VB008

VB009

VB010

VB011

VB012

VB013

SV05

OUT301

HALARM

VB008

VB009

VB010

TIE_G_87P.GCB1.PRO.INDGG01_IND01

CBL_TIE_H_50BF.GCB1.PRO.INDGG01_IND01

CBL_TIE_M_B87C.GCB1.PRO.INDGG01_IND01SV03

RIGHT_BKR_1VB002

VB003RIGHT_BKR_2

RIGHT_BKR_3VB004

OUT303

SV04

CBL_TIE_A_B87C.GCB1.PRO.INDGG01_IND01LEFT_BKR_1

DEVICE_OK

OUT304VB005

SV05

TIE_S_87P.GCB1.PRO.INDGG01_IND01

XFMR3_T_C67.GCB1.PRO.INDGG01_IND01 SV03

RIGHT_BKR_1VB002

VB003RIGHT_BKR_2

DEVICE_OKSV05

OUT303

1

SV04

LEFT_BKR_1

LEFT_BKR_2OUT304VB005

VB006

SV05 DEVICE_OK

TIE_S_87P.GCB1.PRO.INDGG01_IND01

XFMR3_T_C67.GCB1.PRO.INDGG01_IND01

1

52a


52a


52a


52a


52a


52a


52a


52a

1

1

1

SV051DEVICE_OK

SV02

LINE_BKR

DEVICE_OK

OUT302VB001

SV051

TIE_G_87P.GCB1.PRO.INDGG01_IND01

CBL_TIE_H_50BF.GCB1.PRO.INDGG01_IND01

CBL_TIE_M_B87C.GCB1.PRO.INDGG01_IND01SV04

LEFT_BKR_1VB005

VB006LEFT_BKR_2

LEFT_BKR_3VB007

OUT303

SV051DEVICE_OK

SV02

LINE_BKR

DEVICE_OK

OUT302VB001

SV051

SV01

SV01

IN101

OUT301

IN101

SV01

SV02

43TCS

43TCS

43TCS

VB012XFMR1_F_C67.GCB1.MessageQuality


BLACK & VEATCH | Status 9

3.6 TEST PLAN A test plan was created that added to the vendor’s test plan by providing a series of steps and tests to perform prior to performing the vendor’s recommended P2P application testing.

4 Status As of January 26, 2017, the installation was not complete. The schedule indicates at that time that testing may be completed by the time this paper is presented at the conference. Results will be presented at that time.

5 Lessons Learned The biggest lesson learned is that some vendors interpret the VLAN ID as hexadecimal and others as decimal, making the VLAN ID a new type called “hexordecimal”.

Another lesson learned is that vendors implementing their own GOOSE schemes in products that could be implemented in any substation automation system architecture need to provide technical documentation and configurability so that the scheme can be integrated into the overall substation automation system architecture, such as:

Logical network requirements (such as required VLAN ID and whether value is configured as hexadecimal or decimal).

Performance requirements for the application (what performance class from IEC 61850-5 is required so that application performs properly).

GOOSE message settings, such as VLAN ID, GOOSE ID, broadcast MAC address, and APP ID should be user configurable. VLAN priority and retransmission time should be allowed to be modified by the user unless adverse performance results.

IEC 61850 CID files should be provided with instructions on how to download.

6 Bibliography A DTE-About DTE, retrieved 2017/01/24, from https://newlook.dteenergy.com/wps/wcm/connect/dte-web/home/about-dte/common/about-dte/about-dte B Application Note #11, “Introduction to Paralleling of LTC Transformers by the Circulating Current Method”, available from http://www.beckwithelectric.com/docs/app-notes/appn-11.pdf. C Application Note #13, “Advanced Paralleling of LTC Transformers by the Circulating Current Method”, available from https://www.beckwithelectric.com/docs/app-notes/appn-13.pdf. D Application note #24, “Advanced Paralleling of LTC Transformers by DVAr™ Method”, available from http://www.beckwithelectric.com/docs/app-notes/appn-24.pdf. E Preuss, Boles, and Nutter, “Are Fast Flying GOOSE Messages a Myth?”, presented at the 2016 Power & Energy Conference. F IEEE P2030.100™, Draft 11, “Recommended Practice for Implementing an IEC 61850 Based Substation Communications, Protection, Monitoring and Control System”. G Preuss, “Applications for Distributed I/O in Substation Integration and Automation Systems”, presented at the 2009 Western Power Delivery Automation Conference.

https://newlook.dteenergy.com/wps/wcm/connect/dte-web/home/about-dte/common/about-dte/about-dte

https://newlook.dteenergy.com/wps/wcm/connect/dte-web/home/about-dte/common/about-dte/about-dte

http://www.beckwithelectric.com/docs/app-notes/appn-11.pdf

https://www.beckwithelectric.com/docs/app-notes/appn-13.pdf

http://www.beckwithelectric.com/docs/app-notes/appn-24.pdf

gooseing your paralleling scheme

Documents