november 2, 2007 nerc compliance workshop assessing compliance presenter name presenter title...

November 2, 2007

NERC Compliance Workshop

TEXAS REGIONAL ENTITY

AN INDEPENDENT DIVISION OF ERCOT

Assessing Compliance

Presenter NamePresenter Title (Optional)

Audits and Spot Checks

Robert Potts



NERC Compliance WorkshopNovember 2, 2007

3



NERC and ERCOT Protocol and Guide Audits

● Some registered market participants within the ERCOT footprint will be audited every three or six years depending upon their registration. Transmission Owners and Qualified Scheduling Entities will be

audited at a minimum of every three years. Additional audits may be performed if the Texas RE finds it is necessary.

Generator Owners, Generator Operators, Distribution Providers, and possible others will be audited at a minimum of every six years. Additional audits can be scheduled if the Texas RE finds it is necessary.

● The Texas RE will create an audit schedule in the year prior to the actual audit year. This schedule will be sent well in advance of the scheduled audit date and posted on the Texas RE website.


4



How Does One Prepare for an Audit?

The audited entity will receive a pre-audit package from the Texas RE containing the following:

● List of NERC Standards being actively monitored● NERC audit questionnaire● List of Texas RE auditors signed confidentiality documents and

NERC representative confidentiality documents (FERC representatives may be present as well)

● ERCOT Protocol and Guide questionnaire● Reliability Standards Audit Worksheets (RSAWS)● General information request (company profile, hotel info, etc.)● Audit agenda


5



How Does One Prepare For An Audit? (cont.)

● The pre-audit package will be sent to the entity at least two months prior to the audit.

● The entity will answer questionnaires, supply all needed information such as hotel information and company profile.

● The entity will fill out the RSAWS information that describes what documents cover the NERC standards that are being evaluated.

● The entity will decide if there is any conflict with any proposed member of the audit team and notify the Texas RE if there is any.

● Within one month of the stated audit date, the entity will return the information to the audit team leader at the Texas RE.


6



How Does One Prepare For An Audit? (cont.)

● Most audits in 2008 are two-day audits. As the number of actively monitored standards increases, it will

take longer to perform the audits (once scheduled for one day). The more up-front work that is accomplished, the faster the on-

site visit will be.● The onus is on the entity being audited to provide the appropriate

information to prove it is meeting the requirements of the NERC standards and ERCOT protocols and guides.

● Some audits will be tabletop audits that are performed by the Texas RE using information that is received from the entity.

● It is imperative that the most factual information be available.


7



The On-Site Audit – During the Audit

● Audit team will need a work area for several lap-top computers.● Auditors will need the documented evidence to be available and a

subject matter expert to be available to answer questions or provide additional documentation.

● The audit team will split into two or more groups: One group will interview operators. One group will interview IT, relay, communication personnel, or

others.● All subject matter listed in the NERC standards and ERCOT

protocol questionnaires will be addressed at the audit.● If other NERC standard violations are found during the audit, they

will be reported.


8



The On-Site Audit – End of the Audit

● The audit team will meet with management and provide a brief accounting of their findings.

● The audited entity will be told if there are preliminary non-compliance findings.

● The audit team may want to keep all documentation that was issued in the audit or some of the documentation. The right to remove any and all documents from the audit site is

given to the auditors by FERC Order 672.


9



Post-Audit

● Within two months of the on-site audit a draft report will be sent to the audited entity.

● The entity will have an opportunity to review the report and provide comments, if necessary.

● If there are findings of non-compliance, they will be reported to NERC prior to the report being completed.

● Within two weeks of the draft report, a final report will be issued by the Texas RE.


10



Spot Checks

● Spot checks by the Texas RE may consist of Data submittals, Site visits, Observations, Compliance investigations, and/or Tabletop audits.

● Texas RE may decide to focus on one or more NERC standards in a spot check.

● Spot checks may be conducted on-site, by phone, or through a tabletop audit.

● The entity will be advised of a spot check 20 days prior.


11



Audits and Spot Checks

Questions?


12



Complaints or Incident Reports

Robert Potts




13



Submitting a Complaint

● Complaints may be made by ERCOT Operations due to incidents that occur in real-time operations where matters of possible non-compliance cannot be resolved.

● Market participants have the right to file complaints against ERCOT using this process as well.

Refer to Texas RE Complaint Form on Compliance Monitoring and Enforcement Program of ERCOT.com:

http://www.ercot.com/mktrules/compliance/tre/enforcement/index.html


14



Determining a Violation

● When Texas RE receives a compliant, a request for information will be made to the entity that the complaint was about. This will give the Texas RE the chance to view the complaint from both viewpoints prior to making a judgment call on non-compliance

● Once the Texas RE receives the response from the entity, a determination is made to whether or not there is a violation (either NERC and/or protocol or guide).


15



Violations Process

● If the determination is made that a violation did occur, the Texas RE will inform the proper entity (NERC or PUCT).

● A violation notice will be written if the violation is ERCOT protocol- or guide-related. The entity in violation will be notified, and a copy will be sent to the PUCT.

● If the violation is a NERC standard violation, Texas RE will notify the violating entity and copy NERC.

● There are appeals processes that the entity may use in all violations whether ERCOT protocol / guides and/or NERC.


16



Complaints or Incident Reports

Questions?

November 2, 2007

NERC Compliance Workshop



NERC Compliance Workshop Event Investigations

Bob Collins


18



Example

An event that recently happened● Affected 50 million people● Affected eight states and a Canadian province● Estimated cost range between $4 Billion to $10 Billion● Investigation found:

Four causes• Inadequate system understanding• Inadequate situational awareness• Inadequate tree trimming• Inadequate RC diagnostic support

Seven NERC violations● One of the reasons for EPUC 2005

Reference: Final Report on the August 14, 2003 Blackout in the United States and Canada US-Canada Power System Outage Task Force Report – April 2004


19



Sequence of Events

● 2:02 p.m. First 345 kV line in Ohio fails due to contact with a tree in Walton Hills, Ohio.

● 2:14 p.m. An alarm system fails at FirstEnergy's control room and is not repaired.

● 2:27 p.m. A second 345 kV line fails due to a tree. ● 3:05 p.m. A 345 kV transmission line known as the Chamberlain-Harding

line fails in Parma, south of Cleveland, due to a tree. ● 3:17 p.m. Voltage dips temporarily on the Ohio portion of the grid.

Controllers take no action, but power shifted by the first failure onto another 345 kV power (Hanna-Juniper interconnection) line causes it to sag into a tree at 3:32 p.m., bringing it offline as well. While Mid West ISO and FirstEnergy controllers try to understand the failures, they fail to inform system controllers in nearby states.

● 3:39 p.m. A FirstEnergy 138 kV line fails. ● 3:41 A breaker connecting FirstEnergy's grid with American Electric Power

are tripped as a 345 kV power line (Star-South Canton interconnection) and fifteen 138 kV lines fail in northern Ohio. Later analysis suggests that this could have been the last possible chance to save the grid if controllers had cut off power to Cleveland at this time.


20



Sequence of Events (cont.)

● 3:46 p.m. A 345 kV line known as the Tidd-Canton Central line trips. ● 4:06 p.m. A sustained power surge on some Ohio lines begins an

uncontrollable cascade after another 345 kV line (Sammis-Star interconnection) fails.

● 4:09:02 p.m. Voltage sags deeply as Ohio draws 2 GW of power from Michigan.

● 4:10:34 p.m. Many transmission lines trip out, first in Michigan and then in Ohio, blocking the eastward flow of power. Generators go down, creating a huge power deficit. In seconds, power surges out of the east, tripping east coast generators to protect them, and the blackout is on.

● 4:10:37 p.m. The eastern Michigan grid disconnects from the western part of the state. Two 345 kV lines in Michigan trip. A line than runs from Grand Ledge to Ann Arbor known as the Oneida-Majestic interconnection trips. A short time later, a line running from Bay City south to Flint in Consumers Energy's system known as the Hampton-Thetford line also trips.

● 4:10:38 p.m. Cleveland separates from the Pennsylvania grid.


21




● 4:10:39 p.m. 3.7 GW power flows from the east through Ontario to southern Michigan and northern Ohio, more than ten times larger than the condition 30 seconds earlier, causing a voltage drop across the system.

● 4:10:40 p.m. Flow flips to 2 GW eastward from Michigan through Ontario, then flips westward again in a half second.

● 4:10:43 p.m. International connections begin failing. ● 4:10:45 p.m. Western Ontario separates from the east when the Wawa-

Marathon 230 kV line north of Lake Superior disconnects. The first Ontario plants go offline in response to the unstable system.

● 4:10:46 p.m. New York separates from the New England grid. ● 4:10:50 p.m. Ontario separates from the western New York grid.


22



● 4:11:57 p.m. The Keith-Waterman, Bunce Creek-Scott 230 kV lines and the St. Clair-Lambton #1 and #2 345 kV lines between Michigan and Ontario fail.

● 4:12:03 p.m. Windsor, Ontario and surrounding areas drop off the grid. ● 4:13 p.m. End of cascade. 256 power plants are off-line. Eighty-five percent

went offline after the grid separations occurred, most of them on automatic controls.



23



Event, August 14, 2003 – The Night Before


24



Event, August 14, 2003 – The Night Of


25



ERCOT Event – April 17, 2006

● Created a EECP Step 1 Event (RR below 2300 MW)

● Frequency lowest value: 59.782 Hz

● Responsive reserve lowest point: 1539.25 MW

● Maximum load for the day: 52319 MW

● EECP Step 4 ERCOT-ordered statewide rolling blackouts

● Contributing factors: Unseasonable weather – temps in the 90s normally in the 70s Capacity shortage – 15% of units were down for scheduled

maintenance Four large units (1200 MW of capacity) tripped off-line during peak time

● Results of investigation: Created adjustable responsive reserves

• Gives a more accurate status on reserves

Daily grid conditions reports


26



Texas RE Role in Event Investigation

● What governs Texas RE in event investigations are rules and regulations: ERCOT operational guides and protocols NERC standards

● Texas RE role begins after initial investigations by ERCOT ISO.


27



What is a Texas RE Event?

Texas RE significant event list:● Emergency Electric Curtailment Plan (EECP)● Emergency notice issued● Simultaneous loss of any combination of three:

Generation units Transmission lines Autotransformers (138 kV and above)

● Special protection system activation● Remedial action plan or mitigation plan implementation load

shedding● Under-frequency relay operations (dev > 0.3 Hz)● Sustained voltage deviations > 5% on 345 kV systems● First contingency security violation


28



Texas RE Significant Event List (cont.)

● Failure of computer system at ERCOT, QSE, or TO● Voltage collapse on portions of 138 kV or 345 kV systems● Loss of an ERCOT, QSE, or TO control center● Black Start Plan initiation● Uncontrollable loss of 150 MW or greater of firm load shed for more

then 15 minutes● Any event determined by the ERCOT Reliability and ROS WG

Reference: ERCOT Compliance Process Approved by the PUCT, July 20, 2006


29



NERC/DOE Reportable EventsEOP-004 Disturbance Reporting

● Summary of NERC/DOE reportable events: Physical sabotage, terrorism or vandalism Cyber sabotage, terrorism or vandalism Complete operational failure of electrical system Electrical system separation (islanding) Uncontrolled loss of firm system load 300 MW or greater Load shedding 100 MW or greater under emergency operation Voltage reductions 3% or more Public appeals (emergency conditions to reduce demand) Suspected physical attacks Suspected cyber or communications Fuel supply emergencies Loss of electric service (50,000 for one hour or more)

● DOE EIA-417 Report must be filed


30



NERC/DOE Form EIA-417 – Schedule 1

ftp://ftp.eia.doe.gov/pub/electricity/eiafor417.doc

● Items 1-8 completed within one hour of the event

● All others are due within six hours after the event


31



NERC/DOE Form EIA-417 – Schedule 2

● Narrative description due within 48 hours of the event

● A final report due within 60 days


32



Investigation Process

● ERCOT ISO will send an Event Trigger Report. ERCOT system operator supervisor will initiate the report

● Reporting procedure is followed: ERCOT ISO will investigate the event and will report to Texas

RE within 10 business days (detailed report of the event). Texas RE will have one day to announce if it will follow up with a

compliance report• If any operational guide or ERCOT protocol violations found, or

• If any NERC standards violations found.

Market participants may get a request for information about the event and will have five days to respond to the request (ERCOT or Texas RE).


33



References

ERCOT Operation Guides● Section 3.1.4.1 Operational Interfaces – Outages● Section 3.1.5.1 TOP’s Responsibility for NERC/DOE Reportable

Events● Section 3.1.5.3.4 ERCOT Requirement for Reporting Sabotage● Section 4 Emergency Operations● Section 6 Reports and Forms● Section 7 Disturbance Monitoring and System Protection

Relay Setting and Misoperations


34



References

NERC/DOE● NERC Standard EOP-001 Emergency Operation Planning● NERC Standard EOP-002 Capacity and Energy Emergencies● NERC Standard BAL-002 Disturbance Control Standard (DCS)● NERC Standard EOP-004 Disturbance Reporting

Form EIA-417: ftp://ftp.eia.doe.gov/pub/electricity/eiafor417.doc● NERC Standard PRC-004-1 Analysis and Mitigation of

Transmission and Generation Protection System Misoperation● NERC Standard PRC-016-0 Special Protection System

Misoperations


35



Event Investigations

Questions?


36



Data Submittals and Periodic Reporting

Kenneth Kan




37



Topics

● Schedule Control Error (SCE)● Resource Plan Performance Metrics● Disturbance Control Standard (DCS)● Time Error Correction● Vegetation Management● Other Periodic Data Submittals


38



Schedule Control Error

● Protocol sections 6.10.5, 6.10.6, 6.10.12● Applies to QSEs with generation resources● Schedule Control Performance Standard 1 (SCPS1) (one-minute

measure) Measures how well QSEs stay at 60 Hz 100% or greater to pass

● SCPS2 (10-minute measure) Measures how close QSEs keep to their schedules 90% or greater to pass


39



Schedule Control Error (cont.)

● Scores are calculated at beginning of every month.● After 10th calendar day, alleged non-compliance letters are sent to

QSEs and copied to PUCT.● QSEs send responses with explanation/exception requests,

corrective action plans to Texas RE.● After 20th calendar day, Texas RE sends out confirmed non-

compliance notices to QSEs and PUCT. If a QSE fails SCPS2, a copy is sent to Settlements for any

adjustments.● Violation notices are posted on ERCOT website.● Name of violating QSEs are included in monthly Texas RE reports

to board of directors.


40



NERC Standard BAL-001-0 – CPS1 and CPS2

● CPS1 is calculated for ERCOT system.● ERCOT has waiver for CPS2 for NERC requirements.● Monthly reports are sent to NERC.


41



Resource Plan Performance Metrics

● Protocol section 4.10● Measures performance of QSEs submitting status of specific

generation resources and/or loads acting as resources (LaaRs) through the Resource Plan

● Six measured metrics: Resource status Low sustainable limit (LSL) as percentage of high sustainable

limit (HSL) Day-ahead zonal schedule Adjustment period zonal schedule Down bid and obligation Total up ancillary services


42



Resource Plan Performance Metrics (cont.)

● Scores are calculated at the beginning of each month.● Score of 90% or higher is required to pass each metric.

Four consecutive failing months for a metric will result in Level 1 Non-Compliance for that metric.

● Scores are sent out to any QSEs that failed any metrics. QSEs may request their scores.

● QSEs given 10 business days after receiving score to Review data and respond with any exclusion requests, and Submit corrective action plan.


43



Resource Plan Performance Metrics (cont.)

If QSE has Level 1 Non-Compliance (four consecutive monthly failures for a metric):

● Texas RE will send a letter and make telephone contact with Compliance Contact of QSE that is found non-compliant.

● QSE will be given 10 business days to provide written statement and/or evidence as to why further action should not be taken.

● If QSE accepts responsibility for non-compliance, it shall submit a mitigation plan.


44



Disturbance Control Standard

● DCS event defined as loss of 80% or more of single biggest resource: South Texas Project = 1354 MW (maximum) 80% of STP = 1083 MW (minimum)

● Fifteen minutes to recover to pre-disturbance levels or 60 Hz, whichever comes first

● Quarterly DCS Reports to NERC


45



Time Error Correction

● Operating Guides 2.2.7 (Time Error and Time Synchronization) Initiated: Time Error = ± 3 seconds Terminated: Time Error less than ± 0.5 seconds May be postponed if load patterns in immediate future will

correct it Accumulated time error (ATE) should NOT be > 5 seconds

● Five-second ATE rule not in NERC Standard BAL-004-0 (Time Error Correction)

● Reported quarterly to NERC


46



Vegetation Management

● Operating Guides 3.1.5.2● NERC Standard FAC-003-1● Zero-tolerance policy!● Responsibility of transmission owners

TOs are responsible for VM on 345 kV lines and other voltage lines deemed significant to reliability of the transmission grid

● Included in VM program: Inspection requirements Trimming clearances Annual Work Plan (AWP)


47



Vegetation Management (cont.)

● When? TOs send outage reports to the Texas RE on the 20th of each

month to report any outages caused by contact with vegetation.• Email reports to: [email protected]

Annually, TOs self-certify that they performed vegetation maintenance in their AWP according to requirements in VM program.

● Texas RE may conduct spot checks at any time on TO to check plan initiation, progress, site inspections, etc.


48



Vegetation Management

Are there any exceptions in reporting VM outages?● Multiple outages on an individual line, if caused by the same

vegetation, shall be reported as one outage regardless of the actual number of outages within a 24-hour period.

● A single trip followed by a successful automatic re-close within a 24-hour period shall not be a reportable outage.

● Vegetation contacts due to natural disasters or storm-related (Examples: earthquake, fire, tornados, hurricanes and wind shear [micro-bursts] ice storms, hail storms, and floods) are not considered vegetation-related.


49



Other Periodic Data Submittals

Some ERCOT operating guide requirements have NERC standards:● Fault recorder locations● 345 kV disturbance events● Relay mis-operations● Annual under-frequency load shedding (UFLS) survey● Annual load data request (ALDR)● Special protection system (SPS) operations report analysis● SPS mis-operations

● TRE mostly not involved in these but will notify official registration compliance contacts whenever data submittal is needed for NERC assessment


50



Data Submittals and Periodic Reporting

Questions?

november 2, 2007 nerc compliance workshop assessing compliance presenter name presenter title...

Documents

audit audit team

audit schedule

onsite audit

audit agenda slide

nerc compliance workshop

scheduled audit date

audit team leader

preaudit package