D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

1

ELECTRICAL NETWORK:

18 KV NETWORK REVIEW AND STATUS OF THE SPS

CONSOLIDATION PLAN

D. Bozzini on behalf of EN-EL-HT

With the contribution of EN-EL-OP section members

and in particular G. Fernqvist, G. Cumer & C. Jach

ATOP Days - Session 4 - Technical infrastructure

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

2

Outline

PART IMajor events on HV electrical cable network in 2008Major events on HV cells in 2008 Historical trend of failures on the SPS 18 kV networkCASE 1: The 2008 SPS joints crisisCASE 2: BQ1 CompensatorCASE 3: RESTO 1

PART IISPS Consolidation, status and future programBE-TI8 network layoutSMB: what do we do?Failure location on HV cables network

CONCLUSIONS

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

3

Major Events on HV Electrical cables Network

SPS 18 kV network– 6 cable failures on the 18 kV network (Including the 4 ones mentioned by P. Sollander)

• 2 on the pulsed TI8 network, stop of the SPS machine (2001)• 3 on the pulsed network, operation of SPS possible in open loop

(1970)• 1 failure on the SMB1, redundant network available (1970)

BEQ1 Compensator– Failure on a 18 kV cable head of the BEQ1 feeder in BE substation,

fortunately not in operation during failure period (2003)

RESTO 1 Meyrin– Elastimold 18 kV elbow connector on a transformer powering the general

services of Resto 1. (2001)– Near miss failure detected tanks to the preventive maintenance and the

reinforced visual inspections of the installation by the operation section.

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

4

Major Events on CERN HV cells

– ME24 cell explosion ( 1995)

• 10 days stop of the machines feed through the ISOLDE loop

– SE18 near miss ( 1995)

– Preventive action taken• Problem limited to FLUAIR cell type on Meyrin substations and SE18

• Monitoring of temperature and humidity

• Systematic Partial Discharge (PD) corona effects measurements

• Visual inspection inside cells whenever accessible

Comments: In the past 5 years, 2 similar accidents and 3 near miss !

– ME59 switch closing failure during auto-tranfer exercise ( 1968)

• Design life time 25 years, continuous operation at CERN 41 years !

Comments: Remember the 29 July 2006, same type of equipment !

ME9 Jura substation ( 1968) critical installation

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

5

Historical trend of failures on the 18 kV SPS network

1980 1985 1990 1995 2000 2005 20100

1

2

3

4

5

6

7 6 (2008)

Nature of the failuresA) Aged cables, made with XLPE affected by water-treeing effect

B) Not directly the cable but the accessories (joints, terminals)

Remarks for 20084 out of 6 failures affected recently installed SPS network segments ( 2001)

2 remaining failures on old cables

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

6

CASE 1: SPS joints crisis (1) - Failures

BE

BB3 BA4 BA51360 m 1270 m

1595 m

BB5

Foui

lle D

Foui

lle C

Gal

lerie

Foui

lle B

Foui

lle A

Reg

ard

BA

4

Ref dwg SPSEM___0003

D. Bozzini TS/EL/HT

Fault #2EDMS 944129Date:25-jui-2008

Fault 2007EDMS 846414Date:27-mai-2007

Fault #1EDMS 924713Date:24-mai-2008

SPSStable loop

SPSPulsed loop

TI8Line

Fault #3EDMS NADate:26-Dec-2008

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

7

CASE 1: SPS joints crisis (2)- Analysis

BE

BB3 BA4 BA51360 m 1270 m

1595 m

BB5

Foui

lle D

Foui

lle C

Gal

lerie

Foui

lle B

Foui

lle A

Reg

ard

BA

4

Ref dwg SPSEM___0003

D. Bozzini TS/EL/HT

Work done in 2001 by SPIE sub contractors

Work done in 2001 by SPIE technicians

Analysis report #1Cellpack PB-08-27

1 faulty jointDate: 27-jul-2008

Analysis report #2Cellpack PB-07-38

1 faulty jointDate: 25-jul-2008

SPSStable loop

SPSPulsed loop

TI8Line

Analysis report #3Cellpack PB-08-341 faulty joint2 in service jointsDate: 07-sep-2008

Analysis #4CERN1 in service jointDate: sepl-2008

Analysis #5CERN1 in service jointDate: Oct-2008

Analysis #6CERN1 in service jointDate: Nov-2008

Analysis #7CellPack1 faulty jointDate: Feb-2009

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

8

CASE 1: SPS joints crisis (3) - Analysis

Analysis of the joints

Conclusions of ALL 8 joints examinedASSEMBLY PROCEDURE NOT FOLLOWEDPOOR QUALITY OF WORK

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

9

CASE 1: SPS joints crisis (4) - Repair

Corrective actions• Repair of all joints that failed

Preventive actions• Replacement of all joints• Improvement of the shielding continuity across the joints

• 2 new joints for each repair/replacement

• Now we have 80 joints instead of 40

BE

BB3 BA4 BA51360 m 1270 m

1595 m

BB5

Foui

lle D

Foui

lle C

Gal

lerie

Foui

lle B

Foui

lle A

Reg

ard

BA

4Ref dwg SPSEM___0003

D. Bozzini TS/EL/HT

SPSStable loop

SPSPulsed loop

TI8Line

Repair #5 SPIE Oct 2008, 13 joints

Repair #4 CERN Sep 2008, 24 joints

Repair #3 CERN Jun 2008, 1 joint

Repair #2 CERN Mai 2008, 1 joint

Repair #1 CERN Mai 2007, 1joint

Repair #6 SPIE Jan 2009, 1 joint

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

10

CASE 1: SPS shield currents

Preventive action• 68/80 joints have a reinforced

shield continuity• Modified assembly procedure

approved by joint manufacturer

Standard

Reinforced shield continuity

Signs of heating confirmed by the conclusions of the joints analysis

• This triggered a campaign to measure shields currents

• Sources and reasons not yet fully understood

CNGS TI8

100 A/div

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

11

CASE 2: BEQ1 compensator – 18 kV cable terminal failure

ConclusionASSEMBLY PROCEDURE NOT FOLLOWEDPOOR QUALITY OF WORK

Failure on a 18 kV cable terminalDate of installation 2003, place BE, Indoor

Discharge to ground without load

Analysis by third party (Cellpack Laboratory, DE)

Outcome of the analysisLength of insulation dimensions not done according to the procedure

Use of inappropriate tooling (knife) causing serious weaknesses on the insulation

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

12

CASE 3: RESTO 1 - Elastimold 18 kV elbow connector failure

Failure on a 18 kV Elastimold connectorDate of installation 2003, place Bdg 500, Indoor

Discharge to ground, circuit was on load

Analysis by third party (Cellpack Laboratory, DE)

Outcome of analysisPreparatory dimensions not done according to the procedure

Use of inappropriate tooling (knife) causing serious weaknesses on the insulation

ConclusionASSEMBLY PROCEDURE NOT FOLLOWEDPOOR QUALITY OF WORK

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

13

SPS Consolidation - Status on March 2009

18 kV cable network (SMB not included) and substationsMarch-09

Segment Length Stable Pulsed[m] Done Done

BE-B3 350 350 350B3-B4 1300 1300 1300B4-B5 1350 1350 1350B5-B6 1260    B6-B7 550 550 550B7-B1 1100    B1-B2 1520    B2-BE 1100 1100  Total [m] 8530 4650 3550Done [%] 55 42

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

14

SPS Consolidation – Short term work plan

From March 2009 till December 2010• Install cables between BA6-BA6 (2X), BA1-BA2 (2X),

BA2-BE (1X)

• Digging 6.6 km

• Prepare renovation of substations BA4 and BA2

• Define and implement scheme for parallel coupling of the BE-TI8 antenna with BE-BA4 pulsed loop

During shutdown 2010-2011• Commissioning of installed cables

• Renovation and commissioning of substations in BA4 and BA2

• Installation and commissioning of parallel coupling BE-TI8 with BE-BA4 pulsed loop

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

15

SPS Consolidation – Mid term work plan

By the end of shutdown 2012• Installation and commissioning of cables between

BA4-BA1 (2X)

• Digging 1.1 km

• Renovation and commissioning of substations BA1 and BA3

By the end of shutdown 2013• Renovation and commissioning of substation BA6

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

16

BA4-TI8 network layout

Situation• Power line added in 2004 to power the

CNGS experiment• On this line we also have the SPS stable

network for BA4• Not being part of a loop -> no redundancy for

powering CNGS and SPS pulsed network in BA4

TI8

Consequences• If a fault appears on this line (ex: joint to be

replaced), one week of shut down minimum

Remedies• Implement possibility to operate the TI8 line

in // to the SPS pulsed loop from BE to BA4

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

17

SMB

Consequences• If a fault appears on this line, one week of

shut down minimum of the affected segment

How to proceed?• 1) Receive guidelines whether to include

these circuits in the EN/EL SPS consolidation mandate or:

• 2) as recommended by the ATC on 27th Oct. 2006 study and find procedures for a fast repair and shorten DTTR

Situation• Old power lines affected by water treeing

effect ( 1970) • Failure rate going up with the time• Operation is not 100 % redundant (see 4th

ATC minutes 27th October 2006)• Fully operated by TE-EPC, maintained by

EN/EL

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

18

Failure location on HV network

Current procedure• As soon as a failure is detected, ERDF (Electricite Reseau Distribution

France) is contacted and they intervene on average within three days to precisely localize the failure.

Technique• In most of t he cases reflectometry coupled with a pulsed surge generator.

Limits• Time of intervention depends on availability of ERDF. Also to consider that

EDF network diagnostic has priority with respect to CERN.

How to improve• EN/EL has almost all the equipment to perform the same type of diagnostic• A bit old but can be upgraded with a dedicated budget• Today there are weaknesses on expertise, but if necessary on a short mid

term period we can envisage to set up a EN/EL team for such diagnostic

What is needed?• Dedicated personnel, time for set-up and time for training on real cases

D. B

ozzi

ni, E

N-E

L-H

T, A

TOP

Day

s 20

09, 4

-6 M

arch

200

9

19

CONCLUSION

Failures• 6 out of 8 major failures or near miss are related to bad assembly of HV components

• For the SPS, except for the “joint crisis”, 2008 as not been a bad year, 2 faults due to aging cables. One affecting SMB. But…remember what we saw on the video!

• Cells and substations: let us not underestimate the status of ME59 and ME9

Serious problem of HV components assembly quality• Identification of specialized firms in ongoing with good results

• In-house expertise must be improved in order to follow the execution of repair

High currents in SPS shields• Require more time to investigate the sources of these currents

Consolidation of SPS 2009-2010 (assuming LHC will run over the winter)• Installation of cables on two segments (BA5-BA6 and BA7-BA1)

• Re-configure substations in BA2, BA4 and improve the TI8 network layout

• Require solid budget and availability of sufficient and fully qualified personnel

• SMB: We need to know now what to do

• Failure location: there is room for improvement of the MTTR