1 brookhaven science associates control system overview l. r. dalesio epics collaboration march 11,...

1 BROOKHAVEN SCIENCE ASSOCIATES

Control System Overview

L. R. DalesioEPICS Collaboration

March 11, 2008


Outline

• Requirements• Standard Control System Components• Development Opportunities

• Embedded Device Control• Client/Server Architecture for High Level Applications• Relational Database and tools for all configuration

parameters• Near Term Plans• Scope of Work and Budget• Concluding remarks


Control System Requirements – 1 of 3

• Bunch Length 1-40 psecs• 2.6 usec ring revolution• Top off every 1 minute• Top off bunch train 140-300 nsec• Top off damping time 10-50 msecs (no extraction)

• Synchronous reading/writing (latency?jitter?future settings?rate?)

• Manual control of orbit trims, quadrupoles, sextupoles, and insertion devices are asynchronous

• ~10 Hz write/read is suitable for “turning knobs” for a power supply • 5 Hz updates to operators of up to 1000 chosen parameters • Archive up to 6000 parameters at a rate of 2 Hz continually

• What is done elsewhere?• Must scale to support 150,000 physical I/O connections and 400,000 computed

variables • 99.99% availability 24/7



• Transient Recording• Take coherent turn by turn orbit data for up to 800 channels 1024 turns• Latch the last 10 seconds of data from all parameters in the storage ring • Others?• Beam line needs 1 msec archiving over 1 minute for temperatures and positions

• Provide data for all control aspects • 5 KHz RF Feedback on beam phase• 10 kHz orbit feedback, (100 usec loop time)

• 300 BPMs (10 per cell)• 2 * 120 Corrector PS in 90 I/O Controllers (IOC)

• 20 msec equipment protection mitigation (too long?)• 1 Hz model based control • 10 kHz power supply read backs triggered from timing sys• 10’s of Hz Data Collection for RF loop correction.• 80 psecs pulse to pulse timing jitter.

• During top off, some beamlines will need 1.1 - 1.8 psecs of timing jitter• Machine physics thinks this number should be < 2 psecs

• Provide electron detector as event for beam line?



• First need dates• 2008/03 – vibration tests on girders done using stand alone software.

Some vibration sensors will be used online.• 2008/06 – support libera BPM evaluation• Where do the rest of the diagnostics fit• Where does the conventional facility demonstrator fit• Where does the power supply test stand fit• 2008/08 – X-9 upgrade for NSLS beam line• 2008/09 – low level RF lab support• 2009/06 – support water systems• 2009/09 – support thermocouple measuring• 2009/09 – support vacuum test stands• 2010/03 – support high power RF

• Control Room• Must be able to be relocated• Control room in the LOB for commissioning• Control room re-located with accelerator team for operation


Nomenclature Standard

Psy:PI-Ssy:SI-Tsy:TI<Dev:DI>Sg:SgI-SD

Control System Naming Examples

S:C30-VA:G1<SGV:A>Pos:1-Opn

S:C20-FE:B<FV>Pos:1-Cls

S:C20-VA:G1<IP:A>E:1-RB

S:C20-VA:G4<CHMA:B>

S:C20-VA:G4<BLWR>

S:C24-RF:G1<TMP:B>

S:C24-RF:G1<CCG:B>P

S:C24-VA:G1<CCG:D>P

System Device Signal

Comment

cell 30, vacuum, girder 1, sector gate valve A (upstream), open

cell 20 BM front end fast valve

1st sputter ion pump at C20 girder #1

2nd aluminum chamber at C20 girder #4

rf shielded bellows at C20 girder #4

TMP station at 2nd SC cavity insulating vacuum

cold cathode gauge at 2nd SC cavity insulating vacuum

cold cathode gauge at 2nd SC cavity beam vacuum


Control System Software Standards

• EPICS based• Embedded Real-Time Operating System choices: RTEMS which is in use

at LCLS, Spear, and CLS or VxWorks which is in use at APS, SNS, Diamond, and SLS.

• Linux Workstations• Use of standard EPICS engineering tools: Extensible Display Manager,

Channel Archiver, Striptool, Alarm Handler• Evaluate the possibility of participating in the CSS development• Visual Database Configuration Tool w/ Modifications for Table Entry• Use of physics applications: Matlab Middle Layer Toolkit (MMLT) which is

in use at LBL, Spear, Diamond, SLS, CLS, and ASP, and eXtensible Accelerator Language (XAL) which is in use at SNS and Elegant which is in use at APS.

• Use of Beam Line Applications: Matlab in place of SPEC, BluIce, IDL, SynApps


Control System Architecture

Shared Memory

Ethernet – EPICS Channel Access Protocol

Events/ / Timing Data

PLC

CNET

I /O

Field I/O Field I/O…..

Instrumentation Ethernet

ALS

NET

Field I/O Field I/O…..PLCS ,Slow I/O, High Reliability, Low Accuracy, High Density Vacuum, PPS, MPS, Non FOFB PS, Cryo., Facility control

CPU

EVR

I /O

I /O

MTRS

Read Remote Ethernet and serial devices. High density IO, Motor Control Position Control, Image Acquisition, Non FOFB Diagnostics

Operator stations: Displays, Archiving, Alarm Management, Strip charts, Save/Restore Utility

CPU

EVG

CEL L

Timing Master

CPU

EVR

CEL L

BPM IOCs

CPU

EVR

CEL L

PS IOCs

BPM

BPM

BPM

BPM

BPM

BPM

PS

PS

PS

PS

Fast Equipment Protection Signal


Control Hardware Standards for Test Stands

• Dell Linux development workstations• PLC Solutions• Allen Bradley Control Logix• Allen Bradley FlexIO• Building Automation• ALS Needs fast Ethernet based interface board• VME crates• Rittan 4 slot VME64x-2U4S-PS300C-SM 4,395.00• Rittan 7 slot VME64x-4U7S-PS900C-SM 5,510.00• Weiner 9 slot VME195xPO 6,050.00• Weiner 21 slot VME6023/611_JL 7,271.00• CPU Boards• Motorola MVME 5500 3,000.00• PP410 3,500.00• Motor Controllers• Hytec 8601 with motor driver 32 axis ~1,000/axis• Newport XPS Intelligent motion controllers


Areas For Technical Development

• Embedded Controllers need an open standard for high speed, deterministic functions. Work with other labs and board manufacturers to develop one.

• High Level Applications currently tie together functions through data or file structures. To make the components of High Level Applications modular and distributed, a client/server architecture is needed.

• Relational Databases support data management through the life of a project. Development of adequate tools to enter and report this data is required early.


Control Hardware For Beam Synchronous Applications

2 GB Shared Memory – Communicates Cell Data @ 20 usecs

Ethernet – EPICS Channel Access Protocol

Events/ / Timing Data – Beam Trip Event, Synch Data Event

CPU

EVG

CEL L

Timing Master

Physics Applications: Correlation plots, orbit calculations, beam dump data

CPU

EVR

CEL L

BPM IOCs

CPU

EVR

CEL L

PS IOCs

BPM

BPM

BPM

BPM

BPM

BPM

PS

PS

PS

PS

Fast Equipment Protection Signal

Compute Fast Orbit Feedback


Embedded Controllers - Per Cell

BPMController

RF Clock Distribution45 MHz

Fiducial Distribution1 Hz

BPMController

BPMController

BPMController

BPMController

BPMController

100 MB ENET

PSController

PSController

PSController

PSController

VME I/OC EPICS

CoreController

VME Interface

VME I/OC EPICS

Next CellCoreController

VME Interface

VME I/OC EPICS

Prev CellCoreController

VME Interface

2 GBit ENET

+T0

+ - Settling time on BPMs

+0.0 usecs - BPM to Compute Controller

384 bits = 64 bits * 6 BPMs

4.9 usecs = 384 bits over 100 MBit enet

+4.9 usecs - Compute Controllers to each other

10,290 bits = 30 nodes * 384 bits

10.5 usecs = 10,290 bits over 2 GB enet

+15.4 usecs – Compute local matrix

0 usecs

+ 15.4 usecs - Communicate t Power Supply Controllers

24 = 4 PS * 4 bytes each

3.5 usecs

+18.9 usecs – loop complete

settling time for magnets

communicate diagnostic waveforms etc…

+200 usecs – start again


Embedded Controllers - Approach

• Develop a prototype cell controller:• Redundant 2 Gbit communication paths for peer to peer communication• RF timing signals• Verify communication and timing jitter meet requirements

• Develop the interface from the cell controller to a processor for integration into EPICS

• In parallel:• Develop the inexpensive device controller with redundant 100 MBit controllers• Develop the 100 Mbit receive and transmit circuits for the Cell Controllers• Develop the EPICS interface to the Cell Controller


Embedded Controllers - Status

• FY 08• Purchase order in place with LBL (Alex Ratti and Larry Doolittle)• Develop a prototype cell controller:

– Redundant 2 Gbit communication paths for peer to peer communication– RF timing signals– Verify communication and timing jitter meet requirements

• Develop the interface from the cell controller to a processor for integration into EPICS

• FY 09• Develop the inexpensive device controller with redundant 100 MBit controllers• Develop the 100 Mbit receive and transmit circuits for the Cell Controllers• Begin integration of these device controllers

– Libera– Power supply control– LLRF control


LatticeOptics

Beam Resp.Matrix (S)

Model Data

MeasuredOrbit

Orbit Differences

State Data

Application / Family

Control System / Channels

I/O Controllers

Beam Resp. Matrix Diff’s

OpticsDeviations

EPICS Client/Server

Gradient Errs& Corrections

Name Mapping

Mid Level Model Client/Server

ConfigurationParameters

PhysicsApplications

Conversions

Need to develop

Existing

Need to port

Computed Data

High Level Applications – Client/Sever


High Level Applications - Approach

• Install an operational NSLS II simulation using Matlab Middle Layer Toolkit

• Define an interface library for physics applications to communicate to the configuration parameters, calibrations, model data, and measured data.

• Port some subset of the available packages (Matlab Middle Layer Toolkit, Elegant, XAL, Dimad, and SAD) onto the interface library one application at a time.

• Develop a client/server implementation of the interface library.• Use an RDB to store configuration parameters and calibrations.


Relational Database

• Support all configuration data• Provide tools as they are needed throughout the

project.• Make the RDB the source of all configuration data and

derive it from there for all uses.


Relational Database - Approach

• Use the IRMIS database as the basis for our component and wiring database which is currently in use at APS, SNS, CLS, Diamond, and SLS.

• Extend the database to support Optics early through a contract with Tech-X in conjunction with our physicists and control group.• This contract includes a name mapper and lattice input / report tool

• Develop tools through contracts, collaboration, and the hire of a second RDB expert.• We are preparing SOWs for Naming Tool and Wiring Input

• Schedule the time and put in the project rigor to use the RDB and tools as new portions of the project are needed.


WBS Dictionary: Hardware and software for the global controls of the linac, booster ring, transport lines, storage ring, network, relational database, data archiving, timing, fast feedback, and physics application support.

WBS 1.03.05 Global Controls System

~ 10% of Accelerator

WBS Description

Direct $

FTEs/yr Total FTE Costs Total Material

1.03.05/01 – 1.03.05.09Apps: Diagnostics, RF, Power Supply, Vacuum

SubsystemIntegration of: Insertion Devices, Beam lines,

Safety Systems, Facility 7 5.3 M 1.9 M

1.03.05.12 Timing 1.5 1.2 M 0.9 M

1.03.05.11 & 1.03.05.13High Level Applications 6 4.8 M 0

1.03.05.10 Accelerator Control Room 75K 0.5 M

1.03.05.11 Network .5 0.3 M 1.3 M


Near Term Plan

• Start R&D • Contact Nearly in place for development of open-source embedded device controllers with

LBL team (Larry Doolittle and Alex Ratti)• RDB server is purchased and we are ready to Install IRMIS• Tech-X contract ready to go out the door to extend IRMIS to support Optics• Position open for someone to Develop RDB tools for entry and reports• Installed NSLS II Simulation through IOCs• Installed Matlab Middle Layer Toolkit for Commissioning• Two positions in place to Start Development of API for High Level Applications (one starts in

May, the other position should be filled by March)• Make Preliminary Hires

• RDB job ad is placed. A candidate has been indentified and could start by March 15th..• High Level Applications Engineers – Guobao Shen starts May 10, 2nd position starts by March

15th.. • 2 positions are open for project engineers• 1 position is open for an embedded systems expert

• Completed preliminary design• Completed extensions to Visual DCT for Spreadsheet based input


Concluding Remarks

• Global requirements are complete• Equipment control S/W and H/W infra-structure is selected.• A collaboration of LBL and SLS EEs along with NSLS II team are

prepared to begin the open-source, embedded device controller development

• RDB tables and tools have begun building on the success of IRMIS.• High Level Application environment is selected. The Diamond simulation

running under EPICS is installed along with the Matlab Middle Layer Toolkit as a starting point at KEK by Guabao Shen.

• The API for High Level Applications is being developed in conjunction with MMLT, SDDS, and XAL authors.

• Staff Acquisition is a critical activity and we have made good progress.

1 brookhaven science associates control system overview l. r. dalesio epics collaboration march 11,...

Documents