Plans for EPICS in Hall D at Jefferson Lab

Elliott WolinEPICS Collaboration Meeting

Vancouver, BC30-Apr-2009

Jefferson Lab: Newport News, Virginia6 GeV Continuous Electron Beam Accelerator Facility

Superconducting RFQ’s

Three existing experimental halls

Approved 12 GeV upgrade and new hall $310M

GlueX experiment:

200 kHz trigger

3 GB/s off detector

300 MB/s to tape

GlueX Experiment in Hall D

Search for mesons with gluonic excitations

• 200 kHz trigger rate• Deadtimeless readout• 15 kB event size• 3 GB/sec to L3 farm• Factor 10 L3 rejection• 300 MB/s to tape• Data taking in 2014

Experiment vs. Accelerator Operations

• Trigger/DAQ-centric, not controls-centric– trigger and DAQ hardware more problematic– high-speed data path is critical

• Run-oriented– many configuration changes– sometimes every 5 minutes– hardware download/readback speed critical– may bypass normal controls path for speed

5

ROC

ROC

ROC

ROC

ROC

ROC

EMU

EMU

EMU

EMU

EMU

EMU

Level 3 Farm

90% Rejection

EMU

RAID

SILO in Computer

Center

2 PB/year

60ROCs

8EMUs

8EMUs

ManyCores

1EMU

16 TB16 hrs

Event BuilderCrossbar

DAQ at High Luminosity3 GByte/sec

3 GB/s 3 GB/s3 GB/s

300 MB/s300 MB/s

300 MB/s

6

Type Number of Devices

Control Points per Device

Channel Count Control Mechanism

FDC cooling system 20 1 20 PLC

Cryogenic target 40 1 40 PLC

Solenoid magnet 200 1 200 PLC

Gas systems 40 1 40 PLC

Misc 50 1 50 PLC

Beamline Many Varies 200 PLC/EthernetGoniometer 6 axes 10 60 Ethernet

Tagger 750 4 3000 EthernetVXS, VME, HV, LV crates 80 15 1200 Ethernet

HV, LV channels 2300 4 9200 Ethernet

Discriminators 2800 1 2800 Ethernet

Cockroft-Walton bases 2800 5 14000 CAN

Board temperatures 1575 1 1575 SMBus

Misc 200 1 200 Ethernet

Software Channels 200 1 200 Ethernet

TOTAL 32785

Controls/Monitoring Channel Counts

Hardware

Electronics

7

Experiment Control System(ECS)

Supervisory Control and Data Acquisition System

(SCADA)

Detector Control System(DCS)

Hall D Three-Tier Controls Architecture

Allen-Bradley

PLC

Magnets Target Gas Cooling

HVSystem

LVSystem

OtherSystems

High-level state machines and operator GUI’s.

(AFECS)

Communication, archiving, alarms

(EPICS)

Autonomous detector control, low-level state machines.

(PLC, Ethernet, CAN, SMBus)

8

• Detector Control System– 24x7 autonomous operation– All PID and control loops reside here– No EPICS at this level

– Allen-Bradley PLC• engineers prefer simple analog sensors/actuators• will have some Ethernet devices

– High-level manufacturer-supplied controllers• HV, LV• goniometer• CAN, SMBus controllers

9

• SCADA - Supervisory Control and Data Acquisition– EPICS• JLab has extensive experience with EPICS

– accelerator and all three existing halls• Will use Linux-only EPICS, no VxWorks or RTEMS• CSS and/or LabView for non-expert graphics

– few restrictions on expert screens• Integrate with publish/subscribe package (cMsg)• Alarm system – BEAST (SNS) or LASER (CERN)• Many other tools in EPICS toolkit

– archiving – from SNS or JLab– strip charts – StripTool, SNS, JLab or Labview– save-and-restore facility – SCORE from SNS– web – CAML, WebCA– analysis - Labview

10

• Experiment Control System

– AFECS • Agent Framework for Experiment Control Systems• Java agent-based control system from JLab DAQ group• Used for run control• Allows integration of slow controls with run control

– high-level state machine scripting language– hierarchical – similar to SMI++ used at LHC

11

Alarm System• Goals – are these satisfied by BEAST? LASER?

1. All alarms require operator action• otherwise “alarm” is a nuisance

2. Single problem gives single alarm• alarm filtering/reduction, flood suppression

3. Context sensitive alarms• no alarm on device if not being used

4. Alarm shelving• remove from alarm screen

Labview

• Considering for majority of operator screens• Channel Access modules available– client for Linux (ORNL)– waiting for Linux server

• Basically just another GUI builder– like EDM, CSS– superior analysis capabilities

Conclusions

• Using EPICS for SCADA level (middle tier)– Linux only– no control loops in EPICS

• Cherry-picking best applications– Many from SNS/ORNL

• May use Labview extensively

Please give me comments and suggestions!

Backup

15

DetectorLevel 1

Energy SumTrigger

TriggerSupervisor

Module

Trigger

Inputs

Add’lTrigger

Logic

OutputPulser 1

Pulser n

Calib 1

Calib n

Trigger Signal

Full Experiment Trigger

16 bits

16 bits

16

PLC Redundant Chassis LayoutSeven Module Chassis

Power supplyPLC

Redundancy Module

CommunicationModule

Remote ChassisCommunicator

Blank Slots

17

Ten Module Remote Chassis

Field Devices:- temperature sensors- pressure gauges- vacuum gauges- cryogenic level gauges- relays- voltage, current gauges- flow meters- many others

Power supply

Remote ChassisCommunicator

DC Input Module