cls: control system e. matias, r. berg, g. wright, t. wilson, t. johnson, r. tanner and h. zhang...
TRANSCRIPT
![Page 1: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/1.jpg)
CLS: Control System
E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. ZhangCanadian Light Source
University of SaskatchewanInvited Talk – Particle Accelerator Conference 2005 Knoxville, Tennessee
![Page 2: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/2.jpg)
CLS Control System Heritage
• Saskatchewan Accelerator Laboratory (SAL) operated from the late 1960s until 1999.– Control system evolved from
PDP-8 -> PDP-11 -> VAX -> NeXT and Sun workstations.
– IO was based on CAMAC with two CAMAC data highways.
– Some Micro84 PLCs.
• Control System was locally developed running on BSD UNIX.
• Was it open source? Yes, but only because so much
of it was locally developed.
• What to do with the grass field?
![Page 3: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/3.jpg)
CLS Control System Heritage
• 1999 March 31 funding for CLS was approved. Nuclear physics program was discontinued.
• The existing Linac would need to be reconfigured and refurbished.
• Linac Controls:– CAMAC hardware would
need to be replaced.– Power supplies would need to
be upgraded.– RF control would need to be
redesigned.– The old computer hardware
would need to be replaced.• We need to make some
design choices....
![Page 4: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/4.jpg)
CLS Control System Principles
• System design based on highly distributed control.• Extensive use of single board computers (originally used in
SAL).• Target lifetime of 15+ years.• Data communication over Ethernet when possible.• System must be user-friendly. • The accelerator and beamline systems must be maintainable by
a small team. • Reliability and availability of beam are critical to the success of
the facility.• Building an open source control system was not the initial goal, it
was the outcome.
• Accelerator complex must be complete by Dec. 2003 and the first phase of beamlines by Dec. 2004. The project must come in on budget.
![Page 5: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/5.jpg)
• The options: (1) EPICS or (2) Isagraph/Virgo.
• EPICS was selected, since it had:– large built up accelerator and beamline user community;– availability of suitable drivers and utilities;– credibility with the CLS user community; and– good design.
• EPICS Extensions selected include:– EDM,– Accelerator Toolbox, – Gateway and– Data Archiver.
• EPICS extensions that were locally developed:– assortment of drivers,– IOC Auto-Save-Restore,– simple beamline scanning program, and– SQL Alarm Management Database.
Selection of a Distributed Control Platform
![Page 6: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/6.jpg)
• The options: (1) Sun or (2) Linux.
• Linux was selected, since it had:– better hardware
availability, and– fairly equivalent reliability
levels.
• EPICS Extensions selected include:– EDM,– Array Display Tool,– StripTool, and– Knob Manager.
• We are now starting to deploy touch screens running Linux/EDM.
Selection of an Operator Workstation Platform
![Page 7: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/7.jpg)
• PLC hardware extensively used– Telemecanique Momentum– Siemens S7/300– Siemens S7/400– Siemens S7/400 F (safety critical applications)
• Software Written using IEC 61131-3 Programming Language primarily Graphical Function Block Notation
PLC Control
![Page 8: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/8.jpg)
Selection of a Hard Real-Time Operating System
• The Options: (1) RTEMS and (2) VxWorks.
• RTEMS was selected, since it had:– good experience from SAL,– additional flexibility with single board
computers, and– high level of reliability.
• IOCs are CLS/SIL embedded controllers (approx 150) based on the MC68360 25 MHz. Processor.
• Pros and Cons:– No dynamically loaded libraries;
must be linked prior to download.– Large number of IOCs (separation
of function but more points of failure).
![Page 9: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/9.jpg)
• The options: (1) Matlab, (2) SciLab, or (3) root.
• Matlab was selected primarily because of the availability of the accelerator toolbox and staff experience.
• Matlab is commercial, the accelerator toolbox is open source.
• Software originates from ALS and SPEAR III.
• Augmented with other CLS specific utilities.
• Also being used as a commissioning tool for beamlines.
• Special care is required to maintain consistency with other parts of the control system.
Selection of an online scripting/analysis package
![Page 10: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/10.jpg)
• Single board computers (EPICS/RTEMS) used for:– stepper motors,– power supply control,– vacuum equipment
monitoring,– radiation monitors, and– other RS-232 devices.
• PLC hardware/software used for machine protection.
• Industrial PCs with VME used for diagnostics.
• Linux servers used for high-level control, network services and EPICS/PLC interface.
Implementation
![Page 11: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/11.jpg)
Linux and VME
• Using VME hardware connected to a Linux PC.
• SIS1100 PCI card <-> fiber optic link <-> SIS3100 VME module
• Maps VME backplane to IOC memory.
• Advantages:– PC can be physically separated from VME crate.– More than one VME crate per PC.– Multiple applications can access the same crate.– High throughput 25 to 80 Mbytes/sec block
transfer.
• Work ongoing on RTEMS support.
/dev/SIS1100_2(descriptor 2)
VME CRATE 1:Hardware
mapped memory
VME CRATE 2:Hardware
VME CRATE 3:Hardware
Fiber Optic link
FiberOpticLink
Fiber Optic Link
/dev/SIS1100_1(descriptor 1)
/dev/SIS1100_3(descriptor 3)
EPICSApplication
PV recordread
routines
PV recordwrite
routines
Linux IOCsis1100 PCI
card
sis1100 PCIcard
sis1100 PCIcard
![Page 12: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/12.jpg)
• Provides fiber optic signal distribution of triggers.
• VXI based hardware• IOC running EPICS on
RTEMS.• Operator
Interface implemented using Glade.
• Glade wasselected forthe table andfile handlingcapabilities.
Timing System
![Page 13: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/13.jpg)
• Beamline Controls are based on the same software and hardware as the accelerator systems.
• Each beamline is on a separate virtual network.
• The EPICS Gateway provides links between the different networks.
• Matlab is used for scripting.
Beamlines
![Page 14: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/14.jpg)
Interface to Administrative Systems
• Administrative (financials and work-management) databases are based on MS-SQL.
• freeTDS was used to provide an interface into these systems.
• The control-system is a source of alarm-data that can be used to trigger preventative maintenance activities.
• This integration is still a work in progress.
![Page 15: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/15.jpg)
VLANs for: each beamline, machine control, development, office, visitors
VME Crate
(Reflective Memory)
MicroStep
EROCIOC
RTEMS
FieldDev. RS-232
Devices
OPI
Linux
IOCStep Controller
RTEMS
Motors
MicroStep
OPI
Linux
OPI
Linux
Touch PanelOPI Linux
NetworkServer
(bootp, dhcp,auto restore)
Linux
DataArchiveServer
Linux
AlarmServer
MS-Win
MS-SQLServer
MS-Win
PowerEdgeIOC
Linux
PS BoardsIOC
RTEMS
PowerSupplies
EROCIOC
RTEMS
FieldDev.
EthernetDevices
PLC & GPIB
FieldDev.
MagnetsMotors
1GigBridge
IOC
Linux
FieldDev.
ProfibusPLC
System Architecture
![Page 16: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/16.jpg)
Lessons Learned
• The use of open source did not compromise reliability or availability.
• Configuration management is critical to the success of the system.
• Layering of different software packages requires care to ensure the system is reliable.
• Additional flexibility in available in managing upgrades compared to commercial software.
• Dependence on suppliers continuing to support the product is not an issue.
• Our, building automation system is a case in point, where the vendor has discontinued the product and we are now replacing it with EPICS and PLC hardware.
![Page 17: CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited](https://reader036.vdocuments.net/reader036/viewer/2022062802/56649e8f5503460f94b93726/html5/thumbnails/17.jpg)
Winter at the CLS