epics internals nick rees. outline introduction libraries and modules data structures process types...
TRANSCRIPT
EPICS Internals
Nick Rees
Outline• Introduction• Libraries and modules• Data Structures• Process types• Examples of processing
INTRODUCTION
Background• When I accepted this challenge I thought there
was an EPICS internals talk I could copy.– It turns out there wasn’t.
• I decided the aim of the talk was to provide EPICS developers with a vague idea of what to expect to see when they look at a running EPICS system.– It turns out this wasn’t that easy– It also was very difficult to do this in 40 minutes.– By focussing on this I missed out more than I covered.
• The result is something which even I find a bit difficult to understand (and I wrote the talk), so I don’t know about the rest of you.– For the non-programmers out there, good luck.
What is EPICS core?
What is EPICS core?• 109 directories• 1567 files, 239241 lines of all types
– 281 files, 93451 lines of C– 203 files, 49807 lines of C++– 329 files, 34985 lines of header files– 28 files, 14964 lines of html
• Over 20 years of development by a competent world-wide team.
• Not an enormous body of code.• A pretty reliable body of code that does what it is
designed to do.• A reasonably useful, flexible and portable tool
What isn’t EPICS core?• Consistently designed• Designed• Buzzword compliant, or even object-oriented• Well understood by lots of people.• Stylistically consistent.• Pretty, elegant or pure
Another view• The Application Developers guide states:
– “EPICS consists of a set of core software and a set of optional components. The core software, i.e. the components of EPICS without which EPICS would not function, are:
• Channel Access - Client and Server software
• IOC Database
• Scanners
• Monitors
• Database Definition Tools
• Source/Release”
• This talk focuses on the first 4 points, with most emphasis on the IOC Database.
LIBRARIES AND MODULES
Helper libraries• If you write EPICS base software you must know,
understand and use the basic helper libraries it provides. For example:– ellLib – doubly linked list library identical to VxWorks
listLib– gpHash – efficient hashes– freeList – malloc replacement giving efficient handling
of fixed size blocks.– epicsRingBytes – ring buffers.– epicsThread – thead creation and control– epicsMutex – OS independent mutexes– etc...
Essential EPICS code• dbStaticLib.c
– Static database access routines• Used during EPICS development or before iocInit.
• dbAccess.c– Run-time database access routines.
– dbNameToAddr, dbPutField, dbPut, dbGetField, dbGet, dbProcess etc.
• dbConvert.c– Type conversions
• camessage.c– Channel Access server function table implementations.
• dbEvent.c– Implements database event handling
• etc.....
What EPICS libraries are there?
• Common– Cap5– ca– cas– Com– gdd
• Host and IOC:– asHost– asIoc– dbStaticHost– dbStaticIoc
• IOC only– dbIoc– dbtoolsIoc– miscIoc– recIoc– registryIoc– rsrvIoc– softDevIoc– testDevIoc
EPICS library dependencies
Conclusion• If you want to know more, consult the source.
DATA STRUCTURES
dbBase.h• Basic database data structures are in dbBase.h• These define the structures that hold the
database.• The are defined in the dbd and db files, and
created by dbStaticLib.• At run-time, they are accessed by dbAccess
routines.
dbBase.h record tree• dbBase
└ recordTypeList└ dbRecordType
└ recordList└ dbRecordNode└ name└ precord└ dset└ lset etc.└ ...
└ deviceList└ ...
└ dbFldDes[]└ rset└ ...
dbBase.h tree
dbBase definitiontypedef struct dbBase {
ELLLIST menuList;ELLLIST recordTypeList;ELLLIST drvList;ELLLIST registrarList;ELLLIST functionList;ELLLIST variableList;ELLLIST bptList;void *pathPvt;struct dbPvd *ppvd;struct gphPvt *pgpHash;short ignoreMissingMenus;short loadCdefs;
}dbBase;
dbRecordType definitiontypedef struct dbRecordType {
ELLNODE node;ELLLIST attributeList; /*LIST of attributes*/ELLLIST recList; /*LIST of sorted dbRecordNodes*/ELLLIST devList; /*LIST of device support*/ELLLIST cdefList; /*LIST of Cdef text items*/char *name;short no_fields; /* number of fields defined */short no_prompt; /* number of fields to configure*/short no_links; /* number of links */short no_aliases; /* number of aliases in recList */short *link_ind; /* addr of array of ind in papFldDes*/char **papsortFldName;/* ptr to array of ptr to fld names*/short *sortFldInd; /* addr of array of ind in papFldDes*/dbFldDes *pvalFldDes; /*pointer dbFldDes for VAL field*/short indvalFlddes; /*ind in papFldDes*/dbFldDes **papFldDes; /* ptr to array of ptr to fldDes*//*The following are only available on run time system*/struct rset *prset;int rec_size; /*record size in bytes */
}dbRecordType;
dbRecordNode definitiontypedef struct dbRecordNode {
ELLNODE node;
void *precord;
char *recordname;
ELLLIST infoList; /*LIST of info nodes*/
int flags;
}dbRecordNode;
Record (dbCommon) definitiontypedef struct dbCommon {
char name[61]; /* Record Name */char desc[41]; /* Descriptor */char asg[29]; /* Access Security Group */epicsEnum16 scan; /* Scan Mechanism */epicsEnum16 pini; /* Process at iocInit */epicsInt16 phas;/* Scan Phase */epicsInt16 evnt; /* Event Number */epicsInt16 tse; /* Time Stamp Event */DBLINK tsel; /* Time Stamp Link */epicsEnum16 dtyp; /* Device Type */epicsInt16 disv; /* Disable Value */epicsInt16 disa; /* Disable */DBLINK sdis; /* Scanning Disable */epicsMutexId mlok; /* Monitor lock */ELLLIST mlis; /* Monitor List */epicsUInt8 disp; /* Disable putField */epicsUInt8 proc; /* Force Processing */epicsEnum16 stat; /* Alarm Status */epicsEnum16 sevr; /* Alarm Severity */epicsEnum16 nsta; /* New Alarm Status */epicsEnum16 nsev; /* New Alarm Severity */
epicsEnum16 acks; /* Alarm Ack Severity */epicsEnum16 ackt; /* Alarm Ack Transient */epicsEnum16 diss; /* Disable Alarm Sevrty */epicsUInt8 lcnt; /* Lock Count */epicsUInt8 pact; /* Record active */epicsUInt8 putf; /* dbPutField process */epicsUInt8 rpro; /* Reprocess */struct asgMember *asp; /* Access Security Pvt */struct putNotify *ppn; /* addr of PUTNOTIFY */struct putNotifyRecord *ppnr; struct scan_element *spvt; /* Scan Private */struct rset *rset; /* Address of RSET */struct dset *dset;/* DSET address */void *dpvt; /* Device Private */struct dbRecordType *rdes; /* dbRecordType */struct lockRecord *lset; /* Lock Set */epicsEnum16 prio; /* Scheduling Priority */epicsUInt8 tpro; /* Trace Processing */char bkpt; /* Break Point */epicsUInt8 udf; /* Undefined */epicsTimeStamp time; /* Time */DBLINK flnk; /* Forward Process Link */
} dbCommon;
dbAddr definition• Virtually all runtime access to the database is via a dbAddr
handle.
typedef struct dbAddr { struct dbCommon *precord; /* address of record */ void *pfield; /* address of field */ struct dbFldDes *pfldDes; /* address of fldDes */ long no_elements; /* number of elements */ short field_type; /* type of field */ short field_size; /* size of the field */ short special; /* special processing */ short dbr_field_type; /* request type */ /* DBR_STRING,...,DBR_ENUM,DBR_NOACCESS */} dbAddr;
PROCESS TYPES
EPICS and threads• A typical C/C++ programmer’s first reaction is
that EPICS has lots of threads• However, they all have there uses, and once you
understand them they are not so confusing.• The need arises from the need to have non-
blocking processing and strict priorities.• It is actually simpler and more efficient than a lot
of select() calls.
EPICS ThreadsName PrioritycbHigh 71
timerQueue 70
scanOnce 70scan0.1 66
scan0.2 65
cbMedium 64
scan0.5 64scan1 63
scan2 62
scan5 61scan10 60
cbLow 59
CAC-event 51dbCaLink 50
CAS-client 20
CAS-event 19
CAS-TCP 18CAS-beacon 17
CAS-UDP 16
errlog 10taskwd 10
• IOC management:– timerQueue– taskwd– errlog
• Database processing:– cb*– scan*
• Channel Access:– CAS-*– CAC-event– dbCaLink
• Plus various driver threads...
IOC management threads• timerQueue
– Implements all EPICS base delays
• taskwd– Monitors tasks for any suspensions
• errlog– Handles asynchronous processing of log messages.– Forwards log messages to the console or to files at low
priority.
Database processing threads• cb*
– callback tasks – used for event and I/O Interrupt scanned records.
– Calls any specified function asynchronously– 3 priorities (low, medium high)
• scan*– periodic scan tasks– Calls record process() routines at regular intervals.– Higher scan rates have higher priorities.
• + all driver threads– Every asyn port has its own thread
Channel Access threads• CAS-*
– Channel access server tasks• CAS-UDP listens for channel lookups
• CAS-TCP listens for TCP connections
• CAS-beacon looks for CA beacons
• One CAS-client and CAS-event task per client
• dbCaLink– Channel Access client that processes CA requests on
behalf of the database.
• CAC-event– Handles the CA client callback events (I think)
EXAMPLES OF PROCESSING
Example sequence diagram
IOC function callsIOC tasksClient
Task name C filename
C function
CA server call processing• Every CA client message has a function code defined
in caProto.h of the form CA_PROTO_XXXX– E.g.: CA_PROTO_WRITE
• This will convert into a function in one of the CA jump tables (tcpJumpTable or udpJumpTable). Functions are off the form xxxx_action(mp,pPayload,client)– Parameters are pointers to
• CA header block
• Data
• client structure.
– Performs:• Net to host byte swapping
• Access security checks
– E.g.: write_action(mp,pPayload,client)
CA server call processing• xxxx_action often calls a “CA db_access” routine
with underscores, which handles an ugly “old to new” type enumeration conversion– E.g.: db_put_field(pAddr, oldDataType,pPayload,count)
• This calls a dbAccess routine which is part of the database code.– E.g.: dbPutField(pAddr,newDataType,pDate,count)
• This locks the database and implements the appropriate action.
caget from a record
Channel life cycle
caput to an ai record
dbPutFieldlong epicsShareAPI dbPutField(DBADDR *paddr, short dbrType, const void *pbuffer, long nRequest)
{
long status = 0;
dbFldDes *pfldDes = paddr->pfldDes;
dbCommon *precord = paddr->precord;
short dbfType = paddr->field_type;
/* Various sanity tests deleted here... */
dbScanLock(precord);
status = dbPut(paddr, dbrType, pbuffer, nRequest);
if (status == 0) {
if (paddr->pfield == (void *)&precord->proc ||
(pfldDes->process_passive && precord->scan == 0 && dbrType < DBR_PUT_ACKT)) {
if (precord->pact) {
precord->rpro = TRUE;
} else {
/* indicate that dbPutField called dbProcess */
precord->putf = TRUE;
status = dbProcess(precord);
}
}
}
dbScanUnlock(precord);
return status;
}
dbPutdbPut(DBADDR *paddr, short dbrType, const void *pbuffer, long nRequest){ long special = paddr->special; short field_type = paddr->field_type; long no_elements = paddr->no_elements;
if (special) dbPutSpecial(paddr, 0);
if ( IS AN ARRAY ) prset->get_array_info(paddr, &dummy, &offset);
dbPutConvertRoutine[dbrType][field_type](paddr, pbuffer, nRequest, no_elements, offset);
if ( IS AN ARRAY ) prset->put_array_info(paddr, nRequest);
if (special) dbPutSpecial(paddr,1);
if ( not ( VALUE FIELD and PP )) db_post_events(precord, paddr->pfield, DBE_VALUE | DBE_LOG); return;}
camonitor from an ai record
caput callback from an ai record
What’s the stack trace like?Call Locationclone() libc.so.6
start_thread() libpthread.so.0
start_routine(arg) osdThread.c:320
camsgtask(pParm) camsgtask.c:123
camessage(client) camessage.c:2442
write_action(mp,pPayload,client) camessage.c:810
db_put_field(paddr,src_type,psrc,no_elements) db_access.c:1216
dbPutField(paddr,dbrType,pbuffer,nRequest) dbAccess.c:1257
dbProcess(precord) dbAccess.c:630
process(pao) librecIoc.so
writeValue(pao) aoRecord.c:537
write_ao(pao) devAoSoft.c:72
CAS-client thread
Socket recv call (caput starts here)
Process CA messagescaput CA action
caput to an ai recordCall Locationclone() libc.so.6
start_thread() libpthread.so.0
start_routine(arg) osdThread.c:320
camsgtask(pParm) camsgtask.c:123
camessage(client) camessage.c:2442
write_action(mp,pPayload,client) camessage.c:810
db_put_field(paddr,src_type,psrc,no_elements) db_access.c:1216
dbPutField(paddr,dbrType,pbuffer,nRequest) dbAccess.c:1257
dbProcess(precord) dbAccess.c:630
process(pao) librecIoc.so
writeValue(pao) aoRecord.c:537
write_ao(pao) devAoSoft.c:72
CAS-client thread
Socket recv call (caput starts here)
Process CA messagescaput CA action
caput callback to an ai recordCall Locationclone() libc.so.6
start_thread() libpthread.so.0
start_routine(arg) osdThread.c:282
camsgtask(pParm) camsgtask.c:123
camessage(client) camessage.c:2508
write_notify_action(mp,pPayload,client) camessage.c:1823
dbPutNotify(ppn) dbNotify.c:342
putNotifyCommon(ppn,precord) dbNotify.c:236
dbProcess(precord) dbAccess.c:643
process(precord) aiRecord.c:177
recGblFwdLink(precord) recGbl.c:267
dbNotifyCompletion(precord) dbNotify.c:422
callbackRequest(pcallback) callback.c:154
CAS-client task
camonitor from an ai record
Call Locationclone() libc.so.6start_thread() libpthread.so.0start_routine(arg) osdThread.c:282periodicTask(arg) dbScan.c:553scanList(psl) dbScan.c:658dbProcess(precord) dbAccess.c:643process(prec) calcRecord.c:126recGblFwdLink(precord) recGbl.c:265dbScanFwdLink(plink) dbAccess.c:519dbScanPassive(pfrom,pto) dbAccess.c:490dbProcess(precord) dbAccess.c:643process(precord) aiRecord.c:175monitor(prec) aiRecord.c:403db_post_events(pRecord,pField,caEventMask) dbEvent.c:767db_post_single_event_private(event) dbEvent.c:715
scan1 task
Camonitor event
Call Locationclone() libc.so.6start_thread() libpthread.so.0start_routine(arg) osdThread.c:282event_task(pParm) dbEvent.c:941event_read(ev_que) dbEvent.c:865read_reply(pArg,paddr,eventsRemaining,pfl) camessage.c:627cas_send_bs_msg(pclient,lock_needed) caserverio.c:63send() libc.so.6
CAS-event task
caget from an ai record
Call Locationclone() libc.so.6start_thread() libpthread.so.0start_routine(arg) osdThread.c:282camsgtask(pParm) camsgtask.c:123camessage(client) camessage.c:2508read_notify_action(mp,pPayload,client) camessage.c:757read_reply(pArg,paddr,eventsRemaining,pfl) camessage.c:557db_get_field(paddr,buffer_type,pbuffer,no_elements,pfl) db_access.c:543dbGetField(paddr,dbrType,pbuffer,options,nRequest,pflin)dbAccess.c:989dbGet(paddr,dbrType,pbuffer,options,nRequest,pflin) dbAccess.c:1009getOptions(paddr,poriginal,options,pflin) dbAccess.c:393dbFastGetConvertRoutine[field_type][dbrType] (localAddr.pfield, pbuffer, &localAddr)
dbAccess.c:1069
CAS-client task
The end• What did I miss?
– IOC initialisation– Database definition– Static database access– dbStatic– Test facilities– Access security– IOC shell– Registry– OSI– libCom– Build rules– Record, device and driver support
• Basically everything.....
Resources• EPICS Application Developers Guide• Channel Access Protocol Specification
– http://epics.cosylab.com/cosyjava/JCA-Common/Documentation/CAproto.html
• Training slides on EPICS web site• Eclipse is your friend...• “module load ddd” gives you the ddd graphical
debugger.