Rob Lambert, NIKHEF Core Soft, 24th July 2012 1
Decoder DB framework(savannah task #19106)
R. Lambert
(Massive thanks to Gerhard and others for letting me rant at them continuously about this very annoying task)
A Simple Decoder
Rob Lambert, NIKHEF Core Soft, 24th July 2012 2
DecoderRawBankDataObject (Container)
Our Decoders
Rob Lambert, NIKHEF Core Soft, 24th July 2012 3
Client Alg
RawEvent
TES
Decoder
BankA
BankB
BankC
/Event
DataObject (Container)
DataObject (Container)
DataObject (Container)
(Private?) Tool (Handle?)
(Public?) Tool (Handle?)
Each subdetector follows acompletely different strategy.
Data On Demand??
DecodeRawEvent.py + various .opts files
Existing Decoding
Rob Lambert, NIKHEF Core Soft, 24th July 2012 4
Pros Cons
<100 lines of options Uses old-style .opts
Leave all logic to the C++ Forces us to leave all logic to the C++
Easy to re-configure one decoder Impossible to logically reconfigure many decoders
Sets up DoD Forces us to use or interrogate DoD
DoD locations are unsafe to reconfiguration
Tools outside of DoD unknown
Sets up potentially unused algs
Requirements Don’t re-write any decoder C++
Don’t overwrite any current default C++ decoder behaviour
Don’t force the updating or removal of existing user/SD scripts
Handle all the very different subdetector mechanisms
Allow use of the RawEventFormat database
Allow both user *and* high-level reconfiguration
Simplify decoders so they again look like:
Rob Lambert, NIKHEF Core Soft, 24th July 2012 5
DecoderBankADataObject (Container)
Decoder Class Many iterations and re-implementations later…
Hides all the subdetector differences behind one interface
Rob Lambert, NIKHEF Core Soft, 24th July 2012 6
1. New “configurabloid” class
class Decoder(object): “”” A sort of configurabloid which adds itself to a database of possible decoders “”” FullName=“” # the full gaudi name of the low-level configurable Active=False # flags this as an alg to be configured, writing to the TES Banks = [] #list of banks to decode Inputs={} #list of input locations, or {Property:value} dict Outputs={} #list of output locations, or {Property:value} dict
… def setup(self): “”” Configure and return a true gaudi configurable “””
Decoder DB Many iterations and re-implementations later…
Note that inputs/outputs are _read_ from the property if no override of the default is specified… but defaults are incorrect
Rob Lambert, NIKHEF Core Soft, 24th July 2012 7
2. Translate everything in DecodeRawEvent.py to decoder objects
DecoderDB={}
#===========VELO===========Decoder("DecodeVeloRawBuffer/createVeloClusters", active=True, banks=["Velo"], inputs={"RawEventLocations" : ["Other/RawEvent","DAQ/RawEvent"]}, outputs={"VeloClusterLocation" : None}, properties={"DecodeToVeloClusters": True,"DecodeToVeloLiteClusters":False}, conf=DecoderDB)
…
Decoder Conf Many iterations and re-implementations later…
Note how very simple this configurable can be once the subdetector differences are hidden
Rob Lambert, NIKHEF Core Soft, 24th July 2012 8
3. Instrument new configurable to setup the DoD svc if needed
from DAQSys.Decoders import DecoderDB
class DecodeRawEvent(ConfigurableUser): ""“Simple configurable to handle raw event decoding """ __queried_configurables__ = [RawEventFormatConf]
__slots__ = { "DataOnDemand" : False #Add decoding into DoD? , "Sequencer" : None #Add decoding to this sequence? , "DecoderDB" : None #if none, then decoder_db is used , "OverrideInputs" : None #use raw event version to override input locs }
Use case #1 Drop-in replacement for DecodeRawEvent.py
Note that for L0, $L0TCK/L0DUConfig.opts should also be loaded… but that’s not really the same sort of “decoding”
Rob Lambert, NIKHEF Core Soft, 24th July 2012 9
from Gaudi.Configuration import *importOptions(“$STDOPTS/DecodeRawEvent.py”)
from Configurables import DecodeRawEventDecodeRawEvent().DataOnDemand=True
Use case #2 Decode only one bank, in my own sequence
E.g. to bind members in the trigger sequencers
Rob Lambert, NIKHEF Core Soft, 24th July 2012 10
from DAQSys.Decoders import DecoderDB as ddbfrom DAQSys.DecoderClass import decodersForBank
mySeq.Members+=[d.setup() for d in decodersForBank(ddb,“Velo”)]
Use case #3 Use new RawEventFormat to override input locations
E.g.: To re-run Moore >v20r2 on stripped data, and/or make sure default Moore looks first in DAQ/RawEvent!
Rob Lambert, NIKHEF Core Soft, 24th July 2012 11
from Configurables import DecodeRawEvent
DecodeRawEvent().OverrideInputs=“S20”
Use case #4 Decode a second time with slightly different options
E.g.: To decode multiple TCKs in the same DaVinci job
Rob Lambert, NIKHEF Core Soft, 24th July 2012 12
from DAQSys.Decoders import DecoderDB as ddb
newDec=ddb[“OTTimeCreator”].clone(“OTTimeCreator/Ot2”)newDec.Properties[“OutputLevel”]=42newDec.overrideInputs(“Other/RawEvent”)newDec.overrideOutputs(“Other/OTTimes”)
Annoyances Default RawEventLocations doesn’t hold the default
RawEventLocations… gaaaah!!
Rob Lambert, NIKHEF Core Soft, 24th July 2012 13
… same with ODINDecodeToolOutput!
Advantages Configure time introspection:
The database knows which inputs are required and which outputs exist
Spot configuration mistakes The database is smarter than the options file, it spots logical
problems, such as:
Both algs decode LiteClusters. If you run Lite *then* full you will be unable to “put” new output
Rob Lambert, NIKHEF Core Soft, 24th July 2012 14
VLClusters = VLRawBankDecoder("createVLClusters")VLClusters.DecodeToClusters = TrueVLLiteClusters = VLRawBankDecoder("createVLLiteClusters")VLLiteClusters.DecodeToLiteClusters = TrueVLLiteClusters.DecodeToClusters = False
Being Smarter The RICH has a nice configuration framework, which I can
also use in this database:
Which implies Chris’ module could actually add itself to my database. Anything “active” in the database will be configured.
Rob Lambert, NIKHEF Core Soft, 24th July 2012 15
from Configurables import RichToolst=RichTools().smartIDTool()#ensure it's a public tool...tname=t.getFullName().replace("/","/ToolSvc.")
Decoder("Rich::DAQ::RawBufferToRichDigitsAlg/RichRawEventToDigits", active=True, banks=['Rich'], outputs={"RichDigitsLocation":None}, properties={"DecodeBufferOnly":False}, publicTools=[tname], conf=DecoderDB)
t2=RichTools().rawDecoder()t2name=t2.getFullName()
Decoder(tname, active=False, privateTools=[t2name], conf=decoder_db)
Decoder(t2name, active=False, inputs={"RawEventLocations":["Rich/RawEvent","DAQ/RawEvent"]}, conf=DecoderDB)
Conclusions A drop-in replacement for DecodeRawEvent.py exists
This was not an easy task!
Due to the many subdetector differences a configurabloid wrapper class provided the simplest implementation
DAQSys: .DecoderClass, .Decoders, .Configuration
Suggestions:1. Move defining default locations to the constructor!!
2. Homogenize the subdetector decoding C++, names and design
3. Move “options” logic from C++ python
4. Use a service to decode all BanksDataObject(Container), without using the TES, since we never write out these containers (reducing CPU cycles and memory consumption)
Rob Lambert, NIKHEF Core Soft, 24th July 2012 16
End Backups are often required
Rob Lambert, NIKHEF Core Soft, 24th July 2012 17