the front-end electronics for the pierre auger observatory surface array

IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 51, NO. 3, JUNE 2004 413

The Front-End Electronics for the Pierre AugerObservatory Surface Array

D. Nitz and The Pierre Auger Collaboration

Abstract—The Pierre Auger Observatory will study ultrahigh-energy cosmic rays, reaching full sensitivity for primaries with en-ergies above 1019 eV. The southern hemisphere site observatoryis currently under construction. The surface array at this site willcontain 1600 water Cherenkov detector stations distributed over3000 km2. In this paper, the front-end electronics of the surfacedetector stations, which couple analog signal processing with dig-ital triggering, are described. The enviromental constraints, his-tory of the development, and the final design being deployed in thefield are discussed. Performance characteristics of the electronicsare presented.

Index Terms—Analog filters, Cerenkov detectors, cosmic rays,physics, programmable logic devices, testing, triggering.

I. INTRODUCTION

EACH surface array station is a water Cherenkov detector[1], [2] which is solar powered, instrumented with low

power electronics [3], and communicates with the observatorycampus via a custom radio network [4]. A photograph of one ofthe detector stations is shown in Fig. 1. More information canbe found in [5].

Signals from three photomultiplier tubes (PMTs) are sentto the front-end electronics which shapes and digitizes themevery 25 ns with the help of 10-bit analog-to-digital (ADC)converters. Subsequently, the ADC outputs are presented to thetrigger/memory circuitry.

The trigger/memory circuit evaluates ADC outputs for inter-esting trigger patterns, stores the data in buffer memory, andinforms the detector station microcontroller when a trigger oc-curs. The station controller sends triggers, and when requested,event data to the observatory campus via the wireless network.A hierarchical event trigger is used [6], [7] to select events ofinterest and reject background, while keeping within the rateconstraints imposed by the station microcontroller, the commu-nications link bandwidth, and the central data acquisition (DAQ)system. The trigger/memory circuit generates the first of the hi-erarchical trigger levels.

II. DESIGN REQUIREMENTS

The design requirements arise from a combination of environ-mental constraints, power limitations, physics considerations,and negotiated interfaces with other components of the detector.

Manuscript received May 22, 2003; revised December 18, 2003. This workwas supported by the U.S. Department of Energy.

The author is with the Physics Department, Michigan Technological Univer-sity, Houghton, MI 49931 USA.

Digital Object Identifier 10.1109/TNS.2004.828507

Fig. 1. One of the installed detector stations. A 9 phototube is situatedbeneath each of the three hatch covers. (View of the third hatch cover is blockedby the solar panels). The station electronics package is located under the domeshaped cover.

The detector stations are located in a dry desert region, at anaverage altitude of 1400 m, at 35 lattitude, 69 longi-tude. Large ambient temperature swings are possible betweenday and night, summer and winter. Long-term temperature mea-surements in the area indicate that temperatures can range from

to during the year. The electronics is thereforespecified for operation between and .

As the array is located in a sparsely populated area, thereare few places where one can connect to the electrical powerdistribution system. Power for each station is thus supplied viaa pair of 12-V 100-Ah lead acid batteries, which are chargedvia two 50–60 W-peak solar panels. This imposes a strict 10-Wpower budget on the station electronics. The power budget forthe front-end is 1.5 W.

Physics imposes constraints on the measurement specifi-cations. The typical signal from the PMTs for an interestingcosmic ray air shower event contains portions generated byphotons which convert in the tank, electrons (of both signs), andmuons. Other types of particles are rare enough to be negligible.The muons are fewer in number, but give a signal with a sharprise time and a decay time dominated by the attenuation lengthof the Cherenkov photons in the water. Usually, the convertedphotons and electrons are individually of much lower energy

0018-9499/04$20.00 © 2004 IEEE

414 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 51, NO. 3, JUNE 2004

Fig. 2. An example of a simulated muon signal from the high gain output of asingle PMT that would be observed across a 50 input to one of the front-endboard channels. The PMT is operated at a gain of 2�10 . An amplifier in thePMT base further increases the amplitude of the signal by a factor of 32. Onephotoelectron produces a 2-mV signal, when averaged over a 25-ns interval. Thestructure is due to photons arriving at the PMTs experiencing different numbersof reflections at the walls of the detector.

than the muons and result in small signals. However, there aremany more of them, and their superposition yields a relativelysmooth distribution. The arrival time distribution is dispersed,characteristic of the shower geometry and species.

The typical rise time of a muon signal is 10 ns with a decaytime of 70 ns. A simulation of the signal seen at the output ofone of the PMT bases in response to a through-going muon isshown Fig. 2.

When muons are combined with the electromagnetic com-ponent of the air shower, the signal becomes more complex. Asimulation of the response of a single PMT to the passage of atypical air shower front is shown in Fig. 3. The structure in thissignal contains information that can be used to infer attributes ofthe composition of the primary cosmic rays. This leads to a re-quirement that the front-end electronics record a detailed wave-form, rather than simply the integrated or peak signal.

Early measurements indicated that a single vertical muonwould generate a peak signal of photoelectrons per PMTper 25 ns interval. The production detectors perform somewhatbetter than that, providing photoelectrons per PMT inthat interval. However, 500 m from the axis of a hugeshower, the signal reaches photoelectrons per 25 nsinterval per PMT. This is due to the characteristic lateral distri-bution of the particles in the air shower front, which falls offrapidly away from the shower axis. This drives the specificationthat the gain be ADC count per photoelectron, with adynamic range of 15 bits or more, and ADC count RMSnoise.

However, shower fluctuations and uncertainty in determiningthe shower core position mean that even for the largest signals,a precision of a few percent in a single PMT measurement willnot be a limiting factor. Thus, while a dynamic range of 15 bitsis required, a granularity of a few percent is acceptable.

Aliasing noise induced by the ADC will contribute to thesignal degradation. The most important constraint here is the

Fig. 3. An example of a simulated shower signal from a single PMT as seenat the input to one of the front-end board channels under the same operatingconditions as in Fig. 2. Note the difference in horizontal scale between this figureand Fig. 2.

contribution to the peak signal determination for muons. Otherimportant measurements, which integrate over the 3 PMTs andmultiple time bins, are less affected. The 25–30% photostatistic(plus PMT excess noise factor) fluctuations in the vertical muonsignal means that 10% RMS aliasing noise could be acceptable.More conservatively, the aliasing noise should be less than a fewpercent.

Input signals to the front-end board are specified to be be-tween 0 and full scale. This range was chosen to minimizethe effect of pickup noise on the cables between the photomul-tiplier bases and the front-end, while staying within the linearrange of the PMT base amplifier, an Analog Devices AD8012operating with supplies.

50 Ohm termination of the input signals was specified forcompatibility with common instrumentation. SMA connectorswere specified for reliability.

The board interfaces to the bus of the station microcontroller,an IBM PowerPC 403GCX.

III. DESIGN SOLUTION

A block diagram of the front-end is shown in Fig. 4. A pho-tograph is shown in Fig. 5.

A. Analog Section

The amplifier/filter section provides two primary functions:gain adjustment and antialiasing filtering. The schematic for oneof the six identical channels is shown in Fig. 6.

The signals from the PMT bases are connected to 6 inputs onthe front end board. Back-to-back PIN diodes following the se-ries input resistor provide some degree of protection of the inputamplifier in case of voltage spikes on the input. An invertingamplifier stage followed by a noninverting stage acts as a fivepole Bessel filter with an overall gain of . Analog DevicesAD8012 amplifiers are also used here with supplies.The filter cutoff is set at 20 MHz. The aliasing noise hasbeen measured to be 5% with this cutoff. This gives acceptableanti-aliasing, while preserving the time structure of the signals.

Analog Devices AD9203 10-bit 40 MHz semiflash ADCs areused. A final RC filter (not shown) is located at the input to

NITZ: PIERRE AUGER OBSERVATORY SURFACE ARRAY 415

Fig. 4. Block diagram of the front-end electronics board with embedded block diagrams of the trigger circuitry.

Fig. 5. Photo of a production front-end electronics board.

each ADC. The output of the amplifier/filter section has a 0to 1 V full scale range. In order to observe the input pedestaland to allow for small offsets in the amplifier section, the ADCnegative reference input is offset to using a resistordivider.

In order to achieve the desired dynamic range, two outputs aresupplied from each PMT base; a low gain output directly fromthe PMT anode, and an amplified output of the last dynode witha factor of 32 higher gain. Thus, two ADCs are used for eachPMT. This yields a nominal 15-bit dynamic range, with 3% leastbit precision at the overlap.

B. Trigger/Memory Section

The trigger/memory circuit is implemented using two AlteraEP1K100QI2082 programmable logic device’s (PLDs) and anIDT IDT70V3569S6DRI static RAM chip. The design of thetrigger/memory circuit utilizes a modular pipelined approach,driven by the same 40-MHz clock as the station microcontroller.A block diagram of the trigger/memory circuitry is shown inFig. 4.

1) Triggers: The first level trigger must efficiently tag highenergy cosmic ray air showers, while simultaneously rejectingmost lower energy showers and minimizing composition-de-pendent trigger biases, within a rate constraint of 100 Hz.


Fig. 6. Schematic of an amplifier/filter channel. Filter resistor and capacitor values used on the production front-end board are shown. These values have beenmodified from nominal Bessel filter values to compensate for parasitic capacitances on the board.

Since the number of particles in the shower front is correlatedwith the primary particle energy, and the time dispersion of theshower front increases with the distance from the shower axis,for any fixed number of Cherenkov photons detected, those fromhigher energy showers will be more dispersed in time than thosefrom lower energy showers. In addition, because the energies ofthe electrons and photons in the shower are typically a few MeV,while the typical muon energies are about a GeV, the Cherenkovsignals from electrons and photons in the shower are smallerthan those of muons. Heavier nuclei tend to produce air showerswhich have more muons than lighter nuclei.

The level 1 trigger (primarily) uses a low threshold to mini-mize composition bias, and requires the signals to be extendedin time to reject lower energy showers. In order to be fully ef-ficient for a wide range of zenith angle showers and providesufficient diagnostic and monitoring capabilities, the trigger isactually the logical OR of a number of different sub-triggers.

An external trigger is the simplest, allowing triggers froman external source. A random trigger is useful for measuringbackground signals without introducing a trigger bias. It is alsouseful for test purposes.

The basis for the other subtriggers are instances of the multi-plicity trigger, which are fired whenever a specified minimumnumber of the high gain PMT signals are above their respec-tive threshold levels. Alternatively, they can be programmed tofire whenever the sum of a selected set of PMTs is above a sumthreshold.

Three of the multiplicity triggers feed time-over-thresholdtriggers. These implement the primary level 1 trigger algorithmdiscussed above by requiring that the feeding multiplicitytrigger module fire a minimum number of times within asliding window. The required minimum and the window widthare adjustable parameters set in registers.

A fourth multiplicity trigger feeds a single-bin trigger, whichis essentially the limiting case of a time-over-threshold trigger,with the sliding window width set to one bin.

Data is accumulated in a circulating shower memory bufferuntil a trigger occurs. The active buffer is then allowed to con-

tinue accumulating data until the trigger position is ofthe way through the buffer. At that point, an interrupt signal issent to the station controller, the time of the event is registeredby a time tagging circuit, the active memory buffer is closed,and a second buffer is made active. A three-bit synchronizationtag is also recorded by the time-tagging circuit, to ensure thatthe correct time tag is associated with each event.

The shower memory buffers are each 768 words long and64 bits wide. They are stored in internal PLD memory. Doublebuffering is used to minimize dead time.

2) Muon Buffers: Muon buffers are implemented to:1) collect a large sample of single muons for calibration andmonitoring; and 2) provide information about muons on theedges of an air shower, beyond the set of stations with a showertrigger. They record the ADC signals from the three high gainPMT channels when those signals meet the feeding multiplicitytrigger instance conditions. This data is zero-suppressed,therefore a time stamp is recorded for each block of data.

A muon buffer is closed after completing the recording of amuon signal which fills the buffer beyond the three-quarters fullpoint. Alternatively, a timer may be enabled to close the muonbuffers at preset intervals. When a buffer is closed, an interruptsignal is sent to the station controller, the time of the interruptis recorded by the time tagging circuit, and the second bufferis made active. A one-bit synchronization flag is also recordedby the time tagging circuit. Time stamps for the opening andclosing of the buffer are recorded by the trigger/memory circuitto allow reconstruction of the time corresponding to any dataword in the buffer. The muon buffers are stored in the externalIDT memory. They are double buffered, with each buffer 819232-bit words long.

3) Scalers: Two instances of double buffered scalers are im-plemented. Each scaler counts the output of an instance of a mul-tiplicity trigger. Alternatively, any of the other trigger outputsmay be fed to either scaler. As very high trigger rates (whichcan saturate the station controller with high priority interruptroutines) are problematic, this feature is useful for checking thatthe rate that a newly configured trigger would generate is accept-


able before allowing the trigger generate interrupts. The scalerbuffers open and close synchronously with the muon buffers toallow rates to be computed.

4) Input/output: The 60 bits of ADC data from the six ADCsare processed by input modules before the data is presented tothe remainder of the circuit. These modules buffer the ADCdata, or optionally, replace it with test data from a set of reg-isters, counters, or shift registers. Additionally, a digital filtermay be enabled to track and correct for pedestal shifts.

The trigger/memory circuit interfaces to the station mi-crocontroller bus, an IBM PowerPC 403GCX. The circuit iscontrolled by programmed data transfer writes/reads to/fromregisters. Data are transferred from the trigger/memory circuitmemory buffers into the station controller memory via directmemory access operations initiated by the station controller.

5) Code Structure: The PLD code is written in AlteraAHDL and is compiled using the Altera Quartus® compiler.The code is heavily pipelined to achieve maximum safetymargin in timing. A modular structure has been adopted tofacilitate code reuse and effective long term maintenance. Codeis stored in a CVS repository to allow tracking of code changesand to facilitate maintenance and distribution. The source codeis passed through a M4 preprocessor to select any conditionallycompiled sections of the code.

As you will note in Fig. 4, the PLD code is split betweentwo PLDs. PLD A receives only the high gain ADC signals,which are the only ones used in triggering. PLD A stores thehigh gain ADC signals in its internal shower memory buffers. Italso provides the control signals and data written into the muonbuffers. PLD B receives both the high gain and the low gainADC signals. It stores the low gain ADC signals in its internalshower memory buffers and uses the high gain signals for triggergeneration. Any shower trigger generated by one of the PLDs ispassed to the other, so that both PLDs see exactly the same setof triggers.

Cadence® Verilog is used to simulate the operation of thethree chip trigger/memory circuit. A Verilog model of each ofthe PLD chips is exported from the Quartus® compiler. This iscombined with a Verilog model of the memory chip obtainedfrom IDT. The Verilog software test benches include the worstcase timing variations of the microcontroller and ADCs. Thishas been extremely useful in identifying timing violations be-tween the PLDs and the memory chip, or between the PLDsand the station controller. The Verilog test bench code can op-tionally generate test vectors for a pattern generator, enablingthe same test bench to be executed on the actual hardware (seedescription of the timing function test rig below). This allowsextremely detailed (and automated) timing checks of the actualhardware to be performed.

IV. HISTORY

The Auger Observatory has gone through several stages onthe way to the deployment of the final equipment. We denotethese phases as “Engineering Array,” “preproduction,” and“production.” Correspondingly, the front-end board has gonethrough a number of design stages on the way to the productiondesign.

Initially, the analog and digital sections were packaged onseparate boards to facilitate parallel development. This organi-zation persisted through operation in the engineering array. Thetwo sections were merged onto a single board for production.Additionally the form factor of the board was modified to matewith a modified station controller layout and improved RF en-closure, requiring substantial layout changes.

A. Analog Section

In the Engineering Array the analog section used Analog De-vices AD8011 amplifiers instead of the AD8012 amplifiers nowused. Additionally, the full scale input ranges of the low andhigh gain channels differed. This led to partitioning the analogsection into a gain-matching input section followed by a filtersection. A two pole Bessel filter was used.

In the production design, the gain equalization functions havebeen moved entirely to the photomultiplier base. This alloweda 5-pole Bessel filter to be implemented (to increase the band-width without degrading the aliasing performance) without in-creasing the number of amplifiers. The AD8011 amplifiers werereplaced by AD8012 amplifiers, essentially dual versions of theAD8011, in order to reduce component count.

B. Trigger/Memory Section

The development of the trigger/memory circuit progressedthrough two parallel paths.

1) ASIC: One path that was followed was to develop acustom application specific integrated circuit, using a 0.35-CMOS process. This effort was successfully carried out throughthe production of the first prototype iteration of the final design.

2) Engineering Array Configuration: In parallel, a secondpath was taken utilizing an Altera EP20K200RI240-2 PLDand an IDT IDT70V3569S6DRI static RAM chip [8]. Thiscircuit was successfully used in the Auger Engineering Arrayand demonstrated that the desired functionality could be metwith an available commercial PLD. However, this solution farexceeded the cost targets.

3) Production Configuration: We finally adopted a third al-ternative, utilizing two Altera EP1K100QI2082 PLDs and theIDT memory chip. The cost of this option is a factor of two lessthan the first PLD approach described earlier. While still moreexpensive than the ASIC approach, the reduced technical riskand greater flexibility of a programmable device led us to adoptthis solution for production deployment.

C. Implementation Issues

There were several issues which required attention during thedevelopment of the production front-end board.

Noise was a serious problem in the early iterations. This waseventually solved by ensuring good separation of the analogand digital grounds and supplies. In addition, all clockeddigital signals near the analog section were sandwiched be-tween ground/power planes to minimize pickup through straycapacitances.

In early boards there was insufficient noise margin on thedigital ADC outputs to the PLDs. This was resolved by adding


low voltage TTL buffers with series damping resistors betweenthe ADCs and the PLDs.

Parasitic capacitances proved to be an issue for the prepro-duction boards, as parasitic capacitances in the analog sectionrivaled those of the filter capacitors. This lowered the cutoff fre-quency of the filter a factor of two below what was expectedfrom calculations which did not include the parasitic layoutcapacitances.

This was resolved by including the extracted parasitic capac-itances in the filter simulation, and adjusting the filter compo-nents to compensate. Finally, a further revision of the layout wasproduced which both equalized and reduced the parasitic capac-itances of the layout.

V. TESTING

Predeployment testing is important to keep the mean timebetween failures in the field long.

We have adopted a testing procedure that incorporates someelements of environmental stress screening. Our testing is par-titioned into three segments. We have dubbed these the “smoketest,” “environmental screening” (ESS), and “functional test.”If at any stage boards are found to fail to conform to specifi-cations, they are removed from the testing and segregated untilthey can be reworked.

A. Smoke Test

We start with a simple “smoke test,” in which we simplypower the board in a simple test fixture to verify that the supplycurrents are within specifications. This test protects the rest ofour more sensitive test apparatus, and is an easy way to keepobviously noncompliant devices out of the much longer subse-quent test stages.

B. Environmental Stress Screening

During the ESS, two front-end boards are connected to eachof eight ESS test rig boards for insertion into the environmentalchamber. We then cycle the chamber between andfor 48 h. This yields a throughput of 16 front-end boards in 2 d.

C. Functional Testing

After a front-end board has completed the environmentalscreening, it undergoes detailed functional testing. For thiswe use a so-called “Unified Board” (UB), which is just oneof the normal detector station microcontroller boards. Due tothe expected large temperature variations in the field, it is notsufficient to test the boards at a single temperature. Functionaltesting is performed at , , and inside anenvironmental chamber.

After the PLDs are configured, a rather detailed set of func-tional tests are performed. These include writing to and readingback each PLD register; measuring the pedestal, linearity, andgain of each channel; checking the filter cutoff frequency;checking for crosstalk between channels; verifying that thereare no missing or stuck ADC bits; verifying that there are nomissing or stuck databus bits; and pattern testing of the internaland external memories.

Fig. 7. An example of a portion of one ADC trace from a high gain output ofone of the PMTs in an operating tank. The horizontal scale is in ADC time bins,with 25 ns per bin. The vertical scale ADC counts, with 2 mV (at the input tothe front-end board) per count. This tank was � 1 km from the core of a largeshower that triggered the array.

The major components of the functional test rig include anenvironmental chamber, an UB, an arbitrary wave generator toprovide signals to the front-end board analog inputs, and an RFmultiplexer to direct the output of the wave generator to selectedfront-end board inputs. As the insertion and removal of 1600front-end boards under test far exceeds the insertion ratings ofthe connectors on the UB, an inexpensive sacrificial board isinserted between the UB and front-end board under test. Thisboard, which also provides connectors for a logic analyzer, isreplaced after every front-end board insertions. All in-struments, except for the UB, are GPIB controlled. The UB isconnected to the test control computer via a serial link.

In order to maximize testing throughput, a second environ-mental oven is used so that this rig can be run in parallel withthe ESS rig.

D. Timing Function Test Rig

The “Timing Function Test Rig” was created to performdetailed timing tests of the front-end board trigger and memorycircuitry. While not normally used in the production testingof the front-end boards, it has been useful in validating thetrigger/memory circuit design by facilitating detailed auto-mated comparisons between simulation test benches and theactual hardware. The timing function test rig consists of anenvironmental oven (the same one used for the functional testrig), a 250-MHz digital pattern generator, and a logic analyzer.A timing function test board routes signals from the patterngenerator to the front-end board, and from the front-end boardto the logic analyzer.

E. Software

In order to automate the process of front-end board testingand reliably store the results, a number of software packageswere developed using C++.

1) GPIB Under Linux Access Software System (GLASS):GLASS was created for accessing multiple GPIB devices, usingstandard C++ functions. GPIB devices, connected to a controller


PC via a GPIB network, can be accessed from any computerconnected to the same Ethernet network as the controller PC.The address of a GPIB device is a mix of the IP address of thecontroller PC and devices number in the GPIB network.

2) Front-End Electronics Tester (FEET): FEET was createdas a straightforward implementation of the testing proceduresdescribed above. It is based on the GLASS and ROOT [9] pack-ages and is written in C++. Devices which participate in the testsare controlled by GLASS. Results of each test are recorded inthe database FEETDB, according to the serial numbers of thetested board. FEETDB is organized such that results can be ac-cessed in HTML format from any computer connected to theInternet.

VI. STATUS

Production electronics are being produced, tested, deployed,and used. Fig. 7 shows one of the ADC traces recorded in a realevent. Compare this to Fig. 3. The production front-end boardsmeet all specifications. The power consumption is 1 W. Themeasured noise performance is ADC channels RMS.

ACKNOWLEDGMENT

This paper presents work done by members of the surfacedetector electronics front-end subtask of the Auger Collab-

oration. The author, reporting this work in his capacity ascoordinator of that subtask, would like to acknowledge a fewof the many collaboration members who contributed to thework. Z. Szadkowski provided early versions of the PLD code.M. Trombley, J. Darling, and J. Chye made major contributionsto the ASIC development work. A. Dorofeev, J. Anderson, andR. Lewis did much of the development work for the test rigsand associated software. D. Cheam contributed substantiallyto the PLD coding and front-end board testing. K. Bowie hasbeen a mainstay in the front-end board design.

REFERENCES

[1] J. Bluemer, “Status, performance, and perspectives of the Pierre AugerObservatory,” in Proc. 28th ICRC, Tsukuba, 2003.

[2] P. Mazur, “The surface detectors of the Pierre Auger Observatory,” inProc. 28th ICRC, Tsukuba, 2003.

[3] T. Suomijärvi, “Processing of the signals from the surface detector ofthe Pierre Auger Observatory,” in Proc. 28th ICRC, Tsukuba, 2003.

[4] P. D. J. Clark and D. Nitz, “Communications in the Auger Observatory,”in Proc. 27th ICRC, Hamburg, 2001.

[5] Pierre Auger Observatory . [Online]. Available: http://www.auger.org[6] D. Nitz and the Pierre Auger Collaboration, “Triggering and data acqui-

sition systems for the Auger Observatory,” IEEE Trans. Nucl. Sci., vol.45, pp. 1824–1827, 1998.

[7] , “Implementation of the first level trigger for the Auger Observa-tory surface array,” in Proc. 27th ICRC, Hamburg, 2001.

[8] Z. Szadkowski and D. Nitz, “A PLD implementation of the Pierre AugerObservatory first level trigger,” in Proc. 27th ICRC, Hamburg, 2001.

[9] R. Brun and F. Rademakers. ROOT: An object-oriented data analysisframework. [Online]. Available: http://root.cern.ch

the front-end electronics for the pierre auger observatory surface array

Documents