physics division the south pole telescope readout system march 4, 2004 spt eab meeting, u. chicago...

Physics Division

The South Pole The South Pole TelescopeTelescope

Readout SystemReadout SystemMarch 4, 2004March 4, 2004

SPT EAB Meeting, U. ChicagoSPT EAB Meeting, U. Chicago

OUTLINEOUTLINE Overview of SystemOverview of System Technical ChallengesTechnical Challenges Current StatusCurrent Status

Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 2

SPT Readout TeamSPT Readout Team Matt Dobbs + Helmuth SpielerMatt Dobbs + Helmuth Spieler

overall coordination, design, integration, hands on implementationoverall coordination, design, integration, hands on implementation LBNL Engineers John Joseph and Chinh VuLBNL Engineers John Joseph and Chinh Vu

Design, Layout and Commissioning of Osc/Demod and SQUID Controller Design, Layout and Commissioning of Osc/Demod and SQUID Controller boardsboards

John Clarke (UCB) and Sherry Cho (postdoc)John Clarke (UCB) and Sherry Cho (postdoc) SQUID ExpertiseSQUID Expertise

UCB Grad Student Trevor Lanting (A. Lee)UCB Grad Student Trevor Lanting (A. Lee) Cold Frequency Domain SQUID Multiplexer commissioning and Cold Frequency Domain SQUID Multiplexer commissioning and

performanceperformance UCB Grad Student Martin Lueker (W. Holzapfel)UCB Grad Student Martin Lueker (W. Holzapfel)

SQUID shielding, SQUID Controller commissioning, softwareSQUID shielding, SQUID Controller commissioning, software Technician Dennis SeitzTechnician Dennis Seitz

SQUID Quality control and testing (experience from CDMS)SQUID Quality control and testing (experience from CDMS) much useful help and discussions from T. Crawford (Chicago)much useful help and discussions from T. Crawford (Chicago)

extended visits to LBNLextended visits to LBNL


Readout System Readout System RequirementsRequirements

TES Bolometer ReadoutTES Bolometer Readout resolve bolometer noise floor (~10 pA/√Hz)resolve bolometer noise floor (~10 pA/√Hz)

SQUID as 1st amplifier stageSQUID as 1st amplifier stage maintain voltage bias for low (0.5 Ω) impedance sensormaintain voltage bias for low (0.5 Ω) impedance sensor

low amplifier input impedance low amplifier input impedance shunt feedback shunt feedback 1/f knee low enough for drift scanning (~100 mHz)1/f knee low enough for drift scanning (~100 mHz)

AC biasAC bias

DiagnosticsDiagnostics operate SQUID open loopoperate SQUID open loop map SQUID output voltage vs. input currentmap SQUID output voltage vs. input current monitor SQUID DC level during data takingmonitor SQUID DC level during data taking full spectral distribution for each readout channelfull spectral distribution for each readout channel

Full computer controlFull computer control


APEX-SZ is SPT PrototypeAPEX-SZ is SPT Prototype SPT and APEX-SZ use same readoutSPT and APEX-SZ use same readout sharing of resources between APEX-SZ and SPT readout sharing of resources between APEX-SZ and SPT readout

systems provides SPT with early experiencesystems provides SPT with early experience APEX-SZ is 320 channelsAPEX-SZ is 320 channels APEX-SZ baseline: no multiplexingAPEX-SZ baseline: no multiplexing Readout includes all functionality req’d for multiplexingReadout includes all functionality req’d for multiplexing

APEX-SZ timeline (deployment end of this year) means APEX-SZ timeline (deployment end of this year) means electronics will be available early for SPTelectronics will be available early for SPT serves as proof-of-concept for SPTserves as proof-of-concept for SPT time for modifications / upgrades post-APEXtime for modifications / upgrades post-APEX


ImplementationImplementation 8-channel SQUID controller8-channel SQUID controller

mounted directly on cryostatmounted directly on cryostat full computer control + diagnosticsfull computer control + diagnostics one cable per 8-ch controller connects to demodulator boardone cable per 8-ch controller connects to demodulator board

Demodulator + digitizer for each bolometer channelDemodulator + digitizer for each bolometer channel rack-mounted in receiver cabinrack-mounted in receiver cabin 3 VME crates (9U) for 940 bolometers3 VME crates (9U) for 940 bolometers 16 demodulator channels per 9U board16 demodulator channels per 9U board each board includes digitization + computer interfaceeach board includes digitization + computer interface connections to computer optically isolatedconnections to computer optically isolated

Use standard commercial ICs throughoutUse standard commercial ICs throughout low cost per channellow cost per channel


MultiplexingMultiplexing reduce heat transfer through wiring to cold stagereduce heat transfer through wiring to cold stage scalability of system to larger arraysscalability of system to larger arrays minimize complexity of cold wiringminimize complexity of cold wiring reduce number of SQUIDsreduce number of SQUIDs

reduce costreduce cost reduce testing timereduce testing time minimize Prozac usageminimize Prozac usage

increased dynamic range requirements relative to single increased dynamic range requirements relative to single bolometer readoutbolometer readout


Frequency Domain Frequency Domain MultiplexingMultiplexing

complementary to time domain multiplexingcomplementary to time domain multiplexing bolometers are AC biased in superconducting transitionbolometers are AC biased in superconducting transition

ffcarrier carrier well above bolometer bandwidthwell above bolometer bandwidth

bolometers grouped in readout “modules”, 8-32 channelsbolometers grouped in readout “modules”, 8-32 channels each bolometer in module biased at unique feach bolometer in module biased at unique fcarrriercarrrier = 0.5-1 MHz = 0.5-1 MHz each channel has stand-alone capability (no shared oscillators)each channel has stand-alone capability (no shared oscillators)

incident radiation (sky signal) causes variation in Rincident radiation (sky signal) causes variation in RBOLOBOLO

amplitude modulates carrier amplitude modulates carrier transfers signal power to sidebands (~1kHz bandwidth)transfers signal power to sidebands (~1kHz bandwidth) bolometer signals at unique frequencies bolometer signals at unique frequencies can be summed in one wire can be summed in one wire

bias currents are applied as “comb” of carriersbias currents are applied as “comb” of carriers one wire for bias one wire for bias module requires 2 wires module requires 2 wires

frequency-selective demodulation to separate bolometer signalsfrequency-selective demodulation to separate bolometer signals


Frequency Domain Frequency Domain MultiplexerMultiplexer

f500 kHz

f500 kHz

200 Hz

80 dBc

Amp-Modulated by BoloDDS

Amplified

600 kHz 600 kHz f100 Hz

Demodulated

500 kHz

600 kHz

f100 Hz

f500 kHz

200 Hz

80 dBc

600 kHz

Shunt FBSQUID

fMUX Chip

DDS Oscillators

RB

R1

R2

-V2

V1

I

I1

RS

C1 C2 C3 Cn

L1 L2 L3 Ln

2

Vn(fn)

Demodulator

RSRSRS

Rfb

ADCSQUID Controller

TES Bolometers

Series Array SQUIDs

T=4 K T=0.3 K only 2 wires from coldstage



fMUX Chip

DDS Oscillators

RB

R1

R2

-V2

V1

I

I1

RS

C1 C2 C3 Cn

L1 L2 L3 Ln

2

Vn(fn)

Demodulator

RSRSRS

Rfb

ADCSQUID Controller

TES Bolometers

Series Array SQUIDs

Bolometers are AC biased Bolometers are AC biased not sensitive to low frequency noise downstream of bolometernot sensitive to low frequency noise downstream of bolometer

Shunt feedback Shunt feedback provides low input impedance to maintain bolo voltage bias for AC carriersprovides low input impedance to maintain bolo voltage bias for AC carriers

Use of Digital Direct Synthesizers Use of Digital Direct Synthesizers programmable, cheap, and scalable.programmable, cheap, and scalable.

System is bolo noise limited.System is bolo noise limited.



fMUX Chip

DDS Oscillators

RB

R1

R2

-V2

V1

I

I1

RS

C1 C2 C3 Cn

L1 L2 L3 Ln

2

Vn(fn)

Demodulator

RSRSRS

Rfb

ADCSQUID Controller

TES Bolometers

Series Array SQUIDs

test chip of 8-channel Multiplexer (LBNL, UCB, NGC)


Osc/Demod Readout Osc/Demod Readout BoardBoard 16 Channels / board (9u x 400mm)16 Channels / board (9u x 400mm)

Separate oscillators for bolo bias + nullingSeparate oscillators for bolo bias + nulling independent amplitude + phase controlindependent amplitude + phase control use DDS with common clock (both synchronous)use DDS with common clock (both synchronous)

harmonics, spurious signals < 80 dBc (typ < 100 dBc)harmonics, spurious signals < 80 dBc (typ < 100 dBc) noise sidebands low to 10 mHz.noise sidebands low to 10 mHz.

Sampling Demodulator (+35 dBm IP3)Sampling Demodulator (+35 dBm IP3) Parallel channel to monitor SQUID DC output Parallel channel to monitor SQUID DC output

independent of carrier amplitude (flag flux jumping)independent of carrier amplitude (flag flux jumping) 14 Bit ADC’s (4 per 16 channels)14 Bit ADC’s (4 per 16 channels)

oversampling to obtain dynamic rangeoversampling to obtain dynamic range one sample-and-hold per channel one sample-and-hold per channel

Opto-isolated RS485 to PC for ControlOpto-isolated RS485 to PC for Control SQUID Controller commands passed throughSQUID Controller commands passed through

Opto-isolated LVDS to Opto-isolated LVDS to Custom digital PCI I/O board in PC (FNAL design)Custom digital PCI I/O board in PC (FNAL design) data push at 7 Mbits /s data push at 7 Mbits /s

(about 1000 channels x 2.5 kHz = 6 Mbits/s)(about 1000 channels x 2.5 kHz = 6 Mbits/s) Power ~80W per board.Power ~80W per board.


Warm Cabling and Warm Cabling and MechanicsMechanics 9U Osc/Demod boards housed in VME 64x Crates with 9U Osc/Demod boards housed in VME 64x Crates with

custom backplanescustom backplanes 20 boards (16 ch ea.) / crate, 3 crates20 boards (16 ch ea.) / crate, 3 crates 1 clock distribution board per crate 1 clock distribution board per crate

Warm CablingWarm Cabling commercial 10’ SCSI differential cables commercial 10’ SCSI differential cables (Cryostat(Cryostat Readout boards), twisted pairs in common shield Readout boards), twisted pairs in common shield LVDS twisted pair to PCLVDS twisted pair to PC

Power:Power: 5 kW for 60 boards = 920 Channels5 kW for 60 boards = 920 Channels commercial 250 kHz switching power supplies, 1% Vcommercial 250 kHz switching power supplies, 1% Vp-pp-p Ripple Ripple


8ch SQUID 8ch SQUID ControllerController

8 channel SQUID controller8 channel SQUID controller• shunt feedbackshunt feedback• switch open loop/closed loop w. switchable switch open loop/closed loop w. switchable gain, gain, • on board FPGA / DAC for programming on board FPGA / DAC for programming SQUIDSQUID• mates directly to cryostatmates directly to cryostat• analog data out to demodulator boards.analog data out to demodulator boards.

Min

RFB

104VΦ

enoiseam

p

Lin

Signal IN

100

100

10

MOSFET

Cold Warm

Not Shown:• DC Squid bias• DC Flux bias• DC Amp Voltage Offset• SQUID Voltage measure lead

19mm pitch


SQUID pre-amplifersSQUID pre-amplifers Measure current in SC loop of LCRMeasure current in SC loop of LCRbolobolo

100 element Series array SQUIDs from 100 element Series array SQUIDs from NISTNIST AC biased AC biased 1/f noise unimportant 1/f noise unimportant Squid Noise: 0.1 μSquid Noise: 0.1 μΦΦ00/√Hz ≈ 2 pA/√Hz, /√Hz ≈ 2 pA/√Hz,

adequateadequate Expect to receive first two wafers from Expect to receive first two wafers from

NIST next week (destined for APEX-SZ)NIST next week (destined for APEX-SZ) Quality control and testing at Berkeley by Quality control and testing at Berkeley by

Dennis Seitz (experienced with NIST Dennis Seitz (experienced with NIST arrays for CDMS)arrays for CDMS)

SQUID Transimpedance

0

50

100

150

200

250

300

350

400

-0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0

Flux Bias (Phi0)

Ztr

(O

hm

s) ZTrans= 400 Ω

Flux BiasPUT N

IST DRAW

ING H

ERE.


SQUID Magnetic SQUID Magnetic ShieldingShielding

!" # $ %"" & '&

( "" ) %$ $ *+,-. # ' # )


SQUID Magnetic SQUID Magnetic ShieldingShielding

Two part shieldingTwo part shielding 0.056’ Cryoperm chimney style shield0.056’ Cryoperm chimney style shield

(attenuates B, by few 10(attenuates B, by few 1033

Nb film Nb film (pins B field lines, by >15)(pins B field lines, by >15)

Measured performance (prototype)Measured performance (prototype) DC attenuation of few 10DC attenuation of few 1055

improves to >10improves to >1066 for AC B fields for AC B fields Production Drawings near-readyProduction Drawings near-ready Finalizing Mechanical support/connetorsFinalizing Mechanical support/connetors

CalculatedAttenuationfrom Nb Foil

(Martin Lueker)


Excess Loading

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

400 500 600 700 800 900 1000 1100 1200

freq (kHz)

loa

din

g (

oh

ms

)

Excess Loading

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 0.2 0.4 0.6 0.8 1 1.2

dielectric area (mm2)

loa

din

g (

oh

ms

) NbNb22OO55 capacitors capacitors

lossy, spoils lossy, spoils voltage bias.voltage bias.

Solution: use Solution: use external NP0 external NP0 ceramic chip caps.ceramic chip caps.


MUX LC ChipMUX LC Chip

2nd Generation MUX ChipNew Mask for MUX chip, inductors only

1/3 Q

Fabricated byA. Smith, Northrop Grumman Space Tech

1st Generation MUX ChipIntegrated LC’s. 8 channels


Excess Loading

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

400 500 600 700 800 900 1000 1100 1200

freq (kHz)

loa

din

g (

oh

ms

)

Measured Loading (L only with surface mount)

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0 200 400 600 800 1000 1200

freq (kHz)

load

ing

(o

hm

)

OLD LC Chip

New L chip, with Surf mount caps

Loss in LCLoss in LC


100000 1000000

1E-6

1E-5

1E-4BK2 L chip Inductive Coupling

k=0.01

k=0.009

k=0.008k=0.007

k=0.006neighbour

next nearest neighbour

co

up

led

cu

rre

nt (

A)

frequency (Hz)

LC Chip Cross TalkLC Chip Cross Talk


0.01 0.1 1 10 100 1000

1

10

100

1000

Demodulated Noise of Sensor J2 (sensor normal, T=0.3K)

expected level

demodulator floor

sensor

cu

rre

nt n

ois

e (

pA

/Hz1/

2 )

freq (Hz)

NOISE OF SENSOR JUST ABOVE ITS TRANSITION (HIGH VOLTAGE BIAS) WITH CMB ELECTRONICS (fbias=1.5 MHz)

100 Hz Sky signal

1/f dominated byactive filter in demodulator fixed in new Tayloe mixer

measured noise as calculated


0.1 1 10 100

1

10

100

1000

10000

Demodulated Noise of Sensor J2 (T=0.3K)

detector normal in transition in transition in transition in transition demodulator noise floor

curr

en

t no

ise

(p

A/H

z1/2 )

freq (Hz)

NOISE OF SENSOR IN TRANSITION WITH 100Hz SIMULATED SKY SIGNAL WITH CMB ELECTRONICS (fbias = 1.5 MHz)


Readout ChallengesReadout Challenges dynamic range: >80 dB for single bolometerdynamic range: >80 dB for single bolometer high Q LC resonatorhigh Q LC resonator

solved- custom Nb inductors + NP0 chip capacitorssolved- custom Nb inductors + NP0 chip capacitors intermodulation distortionintermodulation distortion

new Tayloe Mixer (35 dBm IP3)new Tayloe Mixer (35 dBm IP3) requires (passive) nulling of bolometer carriers at SQUID inputrequires (passive) nulling of bolometer carriers at SQUID input

low frequency responselow frequency response measured DDS stability, good to 10 mHzmeasured DDS stability, good to 10 mHz low noise Tayloe Mixer low noise Tayloe Mixer

cold SQUID + warm amplifier feedback loopcold SQUID + warm amplifier feedback loop requires short cold-warm wiring lengthsrequires short cold-warm wiring lengths


Intermodulation Distortion Intermodulation Distortion and Nullingand Nulling

Carrier inter-modulation products Carrier inter-modulation products troublesome.troublesome.

Can avoid them or NULL the Can avoid them or NULL the carriers to minimize their effectcarriers to minimize their effect

Carrier nulling at 60 dB Carrier nulling at 60 dB obtainable, accurate to 10 mHz. obtainable, accurate to 10 mHz.

New TAYLOE Mixer on New TAYLOE Mixer on demodulator board has excellent demodulator board has excellent IP3=35 dBm.IP3=35 dBm.


ColdColdWarm Wiring Warm Wiring LengthsLengths

SQUID locked in feedback loop w/ warm ampSQUID locked in feedback loop w/ warm amp Loop gain determines dynamic range.Loop gain determines dynamic range.

original goal: Aoriginal goal: ALOOPLOOP=100 at 1 MHz=100 at 1 MHz

AALOOPLOOP=1 at 100 MHz=1 at 100 MHz

need to maintain phase margin at freq. where Aneed to maintain phase margin at freq. where ALOOPLOOP>1 >1 limits wiring length between cold and warmlimits wiring length between cold and warm depends on max bias freq. x loop gaindepends on max bias freq. x loop gain Example: AExample: ALOOPLOOP=100 at 1 MHz =100 at 1 MHz

round trip wiring length <20 cm for 45 deg phase marginround trip wiring length <20 cm for 45 deg phase margin SQUID arrays relax dynamic range requirementsSQUID arrays relax dynamic range requirements Loop gain requirement set by SQUID non-linearity (intermodulation products) Loop gain requirement set by SQUID non-linearity (intermodulation products)

-- t.b.d. for NIST arrays-- t.b.d. for NIST arrays achieved wire length is 2x 15 cmachieved wire length is 2x 15 cm

can achieve Acan achieve ALOOP LOOP = 50 with some margin= 50 with some margin carrier nulling reduces max. signal -- essentialcarrier nulling reduces max. signal -- essential

COLD | WARM

I1Rfb

SQUID ControllerSeries Array SQUIDs


SPT Readout StatusSPT Readout Status SQUIDsSQUIDs

sample 8-turn NIST arrays tested with our systemsample 8-turn NIST arrays tested with our system first production SQUIDs arriving at Berkeley this monthfirst production SQUIDs arriving at Berkeley this month

SQUID ShieldsSQUID Shields prototypes fabricated at Amunealprototypes fabricated at Amuneal attenuation measured – adequateattenuation measured – adequate designing production cryoperm shields nowdesigning production cryoperm shields now

Cold wiring harnessesCold wiring harnesses prototypes ordered from TechDataprototypes ordered from TechData

SQUID ControllersSQUID Controllers 4-channel analog and 8-channel digitally controlled prototypes tested 4-channel analog and 8-channel digitally controlled prototypes tested

(testing of 8-ch board continues)(testing of 8-ch board continues) Layout revisions of production boards begun Layout revisions of production boards begun

Warm cablingWarm cabling off the shelf SCSI cables, samples procuredoff the shelf SCSI cables, samples procured


SPT Readout StatusSPT Readout Status Oscillator/Demodulator Readout BoardsOscillator/Demodulator Readout Boards

Prototype single channel board fabricated and testedPrototype single channel board fabricated and tested Tayloe Mixer tested as add-on boardTayloe Mixer tested as add-on board

Prototype 16 channel board fabricatedPrototype 16 channel board fabricated functionality verified, need to measure performance, test with systemfunctionality verified, need to measure performance, test with system

VME Crates selected (Wiener), off-the-shelf power supplies selectedVME Crates selected (Wiener), off-the-shelf power supplies selected Custom backplanes: power plane finished, signal/clock plane underway.Custom backplanes: power plane finished, signal/clock plane underway.

FirmwareFirmware slave board firmware written/testedslave board firmware written/tested SQUID Controller / Master board firmware will start soonSQUID Controller / Master board firmware will start soon

Clock Distribution BoardClock Distribution Board prototype fabricated and testedprototype fabricated and tested 9U prototype in the pipeline (very simple board)9U prototype in the pipeline (very simple board)

Digital IO BoardDigital IO Board ““borrowed” Custom IO boards from Fermilab, 2 boards in handborrowed” Custom IO boards from Fermilab, 2 boards in hand

Readout SoftwareReadout Software OO software in C++ on Linux OS, interfaced with experiment Software via TCP/IPOO software in C++ on Linux OS, interfaced with experiment Software via TCP/IP High Level Design Sketches/BrainstormingHigh Level Design Sketches/Brainstorming Many low level Hardware interfaces and tuning algorithms written and in use.Many low level Hardware interfaces and tuning algorithms written and in use.


Readout Status – SystemReadout Status – System individual elements of readout system individual elements of readout system

prototyped and testedprototyped and tested early prototype integrated readout early prototype integrated readout

system tested with bolometersystem tested with bolometer analog SQUID controller, single channel analog SQUID controller, single channel

demodulator…demodulator… noise properties and two channel noise properties and two channel

fMUX testedfMUX tested commissioning 8 channel system with commissioning 8 channel system with

bolometers nowbolometers now testing of “production ready” testing of “production ready”

integrated system forthcoming integrated system forthcoming (this spring/early summer)(this spring/early summer)

Physics Division

The South Pole The South Pole TelescopeTelescope

Readout System Readout System SoftwareSoftwareMarch 4, 2004March 4, 2004

SPT EAB Meeting, U. ChicagoSPT EAB Meeting, U. Chicago


Framework / Framework / InfrastructureInfrastructure

Programming in C++ on Linux OSProgramming in C++ on Linux OS Communication w/ Outside world via TCP/IP only.Communication w/ Outside world via TCP/IP only. Dedicated PC for Data Acquisition, receives digital data Dedicated PC for Data Acquisition, receives digital data

from readout boards, and makes it available on sockets for from readout boards, and makes it available on sockets for SPT control PC to archive (rates up to 100-400 Hz, TBD)SPT control PC to archive (rates up to 100-400 Hz, TBD) also supports streaming data directly to disk at rates up to 2.5 also supports streaming data directly to disk at rates up to 2.5

kHz, for debugging, etc.kHz, for debugging, etc. Readout Tuning and Monitoring PC Readout Tuning and Monitoring PC

algorithms to setup and tune SQUIDs/Bolometersalgorithms to setup and tune SQUIDs/Bolometers monitors noise and DC levels (flux jumping) in all channelsmonitors noise and DC levels (flux jumping) in all channels

knows algorithms to repair channelsknows algorithms to repair channels

provides quality control data on sockets for SPT archiveprovides quality control data on sockets for SPT archive


Sampling Rate of Bolo Sampling Rate of Bolo TimestreamTimestream

sampling pulse can be generated externally or internallysampling pulse can be generated externally or internally the maximum bandwidth is determined by the bolometer time the maximum bandwidth is determined by the bolometer time

constantconstant(hardware allows us to reduce bandwidth below 1 kHz)(hardware allows us to reduce bandwidth below 1 kHz) Bolometer time constant ~ 1 ms Bolometer time constant ~ 1 ms (not well known yet!)(not well known yet!) bandwidth 1/(2πτ) = 160 Hzbandwidth 1/(2πτ) = 160 Hz

by low pass filtering the bolometer timestream at 160 Hz, and sampling at the by low pass filtering the bolometer timestream at 160 Hz, and sampling at the Nyquist frequency 400 Hz, all the timestream information is encoded Nyquist frequency 400 Hz, all the timestream information is encoded

we sample with a 14 bit ADC, providing a LSB accuracy of ~10we sample with a 14 bit ADC, providing a LSB accuracy of ~10 -4-4.. Why use the full bandwidth when the sky signal will be modulated Why use the full bandwidth when the sky signal will be modulated

at slower rates?at slower rates? we can gain further accuracy by over-samplingwe can gain further accuracy by over-sampling Sampling beyond bolometer 3 dB bandwidth allows full reconstruction of Sampling beyond bolometer 3 dB bandwidth allows full reconstruction of

signals within MUX bandwidth for diagnostics. signals within MUX bandwidth for diagnostics. Data storage rate can be reduced by real time averaging during measurementData storage rate can be reduced by real time averaging during measurement

can be done in FPGA on demodulator boardcan be done in FPGA on demodulator board can keep high sample rate and reduce readout rate.can keep high sample rate and reduce readout rate.


Data Flow CartoonData Flow Cartoon

Oscillator/Demodulator Boards(optically isolated from PC / GPS)

DAQ Computer

Control and Monitoring Computer(might be same PC)

read/write

contro

ls

& time st

amp

Data pushed at few MBytes / s

subset of data foronline monitoring &SQUID / Bolo tuning

receiver housekeeping data

RS 485 Serial

To SPT Archive PC


Data RateData Rate Between readout boards and PCBetween readout boards and PC

size of frame for one sample: size of frame for one sample: 2.9 kBytes2.9 kBytes (of which about 40% is header information– see cartoon of frame on next slide)(of which about 40% is header information– see cartoon of frame on next slide)

400 Hz sampling rate 400 Hz sampling rate 1.1 MBytes / second 1.1 MBytes / second 2.5 kHz sampling rate 2.5 kHz sampling rate 7.1 MBytes / second 7.1 MBytes / second

Data rate written to disk (bolo data only)Data rate written to disk (bolo data only) 400 Hz 400 Hz 0.74 MBytes/s 0.74 MBytes/s (62 GBytes / 24 hrs)(62 GBytes / 24 hrs)

100 Hz 100 Hz 0.18 MBytes /s 0.18 MBytes /s (15.5 GBytes / 24 hrs)(15.5 GBytes / 24 hrs)


Size of Data Frame: 2.9 Size of Data Frame: 2.9 kByteskBytes

Data

Fra

me f

rom

O

sc/D

em

od

ula

tor

Data

fro

m o

ne

Cra

te1 Byte CRATE ID9 Bytes GPS Time

1 Byte BOARD ID

1 Byte CHANNEL ID2 Bytes 14 bit data sample

Data

fro

m o

ne

Osc/D

em

od

B

oard

3 cra

tes

16 chan

nels/b

oard

20 boar

ds/cr

ate


Block Diagram: Bolo Data Block Diagram: Bolo Data FlowFlow

SQUID Controller

8 MUX Modules / board

16 DDS Oscillatorsfor Carriers

16 DDS Oscillatorsfor Nulling

16 Demodulators ADC’s

FPGA

Op

to-

Isola

tors

Osc/Demodulator Boards

Digital IO BoardWith FPGA

2 MByte Buffer

Persistent Storageon Hard Disk& Ultrim Tape

Data Acquisition PC

PCI Interface

PC MemorySoftware

ProcessingDataReduction

Bolometer time

stream1 MHz BW

Comb of Carriers

Comb for Nulling

Bolometer time

stream1 kHz BW

Data Cable: Diff. Twist

Pair≤ few

MBytes/s

time interval of encoded data need only be larger than

signal bandwidth,~ 100 Hz.≈0.2 MB/s

we are free to average some number of data samples here, while still maintaining benefits of oversampling.

60 x 16ch Boards

in 3 9u Crates

15 x (8 Mod x 8ch) Boards

RS 485 IODifferential & Isolated

Control (and Monitoring?) PC

PC MemorySoftware

Processing

External SPT Sampling Clock

Control & Bolo / SQUID Tuning

Provides GPSTimestamp

Control

Subset of Data for

Monitoring

Readout Hardware accommodatesboth of these timing Possibilities.

Comment

Flow of Timing Info

Flow of Control Commands

Flow of Bolo Data

Legend

Housekeeping Data

physics division the south pole telescope readout system march 4, 2004 spt eab meeting, u. chicago...

Documents

spt readout systems

spt eab meeting

readout modules

spt prototypespt

lbnlmatt dobbs helmuth

squid controller commissioning

bolometer channelrack

channelseach bolometer