rapid development of radio astronomy instrumentation … · rapid development of radio astronomy...
TRANSCRIPT
Rapid Development of Radio AstronomyInstrumentation using general purpose
open source FPGA boards tools amp libraries
(how to build 12 radio astronomy instruments in two years)
Dan Werthimer Casper Team and Collaborators
httpcasperberkeleyedu
CASPER Centers for Radio Astronomy Signal Processing and Electronics Research
Robert Abiad Henry Chen Terry Filiba Griffin Foster
Suraj Gowda Jason Manley Billy Mallard Peter McMahon
Aaron Parsons Andrew Siemion Mark Wagner
Radio Astronomy Lab
Don Backer Matt Dexter Joeri van Leeuwen
David MacMahon Oren Milgrome Mel Wright Lynn Urry
Berkeley Wireless Research Center
Bob Broderson Chen Chang Kevin Chao
Borivoje Nikolic Brian Richards John Wawrzynek
Collaborators
Xilinx Fujitsu HP Sun Microsystems Agilent NSF NASA NRAO NAIC
Chris Dick Jeff Mock CFA Haystack Caltech Cornell CSIROATNF
JPLDSN South Africa KAT ManchesterJodrell Bank GMRT
Bologna (SKA) Metsahovi ObservatoryHelsinki University
Chalmers (Sweden) Seti Institute
Data pipeline Gregorian
Allen Telescope Arraybull61-meter offset Gregorian (24-meter secondary)
bullrim-supported hydroformed dishes
ATA-42 Operational October 2007
The Problem with the CurrentHardware Development Modelbull Takes 5 to 10 years
bull Cost Dominated by NRE because of custom Boards Backplanes Protocols
bull Antiquated by the time itrsquos released
bull How to buy the hardware at the last minute
bull Each observatory designs from scratch
CASPER Real-time Signal Processing Instrumentation(NSF ATI MRI)
bull Low NRE shared by the community
bull Rapid development (12 instruments 2 years)
bull Open-source collaborative
bull Reusable platform-independent gateware
bull Modular upgradeable hardware
bull Industry standard communication protocols
bull Use switches to solve correlator interconnect
bull Low Cost
Solution
bull Modular Hardware
ndash Low number of board designs
ndash Can be upgraded piecemeal or all together
ndash Reusable
ndash Standard signal processing model which
is consistent between upgrades
Solution use FPGArsquos
1 FPGA = 100 Pentium 1500 the power per opComputational Density Comparison
1000
10000
100000
1000000
10000000
102819
95
311199
7
724199
8
126199
9
419200
1
912002 114200
4
Release Date
(MO
PS
MH
z)l
am
da
^2 Processor Peak
FPGA 32-bit int MAC
FPGA maximum sustained performance
1
10
100
1000
10000
100000
12119
96
61919
97
15199
8
72419
98
29199
9
82819
99
31520
00
10120
00
41920
01
11520
01
52420
02
Release date
MO
PS
(3
2 b
it M
AC
)3X improvementper year
Moores Law for FGPArsquos
FPGA Roadmap
bull Processing power doubling every two years
bull V4 = frac12 power requirements of V2Pro
Manufacturers claim - Xilinx Inc
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
CASPER Centers for Radio Astronomy Signal Processing and Electronics Research
Robert Abiad Henry Chen Terry Filiba Griffin Foster
Suraj Gowda Jason Manley Billy Mallard Peter McMahon
Aaron Parsons Andrew Siemion Mark Wagner
Radio Astronomy Lab
Don Backer Matt Dexter Joeri van Leeuwen
David MacMahon Oren Milgrome Mel Wright Lynn Urry
Berkeley Wireless Research Center
Bob Broderson Chen Chang Kevin Chao
Borivoje Nikolic Brian Richards John Wawrzynek
Collaborators
Xilinx Fujitsu HP Sun Microsystems Agilent NSF NASA NRAO NAIC
Chris Dick Jeff Mock CFA Haystack Caltech Cornell CSIROATNF
JPLDSN South Africa KAT ManchesterJodrell Bank GMRT
Bologna (SKA) Metsahovi ObservatoryHelsinki University
Chalmers (Sweden) Seti Institute
Data pipeline Gregorian
Allen Telescope Arraybull61-meter offset Gregorian (24-meter secondary)
bullrim-supported hydroformed dishes
ATA-42 Operational October 2007
The Problem with the CurrentHardware Development Modelbull Takes 5 to 10 years
bull Cost Dominated by NRE because of custom Boards Backplanes Protocols
bull Antiquated by the time itrsquos released
bull How to buy the hardware at the last minute
bull Each observatory designs from scratch
CASPER Real-time Signal Processing Instrumentation(NSF ATI MRI)
bull Low NRE shared by the community
bull Rapid development (12 instruments 2 years)
bull Open-source collaborative
bull Reusable platform-independent gateware
bull Modular upgradeable hardware
bull Industry standard communication protocols
bull Use switches to solve correlator interconnect
bull Low Cost
Solution
bull Modular Hardware
ndash Low number of board designs
ndash Can be upgraded piecemeal or all together
ndash Reusable
ndash Standard signal processing model which
is consistent between upgrades
Solution use FPGArsquos
1 FPGA = 100 Pentium 1500 the power per opComputational Density Comparison
1000
10000
100000
1000000
10000000
102819
95
311199
7
724199
8
126199
9
419200
1
912002 114200
4
Release Date
(MO
PS
MH
z)l
am
da
^2 Processor Peak
FPGA 32-bit int MAC
FPGA maximum sustained performance
1
10
100
1000
10000
100000
12119
96
61919
97
15199
8
72419
98
29199
9
82819
99
31520
00
10120
00
41920
01
11520
01
52420
02
Release date
MO
PS
(3
2 b
it M
AC
)3X improvementper year
Moores Law for FGPArsquos
FPGA Roadmap
bull Processing power doubling every two years
bull V4 = frac12 power requirements of V2Pro
Manufacturers claim - Xilinx Inc
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Data pipeline Gregorian
Allen Telescope Arraybull61-meter offset Gregorian (24-meter secondary)
bullrim-supported hydroformed dishes
ATA-42 Operational October 2007
The Problem with the CurrentHardware Development Modelbull Takes 5 to 10 years
bull Cost Dominated by NRE because of custom Boards Backplanes Protocols
bull Antiquated by the time itrsquos released
bull How to buy the hardware at the last minute
bull Each observatory designs from scratch
CASPER Real-time Signal Processing Instrumentation(NSF ATI MRI)
bull Low NRE shared by the community
bull Rapid development (12 instruments 2 years)
bull Open-source collaborative
bull Reusable platform-independent gateware
bull Modular upgradeable hardware
bull Industry standard communication protocols
bull Use switches to solve correlator interconnect
bull Low Cost
Solution
bull Modular Hardware
ndash Low number of board designs
ndash Can be upgraded piecemeal or all together
ndash Reusable
ndash Standard signal processing model which
is consistent between upgrades
Solution use FPGArsquos
1 FPGA = 100 Pentium 1500 the power per opComputational Density Comparison
1000
10000
100000
1000000
10000000
102819
95
311199
7
724199
8
126199
9
419200
1
912002 114200
4
Release Date
(MO
PS
MH
z)l
am
da
^2 Processor Peak
FPGA 32-bit int MAC
FPGA maximum sustained performance
1
10
100
1000
10000
100000
12119
96
61919
97
15199
8
72419
98
29199
9
82819
99
31520
00
10120
00
41920
01
11520
01
52420
02
Release date
MO
PS
(3
2 b
it M
AC
)3X improvementper year
Moores Law for FGPArsquos
FPGA Roadmap
bull Processing power doubling every two years
bull V4 = frac12 power requirements of V2Pro
Manufacturers claim - Xilinx Inc
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Allen Telescope Arraybull61-meter offset Gregorian (24-meter secondary)
bullrim-supported hydroformed dishes
ATA-42 Operational October 2007
The Problem with the CurrentHardware Development Modelbull Takes 5 to 10 years
bull Cost Dominated by NRE because of custom Boards Backplanes Protocols
bull Antiquated by the time itrsquos released
bull How to buy the hardware at the last minute
bull Each observatory designs from scratch
CASPER Real-time Signal Processing Instrumentation(NSF ATI MRI)
bull Low NRE shared by the community
bull Rapid development (12 instruments 2 years)
bull Open-source collaborative
bull Reusable platform-independent gateware
bull Modular upgradeable hardware
bull Industry standard communication protocols
bull Use switches to solve correlator interconnect
bull Low Cost
Solution
bull Modular Hardware
ndash Low number of board designs
ndash Can be upgraded piecemeal or all together
ndash Reusable
ndash Standard signal processing model which
is consistent between upgrades
Solution use FPGArsquos
1 FPGA = 100 Pentium 1500 the power per opComputational Density Comparison
1000
10000
100000
1000000
10000000
102819
95
311199
7
724199
8
126199
9
419200
1
912002 114200
4
Release Date
(MO
PS
MH
z)l
am
da
^2 Processor Peak
FPGA 32-bit int MAC
FPGA maximum sustained performance
1
10
100
1000
10000
100000
12119
96
61919
97
15199
8
72419
98
29199
9
82819
99
31520
00
10120
00
41920
01
11520
01
52420
02
Release date
MO
PS
(3
2 b
it M
AC
)3X improvementper year
Moores Law for FGPArsquos
FPGA Roadmap
bull Processing power doubling every two years
bull V4 = frac12 power requirements of V2Pro
Manufacturers claim - Xilinx Inc
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
ATA-42 Operational October 2007
The Problem with the CurrentHardware Development Modelbull Takes 5 to 10 years
bull Cost Dominated by NRE because of custom Boards Backplanes Protocols
bull Antiquated by the time itrsquos released
bull How to buy the hardware at the last minute
bull Each observatory designs from scratch
CASPER Real-time Signal Processing Instrumentation(NSF ATI MRI)
bull Low NRE shared by the community
bull Rapid development (12 instruments 2 years)
bull Open-source collaborative
bull Reusable platform-independent gateware
bull Modular upgradeable hardware
bull Industry standard communication protocols
bull Use switches to solve correlator interconnect
bull Low Cost
Solution
bull Modular Hardware
ndash Low number of board designs
ndash Can be upgraded piecemeal or all together
ndash Reusable
ndash Standard signal processing model which
is consistent between upgrades
Solution use FPGArsquos
1 FPGA = 100 Pentium 1500 the power per opComputational Density Comparison
1000
10000
100000
1000000
10000000
102819
95
311199
7
724199
8
126199
9
419200
1
912002 114200
4
Release Date
(MO
PS
MH
z)l
am
da
^2 Processor Peak
FPGA 32-bit int MAC
FPGA maximum sustained performance
1
10
100
1000
10000
100000
12119
96
61919
97
15199
8
72419
98
29199
9
82819
99
31520
00
10120
00
41920
01
11520
01
52420
02
Release date
MO
PS
(3
2 b
it M
AC
)3X improvementper year
Moores Law for FGPArsquos
FPGA Roadmap
bull Processing power doubling every two years
bull V4 = frac12 power requirements of V2Pro
Manufacturers claim - Xilinx Inc
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
The Problem with the CurrentHardware Development Modelbull Takes 5 to 10 years
bull Cost Dominated by NRE because of custom Boards Backplanes Protocols
bull Antiquated by the time itrsquos released
bull How to buy the hardware at the last minute
bull Each observatory designs from scratch
CASPER Real-time Signal Processing Instrumentation(NSF ATI MRI)
bull Low NRE shared by the community
bull Rapid development (12 instruments 2 years)
bull Open-source collaborative
bull Reusable platform-independent gateware
bull Modular upgradeable hardware
bull Industry standard communication protocols
bull Use switches to solve correlator interconnect
bull Low Cost
Solution
bull Modular Hardware
ndash Low number of board designs
ndash Can be upgraded piecemeal or all together
ndash Reusable
ndash Standard signal processing model which
is consistent between upgrades
Solution use FPGArsquos
1 FPGA = 100 Pentium 1500 the power per opComputational Density Comparison
1000
10000
100000
1000000
10000000
102819
95
311199
7
724199
8
126199
9
419200
1
912002 114200
4
Release Date
(MO
PS
MH
z)l
am
da
^2 Processor Peak
FPGA 32-bit int MAC
FPGA maximum sustained performance
1
10
100
1000
10000
100000
12119
96
61919
97
15199
8
72419
98
29199
9
82819
99
31520
00
10120
00
41920
01
11520
01
52420
02
Release date
MO
PS
(3
2 b
it M
AC
)3X improvementper year
Moores Law for FGPArsquos
FPGA Roadmap
bull Processing power doubling every two years
bull V4 = frac12 power requirements of V2Pro
Manufacturers claim - Xilinx Inc
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
CASPER Real-time Signal Processing Instrumentation(NSF ATI MRI)
bull Low NRE shared by the community
bull Rapid development (12 instruments 2 years)
bull Open-source collaborative
bull Reusable platform-independent gateware
bull Modular upgradeable hardware
bull Industry standard communication protocols
bull Use switches to solve correlator interconnect
bull Low Cost
Solution
bull Modular Hardware
ndash Low number of board designs
ndash Can be upgraded piecemeal or all together
ndash Reusable
ndash Standard signal processing model which
is consistent between upgrades
Solution use FPGArsquos
1 FPGA = 100 Pentium 1500 the power per opComputational Density Comparison
1000
10000
100000
1000000
10000000
102819
95
311199
7
724199
8
126199
9
419200
1
912002 114200
4
Release Date
(MO
PS
MH
z)l
am
da
^2 Processor Peak
FPGA 32-bit int MAC
FPGA maximum sustained performance
1
10
100
1000
10000
100000
12119
96
61919
97
15199
8
72419
98
29199
9
82819
99
31520
00
10120
00
41920
01
11520
01
52420
02
Release date
MO
PS
(3
2 b
it M
AC
)3X improvementper year
Moores Law for FGPArsquos
FPGA Roadmap
bull Processing power doubling every two years
bull V4 = frac12 power requirements of V2Pro
Manufacturers claim - Xilinx Inc
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Solution
bull Modular Hardware
ndash Low number of board designs
ndash Can be upgraded piecemeal or all together
ndash Reusable
ndash Standard signal processing model which
is consistent between upgrades
Solution use FPGArsquos
1 FPGA = 100 Pentium 1500 the power per opComputational Density Comparison
1000
10000
100000
1000000
10000000
102819
95
311199
7
724199
8
126199
9
419200
1
912002 114200
4
Release Date
(MO
PS
MH
z)l
am
da
^2 Processor Peak
FPGA 32-bit int MAC
FPGA maximum sustained performance
1
10
100
1000
10000
100000
12119
96
61919
97
15199
8
72419
98
29199
9
82819
99
31520
00
10120
00
41920
01
11520
01
52420
02
Release date
MO
PS
(3
2 b
it M
AC
)3X improvementper year
Moores Law for FGPArsquos
FPGA Roadmap
bull Processing power doubling every two years
bull V4 = frac12 power requirements of V2Pro
Manufacturers claim - Xilinx Inc
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Solution use FPGArsquos
1 FPGA = 100 Pentium 1500 the power per opComputational Density Comparison
1000
10000
100000
1000000
10000000
102819
95
311199
7
724199
8
126199
9
419200
1
912002 114200
4
Release Date
(MO
PS
MH
z)l
am
da
^2 Processor Peak
FPGA 32-bit int MAC
FPGA maximum sustained performance
1
10
100
1000
10000
100000
12119
96
61919
97
15199
8
72419
98
29199
9
82819
99
31520
00
10120
00
41920
01
11520
01
52420
02
Release date
MO
PS
(3
2 b
it M
AC
)3X improvementper year
Moores Law for FGPArsquos
FPGA Roadmap
bull Processing power doubling every two years
bull V4 = frac12 power requirements of V2Pro
Manufacturers claim - Xilinx Inc
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
FPGA Roadmap
bull Processing power doubling every two years
bull V4 = frac12 power requirements of V2Pro
Manufacturers claim - Xilinx Inc
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Compute Module Diagram
138 bits 300MHz DDR 414Gbs
64 bi
t 30
0 DDR
4GB DDR2 DRAM
128GBs (400DDR)
100BT
Ethernet
5 FPGAs
2VP70FF1704
FPGA
Fabric
MG
T
Memory
Controller
IB4XCX4
20Gbps
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
DR
AM
FPGA
Fabric
MG
T
Memory
Controller
FPGA
Fabric
MG
T
Memory
Controller
FPGA
FabricM
GT
Memory
Controller
FPGA
Fabric
MGT
Memory
Controller
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
IB4XCX4
40Gbps
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Board Interconnect - Upgradable
bull Problem Backplanes are short lived
(S100 Multibus VME ISA EISA PCI PCIx PCIe compactPCI compactPCIe ATCAhellip)
bull Solution Use 10Gbit Ethernet
(10Gbe Infiniband Myrinet Xaui Aurora)
Copper CX4 (40 meters max) or Optical
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Platform-Independent Parameterized Gateware
bull What is Gateware
ndash Design logic of FPGAs
(between hardware and software)
bull Need libraries for signal processing which donrsquot have to be rewritten every hardware generation
bull Matlab Simulink
bull LINUX file IO and process control (Borph)
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Simulink-based Design Tool Flow
bull Simulink Xilinx System Generator Library
bull Custom BEE2 Library Blocksets
bull Software programmable registers
bull BEE Platform Studio
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Biplex Pipelined FFT
(Lynn Urry Aaron Parsons David MacMahon Jeff Mock JPL)
bull Uses 16 the resources of the Xilinx module
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
FFT controls
Simulink Library ndash Aaron Parsons David MacMahon
Verilog Library ndash Jeff Mock
bull Transform length
bull Bandwidth
bull Complex or Real
bull Number of Polarizations
bull Input bit width and output bit width
bull twiddle coefficient bit width
bull Run-time programmable down-shifting
bull Decimate option
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Filter Response
PFB vs FFT
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
PFB vs FFT
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Additional PFB controls
(Aaron Parsons Jeff Mock)
bull Filter overlap
bull Width of filter coefficients
bull Window function for filter (hamming hanning etc)
bull Import filter coefficients for custom filter performance
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Digital Down-Converter
bull Selectable of FIR taps
bull On-the-fly programmable mix frequency
bull Selectable FIR coeff
bull Agile sub-band selection
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
X-Engine Correlation Architecture (Lynn Urry Aaron Parsons)
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
X-Engine Architectureapplied to an arbitrary sized antenna array
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Hardware and Software Librarieslegend
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Applications
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Commercial off-the-shelf
Multicast 10 Gbps (10GE
or InfiniBand) Switch
PFBADCFPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
FPGA DSP
Module
General-purpose CPUs
PFB
PFB
Correlator
Beamformers
Spectrometers
Pulsar timer
Reconfigurable
Compute Cluster
ADC
ADC
Polyphase
Filter Banks
Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources need not be FPGA based
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Applicationsbull VLBI Mark 5B data recorder ndash Haystack NRAO ndash 512 MHz
bull Beamforming ndash ATA SMA ndash
bull SETI ndash Arecibo (UCB)
JPLUCB DSN (Preston Gulkis Levin Jones)
bull Correlators and Imagers
ATA (Aaron Parsons Mel Wright)
PAPER (Reionization Experiment)
Carma Next Gen
MeerKATSKA South Africa
GMRT next gen correlator
Bologna (SKA) FASR
Pulsar Timing and Searching Transient
Greenbank Allen Telescope Array VLA
Swinburne (Parkes) meerKAT Nancay
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
SETI Spectrometers
bull Parkes Southern SERENDIP
bull ALFA SETI Sky Survey (300 MHz x 7 beams)
bull JPL DSN Sky Survey (eventually 20 GHz bandwidth)
Radio Astronomy Spectrometers
bull GALFA Spectrometer ndash Arecibo Multibeam Hydrogen Survey
bull Astronomy Signal Processor ndash ASP ndash Don Backer Ingrid Stairs et al(pulsars)
bull Antenna Holography ATNF China
bull Gavert (DSN education outreach)
bull CMB Bolometer Readout ndash Caltech UCB
bull Fast Readout Spectrometers (Parkes NRAO ATA)
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
128 Million Channel SETI Spectrometer
bull 200 MHz Bandwidth 2 Hz resolution
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
1 GHz bandwidth ldquoPocket Spectrometerrdquo
bull Using ATMEL ADCrsquos at 2 Gsamplessec
bull Performing 4 real FFTrsquos in 1 (complex) biplex pipelined FFT module
bull 2048 channels
bull Uses just 1 ADC 1 IBOB and your laptop
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
NASA MARVEL Mission Mars Volcanic Emission and Life Scoutsub-mm spectroscopy ndash water vents methane
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
4096-Point Spectrometer
Polyphase
FIR
Real rarr
Complex
8K FFT
Power
r2 + i2
Vector
Accumulator(64-bit x 2048)
CPU
TVG
4 x 8-bit
(real)
4 x 18-bit
(real)
2 x 36-bit
(complex)
2 x 36-bit
(real)
1-bit
(serial)
ASIC
FPGA (Emulation)
bull Stream-based 2K Packets
bull Continuous operation
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
4096 channel Mars spectrometer ldquoChip in a dayrdquo FPGA to ASIC
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Portable VLBI backend ndashHenry Chen
bull Interfaces to MARK 5B data recorder
bull 500 MHz spectrum recorder
bull (This makes 4 instruments in 1 year)
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
VLBI Mark 5B Front End500 MHz BW 32 channel filter bank
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
PulsarTransient SearchingTiming
NRAO MeerKAT Swinburne (Parkes) ATA
bull 1 GHz bandwidth polyphase filter banks
1024 channels 30 uS readout full stokes
(Parkes = 300 MHz 7 beams)
bull 1 GHz coherent dedispersion (PFB FFT FFT-1)
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
ATA Flyrsquos Eye Transient Instrument44 fast readout spectrometers 3 weeks to build
Geoff Bower Jim Cordes Griffin Foster Joeri van Leeuwen Peter McMahon Andrew Siemion Mark Wagner Dan Werthimer
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
CASPER FX Architecture
F Engine 0
10GbE Switch
F Engine 1
F Engine N-1
X Engine 0
X Engine 1
X Engine N-1
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Correlators and Beamformers
bull Globally Asynchronous (like a computer cluster)
bull Data is time stamped with 1 PPS at ADC
bull Locally Synchronous Globally Asynchronous
bull Solve problem of correlatorbeamformerinterconnect problem by using 10 Gbe switches (for both interconnect and fast readout)
bull No need for high density complex boards
bull Use Fiforsquos to align data before correlation or beamforminghellip
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Engine Operations
10GbE Buffer X Eng AccumF Engine
ReformatDDCQuanti
zeChanneliz
eX Engine
ADC
F engine
X engine
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
lsar
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
21 lags
300kHz clock
discrete transistors
$19000
1960 ndash First Radio Astronomy Digital Correlator
Sandy
Weinreb
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Correlator processing power
DLB
103
102
10
104
105
106
DXB
70 75 9085 80 95 2000 05 10 2015
VLA
GFlops
1
DCB
LOFAR
SMA
DAS
EVNWSRT
107
103
106
109
ALMA
SKA
EVLA
source Arnold van Ardenne
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Selected correlator quotes
Ray Escoffier
ldquoWith correlator performance having gone up by a factor of
922000 over the last 30 years its only fair that correlator
design engineers salaries should have gone up by a similar
factorrdquo
Sandy Weinreb
ldquoIn 1960 there were no chips just discrete transistors
The $19000 was the cost of the samplers shift registers and
counter It did not include the cost of the 21 accumulators which I
made myself in a few months getting paid $240monthrdquo
Sergei Pogrebenko
ldquoIt is desirable that the output data rate from a data processor is
less than the input data raterdquo
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Ray Escoffier
ldquoWith correlator performance having
gone up by a factor of 922000 over
the last 30 years its only fair that
correlator design engineers salaries
should have gone up by a similar
factorrdquo
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Moores Law ndash Instruments using FPGArsquos 2X per year(1000000 over 20 years)
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Future Spectrometers
2015 4 THz 400 beams 10 GHz each
2020 128 THz 12800 beams
2025 4000 THz 40000 beams
2030 128000 THz 1M beams
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Where to get tested boards
bull Build and test them yourself (cheapest for large Q))
bull iBOB iADC 5 volt cables Digicom Electronics
Mo Ohady modigicomorg
bull Bee2 enclosure and power supply SAE Materials
Mario Salazar mariossaemtlcom
coordinate with Ken Lutz BWRC lutzeecsberkeleyedu
bull iBOB mounting plates front panels ndash make or buy ndash NRAO
bull Other Vendors Rakesh Mehta MTE India
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
New Boardsbull MiniRoach (V5 test board spectrometer)
bull Roach (MeerKat NRAO) (V5 CX4 DRAM SRAM)
bull Bee3 (4V5 CX4 DRAM Microsoft)
bull Quad ADC (200 Msps 8 bit)
bull E2V ADC (5 Gsps 2 25 Gsps 4 125 Gsps)
bull 64 channel ADC board (12 bit 65 Msps)
bull Programmable AttenuatorGain (RAL Oren)
bull 1 PPS distribution (RAL- Oren Milgrome)
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Agilent ADC-FGPA-CX4 board
20 Gsps 8 bit Agilent ADC
XC5VSX240T
6 CX4 ports 120 Gbps xaui100 Gbps infiniband60 Gbps 10 Gbe
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
CASPER the Friendly
bull Group Helping Open-source Signal-processing Technology (GHOST)
ndash Goal to help develop signal processing instrumenation and libraries for the community
ndash Open source hardware gateware and software
ndash Provide training and tutorials
ndash Not so much delivering turn-key instruments
ndash Promote Collaboration
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Collaboration (not turn key instruments)
bull Share Open Source Libraries
bull Workshops
bull Videorsquos and Docrsquos on Tool Flow Libraries
bull Wiki Mailing List
bull Open Source Boards (available from vendors)
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Logistics ndash Presentations
bull Get your PPT PDF to Jason Manley
bull Install on Presentation Laptop
bull DO NOT USE YOUR OWN LAPTOP
bull Please wear microphone (see Jason)
bull We are posting video and ppt to web site
bull Let Jason know if you donrsquot want your presentation posted
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Logistics ndash Location
bull SaturdaySunday at BWRC
BWRC doors locked ndash buzz or call
bull Sunday Night PizzaBeer at Danrsquos ndash all invited
bull Monday Tuesday at Space Sciences Lab
Take Bus to SSL $1 12 mins see program
Monday Tuesday Lunch will be provided
bull Tours of BWRC lab SSL in the afternoons
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Logistics Suggested Donation
$ 50 ( $ 25 for students )
covers morning refreshments two luncheshellip
$ Andrew Siemion
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner
Casper Advisory Board
Matthew Bailes John Ford Yashwant Gupta
Glen Jones Alan Langman Jonathan Weintroub
Brainstorming Session ndash Open Mic ndash Tuesday AM
Constructive Criticism
Ideas for Future Directions Long Term Strategy
How best to collaborate
Organizational Structure
Anything else REPORT
Thanks to Workshop Organizers
Henry Chen Terry Filiba
Griffin Foster Suraj Gowda
Jason Manley Billy Mallard
Peter McMahon Andrew Siemion
Laura Spitler Mark Wagner