Casper Signal Processing Workshop2009

SKA Signal Processing(Preliminary)

Wallace TurnerDomain Specialist for Signal Processing

SPDO

Example Configuration (Phase 2)

2 of 35

Memo 100 identifies the following options:70-200MHz: Sparse AA200-500MHz: Sparse AA

500MHz-10GHz: 3000 15m dishesOr500MHz- 10GHz: 2000 15m dishes with PAFs plus WBSPFOr500MHz-10GHz: 250 Dense AA plus 2400 15m dishes/ WBSPF

Note: On going discussions 15m vs 12m dishes

Example Configuration with Dense AA + SPF

SPDO

Reference Design

3 of 35

SPDO

Dishes+Single Pixel Feeds

4 of 35

American: 6m Hydroformed Dish South Africa: 15m Composite Dish

Canadian: 10m Composite Dish

Note:On going discussions 12m vs. 15m dishRequired sensitivity 10,000 m2K-1

Correlator processor and dump rate proportional to Nant

2 ADC likely to be at antenna (4 bit ?)O/Prate = fs.4bits = 160 G bits/s per antenna

Where fs = sample rate likely to be split into smaller basebands

SPDO

AGN Science Chapter Example

5 of 35

0

200

400

600

800

1000

1200

1400

1600

30 35 40 45 50 55 60

No. dishes

Achieved Tsys K

No. of 15 m dishes needed for AGN and Star Formation for various aperture efficiencies with single pixel feeds

50%

55%

60%

65%

AGN Science ChapterSurvey Speed: 1 x 105 m4K-2deg2

Frequency range 500 MHz to 8GHzTsys now: Not achieved over bandwidthTarget Tsys: 35K

SPDO

Dishes+Phased Array Feeds

6 of 35

Apertif Netherlands: Vivaldi Array ASKAP Australia: Checkerboard Array

PHAD Canada: Vivaldi Array

Note:Some Channelization and Beamforming likely to be at antenna.Maximum Field of View limited by Array size and focal length of dish.Achievable field of view limited by network bandwidth.

SPDO

Example Number PAFs Required

7 of 35

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

40 50 60 70 80 90 100

No. dishes

Achieved Tsys K

No. of 15 m dishes needed for Wide Field Polarimetry for various aperture efficiencies with PAF

50%

60%

70%

80%

Wide Field Polarimetry Science ChapterSurvey Speed: 5 x 108 m4K-2deg2

Max frequency 2GHz

WBSPF would require over 15,000 dishes

Tsys now: 100 KTarget Tsys: 50 K

SPDO

Simplistic View of PAF Processing

• Consider Frequency Domain Beamforming• Channelisation: Nchan ~ 64

– 12 taps gives < 60db aliasing between channels– Processing load ~ (Ntaps + 3*log2(Nchan)) x Nel x 2pol x fs

– Ntaps = 12, Nel = 96 x 2pol & fs = 1.4 GHz for 700MHz bandwidth

Processing load = 8 x 1012 MACS• Beamformer (per antenna):

– Average beams per channel 30– Processing Load = Nbms.Nel.2pol.fs. 4Multiplies = 3 x 1013

Macs

• O/Prate= Nbms.2pol.fs.4bits.8B10B = 420 G bit/s– 42 10 G bit/s optical cables per antenna– Includes 25 % extra bandwidth required for 8B/10B

Encoding• 2000 dishes with PAFs (total 840 T bits/s)Note :*FFT implementation dependent

8 of 35

SPDO

Sparse Aperture Arrays

9 of 35

LOFAR: Netherlands et al

LWA: USA MWA: USA & Australia

Note:Two types of sparse AA required:70MHz – 200 MHz200MHz – 500 MHz

Only solution for EoRHI Science Chapter

SPDO

Dense Aperture Array Station

10 of 35

Assumed Dense Aperture Array~256 tiles x 256 elements per tile2 polarisations per elementSample rate ~ 2.5 G Hz4 bits/ sample56 m diameter250 stations Tsys now 120KTarget 35K Memo 100

Processing Bunker

Dense AA Detail300MHz to 1GHz i.e. 700MHz bandwidthfs=2 x 700 MHz56m diameter array =>2463 m2

44.4 2pol elements per m^2 (30 cm wavelength) = 110,000 elements x 2pol per station, about 2 x 64k elements.Array efficiency 80%, Bore efficiency 75%& Tsys = 35K -> 250 stations for 10,000m2K-

1 sensitivity

SPDO

Simplistic View of Dense Aperture Array Processing

• Consider Frequency Domain Beamforming– Delay is implemented as a phase slope in frequency

domain.– Alternative time domain with tuned lengths of co-ax.

• Channelisation: Nchan 1024,– 12 taps gives < 60db aliasing between channels– Ntaps = 12, Nant = 64k x 2pol & fs = 1.4 GHz for 700MHz

bandwidth– Processing ~ (Ntaps + 3*log2(Nchan)) x Nant x 2pol x fs**=8 x 1015

MACS• Beamformer (per station):

– Average 1437 beams per channel to cover 250 sq degrees FoV

– Processing Load = Nant.2pol.Nbms.fs. 4MACS = 1 x 1018 MACS

• O/Prate**= Nbms.2 pol.fs.4bits.8B10B = 20 T bit/s– Over 2000 10 G bit/s optical cables, 8B/10B Encoding factor

1.25• Up to 250 Dense AA Stations (total 10 peta bits/s)Note :*FFT implementation dependent **Ignores upsampling of channelizer

11 of 35

SPDO

Correlator Processing Loads• Channelisation (700MHz bandwidth):

– SPF 4 x 1014 (105 channels)– PAF 4 x 1012 (4096 fine channels giving ~ 105 total)– Dense AA 6 x 1015 (128 fine channels ~ 105 total)

• Correlation load (700 MHz bandwidth)– SPF load = 6 x 1016 MACS– PAF load = 1 x 1018 MACS– Dense AA = 2 x 1018 MACS

• Correlator Dump Rate (Dish Solution)– 2280 15 m dishes + 40 x 18 dish stations and 3000 km

baseline:– Integration time ~ 200ms for < 1% smearing5 x 106/2 baselines x 105 channels x 4 bytes x 5 Hz = 9 T

Bytes/sNot calculated for other configurations yet

12 of 35

SPDO

Signal Processing Overview Memo 100 Option a*

13* Sparse AAs + 3000 15-m dishes with SPFs

Sparse AAChannelizer Full Stokes Correlator

Pulsar & Transient Processing

Timing

xx,xy,yx,yy

Multipier

Opto-Electrical

Conversion

Optical Rx

Timing

Monitoring & Control

Visibilities

PulsarCandidate

Data

1680Dishes

SparseAA

Local Monitoring

IntegratorDelays

PulsarSearch

& TimingProcessing

Opto-Electrical

Conversion

Opto-Electrical

Conversion

Poly-PhaseFilter

Candidates

Opto-Electrical

Conversion

70 MHz 500 MHz

Full Stokes Correlatorxx,xy,yx,yy

MultipierVisibilities

Integrator

500 MHz 10 GHz

Opto-Electrical

Conversion

Opto-Electrical

Conversion

Poly-PhaseFilter

DishChannelizer

Opto-Electrical

Conversion

Poly-PhaseFilter

Beamformer (core)

BeamFormer

Opto-Electrical

Conversion

Optical Rx

SPF Dish600 CoreDishes

SPF Dish40

Stations

Delays

TBA10 G bit/s

OpticalConnections

7424010 G bit/s

OpticalConnections

TBA10 G bit/s

OpticalConnections

18 Dishes per

station

SPDO

Signal Processing OverviewMemo 100 option b*

14 of 35* Sparse AAs + 2400 15-m dishes with PAFS & WBSPF

Sparse AAChannelizer Full Stokes Correlator

Pulsar & Transient Processing

Timing

xx,xy,yx,yy

Multipier

Opto-Electrical

Conversion

Optical Rx

Timing

Monitoring & Control

Visibilities

PulsarCandidate

Data

1080PAF

SparseAA

Local Monitoring

IntegratorDelays

PulsarSearch

& TimingProcessing

Opto-Electrical

Conversion

Opto-Electrical

Conversion

Poly-PhaseFilter

Candidates

Opto-Electrical

Conversion

70 MHz 500 MHz

Full Stokes Correlatorxx,xy,yx,yy

MultipierVisibilities

Integrator

500 MHz 10 GHz

Opto-Electrical

Conversion

Opto-Electrical

Conversion

PAF/WBSPFChannelizer

Opto-Electrical

Conversion

Optical Rx

SPF Dish600 Core

PAF

SPF Dish40

Stations

Delays

TBA10 G bit/s

OpticalConnections

144,48010 G bit/s

OpticalConnections

TBA10 G bit/s

OpticalConnections

Poly-PhaseFilter

18 PAFS per

station Possibly include a beam former for the core

SPDO

Signal Processing OverviewMemo 100 option c*

15 of 35* Sparse AAs + 250 Dense AA + 2000 15-m dishes with SPFs

Sparse AAChannelizer Full Stokes Correlator

Pulsar & Transient Processing

Timing

xx,xy,yx,yy

Multipier

Opto-Electrical

Conversion

Optical Rx

Timing

Monitoring & Control

Visibilities

PulsarCandidate

Data

SparseAA

Local Monitoring

IntegratorDelays

PulsarSearch

& TimingProcessing

Opto-Electrical

Conversion

Opto-Electrical

Conversion

Candidates

Opto-Electrical

Conversion

70 MHz 500 MHz

Full Stokes Correlatorxx,xy,yx,yy

MultipierVisibilities

Integrator

300 MHz 1 GHz

Opto-Electrical

Conversion

Opto-Electrical

Conversion

Dense AAChannelizer

Opto-Electrical

Conversion

Optical Rx

SPF Dish Delays

TBA10 G bit/s

OpticalConnections

1,000,00010 G bit/s

OpticalConnections

TBA10 G bit/s

OpticalConnections

250Dense AA

1280Dishes

Opto-Electrical

Conversion

Optical Rx Full Stokes Correlatorxx,xy,yx,yy

MultipierVisibilities

Integrator

1 GHz 10 GHz

Opto-Electrical

Conversion

4224010 G bit/s

OpticalConnections

40Stations

Delays

Opto-Electrical

Conversion

Poly-PhaseFilter

Dish WBSPFChannelizer

Poly-PhaseFilter

Poly-PhaseFilter

SPDO

Technology Options• FPGA

– Virtex 6 (available 2010): 2016 x DSP slices clocked at 600 MHz -> 1200 G

MACS

~ 25 G MACs per Watt

1018 MACS requires ~ 106 FPGAS

=> 48 W per device and ~ 48 M Watts for 1018 MACS

Operating cost 1$ per Watt per year => $48M per annum

Plus cost of cooling and delivering power• ASIC

– 22nm (available 2010): 2.5 nW/MHz/Gate > 40 T MACS (4 bit) per device => 25,000

devices Assuming < 50 % gates switching at any one

time: 600kWOperating cost $600k per annum

16 of 35

SPDO

What would F or X unit look like?

17 of 35Baseline Board (front)

Station Board Baseline Board (rear)

Pictures courtesy Brent Carlson

EVLA style boards might be an optiton ?64 ASICS or FPGAs on board (~1.5 kW card)~ 190 boards for Dense AA ASIC correlator14 cards per shelf -> 14 shelvesIs production yield an issue? Could use smaller 8 processor chip board As per ASKAP or UniboardInter-board Communication links increase

SPDO

Multichip Module

18 of 35

• SKADS have developed a promising Multichip Module:4 x 4 antenna array currently,Current RFI Protection shows -57dB per M (in air)

Picture courtesy of Kris Zarb Adami

Could be developed and used in several areas of the SKA(Note that the key components are ADC and Optical I/O, although the others could be useful in some applications.)

SPDO

Correlator Centre Build Cost

19 of 35

0 20000 40000 60000 80000 100000 1200000

10000000

20000000

30000000

40000000

50000000

60000000

70000000

80000000

90000000

100000000

Data Centre Build Cost

50W/sq ft

100W/ sq ft

400W sq ft

Total sq feet of data centre

Bu

ild C

os

t $

ASIC Solution

Kevin Wohlever 2006

ASIC solution• 1 cabinet per 30 sq

ft• Between 80 & 160

cabs?• Factor of 4 for air

con &PSU units, offices

• ~ 50 W per sq ft

SPDO

SPDO Team

Project Director Richard Schilizzi

Project Engineer Peter Dewdney

Executive Officer Colin Greenwood

Project Scientist Joe Lazio

System Engineer Kobus Cloete

Domain Specialist Receptors Neil Roddis

Domain Specialist Signal Transport Roshene McCool

Domain Specialist Computing & Software Duncan Hall

Domain Specialist Signal Processing Wallace Turner

Site Engineer Rob Millenaar

Project Management Officer Billy Adams

Industry Relations Manager Phil Crosby

Office Manager Lisa Bell

20