msc thesis - tu delftce-publications.et.tudelft.nl/publications/332_the... · 2012-07-19 · msc...

Computer EngineeringMekelweg 4,

2628 CD DelftThe Netherlands

http://ce.et.tudelft.nl/

2008

MSc THESIS

The Brandaris128 Camera

Frederic Lemmel

Abstract

Faculty of Electrical Engineering, Mathematics and Computer Science

CE-MS-2008-05

The use of ultrasonic imaging of the cardiovascular structures be-comes more and more important in the biomedical field. The im-agery is improved with the help of an ultrasound contrast agent thatis composed of microbubbles. In order to visualize the anatomy moreaccurately, it is necessary to study the behaviour of these bubblesduring ultrasound excitations. For this purpose, the Brandaris128high-speed camera was developed to be used in this field of research.The Brandaris128 is able to capture and store 6 experiments com-posed of 128 frames at the speed of 25 Mfps, and thus can observethe microbubbles through a microscope. The camera can reach thishigh-speed by use of a turbine gas, that rotates a mirror sweepingthe bubble’s over 128 aligned photo sensors. The setup is composedof 32 electronics cards which each contains four CCDs, the necessarycomponents to process the images and to communicate with an ex-ternal PC that has an embedded memory and a FPSLIC from Atmel.The FPSLIC is a System-on-Chip that controls and commands thecards from inside. The main limitation of the setup is that only 768frames (6 exp. x 128 frames) can be stored in the camera, because ofits limited amount of memory. When the memory is full, its content

is transfered to an external PC via USB2 connections. This process takes too much time (6 s) in comparisonwith the acquisition time (half a second). Furthermore, the camera can run during a limited amount oftime (30 s), because of the heat generated by the rotating mirror (it turns at 20000 rps or 1.2 Mrpm !) andtherefore needs to be cool down during several minutes.As a solution to store more images in the Camera, we have studied and used the flexibility provided bythe FPSLICs, by implementing a Region of Interest. Only the relevant part of the frames is than stored inthe memory, and thus the camera can contain more images. We have also made some proposals for futureimprovements, that can be implemented in the current Brandaris128 camera or in a complete new design.


THESIS

submitted in partial fulfillment of therequirements for the degree of

MASTER OF SCIENCE

in

COMPUTER ENGINEERING

by

Frederic Lemmelborn in Strasbourg, France

Computer EngineeringDepartment of Electrical EngineeringFaculty of Electrical Engineering, Mathematics and Computer ScienceDelft University of Technology


by Frederic Lemmel

Abstract

The use of ultrasonic imaging of the cardiovascular structures becomes more and more im-portant in the biomedical field. The imagery is improved with the help of an ultrasoundcontrast agent that is composed of microbubbles. In order to visualize the anatomy more

accurately, it is necessary to study the behaviour of these bubbles during ultrasound excitations.For this purpose, the Brandaris128 high-speed camera was developed to be used in this field ofresearch. The Brandaris128 is able to capture and store 6 experiments composed of 128 framesat the speed of 25 Mfps, and thus can observe the microbubbles through a microscope. Thecamera can reach this high-speed by use of a turbine gas, that rotates a mirror sweeping thebubble’s over 128 aligned photo sensors. The setup is composed of 32 electronics cards whicheach contains four CCDs, the necessary components to process the images and to communicatewith an external PC that has an embedded memory and a FPSLIC from Atmel. The FPSLIC isa System-on-Chip that controls and commands the cards from inside. The main limitation of thesetup is that only 768 frames (6 exp. x 128 frames) can be stored in the camera, because of itslimited amount of memory. When the memory is full, its content is transfered to an external PCvia USB2 connections. This process takes too much time (6 s) in comparison with the acquisitiontime (half a second). Furthermore, the camera can run during a limited amount of time (30 s),because of the heat generated by the rotating mirror (it turns at 20000 rps or 1.2 Mrpm !) andtherefore needs to be cool down during several minutes.

As a solution to store more images in the Camera, we have studied and used the flexibilityprovided by the FPSLICs, by implementing a Region of Interest. Only the relevant part ofthe frames is than stored in the memory, and thus the camera can contain more images. Wehave also made some proposals for future improvements, that can be implemented in the currentBrandaris128 camera or in a complete new design.

Laboratory : Computer EngineeringCodenumber : CE-MS-2008-05

Committee Members :

Advisor: Prof. Georgi N. Gaydadjiev, CE, TU Delft

Advisor: Prof. Nico de Jong, ThoraxCenter, Erasmus MC

Chairperson: Prof. Kees Goossens, CE, TU Delft

Member: Prof. Said Hamdioui, CE, TU Delft

Member: Rik Vos, ThoraxCenter, Erasmus MC

i

Member: Frits Mastik, ThoraxCenter, Erasmus MC

Member: Dr. Fernando Kuipers, NAS, TU Delft

ii

For Stephanie and my family.

iii

Contents

List of Figures x

List of Tables xi

Acknowledgements xiii

Abbreviations xv

1 Introduction 11.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Objectives and Contributions . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 The Brandaris128 System 52.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 Optics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.3.1 Lenses and rotating mirror . . . . . . . . . . . . . . . . . . . . . . 92.3.2 CCD Mounting block . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.4 Illumination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.5 Flow control system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.6 Timing Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.6.1 Start, Flush, Transfer signals . . . . . . . . . . . . . . . . . . . . . 132.6.2 Triggers signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.7 CCD Controller Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.7.2 CCDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.7.3 DAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.7.4 ADCs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.7.5 SRAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.7.6 USB2 Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.7.7 FPSLIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.7.8 FPGA Configuration EEPROM . . . . . . . . . . . . . . . . . . . . 22

2.8 External Computer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.9 USB Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.10 JTAG Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.10.2 Boundary Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.10.3 Test Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.10.4 TAP Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

v

2.10.5 Boundary Scan Register . . . . . . . . . . . . . . . . . . . . . . . . 252.10.6 Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.10.7 EEPROM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3 Basic Operations of the System 293.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.2 Text Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.3 Acquisition process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.3.1 Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.3.2 Readout phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.4 Memory Transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4 The FPSLIC Device 354.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 FPGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.2.1 Input/Ouput Pins . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.2.2 Core Logic and FreeRam Cells . . . . . . . . . . . . . . . . . . . . 364.2.3 FPGA - Cache Logic . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.3 AVR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.3.1 Program SRAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.3.2 Two-wire serial Interface (I2C) . . . . . . . . . . . . . . . . . . . . 374.3.3 UART . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.3.4 Timers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.3.5 Bidirectional IO ports . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.4 FPGA - AVR Bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.4.1 IOSEL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.4.2 Interrupts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.5 Dual Port SRAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404.6 AVR - Cache Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5 Design Software 415.1 FPGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.1.1 RTL Design and Simulation . . . . . . . . . . . . . . . . . . . . . . 415.1.2 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415.1.3 Place & Route . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415.1.4 Post-Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425.1.5 Bitstream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.2 AVR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.3 FPSLIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.4 External PC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

6 Improvement - Region Of Interest 456.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.2 Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

6.2.1 Pixel’s coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.2.2 Roi’s coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

vi

6.3 The ROI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496.3.1 Choice of a ROI . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496.3.2 Pre-defined ROI . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6.4 Simulation Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

6.6.1 Hardware testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546.6.2 Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546.6.3 Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

6.7 Consequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566.7.1 In the FPGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566.7.2 In the AVR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566.7.3 In the external PC . . . . . . . . . . . . . . . . . . . . . . . . . . . 566.7.4 In the timing controller . . . . . . . . . . . . . . . . . . . . . . . . 56

7 Conclusion 577.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577.2 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577.3 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

7.3.1 Software - Improvements . . . . . . . . . . . . . . . . . . . . . . . 587.3.2 Hardware - Improvements . . . . . . . . . . . . . . . . . . . . . . . 59

Bibliography 62

A VHDL Appendix 63A.1 Original HORCOUNTER : horcounter.vhd . . . . . . . . . . . . . . . . . . . . 63A.2 Modified HORCOUNTER : horcounter.vhd . . . . . . . . . . . . . . . . . . . . 66A.3 CCD ADC : ccd adc.vhd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74A.4 BS62 BLOCKS : BS62 blocks.vhd and BS62 1block.vhd . . . . . . . . . . . 79A.5 net2270 : net2270.vhd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

A CCD Appendix 85A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85A.2 CCD Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85A.3 CMOS Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87A.4 Comparison CCD & CMOS . . . . . . . . . . . . . . . . . . . . . . . . . . 87

vii

List of Figures

1.1 A Bubble recorded in the Brandaris128. . . . . . . . . . . . . . . . . . . . 3

2.1 A sequence of 128 frames from the Brandaris Camera with three bubbles 52.2 The Brandaris128 with its optical microscope . . . . . . . . . . . . . . . . 62.3 The Horse in Motion, Muybridge, 1878 . . . . . . . . . . . . . . . . . . . . 72.4 Block diagram of the subsystems in the Brandaris128. . . . . . . . . . . . 72.5 The Brandaris128 high-speed camera system. . . . . . . . . . . . . . . . . 82.6 Optical schematics of the Brandaris Camera . . . . . . . . . . . . . . . . . 92.7 CCD Mounting Blocks, twelve CCDs are mounted into four Mounting

Blocks. CCD #2, #4, #7, #10 are placed directly in the light of theexperience. The other CCDs receive the light from a ‘surface mirror’. . . . 10

2.8 A 6.2 Mfps recording of a single event in the xenon flash tube. The framenumber is indicated on the top left corner of each frame. It is manifestedthat the enlightenment by the xenon flash is not constant during the timeof a experiment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.9 Schematic Diagram of the Flow Control System . . . . . . . . . . . . . . . 122.10 Turbine response concerning an experiment using helium gas. The tur-

bine requires 10s to be stable. At the time indicated by the arrow, sixrecordings were captured. . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.11 The interactions between the ‘Timing controller’, the external PC and theBrandaris128. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.12 Timing diagram of an experiment with 128 frames. . . . . . . . . . . . . . 142.13 Timing diagram of multiple experiments with the triggers . . . . . . . . . 142.14 A C3 block diagram with a FPSLIC, a RS232 and USB2 controller and

four CCDs, ADCs and SRAM entities. . . . . . . . . . . . . . . . . . . . . 162.15 CCD - Block diagram and Pin configuration . . . . . . . . . . . . . . . . . 162.16 CCD - Optical Black Position. With 7 ‘black sensors’ in front and 30 in

rear in the horizontal direction and with 14 ‘black sensors’ in front and 1in rear in the vertical direction. . . . . . . . . . . . . . . . . . . . . . . . . 17

2.17 ADC, Functional Block Diagram. . . . . . . . . . . . . . . . . . . . . . . . 182.18 SRAM, Functional Block Diagram . . . . . . . . . . . . . . . . . . . . . . 202.19 USB2 controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.20 USB2 architecture with two levels of USB2 hubs . . . . . . . . . . . . . . 232.21 The ‘System Logic’ (the circuit to test) and ‘Test Logic’ (the circuit that

applies the tests) from the chip-level. . . . . . . . . . . . . . . . . . . . . . 242.22 The ‘Test Logic’ with the TAP controller, Instruction and Data Registers. 252.23 TAP controller FSM with the DR (Data Register) and IR (Instruction

Register) states. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.24 The Boundary Scan Register . . . . . . . . . . . . . . . . . . . . . . . . . 262.25 Inside a B/S Cell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

ix

3.1 Signals generated by HORCOUNTER. The ADC reads two times the samesignal at the rising edge of SHP and SHD, the signal is converted at therising edge of DATACLK. The SRAM blocks can write data when WE isset at ‘1’. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.1 FPSLIC - from inside . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 FPSLIC - Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.3 Logic Cell Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.4 The ‘busing network’ with 2304 ‘Core Cells’. . . . . . . . . . . . . . . . . 384.5 Inside a Core Logic Cell . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.1 The different programming phases. . . . . . . . . . . . . . . . . . . . . . . 425.2 Harware/Software Co-Verification . . . . . . . . . . . . . . . . . . . . . . . 44

6.1 CCDs positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466.2 Execution of VERUNIT. Shift a complete row of photosites into the ‘hori-

zontal register’. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466.3 Execution of HORCOUNTER. Shift a complete row of pixels from the ‘hori-

zontal register’ to the ADC. At every four clock cycles, the X-coordinateof the pixel is incremented. . . . . . . . . . . . . . . . . . . . . . . . . . . 47

6.4 ROI vertically flipped inside four CCDs. . . . . . . . . . . . . . . . . . . . 486.5 ROI with vertical symmetry . . . . . . . . . . . . . . . . . . . . . . . . . . 496.6 The different chosen ROI’s configuration, ‘F’, Full Frame: 6 exp. could be

stored in the SRAM, ‘H0’ to ’H3’, Half Frame: 12 exp. could be stored,‘Q1’ to ‘Q3’, Quarter Frame: 24 exp. could be stored. . . . . . . . . . . . 50

6.7 A Region of Interest with its two determinant point, ROI Begin (withthe coordinates ROI Begin x and ROI Begin y) and ROI End (with thecoordinates ROI End x and ROI End y). . . . . . . . . . . . . . . . . . . 52

6.8 X and Y coordinates determined in HORCOUNTER . . . . . . . . . . . . . . . 536.9 An image captured without a ROI. . . . . . . . . . . . . . . . . . . . . . . 546.10 Selection of a ROI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546.11 An image captured with a ROI. . . . . . . . . . . . . . . . . . . . . . . . . 55

7.1 A daisy chain that connects the JTAG connectors to the 32 C3. . . . . . . 59

A.1 Diagram showing 768(H) x 490(V) element interline CCD image sensorand an equivalent unit cell circuit. . . . . . . . . . . . . . . . . . . . . . . 85

A.2 CCD - Pixels transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86A.3 A CMOS sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

x

List of Tables

2.1 Comparison of the main characteristics of the Brandaris128 to two com-mercially available camera types. . . . . . . . . . . . . . . . . . . . . . . . 6

3.1 Example of some console commands . . . . . . . . . . . . . . . . . . . . . 30

6.1 Resources utilization before the ROI’s implementation . . . . . . . . . . . 556.2 Resource utilization after ROI’s implementation . . . . . . . . . . . . . . . 55

xi

Acknowledgements

I would like to thank my advisor Georgi Gaydadjiev, for his help and advices during mythesis.

I also want to thank Nico de Jong, Rik Vos and Frits Mastik from Erasmus MC fortheir help and encouragements.

Finally, I would to acknowledge all the people of the ‘Biomedical Engineering’ fortheir warm welcome and the good time that I have past with them.

Frederic LemmelDelft, The NetherlandsAugust 8, 2008

xiii

Abbreviations

ADC Analog to Digital Converter

B/S Cell Boundary Scan Cell

C3 CCD Controller Card

CCD Charge-Coupled Device

CDS Correlated Double Sampler

CMOS Complementary Metal-Oxide Semiconductor

DAC Digital to Analog Converter

DMA Direct Memory Access

EDIF Electronic Design Interchange Format

EEPROM Electrically-Erasable Programmable Read-Only Memory

FPGA Field-Programmable Gate Array

FPS Frames Per Second

FPSLIC Field Programmable System Level Integrated Circuit

FSM Finite State Machine

I2C Inter-Integrated Circuit Bus

IC Integrated Circuit

ICE In-Circuit Emulator

IOSEL Input/Output Selection Line

JTAG Joint Test Action Group

LSB Least Significant bit

MFC Mass-Flow Controller

xv

PCB Printed Circuit Board

ROI Region Of Interest

RPS Rotation-Per-Second

RISC Reduced Instruction Set Computer

RTL Register Transfer Level

SoC System-On-Chip

SRAM Static Random Access Memory

TAP Test Access Port

UART Universal Asynchronous Receiver Transmitter

VHDL Very High speed integrated circuit Hardware Description Language

xvi

Introduction 1“Any sufficiently advanced technology

is indistinguishable from magic.”- Arthur Charles Clarke

Nowadays, the use of ultrasonic imaging of the cardiovascular structures becomesmore and more important in the biomedical field. It provides a quick image of thecardiovascular system and more and more accurate representation of the anatomy (e.g.cancer, ...). The use of ultrasound is generally less expensive and safer than the otherimaging techniques (such as X-Ray and MRI).

In order to visualize the small arterial and venule networks, an ultrasound contrastagent[1] is injected in the blood stream. This agent is composed of microbubbles whichreflect the sound waves and improve distinction of the details in the imagery.

The ‘Biomedical Engineering’ department in the Erasmus MC in Rotterdam workson methods and technics to optimize ultrasonic imaging. For that, they need to increasetheir knowledge about the specific bubble behavior.

Two methods are used to study the microbubbles: an acoustical or an optical ap-proach. The acoustic approach consists to capture and analyze the acoustic response ofa bubble submitted to an ultrasonic field without looking to its appearance. The opticalapproach consists to see how the bubble behaves during ultrasound excitation.

The optical technique needs a camera able to record the dynamic of a bubble sub-mitted to ultrasound at the clinical relevant frequencies (between 1 and 10 MHz). Fromthe ‘thumb rule’ of the Nyquist-Shannon theorem, it is convenient to measure five timesthe studied bubble during its vibrating period. At the frequency of 5 MHz, the bestsampling frequency would be 25 MHz.

To achieve this goal a very high-speed camera, the ‘Brandaris128’[5], was developedby the ‘Biomedical Engineering’[6] department in collaboration with the ‘Physics ofFluid’ department at the University of Twente. This camera is able up to record 25 Mfps.

The device reaches this speed by using a rotating mirror concept; a gas turbinerotates a mirror that sweeps the bubble’s image over 128 aligned photo sensors.

1.1 Problem Statement

In order to study the bubbles’ dynamic with the Brandaris128, it is necessary to recordand study the response of a bubble that is excited by multiple ultrasound pulses. Themore of pulses that we can applied and the more of image that we can have, the betterit is.

The Brandaris128 achieves these objectives but still has some electronic and mechan-ical constraints:

1

2 CHAPTER 1. INTRODUCTION

• The camera is able to store in its embedded memory a maximum of 768 framescomposed of only 6 experiments with 128 consecutive frames. This restriction isinherent to the limited size of the memory in the camera. This memory is directlyintegrated in the Brandaris128 and cannot be extended. When the memory is full,its content is transfered to an external computer which will process the images. Ittakes around 6 s to copy the data from the memory to the PC.

• Compressed gas is used to drive the turbine that rotates a mirror (see section 2.1).Before to be operational, the camera takes a while because of the gas inertia (seesection 2.5). When the turbine turns, the setup is stable during a short time(between 30 s at 25 Mfps and several minutes at 6 Mfps) due to the heat up of theturbine. After this time, the turbine needs to be stopped during several minutesin order to cool down. These particularities of the camera cannot be changed.

The acquisition of 128 images in the camera takes around 40 ms and then to capture6 experiments it takes around half a second (480 ms). The time to transfer the memory(around 6 s) and to cool down the turbine (several minutes) is tremendously longer thanthe acquisition time and consequently the number of consecutive captured experimentis reduced.

The limitations of 6 experiments is partially solved by the ‘segmented mode’. Itconsists to record a video by reading subsets of segments of sensors and thus the usercan store more experiment with less frames (e.g. 12 experiments with 64 frames). Thissolution does not really solve the problem because of the diminution of recording frames.

1.2 Objectives and Contributions

Since 2003, the Brandaris Camera is involved in many different experiments by the‘Biomedical Engineering’ and the ‘Physics of Fluids’ departments. As a research tool,the camera needs to constantly evolve to fit the need of its users.

An external company, ‘AED Electronics’ was in charge to integrate the electroniccomponents chosen by the universities of Rotterdam and Twente. AED has also selectedand implemented the FPSLIC technology from Atmel Corp in the electronic boards ofthe Brandaris128. Several FPSLICs are integrated in the apparatus and they are theheart of the system.

The FPSLIC is a System-on-Chip composed of a FPGA, a microcontroller both fromAtmel and other elements (see section 4). Unfortunately, the functionalities and thepossibilities of the FPSLIC are not well documented for the ‘Thoraxcenter’ and the‘Physics of Fluids’ departments. The departments do not have the knowledge to fullyuse the capability of the camera.

The objectives of this thesis are :

1. To study and to understand how the Brandaris Camera works from the inside andhow the elements of the setup interact together. We have specifically focused ourinterest on the FPSLIC technology and the electronic components that are involvedwith it.

1.3. THESIS ORGANIZATION 3

2. To make proposals to exploit the potential of the camera by modifying the softwarepart of the setup (via the FPSLIC for example). Some ideas are also proposed inthe hardware part that can be used for a new Brandaris camera.

We have made some contributions about the Brandaris Camera, they are as follows:

• Implementation of a Region of Interest and estimation of the computational costto implement it. The benefits of defining a ROI (fig. 1.1) around a bubble is thatonly the interesting part of the images is stored in the memory and thus more than6 experiments can be stored in the camera before having to transfer its contentsto the external computer.

(a) Without a ROI (b) With a ROI

Figure 1.1: A Bubble recorded in the Brandaris128.

• Study of how the JTAG interface that is integrated in the electronic boards of theCamera is used to push a new firmware into a FPSLIC. We propose an idea inorder to simplify the transfer of a firmware into several FPSLICs. Now, a user thatwants to update the firmware for several electronic boards needs to do it manually.This manual operation is time consuming (see section 2.10.7).

• Study of the implementation and utilization of the reconfigurable properties of theFPSLIC. The FPSLIC can read and write its own firmware and is able to modifyit independently of the JTAG interface. A comparison with the JTAG techniqueis also presented.

• Make proposals to improve the communication system (see section 2.9) between thecamera and the external computer.

• Study of the new functionalities that can be implemented in the software part of theBrandaris Camera and where and when they can be executed.

1.3 Thesis Organization

The organization of the Thesis is divided as follows:

• Chapter 2 : The complete Brandaris setup, its characteristics and the interactionsbetween its elements are described. The explanations are spread over several fields:optics, gas flow and electronics.

4 CHAPTER 1. INTRODUCTION

• Chapter 3 : The main operations that are performed in the Camera are explainedin this section.

• Chapter 4 : The FPSLIC system-on-chip is described in a more detailed way.This section tells us about the possibilities of this component.

• Chapter 5 : A description of the different software packages that are utilized toprogram the Brandaris setup is given.

• Chapter 6 : In this section, the implementation of the ROI is explained and itsconsequences on the actual setup.

• Chapter 7 : In the conclusion, some future directions are proposed to improvethe actual Brandaris128 by using the flexibility of its components or by building anew improved high-speed camera.

The Brandaris128 System 22.1 Background

The Brandaris128 is a high-speed camera that is able to capture videos at the maximumspeed of 25 Mfps. The camera can record a sequence of 128 consecutive black&whiteframes at the resolution of 500x292 pixels (fig. 2.1). A maximum of 6 experiments of 128full-frames can be stored in the device before to be transferred to an external PC.

Figure 2.1: A sequence of 128 frames from the Brandaris Camera with three bubbles

The Brandaris128 was designed upon the frame of the Cordin 119[3] camera fromCordin Corp and has a size of 150 cm(w) x 150 cm(h) x 20 cm(d) and a weight of 140 kg(see fig. 2.2). The name of apparatus was chosen in tribute of the ‘Brandaris Lighthouse’[20] which sweeps its light around, on the island of Terschelling, the Netherlands.

Some cameras that are available in the market (see table 2.1) can acquire a videoat a higher speed than the Brandaris128 (e.g. 200 fps) but they record less frames.Other cameras are slower than the Brandaris128 and they cannot sample correctly thebehaviour of the bubbles submitted at the ultrasound frequency of 5 MHz. And a numberof cameras are not digital and thus require more time to produce a movie.

By its unique characteristics comparing to the high-speed cameras on the market,the Brandaris128 fits its principal objective, study the effects of the microbubbles in abiomedical environment.

In order to acquire 25 Mfps, the Brandaris128 camera uses an established cinemato-graphic principle invented in 1878 by Muybridge[16].

5

6 CHAPTER 2. THE BRANDARIS128 SYSTEM

Figure 2.2: The Brandaris128 with its optical microscope

Rotating mirror Electronic Brandaris128 ShimadzuHypervision HPV-1

Media Photographic Image intensifier CCD CCDFilm + CCC

Framing rate 25 200 25 1(Mfps)

No. Frames 130 8 128 100No. of runs 1 2 6Turnover (s) 1800 60 1Total number 130 16 768

of framesResolution 500x292 312x260

Time between runs 60 s 4 µs 20 msTime between N/A N/A 17 µssubsegments

Table 2.1: Comparison of the main characteristics of the Brandaris128 to two commer-cially available camera types.

Multiple independent cameras receive consecutively a short exposure of the images inline. Muybridge recorded the first movie of a galloping horse by employing 12 cameras(Fig. 2.3) and proved that a horse in full gallop loses contact with the ground. Thistechnique was also used to record the first nuclear explosion.

2.2. OVERVIEW 7

Figure 2.3: The Horse in Motion, Muybridge, 1878

2.2 Overview

The Brandaris128[2] is an apparatus that comprises different elements placed inside andoutside of the system. These components are involved in different physical processes(optical, electronic, gas, ...) and they interact together (fig. 2.4).

Figure 2.4: Block diagram of the subsystems in the Brandaris128.

This chapter describes and explains the different subsystems that composes the Bran-daris camera (fig. 2.5). The multiple elements are as follows :

• A microscope : The microscope is used to magnify the bubbles which have a typicaldiameter of 1-10µm. The support in which the bubbles are observed contains oneor more ultrasound transducers. They are used to generate a ultrasound beamthat is applied to the bubbles.


• A relay lens : The beam light arriving from the microscope is guided to a relaylens that focuses the illumination to the three-sides mirror.

• A three-sides mirror : The beam light of the experiment arrives to a rotatingthree-sides mirror which projects the illumination to a lens bank.

• A lens bank : The light beam from the mirror is projected to a bank of 128 lenses.

• 32 CCD cards : The 128 CCDs are connected to 32 CCD Controller cards or C3.These cards contain the electronics component that are involved in the capture,conversion, storage and transmission of the frames.

• A network of USB2 hubs : The 32 C3 are connected to an external PC via an USBnetwork composed of USB2 hubs.

• An external computer : This PC is the central point to manage the Brandariscamera. It sends its commands through the USB network in order to: initializeand configure the C3, start the acquisition of the experiments, transfer the memorycontent to the external PC, ... .

Figure 2.5: The Brandaris128 high-speed camera system.

2.3. OPTICS 9

2.3 Optics

2.3.1 Lenses and rotating mirror

The ‘Optic Path’ of the Brandaris Camera is the trajectory of an image captured by the‘external microscope’ that is driven by the ’relay lens’ to the ‘three-sides mirror’. At thismoment, the beam light is successively projected to the CCDs with the help of mirrorthough a ‘lens bank’ (fig. 2.6).

The mirror is connected to a gas turbine driven with high-pressurized gas (see sec-tion 2.5) and a photodiode laser counts the number of rotations of the three-sides mirror.

The ‘lens bank’ is used to project the light to the CCDs. Thus a steady image isobtained instead of a sweep picture because the mirror sweeps the sensors at high-speedover the lenses.

Figure 2.6: Optical schematics of the Brandaris Camera

A CCD sensor is embedded in the microscope. It is used to directly see the imageproduces by device on an external monitor. The CCD used is the same model of theCCDs inside the Brandaris128.

2.3.2 CCD Mounting block

Because of the support size of the CCDs, the placement of the 128 photo sensors in onestraight line at an arc circle of 73◦ is not possible. The solution to insert so many CCDsinto this confined space is to use the ‘CCD Mounting blocks’.

A CCD Mounting block contains three CCDs and twelve CCDs are mounted on fourMounting blocks (fig 2.7). On each block, one CCD is placed directly in the light of theimage from the three-sides mirror, the two other CCDs are mounted in such a way that


they receive the beam light reflected via a ‘surface mirror’. The images obtained fromthe ‘reflected’ CCDs are then vertically flipped.

The mirrored images are slightly less luminous than the direct images because thelight beam is reflected by the ‘surface mirror’ in the CCD mounter. This luminosityproblems is not correct.

Figure 2.7: CCD Mounting Blocks, twelve CCDs are mounted into four Mounting Blocks.CCD #2, #4, #7, #10 are placed directly in the light of the experience. The other CCDsreceive the light from a ‘surface mirror’.

The 128 CCDs are not perfectly placed in the mounting blocks. The consequence isthat the images acquired are not pixel aligned. The position of the CCDs do not changein time, it means that the misalignments are constant for each sensor.

When the CCDs are mounted, an automatic detection locates precisely the emplace-ment of the sensors. From this operation, three parameters that distinguish each CCDare calculated: the translation, rotation and zooming variables. The detection is donein the external PC (see section 2.8).

The flipped and misaligned frames are corrected during an automated post-processingstep in the external PC (see section 2.8).

2.4 Illumination

During the illumination phase, the CCDs are lightened during a very shot amount oftime, between 40 ns at the speed of 25 Mfps and 167 ns at 6 Mfps. Consequently, anextra lighting is necessary to light up the subject to study.

2.5. FLOW CONTROL SYSTEM 11

For this purpose, a Xenon flash lamp1 illuminates the bubble. However this flash lightis not constant during the time of one experiment and the 128 CCDs are not exposedwith the same flash intensity (fig. 2.8). This non-uniform flash lighting is not corrected.

Figure 2.8: A 6.2 Mfps recording of a single event in the xenon flash tube. The framenumber is indicated on the top left corner of each frame. It is manifested that theenlightenment by the xenon flash is not constant during the time of a experiment.

2.5 Flow control system

The flow of compressed gas used to rotate the three-side mirror is temporary stored in anexternal buffer and controlled by the Mass Flow Controller (fig. 2.9). Air gas can be usedto spin the mirror at the maximum speed of 5000 rps, it corresponds to an acquisitionspeed of 6 Mfps. Helium gas can spin the mirror up to the speed of 20000 rps2 in orderto sustain the acquisition speed of 25 Mfps. The Helium was chosen for its low frictionproperties during the very high-speed rotations

Before to start to turn the mirror, it is necessary to fill the Camera by injecting gasinside in order to limit the gas friction around the rotating mirror. This process takesfew seconds (around 10 s). After that, the gas flow starts to rotate slowly the mirror tillits speed is stabilized and functional (fig. 2.10). This process takes around 10 s.

Depending of the gas used, the turbine can turn during a limited amount of time,between 20 s with Helium gas (for a frame rate of 25 Mfps) and several minutes withAir gas (for a frame rate of 6 Mfps) because of the heating problem.

1EG&G FX-1163 flash lamp from Perkin-Elmer Optoelectronics, Salem, Massachusetts

2It corresponds to 1.2 million rotations per minute !


Figure 2.9: Schematic Diagram of the Flow Control System

Figure 2.10: Turbine response concerning an experiment using helium gas. The turbinerequires 10s to be stable. At the time indicated by the arrow, six recordings werecaptured.

2.6 Timing Controller

The ‘Timing controller’ (fig. 2.11) is another external PC that produces the signalsSTART, FLUSH, TRANSFER that are used to launch and control the acquisition ofimages in the C3. The controller triggers the flash illumination and the ultrasonic equip-ment for the experiment.

The ‘Timing controller’ uses the mirror ticks (measured with a photodiode laser) tosynchronize the generation of these triggers depending of the speed of the turbine.

2.6. TIMING CONTROLLER 13

USB

Brandaris

External PC

Timing Controller

START

Rundown turbine

FLUSH

ultrasoundtrigger

!ash illumination

TRANSFER

mirrorpulse

mirrorpulse

Figure 2.11: The interactions between the ‘Timing controller’, the external PC and theBrandaris128.

2.6.1 Start, Flush, Transfer signals

The signals START, FLUSH and TRANSFER are generated by the ‘Timing Controller’and they are dispatched in parallel to the FPSLICs (see section 2.7.7) inside the 32 C3.

These signals correspond to different phases in the acquisition process (fig. 2.12).Their properties are as follow :

• START : This signal is sent when the mirror speed is stable and the setup is readyto acquire the experiments. When the 32 FPSLICs receive the START signal, theyare synchronized and are prepared to capture the experiences.

• FLUSH : The FLUSH signal arrives just before the CCDs are illuminated (seesection 2.4) and produces in the C3s the necessary signals to flush the currentcontent of their four CCDs. This operation takes 4.9 µs. After the ‘flushing’process, the CCDs can be illuminated and are prepared to receive the illumination(see section 2.7.2).

• TRANSFER : The TRANSFER signal arrives after the ‘Illumination’ phase andindicates in the C3 that the image captured by the CCDs is ready to be transfered.This ‘transfer’ phase takes 9.5 µs. After that, the images stored in the sensors areconverted and stored in the embedded memory of the Brandaris (see sections 2.7.2and 2.7.4). This phase is called ‘readout’ and takes 33.66 ms.


Figure 2.12: Timing diagram of an experiment with 128 frames.

It is necessary to be certain that the ‘flushing’, ‘illumination’ and ‘transfer/readout’phases must follow each other and not overlap.

2.6.2 Triggers signals

The ‘Timing Controller’ triggers the ultrasound equipment and the flash light. All thesetriggers are synchronized to be effective during the illumination of the CCDs. Thefigure 2.13 shows the different signals generated by the ‘Timing Controller’ for multipleexperiences.

Due to the speed of sound, the ultrasound takes few microseconds to go from thetransducer(s) to the bubble to study. For this reason, the trigger(s) for the ultrasoundneed to be generated before the beginning of the illumination. A preprogrammed wave-form generator produces the ultrasound signal.

Figure 2.13: Timing diagram of multiple experiments with the triggers

2.7 CCD Controller Card

2.7.1 Introduction

The Brandaris camera contains 32 C3 that are placed in a circle arc. These cards containsseveral components described in the next sections:

2.7. CCD CONTROLLER CARD 15

• One FPSLIC : The FPSLIC from Atmel Corp. is a System-on-Chip that containsan AVR microcontroller and an FPGA. This reconfigurable ‘processor’ is connecteddirectly to the external components (CCDs, ADCs, SRAMs, USB2 controller, ...)and controls them.

• Four CCDs : These four high-sensitives sensors (CCD A, ..., CCD D) are used tocapture images.

• One DAC : A reprogrammable Digital-Analog converter use to amplify the signalsinside the CCDs.

• Four ADCs : The ADCs (ADC A, ..., ADC D) convert the analog data from theCCDs.

• Eight SRAM blocks : These eight SRAM blocks (SRAM A1, SRAM A2, ... ,SRAM D1, SRAM D2) are grouped into four entities (SRAM A, ... , SRAM D) oftwo SRAM blocks. They are the embedded memory of the C3.

• One USB2 Controller : The controller is used for the input/output transmissionbetween the C3 and the external PC through the USB2 network.

• One RS232 Controller : For the transmission via the RS232 connector.

• One reprogrammable memory : This element contains an EEPROM to store thefirmware of the FPGA part of the FPSLIC and the electronics to configure thismemory via the AVR part of the FPSLIC.

The FPSLIC component controls the C3, this SoC receives and executes the com-mands arriving from the external via the USB2 connections.

The outputs of the four CCDs (CCD A, ..., CCD D) are directly linked to the inputsof the four ADCs (ADC A, ..., ADC D). The output of each ADC is connected to a SRAMentity or two SRAM blocks (e.g. ADC B is associated with SRAM B1 and SRAM B2).As a consequence of this pipelined architecture, the signals generated by the four CCDsrun in parallel to their corresponding ADC. Then these four signals are converted andsent to the four SRAM entities (fig. 2.14).

The FPSLIC is in charge to receive START, FLUSH and TRANSFER and generatesthe signals to synchronize the CCDs, ADCs and SRAMs blocks (see section 3.3).

2.7.2 CCDs

Principle

The 128 ‘ICX055AL’[4] CCDs are manufactured by Sony Corp. These black&whiteimage sensors have a high-sensitivity and an excellent anti-blooming properties. Becauseeach CCD is exposed during a very short time to the light of the experiments, the highsensitivity is a very important property.

A chip measures 6.00 mm x 4.96 mm with an optical size of 1/3-inch format and iscomposed of several elements (fig. 2.15) :


!ush/transfer

Gain/O"set

RS232Controller

USB 2.0Controller

start!ush

transfer

CCDA

ADCA

SRAM A

SRAM B

SRAM C

SRAM D

ADCB

ADCC

ADCD

CCDB

CCDC

CCDD

1

2

1

2

1

2

1

2

FPSLIC

Figure 2.14: A C3 block diagram with a FPSLIC, a RS232 and USB2 controller and fourCCDs, ADCs and SRAM entities.

12345678

9 10 11 12 13 14 15 16

Note

Note) : Photo sensor

VTU

O VSS

VG

G

DNG V

1

V2

V3

V4

VDD DN

G

BUS

V L GR

CN H1

H2

Horizontal register

retsiger lacitr eV

Figure 2.15: CCD - Block diagram and Pin configuration

• Multiple ‘photo sensors’ : These sensors capture the light and convert it in electricalsignal. A ‘photo sensor’ unit measures 9.8 µm x 6.3 µm.

• Several ‘vertical registers’ : These registers are connected to the small photo sensorsin the CCD in order to receive their signals. The signals stored in the verticalregisters are also shifted to the horizontal register.

• One ’horizontal register’ : It receives the analog signals from the ’vertical registers’and shifts its content to the output pin VOUT .

The CCD is covered by a black layer which reduces its size from 537 x 597 effectivephoto sensors to a total of 500 x 582 photo sensors (fig. 2.16).

These CCDs are originally designed for the video cameras and for this purpose theycan be used in interleaved mode. In the Brandaris128, this mode is not used, it meansthat two vertical photo sensors are merged to produce one vertical pixel. One horizontal


Pin 1

V

7

30

1

14

Pin 9H

537

500

597 582

Figure 2.16: CCD - Optical Black Position. With 7 ‘black sensors’ in front and 30 inrear in the horizontal direction and with 14 ‘black sensors’ in front and 1 in rear in thevertical direction.

photo sensors produces one horizontal pixel. Finally, an image acquired by these CCDswill have an arbitrary size of 540 x 300 effective pixels to a total of 500 x 292 pixels.

The covered pixels or ‘black pixels’ capture the so-called ‘black noise’. This noise is anunwanted interference in the signal captured by the photo sensors. It is a consequence ofthe electron’s agitation and is a function of the ambient temperature. The signal comingfrom the ‘black pixels’ is used to neutralize the ‘black noise’ via an offset operation inthe ADCs.

The Signals

The ‘vertical registers’ in a CCD are driven by the four ‘Vertical Register Transfer Clock’pins : V01, ... V04 (see fig 2.15).

These pins receive the ‘Vertical Clock Signals’ and execute some operations in the‘vertical registers’ and ‘photo sensors’. These signals can do the following operations :

• Flush the ‘photo sensors’. Their content are erased.

• Transfer the electrical data stored inside the ‘photo sensors’ to the ‘vertical regis-ters’.

• Shift the ‘vertical register’. An element stored in these registers is shifted to the‘horizontal register’.


The signals involved in the pins arrived at the same time in the ‘vertical registers’and thus all the operations are done in parallel.

The ‘horizontal register’ is driven by the ‘Horizontal Register Transfer Clock’ (H01

and H02) and the ‘Reset Gate Clock’ (RG) pins. These pins received also the ‘horizontalclock signals’ to shift its content of the ‘horizontal register’ to VOUT , its output pin.

2.7.3 DAC

A MAX521[19] DAC is included in each C3. This component is programmable via a2-wire serial interface (I2C) and generate an analog signal.

This signal is directed to the four CCDs in a C3 and is used to amplify the dataacquired in the photo sensors. This is a pre-amplification.

2.7.4 ADCs

Principle

The four analog analog-to-digital convertors, AD9843A[9] from Analog Devices, are con-nected between the CCDs and the SRAM blocks (see fig. 2.14). They convert an analogsignal into a 10 bits digital signal at the frequency of 20 MHz. Nevertheless only the 8less significant bits are stored in the memory. Then a pixel is represented with 256 graylevels.

DATACLKSHDSHP

BANDGAPREFERENCE

2:1MUX DOUT

AUX2IN

CLPDM

CCDIN

OFFSETDAC

PBLK

AUX1IN

VRTVRB

INTERNALTIMING

INTERNALBIAS

2dB~36dB

AVDD

DVDD

DVSS

AVSS

DRVDD

DRVSS

10

8CML

DIGITALINTERFACE

SDATASCKSL

CLPOB

10CDS

VGA

CLP

BUF2:1MUX

CLP

AD9843A

4dB 6dB

INTERNALREGISTERS

CLP

10-BITADC

6

Figure 2.17: ADC, Functional Block Diagram.

The functionalities of the ADC (fig. 2.17) are:

• Correlated Double Sampling : The input analog signal ( from the pin CCDIN) issampled two times by the CDS element: during its reference level and its data levelof the analog signal. The ADC subtracts these two values and uses the result inorder to reduce the signal noise.

• Gain Amplifier : The ADC can apply a gain in the processed signal. This is apost-amplification contrary to the optional DAC amplification (see section 2.7.3).


• Black Level Clamp : When a ‘black pixel’ is read in the ADC, the analog processorcompares the input signal to an internal reference ‘black level’. This is used tooffset the black noise in the non-‘black pixels’.

The Signals

In the C3, the FPSLIC is connected to the four ADCs and generates the signals necessaryto apply the following functionalities:

• The CDS : Two clock signals are required to read two times the CCD signal in theCDS mode: SHP and SHD and one clock signal is necessary to produce a digitalsignal: DATACLK.

• Black Clamping : The signal CLPOB indicates that the current input CCD signalis a black pixel or not.

• Amplification : The signals SL, SCK and SDATA are used to configure the gainfunction in the ADC.

2.7.5 SRAM

Principle

Each C3 card contains 2 x 4 ‘BS62LV4001’[12] CMOS SRAM components from BSI. ASRAM (fig. 2.18) component can store 512 KB. Each ADC is connected to two SRAMunits that composed a SRAM entity of 1 MB. Thus a maximum of 1 MB of data can bestored for each CCD.

The SRAM stores the images captured from the CCDs at their effective resolutionof 540x300 pixels and can store a maximum of 6 images.

The FPSLIC and the ADCs (see sections 2.7.7 and 2.7.4) run and generate the pixelsat the frequency of 20 MHz (clock period of 50 ns). Unfortunately, the SRAM blockshave a latency of 70 ns to read and write the data. Thus, the embedded memory needsat least two clock cycles at the frequency of 20 MHz to be read and written.

The SRAM blocks are the main bottleneck in the Brandaris camera. They reducethe speed during the reading and writing operations in the memory and they restrainthe number of images that can be stored in the C3.

Signals

In a C3, the eight memory blocks are connected to the FPSLIC that generates thefollowing signals to control the SRAM :

• Address : The signal address is coded in 19-bits (219 = 523288 Bytes = 512 KB) forone SRAM block. With two memory blocks, an address of 20-bits is used, the first512 KB are assigned for the first SRAM block and the other 512 KB are assignedfor the second SRAM block.

• Chip Enable Input : The signal CE activates or deactivates the SRAM.


• Write Enable Input : The signal WE enables or disables writing in the memory.

• Output Enable Input : The signal OE enables or disables reading in the memory.

• Bidirectional ports : Eight bidirectional signals (DQ0, ... DQ7 ) are used to readand write the content of the memory block. The eight output pins from an ADCthat convey the eight LSB bits are directly connected to the eight bidirectionaltheir port of corresponding SRAM blocks. The digitized signal from a CCD isdirectly stored in an SRAM block (Fig. 2.14).

Address

Input

Bu!er

Row

Decoder

Memory Array

2048 X 2048

Column I/O

Sense AmpWrite Driver

Column Decoder

Data

Bu!erOutput

Address Input Bu!er

Data

Bu!erInput

Control

GndVdd

OEWECE

DQ5DQ4

A13A17A15A18A16A14A12

A7A6A5A4

8

8

8

8

DQ7DQ6

DQ3DQ2DQ1DQ0

16

256

2048

204822

A11 A9 A8 A3 A2 A1 A0 A10

Figure 2.18: SRAM, Functional Block Diagram

2.7.6 USB2 Controller

Principle

The ‘Net2270’[13] USB2 controller from NetChip Technology Inc. is used to receive andsend the data between the FPSLIC in a C3 and the external PC through the USB network(see section 2.9) at the maximum speed of the USB2 interface which is 480 Mbps.

The controller contains a buffer to store the data that are emitted or received. TheFPGA is always informed if this buffer is full or not.

Signals

In a C3, the FPSLIC drives the USB2 (fig. 2.19) controller by sending these signals:


Page 4 of 8 (23-Apr-2002)

cordin128_L2.UTSCH - CPU

A A

B B

C C

D D

8

8

7

7

6

6

5

5

4

4

3

3

2

2

1

1

:FO:EGAPLAITINI

COMPANY:

COUNTRY:

16-7-2001

22-04-2002

CCD ERASMUS - INTERFACES

4 8

L2

AED ELECTRONICSBERTELINDISLAAN 3WAALRETHE NETHERLANDS

MVB/JL

CITY

PROJECT:

:GNE:ETAD:VER

ADDRESS:

48 VBUS

49 RREF

12 XOUT

11 XIN

3 SMC

64 TEST3

2 TEST2

1 TEST1

39 EOT

38 DACK

13 DMAWR

36 DMARD

45 IOW

44 IOR

46 CS

40 ALE

43 RESET

4 LA4

5 LA3

6 LA2

7 LA1

8 LA0

54VS

S160

VSS2

63VS

S319

VSSI

O1

24VS

SIO

229

VSSI

O3

9VS

SC1

41VS

SC2

61PV

SS50

AVSS

152

AVSS

2

55RSDP

56DP

58DM

59RSDM

53RPU

47IRQ

37DREQ

35LCKO

34LD15

33LD14

31LD13

30LD12

28LD11

27LD10

26LD9

25LD8

23LD7

22LD6

21LD5

20LD4

18LD3

16LD2

15LD1

14LD0

57VD

D17

VDD

IO1

32VD

DIO

210

VDD

C1

42VD

DC

262

PVD

D51

AVD

D

U35UT2270

21

X2 30MHZ

C21 33P

C2233P

GND

R15

9K09

GND

R16

47K

GND

GND

123456

J5

USB_CONNECTOR

R171K5R40

35E7

R4135E7

R42

1M

C26

10N

GND

L4

42 ohm

L5

42 ohmAT 100MHZ

VCC3

L61 uH

R431k

VCC

3

USB_IRQUSB_DREQ

USB_D0USB_D1USB_D2USB_D3USB_D4USB_D5USB_D6USB_D7

USB_A0

GND

GND

R4410k

VCC3

USB_DACK

USB_CSUSB_RDUSB_WR

USB_RESET

C23

100N

C24

100N

C25

100N

VCC3 VCC3 VCC3

GND GND GND

C95

100N

C112

100N

C130

100N

VCC3 VCC3 VCC3

GND GND GND

GND

C139

100N

C140

22u/16v

C141

22u/16v

C142

22u/16v

VCC3 VCC3

GND GND

A1A2A3A4A5A6A7A8A9

A10A11A12A13A14A15A16A17A18A19A20A21A22A23A24A25A26A27A28A29A30A31A32

J11.A

DIN_64AC

GNDGNDGNDGNDGND

-12V_IN-12V_IN

GNDGND

GNDGNDGNDGNDGND

STARTGND

FLUSHGND

TRANSFERGND

C1C2C3C4C5C6C7C8C9

C10C11C12C13C14C15C16C17C18C19C20C21C22C23C24C25C26C27C28C29C30C31C32

J11.B

DIN_64AC

GND

TRANSFER

GND

FLUSH

-12V_IN

START

-12V_IN

GND

GND

GND

GNDGNDGNDGND

GND

GND

GND

GND

GND

GND

3V3_IN

3V3_IN3V3_IN

3V3_IN3V3_IN

3V3_IN

+20V_IN+20V_IN

+20V_IN

+20V_IN

+20V_IN+20V_IN

+20V_IN+20V_IN

+20V_IN

+20V_IN

+20V_IN+20V_IN

3V3_IN

3V3_IN

3V3_IN3V3_IN

3V3_IN3V3_IN

C143

220p

L71 uH

R2

4K7

GND

3RD OVERTONE CIRCUIT

Figure 2.19: USB2 controller

• input/output indication : the signals IOR and IOW indicate whether the controllerneed to receive or send a packet of 8 bits to the USB network.

• data : The signals LD0 to LD15 are used to send or receive at packet of data fromthe USB2 network. In the current implementation of the Brandaris128, only thefirst 8 signals (LD0 to LD7 ) are used.

• acknowledgement : The signal DACK indicates whether the internal buffer of theUSB controller is full or not.

2.7.7 FPSLIC

An AT94k FPSLIC[11] from Atmel corp. is integrated in each C3. The FPSLIC is a SoCcomposed by : A 8-bit AVR microcontroller from Atmel, an AT40k FPGA from Atmel,a SRAM memory and different other components (see section 4).

The FPSLIC is able to execute different operations (see section 3) in order to controland to coordinate the electronic components in a C3.

The FPSLIC was originally programmed to work at 20 MHz. Unfortunately at thisspeed the electronic components in C3 are unstabled. This problem is not yet clearlysolved and for sake of simplicity, the FPSLIC runs now at the speed of 10 MHz.


2.7.8 FPGA Configuration EEPROM

Each C3 contains a AT17LV010-10DP ‘FPGA Configuration EEPROM’[10] from Atmel.This component contains an EEPROM to store the firmware of the FPSLIC and also ananother element that can reprogram this memory via the AVR part of the FPSLIC.

The reprogrammable functionality of the component is not yet used and implementedin the FPSLIC.

2.8 External Computer

An external computer centralizes the data from the experiments and controls the 32C3 cards through a USB architecture. Thus the PC can sends its commands to the 32FPSLICs.

The main functions of the computer are described as follow :

1. Configuration of the ADCs and DAC.

2. Start the Capture Process (See section 3.3).

3. Reception of the 6 x 128 frames from the C3 cards.

4. Detection of the position of the 128 CCDs (see section 2.3.2).

5. Correction of the unaligned and flipped images.

2.9 USB Structure

The 32 C3 cards of the Brandaris are connected to the external PC via a network ofhubs. Two levels of USB hubs are used to achieve the connections (Fig. 2.20).

The theoretical maximum data rate of the USB2 is 480 Mbits/s (or 60 MB/s). Be-cause of the limited speed of the SRAM, each C3 has a transfer rate of 5 MB/s. Anotherbottleneck is created in the USB structure and the effective transfer rate of the systemwas found to be 20 MB/s.

A possible solution to resolve the congestion due to the USB2 structure is to use anEthernet architecture (see section 7.3.2).

2.10 JTAG Interface

2.10.1 Introduction

The JTAG or ‘Boundary Scan Standard’ is an approach to test the pins of an electroniccircuit without using a probe. An external device can have access to these pins by usinga JTAG interface. Such an interface is integrated in the C3 and can have access to theFPSLIC.

In the Brandaris camera, the JTAG interface is used to connect an external PC via aspecific cable in order to debug the AVR by reading or writing directly the microcontroller

2.10. JTAG INTERFACE 23

. . .

. . .. . .. . .

. . .

32 C38 USB2 Hubs

4 USB2 Hubs

Computer

1

1

4

5

1

2

2

3

4

29

830

31

32

External PC

Figure 2.20: USB2 architecture with two levels of USB2 hubs

pins. The interface is also used to update the firmware contained in the EEPROM ofthe FPGA in the FPSLIC.

2.10.2 Boundary Scan

The ‘Boundary Scan Standard’[15] or the ANSI/IEEE Standard 1149.1 defines a methodto read, write and interact (via some simple instructions) with the pins of an inte-grated circuit. The IC or in this case the FPSLIC is embedded in a ‘System Test Logic’(fig. 2.21), containing a ‘test logic’. The ‘test logic’ contains the necessary circuitry totest the ‘System Logic’.

2.10.3 Test Logic

The ‘Test Logic’ (fig. 2.22) circuitry is composed of the following elements :

• An Instruction Register. It contains the instruction that the ‘Test Logic’ willexecute. This register is serially loaded and unloaded with a shift operation.

• A Data Register. This register contains the data necessary for the current instruc-tion. This register is also read and written via a shift operation.

• A TAP controller. The controller is in charge to manage the different elements viaseveral control and clock signals.


Figure 2.21: The ‘System Logic’ (the circuit to test) and ‘Test Logic’ (the circuit thatapplies the tests) from the chip-level.

This circuit has also access to four or five TAPs, these ports are provided by theJTAG interface. The description of the TAPs is as follow :

• TCK (Test clock input). It is a clock used to synchronize the test circuit.

• TMS (Test Mode Select Input). It selects the operation to be execute in the TAPcontroller.

• TDI (Test Data Input). This is the input that is sent serially.

• TDO (Test Data Output). It is the equivalent to TDI but for the output.

• TRST (Test Reset Input). It is an optional reset pin.

2.10.4 TAP Controller

The TAP controller controls the entire ‘Test Logic’ circuit. From the signals TMS, TCKand the optional TRST, the TAP controller generates the clock and controls signals tocommand the Intruction and Data Registers and defines when these registers are readand wrote via TDI and TDO (see fig. 2.22).

A FSM (fig. 2.23) drives the TAP controller. At each rising edge of TCK a new statecan be reached and each state can generate a clock or control signals.

These states transition are used to execute multiple functions, for example:


Figure 2.22: The ‘Test Logic’ with the TAP controller, Instruction and Data Registers.

• Test-Logic Reset: Reset the Instruction and Data Registers. By applying a ‘true’signal TMS during at least five rising edges of TCK, then this state is reached anda reset is applied.

• Run-Test/Idle: Execute the instruction in the Instruction Register with its DataRegister.

• Capture-DR/IR: Read a bit from the Data Register or Instruction Register.

• Shift-DR/IR: Shift a bit from the Data or Instruction Registers.

• Update-DR/IR: Insert a bit from TDI into the Data or Instruction Registers.

2.10.5 Boundary Scan Register

Into the ‘System Test logic’, the input and output pins of the ‘System Logic’ are con-nected to a ‘Boundary Scan Register’ (fig. 2.24). This register is composed by a daisy-chain of B/S cells. These cells are connected to the input and output ports of the systemlogic and they can inserted or intercepted the signals from these ports.

The B/S Cells (fig. 2.25) are driven by the signals generated in the TAP controller.Depending of the signals TCK, TDM and the clock signals ShiftDR and UpdateDR, aB/S cell has multiple functioning modes:

• Normal Mode: The B/S is transparent and does not interact with the systemlogic if the mode signal or TMS is equalled at 0,

• Scan Mode: The data contains in the scan path of the daisy chain are shifted ofone element to TDO at each TCK pulse if TMS and ShiftDR are equalled at 1.


Figure 2.23: TAP controller FSM with the DR (Data Register) and IR (InstructionRegister) states.

Figure 2.24: The Boundary Scan Register

• Capture Mode : The signals from the output pins of the System logic circuitare loaded in the scan path of the daisy chain if TMS is equalled at 1, ShiftDR isequalled At 0 and one clock pulse of ClockDR is applied.

• Update Mode : A signal previously stored in the scan path (via TDI ) is insertedin the input pins of the System logic if TMS is equalled at 1 and one pulse ofUpdateDR is applied.


Figure 2.25: Inside a B/S Cell

2.10.6 Instructions

The Test Logic Circuit executes a simple instruction stored in the ‘Instruction Register’when the current state of the TAP Controller (see fig. 2.23) stays at Run-Test/Idle :

• SAMPLE: select a signal from the Boundary Scan Register and read it.

• PRELOAD: insert a signal into the Boundary Scan Register.

• BYPASS : bypassing the test circuit.

• IDCODE : force an device-identification register.

• USERCODE : put a data in the EEPROM.

2.10.7 EEPROM

The EEPROM which contains the firmware of the FPGA is updated via the JTAGconnector. An external PC uploads a new firmware by using a JTAG cable connectedbetween the parallel port of the PC and the JTAG connector in the C3. Specific software(see section 5) transfers the bitstream to the card.

In order to push a firmware into the FPGA, the TAP instruction USERCODE is involved.This task takes around 1 minute for one card.

Because of the proximity of the C3 cards the Brandaris, it is difficult to reach theJTAG connectors on the card. The complete uploading process for the 32 C3 is thereforeincreased at between 2 and 4 hours.

A possible solution for this problem is to connect the 32 JTAGs connector into adaisy chain (see section 7.3.2) or to use the reconfigurable properties of the FPSLICcombined with the AT17k component (see section 7.3.1).

Basic Operations of the System 33.1 Introduction

The Brandaris camera is composed of 32 C3 that are connected via a USB connector toan external PC. This computer controls the C3 via the software ‘Winconsole’ that sendssimple text commands to the 32 FPSLICs in the C3.This program was developed by the‘Biomedical Engineering’ department.

The text commands are received in the AVR part of the FPSLIC and are processed inthe AVR and FPGA part of the FPSLIC. The programming languages used are VHDLfor the FPGA (see section 5.3) and C for the AVR (see section 5.2).

The aim of the FPSLIC is to control and to synchronize the electronic elements inthe C3 during the main operations of the camera: acquisition of images and transfer ofimages from the system to the external PC.

This section details the main operations between the FPSLIC and its electroniccomponents.

3.2 Text Console

In order to control the C3, a ‘Text console’ program is implemented in the FPSLICs.This program is executed by the AVR part of the FPSLIC and is used as an interfacebetween the external PC and the C3 through the USB2 connections. The ‘Text console’processes and sends the appropriate response(s) to the computer.

The PC sends its commands to the 32 FPSLICs at the same time, thus the operationsin the FPSLICs are executed in parallel in the C3. It is also possible to manage the ‘Textconsole’ of a C3 via its RS232 connection. These interface are used for testing the cards.

The main commands of the text console are described (see table 3.1) are :

• adc : This command reads and modifies the internal configuration of the four ADCsin the C3 and thus controls the gain and different parameters that are applied insidethese ADCs.

• dac : This command reads and modifies the internal configuration of the DACin the C3. The DAC generates an analogical current that is used to amplify thesignals in the CCDs.

• capture : Before starting multiple acquisitions of images (see section 3.3), the PCsends a command capture with the number of TRANSFER signals to wait beforeto start the transfer and readout phase (see fig. 2.12).

• getid/setid : These commands are used to read or to write a unique identifiernumber in the C3.

29

30 CHAPTER 3. BASIC OPERATIONS OF THE SYSTEM

Command Parameters Meaningadc read ADC readadc <channel> <address> <data> ADC writedac on/offdac <dac0> ... <dac7> Insert new parametersaddress <address> Set the memory addresscapture <ta1> <tb1> <tc1> <td1> ...

<ta6> <tb6> <tc6> <td6> Defines the TRANSFER ticks to waitdebug on/off Debug modegetid Read the C3 IDsetid <id> Set the C3 ID

Table 3.1: Example of some console commands

3.3 Acquisition process

During the acquisition of images, several successive processes are involved. Before anillumination, the photo sensors in the CCDs are flushed in order to store the electricalsignal. This is called the flushing phase. The next step or transfer phase consiststo transfer the signals captured in the photo sensors to their corresponding verticalregisters. During the last step or readout phase, the signals are alternatively shiftedfrom the vertical registers to the horizontal register and from the horizontal register tothe ADC (see section 3.3.2).

The External PC launches the capture process in the 32 C3. The signals START,FLUSH and TRANSFER are used to synchronize the different steps involved during theacquisitions.

All the operations involved in the acquisition process were designed to be functionalat 20 MHz (but are used at the frequency of 10 MHz).

3.3.1 Principle

The acquisition process of 6 experiments with 128 frames each is divided in eight stages:

1. Before to start the acquisitions, the external computer initializes the 32 C3 andconfigures the ADCs and DACs with the commands adc and dac (see table 3.1).

2. After the configuration of the C3, the PC sends the same command capture tothe 32 cards. This command indicates the absolute number of TRANSFER ticksto wait before to start the transfer and readout phases in the four CCDs in the C3.For example the instruction : capture 1 1 1 1 2 2 2 2 ... 6 6 6 6 meansthat in the 32 C3 , the four CCDs (CCD A to CCD D) will transfer the electricalsignals that were captured in their photo sensors to the vertical registers at the1st, 2nd, ... and 6th TRANSFER ticks.With the instruction capture 1 7 1 7 2 8 2 8 ... 6 12 6 12, it means thatin the 32 C3, the electrical signals stored in the photo sensors of the CCD A

3.3. ACQUISITION PROCESS 31

and CCD C will be transferred in the vertical registers at the 1st, 2nd, ..., 6thTRANSFER ticks and the signals in the photo sensors of the CCD B and CCD Dwill be transferred at the 7th, ..., 12th TRANSFER ticks.

3. In each C3, the AVR in the FPSLIC receives the capture command and collectsthe absolute number of TRANSFER ticks to wait for the four CCDs. The micro-controller stores these ticks in an internal table. Immediately after that, the AVRswait the START signal that synchronizes the 32 cards.

4. When the START signal is arrived, the AVR activates the FPGA in order to listenwhen FLUSH signal arrives. When the signal is became, the FPGA manages theflushing mechanism by sending the appropriate signals to flush the CCDs. Onlythe active CCDs that will be read at the next TRANSFER phase are flushed.

5. In the meantime, the AVR reads from the internal table the number of TRANSFERticks to wait before to start the ‘transfer’ phase for the first experiment. Thisvalue is sent to a counter implemented in the FPGA that counts the TRANSFERticks and determines whether or not the waiting signal is arrived. Only the activeCCDs that are concerned for this TRANSFER tick are processed. The AVR knowsat every time if the FPGA has received the appropriate number of TRANSFERsignals. If it is not the case then the AVR loops until the condition is true.

6. The AVR starts again the transfer counter (in the FPGA) for the next acquisition.In the meantime, the microcontroller stops the ability of the FPGA to read and toprocess the FLUSH signals and to start the ‘Transfer Phase’ in the specified CCDs.Again, only the active CCDs for this TRANSFER tick are processed. When thisprocess is done, the AVR starts the readout phase.

7. During the readout phase, the AVR activates the FPGA that generates the corre-sponding signals to shift the analogical signals from the active CCDs to the ADCs,to process these signals in the ADCs and to store the digital signals into the SRAMblocks.

8. When the readout is finished, the FPGA waits for the next FLUSH signal for theactive CCDs that would be processed.

9. Restart to step 6.

3.3.2 Readout phase

The Readout phase consists to execute successively these operations :

1. To shift simultaneously one element of the ‘Vertical Registers’ to the ‘HorizontalRegister’ in the CCDs.

2. To process consecutively all the elements of the ‘Horizontal Register’ in the CCDsin this way :

(a) In the CCD, to shift one element of the ‘Horizontal Register’ to the ADC.


(b) In the ADC, to convert the analogical signal to a digital signal and to transferit to the SRAM block.

(c) In the SRAM, to enable or disable the writing mode of the SRAM blocks andto increment the memory address

3. Restart to step one.

All these steps are driven and synchronized by only two VHDL entities : VERUNITand HORCOUNTER.

VERUNIT entity

This VHDL entity is duplicated four times (VERUNIT A to VERUNIT D) and is used tocontrol the vertical registers in the four CCDs (CCD A to CCD D) independently ofeach other. They produce the signals that control the ‘Vertical Registers’ inside theCCDs. They can flush and transfer the photo sensors to the ’vertical registers’ and alsoshift the ‘vertical registers’ to the ’horizontal register’ (see section 2.7.2).

At 20 MHz, the process takes 4.9 µs to flush the ‘photo sensors’ in the CCDs, 9.5 µsto transfer the data from the ‘photo sensors’ to the ‘vertical registers’ and 4.2 µs to shiftone pixel from the ‘vertical registers’ to the ‘horizontal registers’.

HORCOUNTER entity

A unique VHDL entity HORCOUNTER (see appendix A.1) generates the signals necessaryto shift the ‘horizontal register’ in the CCDs and to process the data simultaneously inthe four CCDs, ADCs and SRAM entities.

These signals generated in HORCOUNTER (fig. 3.1) are used in the following electronicselements :

• CCD : H1, H2 and RG shift one element from the ‘horizontal register’ to theoutput pin VOUT of the four CCDs.

• ADC : The analogical signal from the CCDs is read two times in the ADCs, atthe rising edge of SHP and SHD. A digital signal is produced at the rising edge ofDATACLK. The signals CLPOB and CLPDM indicate if the pixel that is currentlyread is a black pixel or not.

• SRAM : WE is used to activate or deactivate the ‘writing mode’ in the SRAM.The signal AINC increments the memory address in the SRAM blocks.

Physical connections to these signals

The signals H1, H2, RG, SHP, SHD, DATACLK, CLPOB and CLPDM produced byHORCOUNTER in the FPGA are sent to the output pins of the FPSLIC. These pins aredirectly linked to the four CCDs and four ADCs. Consequently, the four CCDs and ADCsin a C3 do always the same operations simultaneously. It is not possible to control thenindependently of each other.

3.4. MEMORY TRANSFER 33

The signal WE is directed to the VHDL entity RAMLOGIC and is used to generatethe necessary signals to activate/deactivate independently the 4x2 SRAM blocks.

The signal AINC is directed to the VHDL entity COUNTER21. This entity in-crements the signal SIG ADDRESS that contains the current memory address for theSRAM blocks. SIG ADDRESS is composed of 20 bits and is sent to 20 output pins ofthe FPSLIC. The eight SRAM blocks are directly connected to these FPSLIC pins andtherefore, the SRAM blocks are pointed to the same memory address at the same time.

It is not possible to generate and to assign four different memory address for the fourSRAM entities because it would be necessary to used 4 x 20 pins in the FPSLIC andthis component has a restricted number of available pins.

Explanations

The entity HORCOUNTER takes four clock cycles to read, convert and store one pixel. Thisoperation is repeated 540 times for the 540 pixels in a row. The procedure to processone pixel is explained step by step as follows (see fig. 3.1):

• 1st Cycle : An analog signal (corresponding to the current processed pixel) isproduced by the four CCDs simultaneously (at the rising edge of H2 ). The signalis sampled for the first time in its corresponding ADC (at the rising edge of SHP).In the mean time, the SRAM blocks store the previous digital signal (when WE isset at 1) processed by the ADCs. The SRAM blocks can be activated independentlyof each other.

• 2nd Cycle : The active SRAM blocks are still set in the writing mode (WE is setat 1).

• 3nd Cycle : The second part of the CCD’s analogical signal is produced (at risingedge of H1 ). This signal is sampled for the second time in its the correspondingADC (rising edge of SHD). The SRAM blocks are not able to write any data (WEis set at 0). The memory address of the SRAM blocks is incremented (rising edgeof AINC ). The same address is used for 4x2 SRAM blocks.

• 4nd Cycle : The four ADCs generates simultaneously a digital signal (at the risingedge of DATACLK ). The active SRAM blocks are able to write these signals (risingedge of WE ).

3.4 Memory Transfer

In this phase, the data stored in the SRAM blocks in the 32 C3 are transferred at thesame time to the external PC via the USB port.

A DMA implemented in the FPGA coordinates the memory transfer to the USB2controller. The DMA reads sequentially the four SRAM blocks for the same address andsends the data to the USB2 controller. After reading the blocks, the address memory isincremented and the controller sends the data to the external PC.


H1

H2

RG

SHP

SHD

DATACLK

CLPOB

CLPDM

WE

AINC

SYSCLK

Hori

zonta

l re

gis

ter

AD

CSR

AM

initialization Pixel1

Pixel2

Pixel20

Pixel21

Pixel529

Pixel530

1st cycle

2nd cycle

3ndcycle

4ndcycle

. . . . . .

. . . . . .

. . . . . .

black pixels black pixels

black pixels black pixels

Figure 3.1: Signals generated by HORCOUNTER. The ADC reads two times the same sig-nal at the rising edge of SHP and SHD, the signal is converted at the rising edge ofDATACLK. The SRAM blocks can write data when WE is set at ‘1’.

The data are stored in an internal buffer in the USB2 controller before being sent tothe network of hubs. If the buffer is full, then DMA waits before continuing.

The entire process takes around 1 second for 128 frames because of the limited speedof the SRAM.

The FPSLIC Device 44.1 Introduction

This section explains what are the general functionalities of the FPSLIC and the capa-bilities that can be use in the Brandaris Camera.

The FPSLIC AT94K40AL-25DQC is a SoC composed of an FPGA, AVR, embeddedmemory, an AVR/FPGA bus and a couple of other components (Fig. 4.1). With theintegration of all these elements, the FPSLIC has a low power consumption and has ancost effective price.

Figure 4.1: FPSLIC - from inside

All the components are connected together and communicated easily (fig. 4.2). Con-sequently, the FPSLIC is flexible by integrating some new functionalities in the FPGAin collaboration with the AVR.

35

36 CHAPTER 4. THE FPSLIC DEVICE

Figure 4.2: FPSLIC - Architecture

4.2 FPGA

The FPGA part of the FPSLIC is an AT40K from Atmel with a theoritical maximun of40000 programmable gates. It can run at the speed of 25 MHz. In the Brandaris128,the speed of the FPGA was defined at 20 MHz but for some stability problems it runsat 10 MHz (see section 2.7.7).

4.2.1 Input/Ouput Pins

The AT40K FPGA contains about 160 programmable Input/Output pins that are usedto connect almost all the electronics components. Now, 121 I/O pins are used in theFPGA part in the C3 cards.

The number of used pins in the FPGA is almost satured, it is not possible to connectdirectly more electronic components.

4.2.2 Core Logic and FreeRam Cells

The FPGA contains 2304 flexible ‘Core Cells’ (48 x 48) which have eight input/outputpads. These cells are connected together (fig. 4.3) and are able to communicate with eachother. They form a network of cells. This network is connected to a ‘busing network’(see fig. 4.4) which is connected to the input/ouput pins of the FPGA. The ‘Core LogicCells’ can access to these pins through the network.

The ‘Core Logic Cells’ are reprogrammable via their internal structure (Fig. 4.5), the’Core Cell’ can implement any pair of Boolean functions of (the same) three inputs orany single Boolean function of four inputs.

The AT40K FPGA contains several row and column repeaters for the communicationbus into the cells’ architecture. At the intersection of these repeaters, a ‘FreeRam cells’is connected (Fig. 4.4). The ‘FreeRam’ is a structure that can store a small quantity ofSRAM. The Freeram blocks are accessible and can be used in the FPGA.

4.3. AVR 37

snoitcennoC suB-ot-lleC )b(snoitcennoC lleC-ot-lleC )a(

WXYZL

WXYZLYX

Y

Y

Y

X

X X

CELL CELL CELL

CELL CELL CELL

CELL CELL CELL

CELL

Figure 4.3: Logic Cell Connections

In the current Brandaris implementation, the ‘Freeram cells’ are not utilized.

4.2.3 FPGA - Cache Logic

The FPGA contains a ‘Cache Logic’ that can be used to reprogram on-the-fly the ‘CoreLogic Cells’. The FPGA can be reconfigured fully or partially.

4.3 AVR

The AVR microcontroller has multiple integrated functions: it can have access to aProgram SRAM, a two-wire serial Interface, two UARTs, some Timers and has also fewbidirectional I/O ports.

4.3.1 Program SRAM

The AVR microcontroller can have access to its own ‘Program SRAM’ memory with acapacity of 32 Kbytes at the speed of 15 ns. The SRAM is only reachable by the AVRand is used during the execution of programs in the microcontroller.

4.3.2 Two-wire serial Interface (I2C)

The AVR is able to access to a ‘two-wire serial interface’ via an I2C bus. This protocolallows the communication between multiple masters and slaves. The AVR is acting as amaster.


= I/O Pad

= AT40K Cell

= Repeater Row

= Repeater

= RAM Block

Interface to AVR

Figure 4.4: The ‘busing network’ with 2304 ‘Core Cells’.

In the Brandaris Camera, the AVR communicates via this protocol to a DAC thatare used to configure the ADCs. In the C3, the AVR is connected to the AT17k com-ponent (see section 2.7.8) via the ‘two-wire serial interface’ and can thus reprogram theEEPROM (see section 7.3.1).

4.3.3 UART

Two full-duplex UARTs are available in the AVR, they are used during the serial com-munication. In the Brandaris128, the UARTs are used to allow the connection of anexternal PC to a C3 (via an RS-232 port).

4.3.4 Timers

The AVR has access to three ‘timers/counters’. They are driven by a clock which isindependent of the clock in the microcontroller. When a timer has finished to count, itproduces an interrupt that can be used. A WatchDog timer is also included. When thistimer stops to count it automatically reset the AVR and the FPGA. These timers arenot implemented in the current version of the Brandaris128.

Currently, the AVR can be run at different clock speed (at 20 MHz with some stabilityproblems or at 10 MHz). A waiting loop implemented to work at 20 MHz will not countthe same amount of time than at 10 MHz. By using a timer/counter, we can achieve a

4.4. FPGA - AVR BUS 39

The Cell

TUOTUO

RESET/SETCLOCK

FB

1 0Z

D

Q

"1" NW NE SE SW "1"

"1""1""0"

YWX

X YWZ

"1" N E S W

TUL 1 X 8TUL 1 X 8

X Y

NW NE SE SW N E S W

V1

H1

V2

H2

V3

H3

V4

H4

V5

H5

"1" OEH OEV L

Figure 4.5: Inside a Core Logic Cell

more reliable behavior that is independent of the speed of the AVR.

4.3.5 Bidirectional IO ports

The AVR has some available ports that can be used to connected multiple externalcomponents.

4.4 FPGA - AVR Bus

An FPGA/AVR Interface is integrated in the FPSLIC. Two methods are used to allowthe communications between the FPGA and the AVR: the IOSELs and the interrupts.

4.4.1 IOSEL

Inside the FPSLIC, the AVR can communicate with the FPGA by sending a request viaan IOSEL. The microcontroller can send at any time a request in order to transmit orto receive data from the FPGA by using the 16 available Input/Output lines.

In the current implementation of the Brandaris128, 15 IOSELs are used to commu-nicate between the AVR and the FPGA. One IOSEL is available for some improvementsto implement some extra functionalities.

4.4.2 Interrupts

The FPGA can communicate autonomously with the AVR via a panel of 16 interruptsthat are easily accessible in the microcontroller.


4.5 Dual Port SRAM

The FPSLIC contains a memory that is shared by the FPGA and the AVR. This dual-port SRAM has a capacity of 16 KB and an access time of 15 ns. This functionality isnot used in the Brandaris128.

4.6 AVR - Cache Mode

The AVR has direct access to the ‘Cache Mode’ in the FPGA. The microcontroller caneasily reprogrammed the FPGA partially or fully. This capability is not implemented inthe Brandaris128.

Design Software 5During this project, multiple software packages were involved to study and to modifythe code inside the AVR and the FPGA. This chapter explains the different relationsand methods that are used to program.

5.1 FPGA

Since the original FPGA code for the Brandaris Camera was designed by the companyAED Electronics, the software package necessary during the different programming steps(Fig. 5.1) were not available at the Erasmus MC.

In order to achieve this work, we used some software provided by ‘TU Delft’.

5.1.1 RTL Design and Simulation

In order to modify the current VHDL code of the Brandaris128, it is necessary, after anychanging in the code, to make some new simulations. We use ModelSim1 to achieve thisgoal.

At this time, the propagation delays inherent to any electronic circuit are not imple-mented in the VHDL code.

5.1.2 Synthesis

During synthesis phase, the software Synplify2 compiles and optimizes the previousVHDL in order to fit with the FPGA technology from Atmel. It means that the VHDLcode is splitted in small entities that correspond to the ‘Core Logic Cells’ (see sec-tion 4.2.2).

At this point, some constraints are applied to the clock frequency (at 20 MHz or10 MHz) or to the input/output pins of the FPGA and give a first estimation of thedelays and the critical path in the program.

5.1.3 Place & Route

The place & route step is done by Figaro3. This software[17] is specific for the FPGAand FPSLIC Atmel’s technology. It can read the file that was generated during thesynthesis phase and can import the clock and input/ouput constraints.

The aim of Figaro is to place correctly the Core Cell’s macro previously defined intothe FPGA’s architecture. It means that the ‘Core cells’ are connected between them

1ModelSim from Mentor Graphics Corp., http://www.model.com/

2Synplify from Synplicity, http://www.synplicity.com/

3Figaro Place&Route, Atmel Corp., http://www.atmel.com/products/FPSLIC/

41

42 CHAPTER 5. DESIGN SOFTWARE

Figure 5.1: The different programming phases.

and to the busing network in order to have access to the FPGA’s pins (see fig. 4.4). Theconnections between the ‘Core cells’ are routed to fit with timing constraints.

After the place&route phase (which takes few minutes), the delays taken by thesignals to cross the FPGA from inside the ‘Core cells’ and between the cells is known.

5.1.4 Post-Simulation

After the previous phase, Figaro can export the result of the place&route to an ensembleof new VHDL entities that consider the delays in the circuit. ModelSim is used againto test the code via a setup of testbenches. The results of this ‘Post-Simulation’ takesaccount of the timing delays. If the simulations do not fit with the expected behavior,then it is necessary to restart the previous steps.

5.2. AVR 43

5.1.5 Bitstream

When all the previous phases are correct, then the FPGA’s bitstream generated by Figarois extracted. Furthermore, it will be used in combination with the AVR bitstream.

5.2 AVR

The microcontroller is programmed with the C language. Two kinds of software areavailable for this task: AVR GCC4: an AVR compiler from C to AVR Assembler, thissoftware is derived from GCC, or AVR Studio5, which is a complete Integrated Develop-ment Environment. This environment enables the use of an external hardware debugger,the JTAG In-Circuit Emulator. This emulator is a 4-wire Test Access Port (TAP) (seesection 2.10) that is able to access to the port of the FPSLIC via the JTAG interface.

For this project, the simple AVR GCC compiler (not the most recent version) wasused for compatibility reasons.

5.3 FPSLIC

Another solution, is to use the package of software provides with ‘System Designer’6

from Atmel.It contains all the software necessary for the:

• FPGA programming: ModelSim for the VHDL simulation, Precision RTL7 for thesynthesis.

• AVR programming : AVRStudio for the AVR C development with the externalFPSLIC JTAG ICE in order to enable the debugging of the microcontroller code.

• Hardware and Software co-verification : With the co-verification, the programsimplemented in the FPGA (hardware part) and in the AVR (software part) arecontinuously developed and tested in parallel (see fig. 5.2). It is not necessary asbefore to test the VHDL code independently of C code. The development is thenspeed up tremendously.

We do not use the complete solution ‘System Designer’ because it is costly. Thesoftware ModelSim and Synplify are provided by TU Delft.

5.4 External PC

The main program in the external PC, ‘WinConsole’, is designed by Visual Studio 6under Windows XP. The USB2 driver that communicates with the PC was developed inthe dept. of ‘Biomedical Engineering’ of Erasmus MC.

4AVR Gcc, http://www.avrfreaks.net/index.php?module=FreaksTools&func=viewItem&item id=145

5AVR Studio, Atmel Corp., http://www.atmel.com/dyn/Products/tools card.asp?tool id=2725

6System Designer, Atmel Corp. , http://www.atmel.com/dyn/products/tools card.asp?tool id=2752

7Precision RTL, Mentor Graphics Corp., http://www.mentor.com/products/fpga pld/synthesis/precision rtl/

44 CHAPTER 5. DESIGN SOFTWARE

Figure 5.2: Harware/Software Co-Verification

Improvement - Region OfInterest 66.1 Introduction

As said before, the main restriction of the Brandaris128 is the limited number of framesthat can be stored in its embedded memory. The experiments done in the Brandariscamera consist to excite one or multiple microbubbles through one or several ultrasoundbeams. These microbubbles have a relatively stable and predictable position during theobservation.

The aim of this thesis is to improve the camera in order to save more images andthus captured more experience. To achieve this goal, we have implemented the conceptof ‘Region of Interest’. Because the bubbles are stable, a ROI could be applied aroundthe studied objects. Only the relevant data in the ROI will be stored in the memory ofthe camera.

In this section, we explain the principle of the ROI, how to choose this area andalso the implementation, results and consequences of the integration of the ROI in theBrandaris camera.

6.2 Principle

Before starting the implementation of the ROI, it is necessary to apply for each readoutpixel this three following steps :

1. Determinate the coordinates of the current readout pixel.

2. Determinate whether this pixel is inside or outside of the ROI by comparing theircoordinates.

3. Determinate the action to apply to the pixel depending on its position.

(a) Inside the ROI : This phase is identical to a normal acquisition. The processedpixel is saved in the embedded memory.

(b) Outside the ROI : The processed pixel is not saved and the memory is notactivated.

6.2.1 Pixel’s coordinates

In order to determine whether a bubble is inside the ROI or not, it is first necessary todefine the coordinates of the current processed pixel in the CCDs.

In a C3, the four CCDs are always placed with the same orientation. In other words,their output pins are directed in the same direction. An example of the current and

45

46 CHAPTER 6. IMPROVEMENT - REGION OF INTEREST

correct placement of the CCDs is illustrated in Fig. 6.1(a). An example of a wrongplacement is presented in Fig. 6.1(b) (this configuration is not used in the camera).

A B C DVout Vout Vout Vout

(a) Correct positions of the CCDs

A B C DVout

Vout

Vout

Vout

(b) Wrong positions of the CCDs

Figure 6.1: CCDs positions

Moreover, the CCDs are simultaneously readout. It means that when a photo-site/pixel is processed in a CCD, then exactly the same pixel with the same coordinatesis also processed at the same time in the other CCDs.

The signals generated alternatively by the VHDL entities VERUNIT and HORCOUNTERare used to shift out the data from the photosites in the CCDs (see section 3.3.2).

Pixel’s coordinates deduce from VERUNIT

After the execution of VERUNIT, one pixel from all ‘vertical registers‘ is shifted into the‘horizontal register’. In other words, a complete row of pixels in the CCDs is moved inthe ‘horizontal register’. When the execution of VERUNIT is finished, the Y-coordinateof the processed pixel is incremented (see fig. 6.2).

. . .. . .

. . .. . .

before shifting

after shifting

vert.reg.

vert.reg.

vert.reg.

horiz. reg.

Figure 6.2: Execution of VERUNIT. Shift a complete row of photosites into the ‘horizontalregister’.

Pixel’s coordinates deduce from HORCOUNTER

After the accomplishment of VERUNIT and before starting HORCOUNTER, an entire row ofpixels is contained in the ‘horizontal register’. Thus, the X-coordinate of the pixel in

6.2. PRINCIPLE 47

this row is set at ‘0’.During the execution of HORCOUNTER, every four clock cycles a pixel is pushed out

of the CCDs and is sent directly to the ADCs (see section 3.3.2). It means that everyfour clock cycles, the X-coordinate of the pixel that will be processed by the ADC isincremented (see fig. 6.3).

This operation is repeated 540 times in order to extract 540 pixels from the ‘horizontalregister’. In reality, this register contains only 537 pixels (see section 2.7.2), the last threepixels are only produced to have an image of 540 x 300 pixels.

. . .. . .

. . .. . .

before shifting

after shifting

vert.reg.

vert.reg.

vert.reg.

horiz. reg.

Figure 6.3: Execution of HORCOUNTER. Shift a complete row of pixels from the ‘horizon-tal register’ to the ADC. At every four clock cycles, the X-coordinate of the pixel isincremented.

6.2.2 Roi’s coordinates

In order to determinate if the current processed pixel is inside or outside of the ROI, itis necessary to compare its coordinates to those of the ROI in the four CCDs in a C3.

From the last section, we know that the coordinates of the current pixel are the samefor in the four sensors. As said before, the CCDs are placed on the CCD MountingBlocks (see section 2.3.2). Consequently, the images captured from the microscope areprojected directly or via a mirror to the CCDs. The frames that are acquired by theCCDs are displayed correctly or are vertically flipped.

The ROI is also affected by this phenomena. This problem is only corrected in theexternal PC. Thus, the absolute coordinates of the ROI in the four CCDs are not thesame and two different ROI are applied in the sensors : the ROI that was defined andthe same ROI but vertically flipped. An example is illustrated in fig. 6.4.

Two solutions are feasible to resolve this problem. They have both pros and cons.


frames with ROI displayed in the external PC

frames with ROI projected on the CCDs via or not a mirror

Figure 6.4: ROI vertically flipped inside four CCDs.

Solution 1 : Normal ROI

If two ROIs appear (normal and flipped), the solution is to execute two distinct ‘readout’phases : the first time to acquire the images from the CCDs having a normal ROI andthe second time with the CCDs having a flipped ROI. It could be done easily because wecan control the ‘vertical registers’ and the SRAM blocks independently from each other.

The advantage of this method is that we can define a ROI that fits precisely themicrobubbles. The disadvantage is that the complete time to ‘readout’ the four CCDsis doubled.

Solution 2 : ROI vertically symmetric

In order to avoid to ‘readout’ two times the CCDs, we can used a ROI having a verticalsymmetry (see fig. 6.5). Because the ROI is symmetric, when we applied a verticalflipped, we obtain the same ROI. Thus, only one ‘readout’ session is necessary to readthe four CCDs.

The benefit of this method is that it runs at the normal speed. The disadvantage isthat the ROI needs to have a vertical symmetry.

Solution retained

Finally, the second solution will be used in the Brandaris camera. This is a simpleimplementation without having any effects on the current speed of the camera duringthe acquisition.

Before starting an experiment, the user of the Brandaris128 can place accurately themicrobubbles and thus can choose an optimal position taking in account the verticalsymmetry of the ROI.

6.3. THE ROI 49

frames with ROI displayed in the external PC

frames with ROI projected on the CCDs via or not a mirror

Figure 6.5: ROI with vertical symmetry

6.3 The ROI

In our implementation, the user of the Brandaris camera can decide to apply or nota ROI and can choose which ROI is appropriate for the experiences. Moreover, theseoperations must be done quickly and can be repeated during the operational time of thecamera.

This section explains how to apply and choose a ROI. For sake of simplicity, only arectangular ROI having a vertical symmetry will be used.

6.3.1 Choice of a ROI

We have two possibilities to choose a ROI :

• By capturing images from the Brandaris128, sending them to the external PC andchoosing an appropriate ROI. Unfortunately, this method requires too many stepsand is too long comparing to the running time of the turbines (around 30 s).

• By choosing a ROI directly from the external monitor connected to the microscope.This could be done by placing a transparent layer that contains a ROI having pre-defined coordinates to the monitor. Even if the placement of the layer and thechoice of the ROI are arbitrary, we could still choose the most convenient ROI (seesection 6.3.2). This method will be used because this is the easiest and fastestapproach.

6.3.2 Pre-defined ROI

During a ultrasound excitation, the user of the camera knows by experience whether thestudied bubbles will be steady or they will move to a specific direction. Furthermore,he or she needs to take into account that the CCDs have some misalignments (seesection 2.3.2).


With these informations, he or she can select the most appropriate ROI (see fig. 6.6)that will encompass the bubbles. This area includes a small margin at these bordersbecause of the misalignments.

Figure 6.6: The different chosen ROI’s configuration, ‘F’, Full Frame: 6 exp. could bestored in the SRAM, ‘H0’ to ’H3’, Half Frame: 12 exp. could be stored, ‘Q1’ to ‘Q3’,Quarter Frame: 24 exp. could be stored.

Each of these pre-defined ROI will be printed on a transparent layer that fit to thesize of the external monitor.

6.4 Simulation Methodology

Before implementing the ROI concept, our first task was to clearly understand how thedifferent VHDL entities interact in the FPGA.

For this reason, we have developed some extra new VHDL entities that will simulatethe electronic components of the C3. These entities are described as follow :

• The couple CCD and ADC : A special VHDL entity CCD+ADC (see appendix A.3) isdedicated to simulate these two components during the ‘readout’ phase. The entityreceives and decodes the signals that are generated by HORCOUNTER (H1, H2 andRG) and by VERUNIT (V1, ..., V4 ). The entity CCD+ADC reads an external file thatcontains a fake image and uses these data to simulate the signals that are readoutand processed by the couple CCD/ADC. The implementation is straightforward,the flushing and transfer behaviors of the CCDs and the gain, double sampling andblack clamping modes of the ADCs are not used.

The system follows the principles defined in Fig. 3.1.

• The SRAM blocks : This VHDL entity (see appendix A.4) simulates the operationsdone in the 4x2 SRAM blocks. It can read and write at different virtual addresses.This virtual memory can be imported and exported directly from ModelSim.

6.5. IMPLEMENTATION 51

• The USB2 Controller : This entity (see appendix A.5) simulates the USB controllerto transfer data from the SRAM to the USB2 port. The data that are generatedby this entity are saved into an external file.

All the external files (images, virtual SRAM, ...) that are used in these VHDL entitiesare generated or read in Matlab1.

We have finally written multiple testbenches in VHDL that simulates the signalsproduced by the AVR part (for example with IOSEL, see section 4.4.1). The behavior ofthe current implementation of the Brandaris128 and the integration of the ROI’s abilitywere tested with these special VHDL entities.

6.5 Implementation

The implementation of the ROI functionality is divided in three steps: to calculate thecoordinates of the actual processed pixel in the CCDs, to determinate whether this pixelis inside or outside the ROI and finally depending on the previous result to save or notthe processed pixel inside the SRAM.

As said before, the Y-coordinates of the processed pixels are calculated after theaccomplishment of VERUNIT and the X-coordinates of these pixels are determinated dur-ing the execution of HORCOUNTER (see section. 6.2.1). Furthermore, the ‘readout’ phaseconsists to execute alternatively VERUNIT and HORCOUNTER. Consequently, we can thencompute the X and Y coordinates of the processed pixel entirely in HORCOUNTER.

The procedure to count the pixel’s coordinates in an improved HORCOUNTER (seeappendix A.2) would be to set the X-coordinate at ’0’ and to increment the Y-coordinatewhen the VHDL entity is started. During the execution of HORCOUNTER, every four clockcycles in the FPGA, the X-coordinate is incremented. The counting process is finishedwhen the X-coordinate is equalled to ‘540’.

In the meantime, this enhanced HORCOUNTER will also determine whether the currentprocessed pixel is inside or outside of the ROI and will save it or not in the memory. In or-der to achieve these tasks and before starting the ‘readout’ phase, the entity HORCOUNTERwill receive the coordinates of the ROI (see fig. 6.7) and a signal to enable or disable theROI. These parameters are sent by using an IOSEL (see section 4.4).

As illustrated in fig. 3.1, the entity HORCOUNTER contains a loop that is executed 540times. Each loop is divided in four sub-cycles (a to d) or clock cycles. This scheme isbased on the original VHDL entity (see section 3.3.2).

The improved HORCOUNTER generates and uses internally several new signals. Thesesignals are : CCD X and CCD Y (coded in 10 bits) that contains the coordinates of thepixel that will be processed at the next step, IS INSIDE X and IS INSIDE Y (coded in1 bit) that determinates whether or not the current pixel is inside the ROI in its X andY coordinates, IS INSIDE ROI (coded in 1 bit) that specifies whether the pixel is inthe ROI and finally the modified WE and AINC that are used to save or not the pixelin the SRAM blocks.

The different steps of HORCOUNTER are illustrated in fig. 6.8 and are organized asfollow :

1Matlab from The MathWorks, Inc.


ROI_A

ROI_B

Inside the ROI

Outside the ROI

Figure 6.7: A Region of Interest with its two determinant point, ROI Begin (withthe coordinates ROI Begin x and ROI Begin y) and ROI End (with the coordinatesROI End x and ROI End y).

1. When the entity is started, the X-coordinate of the processed pixel is set at ‘0’ andthe Y-coordinate is incremented.

2. (a) 1st sub-cycle d : The X-coordinate or CCD X is incremented. It contains thepixel’s coordinates that will be process at the next sub- or clock-cycle.

(b) 2nd sub-cycle a : At this step, the signals IS INSIDE Xand IS INSIDE Y are calculated by using the coordinatesof the pixels. These following computations are done :IS INSIDE X = (CCD X ≥ ROI BEGIN X) AND (CCD X ≤ ROI END X)and IS INSIDE Y = (CCD Y≥ ROI BEGIN Y) AND (CCD Y≤ ROI END Y).

(c) 3nd sub-cycle b : The signal IS INSIDE ROI is calculated by using the for-mula : IS INSIDE ROI = IS INSIDE X AND IS INSIDE Y .

(d) 4nd sub-cycle c : The modified signals WE and AINC are produced. IfIS INSIDE ROI is equalled at ‘0’ then these signals are set also at ‘0’. IfIS INSIDE ROI is equalled at ‘1’ then these signals have the same valuesthan the ‘old’ version of WE and AINC.

3. At then end of HORCOUNTER, the X-coordinate of the processed pixel is equalled at‘540’.

6.5. IMPLEMENTATION 53

sysclk

SHP

CCD_X

IS_INSIDE_X

IS_INSIDE_Y

IS_INSIDE_ROI

new WE

new AINC

CCD_Y

DATACLK

original WE

original AINC

a b c d a b c d a b c d a b c d a b c d a b c d a b c d a b c d a b c d a b c dsub-cycles

OUT OUT OUT OUT OUT OUTIN IN IN ININIT

0

75

1 2 134 135 136 406405 407 539 540

Figure 6.8: X and Y coordinates determined in HORCOUNTER


6.6 Results

6.6.1 Hardware testing

The new implementation of HORCOUNTER with the ROI ability was tested on an externalbackplane. This backplane contains a complete C

3 card and a clock generator. TwoCCDs were connected to the C

3 with only one CCD with a lens. The clock generatorprovides the following signals: the clock frequency at 10 MHz, the START, FLUSH andTRANSFER at the frequency of 10 kHz.

6.6.2 Images

From this platform test, we have captured an image (fig. 6.9) via the CCD with a lens.This picture is stored in the embedded SRAM blocks with an arbitrary size of 540x300pixels because of the black pixels (see fig. 6.9(a)). When the image is displayed on theexternal PC, these blacks pixels are deleted (see fig. 6.9(b)).

In these examples, the external PC does not apply any corrections concerning themisalignment.

(a) Image stored in the embedded SRAM (with

black pixels).

(b) Image displayed in the external PC (without

black pixels).

Figure 6.9: An image captured without a ROI.

Before appling a ROI, we have selected a vertically symmetrical area that we wantto use (fig. 6.10).

Figure 6.10: Selection of a ROI.

6.6. RESULTS 55

By activating the ROI ability and sending its coordinates to the Brandaris camera,we have obtained this image: fig. 6.11. Only the relevant part of this image is saved inthe embedded memory of the C3 (fig. 6.11(a)). In the external PC, the smallest imageis inserted into a frame of 540x300 gray pixels (see fig. 6.11(b)). By using this size ofimage, the management of the frames in the external PC is simplified.

(a) Image stored in the embedded SRAM (with

black pixels).

(b) Image displayed in the external PC (without

black pixels).

Figure 6.11: An image captured with a ROI.

6.6.3 Cost

The original implementation of the Brandaris Camera without the ROI ability doesnot use excessively the resources of the FPSLIC (see table 6.1). Only one third of the‘Logic Cells’ are used and the ‘Freeram’ and the dual-port SRAM cells are not used (seesection 4).

Resource UtilizationLogic Cells 777 of 2304 (34 %)Flip-Flops 251

Table 6.1: Resources utilization before the ROI’s implementation

The new version of the Brandaris Camera that takes into account the ROI has alow overload (see table 6.2) comparing to the original implementation. Only one VHDLentity (HORCOUNTER) was modified and thus did not increase the complexity of the system.

Resource UtilizationLogic Cells 896 of 2304 (39 %)Flip-Flops 294

Table 6.2: Resource utilization after ROI’s implementation

No speedup can be noticed during the acquisition phase. Indeed the new improvedHORCOUNTER consumes exactly the same number of clock cycles as before.


It is also interesting to indicate that in both cases, the utilization of the logic cellsis always below 50 %. According to the literature[17], above this threshold, the routingphase of the logic cells in the ‘Place&Route’ can become difficult or impossible.

6.7 Consequences

6.7.1 In the FPGA

The implementation of the ROI in the Brandaris Camera is done by changing only oneVHDL entity, HORCOUNTER. The other entities involved in the generation of the verticalclock signals, flush and charge signals (in VERUNIT), the transfer of the SRAM to theUSB2 controller and the other internal counters are not modified.

6.7.2 In the AVR

Because more than 6 experiments can be stored in the Brandaris Camera, the programin the AVR part needs to be modified. With our pre-determined configuration of ROIs,we are able to store up to 24 experiments with 128 consecutive frames in the SRAMblocks.

In order to record 24 experiments, we need to use the command capture in the textconsole with these parameters : ‘capture 1 1 1 1 ... 6 6 6 6 ... 24 24 24 24’.This command is too long to be use (261 characters) in the console.

For this reason, we have introduced a new command, captureexp. The syntax issimple, ‘captureexp 24’ generates an internal table to count the Transfer signals for24 experiments. And ‘captureexp 24 segment 2’ initializes the internal table to countthe transfer signals for 2 segments that can acquire each 24 segmented experiments of64 frames.

6.7.3 In the external PC

The program ‘Winconsole’ in the external PC needs to manage the increased numberof experiments. The frames that are downloaded from a region of interest are smallerthan the regular frame. In order to facilitate the correction of the frame’s position (bytranslation, rotation and zooming), the cropped image is copied into an emptied fullframe and can thus be processed normally.

6.7.4 In the timing controller

The timing controller needs to generate more FLUSH, TRANSFER and triggers signalsfor the experiments. The modification of these parameters is straightforward since theparameters can be edited in a text file.

Conclusion 77.1 Summary

The Brandaris128 is a very high-speed camera that was mainly designed to study themechanisms and the behaviour of the ultrasound contrast agents submitted to an ul-trasound field. By understanding their acoustic responses, it is possible to improve theultrasound imagery.

In Chapter 1, we have described the main limitation of the Brandaris128 which isthe limited number of experiments that can be stored in the camera.

In Chapter 2, we have presented the complete system that composes the Bran-daris128. The most important components of the setup were described and also theirinteractions : the rotating mirror, the CCD controller cards which contains the electronicof the camera, the external PC to process the images, the USB connections in order tocommunicate between the camera and the computer, the timing controller which gen-erates the synchronization signals (START, FLUSH and TRANSFER) and the JTAGinterface.

In Chapter 3, we have described the main operations that are done in the Bran-daris128 and also the electronic components that are involved during the functioning ofthe camera.

In Chapter 4, we have given an overview of the capabilities of the FPSLIC.In Chapter 5, we have made a presentation of the software packages that were used

to reprogram the different parts of the Brandaris128: in the FPGA, in the AVR and inthe external computer.

In Chapter 6, we have described a solution about the restricted number of experiencesthat can be stored in the camera. By using a ROI, we can increase the number of imagesacquisition. We have also explained how we have implemented the ROI.

7.2 Discussions

The Brandaris128 is a device with an hardware architecture that cannot be modified: thegas turbine, the embedded memory, the USB connections ... . Most of the limitations ofthe camera come from this constitution, for example only 6 experiments with 128 framescan be stored in the Brandaris128.

By studying the capabilities of the FPSLIC - the central component in the C3 - wecan use its flexibility to improve the camera. Furthermore, we have implemented theconcept of ROI in the Brandaris camera.

57

58 CHAPTER 7. CONCLUSION

7.3 Future Directions

The Brandaris128 was designed in 2003, the camera is still actively used at Erasmus MCand University of Twente. Thanks to the FPSLIC cores, the setup may evolve dependingon the needs.

Two kinds of improvements can be done: by modifying the software to make useof the flexibility of the SoC and the external PC, or by designing a completely newhardware configuration.

7.3.1 Software - Improvements

The software’s improvements can be done by reprogramming the FPGA, AVR or theexternal PC.

Reconfigurable Processor

Each C3 contains an AT17k reprogrammable component, that is connected to the AVRthrough an I2C bus. This component is able to read and write the EEPROM thatcontains the firmware of the FPGA.

The benefit of this capability is that the AVR could reprogram the FPGA with a newbitstream coming from the USB2 network. In that case, it would not be necessary toopen the Brandaris camera and to reprogram individually each C3 cards, if the firmwareshould be updated. This process would take a couple of minutes instead of a few hours.

The external PC would manage the entire operation and could easily and quicklyupload a new firmware corresponding to a new setup of functionalities. The feasibilityto juggle between multiple Brandaris’ implementation will increase the flexibility of theCamera.

Correction of luminosity

During a typical acquisition of images and before starting a ‘readout’, the main task ofthe FPSLICs is to count the FLUSH and TRANSFER signals. During this time, theSRAM blocks are not used.

We could use this interval to read the data stored in the embedded memory from aprevious acquisition (the so-called ‘reference acquisition’) and to extract some parametersthat can be used for the next acquisitions. For example, we could extract the luminosityfrom the 128 frames from the ‘reference acquisition’ and to configure the ADCs to adjustthe luminosity of the frames for the next experiments.

It could be a very convenient solution for the problem of non-uniform luminosity (seesection 2.4).

The capabilities of the FPSLIC can be used to achieve these operations, by usingfor example the dual-port RAM (see section 4.5) to communicate between the AVR andFPGA parts.

This available interval of time is unfortunately relatively short. A complete frame of540x300 pixels cannot be completely read, because of the memory’s speed. A solutionwould be to read every 4 pixels in the memory for example.

7.3. FUTURE DIRECTIONS 59

The Brandaris128 at 10 MHz

The actual implementation of the Brandaris camera is designed to work at 20 MHz. Ittakes into account that the SRAM works at 70 ns (around 14.3 MHz) and needs at leasttwo clock cycles at 20 MHz to be able to read or write the data.

Because of some instability in the electronics components in the C3, the setup runsat only 10 MHz and is thus twice slower. Furthermore at the speed of 10 MHz, only oneclock cycle instead two clock cycles is necessary to read and write data in the SRAMblocks We could use this particularity to speed up the readout phase at 10 MHz.

7.3.2 Hardware - Improvements

Concerning the hardware part, we have also proposed some ideas to improve or build anew high-speed camera based on the Brandaris128.

JTAG network

In order to update the firmware of the FPSLICs, a computer needs to be physicallyconnected to the JTAG connector of the 32 C3. This operation is very long (around 2or 3 hours).

A solution to speed-up this phase would be to deploy a JTAG network or daisy-chainbetween the cards (fig. 7.1) by joining with a wire the JTAG output of a card to theJTAG input of the next C3.

By using the JTAG instructions (see section 2.10.6), we could push a new firmwarein the C3. Each card have a different device-identification register (instruction IDCODE).To perform this task, it is necessary to write a new program that upload an identicalfirmware through the serial JTAG network inside the EEPROM of the FPSLICs.

It could take around 30 minutes to update the 32 FPSLICs. This technique is fasterthan the manual method but slower than by using the reconfigurable capability (seesection 7.3.1) of the FPSLIC. This is because the deployment of a new firmware is doneserially with the JTAG and in parallel with the reprogrammable AT17k.

TMS

TCK

TDI

TDO TMS

TCK

TDI

TDO TMS

TCK

TDI

TDO

TCK

TMS

TDI

. . .

FPSLIC #1 FPSLIC #2 FPSLIC #32

Figure 7.1: A daisy chain that connects the JTAG connectors to the 32 C3.

60 CHAPTER 7. CONCLUSION

Network communication

The SRAM blocks have a maximal throughput of 114.3 Mbits/s (8 bits RAM with anaccess time of 70 ns). When the 32 C3 send their data to the external PC, the maximalthroughput is then 3.6 Gbit/s on the computer.

The USB2 network with its throughput of 480 Mbit/s is clearly overloaded. A solu-tion would be to integrate a gigabit ethernet controller in the C3, to connect them to anethernet switch that can ‘load balance’ the data flow to one or several gigabit ethernetcards into the external computer.

SRAM

The current SRAM blocks in the Brandaris128 is the main limitations of the setup. With1 Mbytes of memory dedicated per CCDs, it is only possible to store 6 full frames (atthe resolution of 540x300) in the SRAM. The slow speed of the SRAM (70 ns) doesn’tfit with a clock rate of 20 MHz (period of 50 ns).

The solution would be to integrate a new generation of SRAM blocks with a biggercapacity and a shorter access time (inferior at 50 ns).

Photo sensors

The photo sensors in the Brandaris128 are a critical part of the system. They need tohave a high sensibility in order to capture correctly the images. This characteristic iscompulsory to study the fluorescence imaging in the microbbubles[8].

The CCD technology is a good candidate for this aspect. The following sensorsfrom Sony Corp. could be used : ICX429ALL (black&white images with a resolutionof 752x582 pixels) and ICX055BL ( same circuit than the ICX055AL but with a highersensibility).

With the ROI capability, we could also use a CMOS sensor (see section A). In thistechnology, the pixels in the sensor are directly accessible without using a vertical orhorizontal register. Consequently, only the relevant area (the ROI) in the sensor couldbe extracted and the readout phase will be faster. In general, the CMOS sensors have aless sensitivity than the CCDs.

Bibliography

[1] C.T. Chin, Modelling the behaviour of microbubble contrast agents for diagnosticultrasound, Ph.D. thesis, University of Toronto, 2001.

[2] C.T. Chin, C. Lancee, J. Borsboom, F. Mastik, M.E. Frijlink, N. de Jong, M. Ver-sluis, and D. Lohse, Brandaris 128: A digital 25 million frames per second camerawith 128 highly sensitive frames, Review of Scientific Instruments 74 (2003), 5026.

[3] Cording Company, Cordin company, high speed imaging solutions,http://www.cordin.com.

[4] Sony Corp., Icx055al, 1/3-inch ccd image sensor for ccir b&w camera, Applicationnote, Sony Corp., Tokyo, Japan (2000).

[5] Rotterdam EramusMC, The brandaris128 website, www.brandaris128.nl.

[6] Rotterdam ErasmusMC, Dept. of biomedical engineering, thoraxcenter, erasmus mc,the netherlands, http://www.erasmusmc.nl/ThoraxcenterBME/.

[7] ER Fossum and L.C. Photobit, CMOS image sensors: electronic camera-on-a-chip,Electron Devices, IEEE Transactions on 44 (1997), no. 10, 1689–1698.

[8] E.C. Gelderblom, High speed fluorescence imaging of encapsulated microbubbles,Master’s thesis, Physics of Fluids, Applied Physics, University of Twente, 2008.

[9] Analog Devices Inc., Ad9843a, complete 10-bit 20 msps, ccd signal processor, Ap-plication note, Analog Devices Inc. (2000).

[10] Atmel Inc., At17lv010-10dp, space fpga configuration eeprom, Application note, At-mel Inc, San Jose, California (2002).

[11] , Fpslic on-chip partial reconfiguration of the embedded at40k fpga, Applica-tion note, Atmel Inc, San Jose, California (2002).

[12] Brilliance Semiconductor Inc., Bs62lv4001, low power/voltage cmos sram, 512k x 8bit, Application note, BSI Inc. (2002).

[13] NetChip Technology Inc., Net2270, 16-bit usb 2.0 high-speed programmable periph-eral controller, Application note, NetChip Technology Inc., Moutain View, Califor-nia. (2001).

[14] D. Litwiller, CCD vs. CMOS, Photonics Spectra (2001).

[15] CM Maunder and RE Tulloss, An introduction to the boundary scan standard: An-si/ieee std 1149.1, Journal of Electronic Testing 2 (1991), 27–42.

[16] E. Muybridge, Muybridge’s Complete Human and Animal Locomotion: All 781Plates from the 1887 Animal Locomotion, Courier Dover Publications, 1979.

61

62 BIBLIOGRAPHY

[17] K. Nasi, T. Karouhalis, M. Danek, and Z. Pohl, Figaro–an automatic tool flowfor designs with dynamic reconfiguration, International Conference on Field Pro-grammable Logic and Applications (2005), 590–593.

[18] E. Oda, Y. Ishihara, and N. Teranishi, Blooming suppression mechanism for an in-terline CCD image sensor with a vertical overflow drain, 1983 International ElectronDevices Meeting 29 (1983).

[19] Maxim Integrated Products, Max251, quad/octal, 2-wire serial 8-bit dacs with rail-to-rail outputs, Application noteMaxim Integrated Products (1996).

[20] Terschelling, The famous brandaris lighthouse at terschelling,http://www.vuurtorens.net/.

VHDL Appendix AA.1 Original HORCOUNTER : horcounter.vhd

l ibrary IEEE ;use IEEE . STD LOGIC 1164 . a l l ;use work . user pkg . a l l ;

entity HORCOUNTER i sport (CLK : in STD LOGIC;

RESET : in STD LOGIC;ENABLE1 : in STD LOGIC;ENABLE2 : in STD LOGIC;D : in STD LOGIC VECTOR(7 downto 0 ) ;BUSY : out STD LOGIC;AINC : out STD LOGIC;H1 : out STD LOGIC;H2 : out STD LOGIC;RG : out STD LOGIC;SHP : out STD LOGIC;SHD : out STD LOGIC;DATACLK : out STD LOGIC;CLPDM : out STD LOGIC;CLPOB : out STD LOGIC;PBLK : out STD LOGIC;WE : out STD LOGIC) ;

end HORCOUNTER;

architecture BEHAVIOUR of HORCOUNTER i scomponent DFF

port (CLK : in STD LOGIC;D : in STD LOGIC;Q : out STD LOGIC) ;

end component ;

signal s i g s t a t e : STD LOGIC VECTOR(1 downto 0 ) ;

63

64 APPENDIX A. VHDL APPENDIX

signal s i g b l a c k : STD LOGIC;signal s i g busy : STD LOGIC;signal s i g a i n c : STD LOGIC;

begin

process (CLK,RESET)variable QINT : STD LOGIC VECTOR(11 downto 0 ) ;variable QBLK : STD LOGIC VECTOR(6 downto 0 ) ;begin

i f RESET= ’1 ’ thenQINT := ”000000000000” ;QBLK := ”0000000” ;

e l s i f r i s i n g e d g e (CLK) theni f ENABLE1= ’1 ’ and ENABLE2= ’1 ’ then−− ( re ) s t a r t counterQINT(11 downto 4) := D;QINT(3 downto 0) := ”0000” ;−− ( re ) s t a r t b l a c k counter (80 c y c l e s = 20 p i x e l s )QBLK := ”1010000” ;

e l s i f not (QINT = ”000000000000” ) thenQINT := DEC(QINT) ;i f not (QBLK = ”0000000” ) then

QBLK := DEC(QBLK) ;end i f ;

end i f ;end i f ;

i f (QINT = ”000000000000” ) thens i g busy <= ’0 ’ ;

elses i g busy <= ’1 ’ ;

end i f ;

i f (QBLK = ”0000000” ) thens i g b l a c k <= ’0 ’ ;

elses i g b l a c k <= ’1 ’ ;

end i f ;

s i g s t a t e <= QINT(1 downto 0 ) ;end process ;

process ( s ig busy , s i g s t a t e , s i g b l a c k )begin

A.1. ORIGINAL HORCOUNTER : HORCOUNTER.VHD 65

−− Opt i ca l b l a c k clampCLPDM <= not s i g b l a c k ;CLPOB <= not s i g b l a c k ;PBLK <= ’1 ’ ;

−− busy output s i g n a lBUSY <= s ig busy ;

i f ( s i g busy = ’0 ’ ) then−− Counter i n a c t i v e .H1 <= ’1 ’ ;H2 <= ’0 ’ ;RG <= ’0 ’ ;SHP <= ’1 ’ ;SHD <= ’1 ’ ;DATACLK <= ’1 ’ ;WE <= ’0 ’ ;s i g a i n c <= ’0 ’ ;

else−− Counter a c t i v e .case s i g s t a t e i s

when ”00” =>

H1 <= ’0 ’ ;H2 <= ’1 ’ ;RG <= ’0 ’ ;SHP <= ’1 ’ ;SHD <= ’1 ’ ;DATACLK <= ’1 ’ ;WE <= ’1 ’ ;s i g a i n c <= ’0 ’ ;

when ”11” =>


when ”10” =>

H1 <= ’1 ’ ;H2 <= ’0 ’ ;RG <= ’1 ’ ;SHP <= ’1 ’ ;


SHD <= ’1 ’ ;DATACLK <= ’0 ’ ;WE <= ’0 ’ ;s i g a i n c <= ’1 ’ ;

when ”01” =>


when others => null ;end case ;

end i f ;end process ;

−− de lay s i g a i n c ha l v e a c l o c k per iodde l ay a in c : d f f

PORTMAP(CLK, s i g a i n c , AINC) ;

end BEHAVIOUR;

A.2 Modified HORCOUNTER : horcounter.vhd

−− Modif ied HORCOUNTERl ibrary IEEE ;use IEEE . STD LOGIC 1164 . a l l ;use IEEE . numer ic std . a l l ;use work . user pkg . a l l ;

entity HORCOUNTER i sport (

CLK : in STD LOGIC;RESET : in STD LOGIC;ENABLE1 : in STD LOGIC;ENABLE2 : in STD LOGIC;D : in STD LOGIC VECTOR(7 downto 0 ) ;

ROI ENABLE : in STD LOGIC;LOAD EN : in STD LOGIC;RESET LOAD : in STD LOGIC;RESET ROI : in STD LOGIC;

A.2. MODIFIED HORCOUNTER : HORCOUNTER.VHD 67

BUSY : out STD LOGIC;AINC : out STD LOGIC;H1 : out STD LOGIC;H2 : out STD LOGIC;RG : out STD LOGIC;SHP : out STD LOGIC;SHD : out STD LOGIC;DATACLK : out STD LOGIC;CLPDM : out STD LOGIC;CLPOB : out STD LOGIC;PBLK : out STD LOGIC;WE : out STD LOGIC

) ;end HORCOUNTER;

architecture BEHAVIOUR of HORCOUNTER i scomponent DFF

port (CLK : in STD LOGIC;D : in STD LOGIC;Q : out STD LOGIC) ;

end component ;

signal s i g s t a t e : STD LOGIC VECTOR(1 downto 0 ) ;signal s i g b l a c k : STD LOGIC;signal s i g busy : STD LOGIC;signal s i g a i n c : STD LOGIC;signal s i g we : STD LOGIC;

constant IMAGE WIDTH : STD LOGIC VECTOR := ”1000011100” ; −− 540constant IMAGE HEIGHT: STD LOGIC VECTOR := ”100101100” ; −− 300

−− CCD’ s coord ina t e sSIGNAL CCD X : STD LOGIC VECTOR(9 downto 0) := ”0000000000” ;SIGNAL CCD Y : STD LOGIC VECTOR(8 downto 0) := ”000000000” ;SIGNAL CCD HOR : STD LOGIC VECTOR(11 downto 0) := ”000000000000” ;

−− ROI’ s coord ina t e sSIGNAL BEGIN X, END X : STD LOGIC VECTOR(9 downto 0 ) ;SIGNAL BEGIN Y, END Y : STD LOGIC VECTOR(8 downto 0 ) ;

SIGNAL i s i n s i d e X , i s i n s i d e Y : STD LOGIC := ’ 0 ’ ;SIGNAL i s i n s i d e ROI : STD LOGIC := ’ 0 ’ ;


begin

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− ROI, Define the coord ina t e s o f the ROI−− Implementation #1−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

PROCESS(CLK, RESET LOAD)VARIABLE LOAD I : s t d l o g i c v e c t o r (1 downto 0) := ”00” ;BEGIN

i f RESET LOAD = ’1 ’ thenLOAD I := ”00” ;BEGIN X <= ”0000000000” ; −− = 0END X <= ”1000011011” ; −− = 539BEGIN Y <= ”000000001” ; −− = 1END Y <= ”100101100” ; −− = 300

e l s i f r i s i n g e d g e (CLK) theni f ENABLE2 = ’1 ’ and LOAD EN = ’1 ’ then

i f LOAD I = ”00” thenBEGIN X(7 downto 0) <= D(7 downto 0 ) ;LOAD I := ”01” ;

e l s i f LOAD I = ”01” thenBEGIN X(9 downto 8) <= D(1 downto 0 ) ;BEGIN Y(5 downto 0) <= D(7 downto 2 ) ;LOAD I := ”10” ;

e l s i f LOAD I = ”10” thenBEGIN Y(8 downto 6) <= D(2 downto 0 ) ;END X(4 downto 0) <= D(7 downto 3 ) ;LOAD I := ”11” ;

elseLOAD I := ”00” ;END X(9 downto 5) <= D(4 downto 0 ) ;END Y <= s t d l o g i c v e c t o r ( unsigned (IMAGE HEIGHT)

+ 1 − unsigned (BEGIN Y) ) ;end i f ;

end i f ;end i f ;

END PROCESS;

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− ROI, Cycle #1−− Ca l cu l a t e s CCD X, CCD Y from HORCOUNTER−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−CCD POS X : Process (CLK, RESET)


begini f RESET ROI = ’1 ’ then

CCD HOR <= ”000000000000” ;e l s i f r i s i n g e d g e (CLK) then

i f ENABLE1 = ’1 ’ and ENABLE2 = ’1 ’ then−− Horcounter s i g n a lCCD HOR <= ”000000000001” ;

elseCCD HOR <= INC(CCD HOR) ;

end i f ;end i f ;

end process ;

CCD X <= CCD HOR(11 downto 2 ) ;

CCD POS Y : PROCESS(CLK, RESET)variable CCD VER : STD LOGIC VECTOR(8 downto 0) := ”000000001” ;BEGIN

i f RESET ROI = ’1 ’ thenCCD VER := IMAGE HEIGHT;

e l s i f r i s i n g e d g e (CLK) theni f ENABLE1 = ’1 ’ and ENABLE2 = ’1 ’ then−− Horcounter s i g n a l

i f CCD VER = IMAGE HEIGHT thenCCD VER := ”000000001” ;

elseCCD VER := INC(CCD VER) ;

end i f ;end i f ;

end i f ;

CCD Y <= CCD VER;END PROCESS;

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− ROI, Cycle #2−− Determinates whether the curren t PIXEL−− i s i n s i d e o f the ROI or not ( s t ep 1)−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−IN ROI X : PROCESS(CLK, RESET)BEGIN

i f RESET ROI = ’1 ’ theni s i n s i d e X <= ’0 ’ ;


e l s i f r i s i n g e d g e (CLK) theni f ( unsigned (CCD HOR(11 downto 2) ) >= unsigned (BEGIN X)

and unsigned (CCD HOR(11 downto 2) ) <= unsigned (END X)) theni s i n s i d e X <= ’1 ’ ; −− In s i d e the ROI

elsei s i n s i d e X <= ’0 ’ ; −− Outside the ROI

end i f ;end i f ;

END PROCESS;

IN ROI Y : PROCESS(CLK, RESET)BEGIN

i f RESET ROI = ’1 ’ theni s i n s i d e Y <= ’0 ’ ;

e l s i f r i s i n g e d g e (CLK) theni f ( ( unsigned (CCD Y) >= unsigned (BEGIN Y)

and unsigned (CCD Y) <= unsigned (END Y)) ) theni s i n s i d e Y <= ’1 ’ ; −− In s i d e the ROI

elsei s i n s i d e Y <= ’0 ’ ; −− Outside the ROI

end i f ;end i f ;

END PROCESS;

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− ROI, Cycle #3−− Determinates whether the curren t PIXEL−− i s i n s i d e o f the ROI or not ( s t ep 2)−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−ROI Cycle3 : PROCESS(CLK, RESET)BEGIN

i f RESET ROI = ’1 ’ theni s i n s i d e ROI <= ’1 ’ ;

e l s i f r i s i n g e d g e (CLK) theni s i n s i d e ROI <= ( i s i n s i d e X and i s i n s i d e Y )

OR not ROI ENABLE;end i f ;

END PROCESS;

process (CLK,RESET)variable QINT : STD LOGIC VECTOR(11 downto 0 ) ;variable QBLK : STD LOGIC VECTOR(6 downto 0 ) ;begin

i f RESET= ’1 ’ then


QINT := ”000000000000” ;QBLK := ”0000000” ;

e l s i f r i s i n g e d g e (CLK) theni f ENABLE1= ’1 ’ and ENABLE2= ’1 ’ then−− ( re ) s t a r t counterQINT(11 downto 4) := D;QINT(3 downto 0) := ”0000” ;−− ( re ) s t a r t b l a c k counter (80 c y c l e s = 20 p i x e l s )QBLK := ”1010000” ;

e l s i f not (QINT = ”000000000000” ) thenQINT := DEC(QINT) ;i f not (QBLK = ”0000000” ) then

QBLK := DEC(QBLK) ;end i f ;

end i f ;end i f ;

i f (QINT = ”000000000000” ) thens i g busy <= ’0 ’ ;

elses i g busy <= ’1 ’ ;

end i f ;

i f (QBLK = ”0000000” ) thens i g b l a c k <= ’0 ’ ;

elses i g b l a c k <= ’1 ’ ;

end i f ;

s i g s t a t e <= QINT(1 downto 0 ) ;end process ;

process ( s ig busy , s i g s t a t e , s i g b l a c k )begin

−− Opt i ca l b l a c k clampCLPDM <= not s i g b l a c k ;CLPOB <= not s i g b l a c k ;

PBLK <= ’1 ’ ;

−− busy output s i g n a lBUSY <= s ig busy ;

i f i s i n s i d e ROI = ’1 ’ then−− i n s i d e ROI, o r i g i n a l code


i f ( s i g busy = ’0 ’ ) then−− Counter i n a c t i v e .H1 <= ’1 ’ ;H2 <= ’0 ’ ;RG <= ’0 ’ ;SHP <= ’1 ’ ;SHD <= ’1 ’ ;DATACLK <= ’1 ’ ;WE <= ’0 ’ ;AINC<= ’0 ’ ;

elsecase s i g s t a t e i s −− Counter a c t i v e .

when ”00” =>

H1 <= ’0 ’ ;H2 <= ’1 ’ ;RG <= ’0 ’ ;SHP <= ’1 ’ ;SHD <= ’1 ’ ;DATACLK <= ’1 ’ ;WE <= ’1 ’ ;AINC<= ’0 ’ ;

when ”11” =>


when ”10” =>


when ”01” =>

H1 <= ’1 ’ ;H2 <= ’0 ’ ;RG <= ’0 ’ ;SHP <= ’0 ’ ;SHD <= ’1 ’ ;


DATACLK <= ’1 ’ ;WE <= ’1 ’ ;AINC <= ’0 ’ ;


end i f ;else−− ou t s i d e ROI, modi f ied codei f ( s i g busy = ’0 ’ ) then

−− Counter i n a c t i v e .H1 <= ’1 ’ ;H2 <= ’0 ’ ;RG <= ’0 ’ ;SHP <= ’1 ’ ;SHD <= ’1 ’ ;DATACLK <= ’1 ’ ;WE <= ’0 ’ ;AINC <= ’0 ’ ;

elsecase s i g s t a t e i s −− Counter a c t i v e .

when ”00” =>

H1 <= ’0 ’ ;H2 <= ’1 ’ ;RG <= ’0 ’ ;SHP <= ’1 ’ ;SHD <= ’1 ’ ;DATACLK <= ’1 ’ ;WE <= ’0 ’ ; −− modi f iedAINC<= ’0 ’ ;

when ”11” =>

H1 <= ’0 ’ ;H2 <= ’1 ’ ;RG <= ’0 ’ ;SHP <= ’1 ’ ;SHD <= ’0 ’ ;DATACLK <= ’0 ’ ;WE <= ’0 ’ ; −− modi f iedAINC<= ’0 ’ ;

when ”10” =>

H1 <= ’1 ’ ;H2 <= ’0 ’ ;RG <= ’1 ’ ;SHP <= ’1 ’ ;SHD <= ’1 ’ ;DATACLK <= ’0 ’ ;


WE <= ’0 ’ ;AINC<= ’0 ’ ; −− modi f ied

when ”01” =>

H1 <= ’1 ’ ;H2 <= ’0 ’ ;RG <= ’0 ’ ;SHP <= ’0 ’ ;SHD <= ’1 ’ ;DATACLK <= ’1 ’ ;WE <= ’0 ’ ; −− modi f iedAINC <= ’0 ’ ;


end i f ;end i f ;

end process ;

end BEHAVIOUR;

A.3 CCD ADC : ccd adc.vhd

−− ICX055AL (CCD) + AD9843A (ADC)

l ibrary IEEE ;use IEEE . STD LOGIC 1164 . a l l ;use IEEE . numer ic std . a l l ;use work . user pkg . a l l ;use std . t e x t i o . a l l ;

ENTITY CCD ADC ISGENERIC(CCD id : i n t e g e r := 0 ) ;PORT (−− CCD parametersV1 : IN s t d l o g i c ;V2 : IN s t d l o g i c ;V3 : IN s t d l o g i c ;V4 : IN s t d l o g i c ;H1 : IN s t d l o g i c ;H2 : IN s t d l o g i c ;RG : IN s t d l o g i c ;SUB : IN s t d l o g i c ;

−− ADC parameters

A.3. CCD ADC : CCD ADC.VHD 75

DATACLK : IN s t d l o g i c ;PBLK : IN s t d l o g i c ;CLPOB : IN s t d l o g i c ;SHP : IN s t d l o g i c ;SHD : IN s t d l o g i c ;SLPDM : IN s t d l o g i c ;SL : IN s t d l o g i c ;SDATA : IN s t d l o g i c ;SCK : IN s t d l o g i c ;THREE STATE : IN s t d l o g i c ;D : OUT s t d l o g i c v e c t o r (7 downto 0)

) ;end CCD ADC;

ARCHITECTURE a r ch i OF CCD ADC IS

−− conver t s s t d l o g i c in t o a charac t e rfunction chr ( s l : s t d l o g i c ) return cha rac t e r i svariable c : cha rac t e r ;begin

case s l i swhen ’U’ => c := ’U’ ;when ’X’ => c := ’X’ ;when ’ 0 ’ => c := ’ 0 ’ ;when ’ 1 ’ => c := ’ 1 ’ ;when ’Z ’ => c := ’Z ’ ;when ’W’ => c := ’W’ ;when ’L ’ => c := ’L ’ ;when ’H’ => c := ’H’ ;when ’− ’ => c := ’− ’ ;

end case ;return c ;

end chr ;

−− conver t s s t d l o g i c v e c t o r in t o a s t r i n g ( b inary base )−− ( t h i s a l s o t a k e s care o f the f a c t t h a t the range o f−− a s t r i n g i s na tura l wh i l e a s t d l o g i c v e c t o r may−− have an i n t e g e r range )

function s t r ( s l v : s t d l o g i c v e c t o r ) return s t r i n g i svariable r e s u l t : s t r i n g (1 to s lv ’ l ength ) ;variable r : i n t e g e r ;

begin


r := 1 ;for i in s lv ’ range loop

r e s u l t ( r ) := chr ( s l v ( i ) ) ;r := r + 1 ;

end loop ;return r e s u l t ;

end s t r ;

function t o s t d l o g i c ( c : cha rac t e r ) return s t d l o g i c i svariable s l : s t d l o g i c ;begin

case c i swhen ’U’ =>

s l := ’U’ ;when ’X’ =>

s l := ’X’ ;when ’ 0 ’ =>

s l := ’ 0 ’ ;when ’ 1 ’ =>

s l := ’ 1 ’ ;when ’Z ’ =>

s l := ’Z ’ ;when ’W’ =>

s l := ’W’ ;when ’L ’ =>

s l := ’L ’ ;when ’H’ =>

s l := ’H’ ;when ’− ’ =>

s l := ’− ’ ;when others =>

s l := ’X’ ;end case ;

return s l ;end t o s t d l o g i c ;

−− conver t s a s t r i n g in t o s t d l o g i c v e c t o rfunction t o s t d l o g i c v e c t o r ( s : s t r i n g ) return s t d l o g i c v e c t o r i s

variable s l v : s t d l o g i c v e c t o r ( s ’ high−s ’ low downto 0 ) ;variable k : i n t e g e r ;

begink := s ’ high−s ’ low ;

for i in s ’ range loops l v (k ) := t o s t d l o g i c ( s ( i ) ) ;k := k − 1 ;

A.3. CCD ADC : CCD ADC.VHD 77

end loop ;return s l v ;

end t o s t d l o g i c v e c t o r ;

−− S i gna l sSIGNAL VIDEO : s t d l o g i c v e c t o r (7 downto 0 ) ; −− ana log i c s i g n a lCONSTANT FROM FILE : STD LOGIC := ’ 1 ’ ; −− 1 => read f i l e s

BEGIN

−− genera te in format ions f o r the CCD sensorCCD Data : PROCESS(SHD, THREE STATE)

variable data : s t d l o g i c v e c t o r (7 downto 0) := ”00000000” ;

variable exp id : i n t e g e r := 0 ; −− i d . exper i encevariable ccd name : s t r i n g (1 to 2 ) ;variable exp name : s t r i n g (1 to 1 ) ;

variable f s t a t u s : FILE OPEN STATUS ;FILE CCD FILE : TEXT;variable l i n e b u f : l i n e ;variable r e ad i n f o : i n t e g e r ;variable f i l e name : s t r i n g (1 to 29 ) ;variable i s opened : s t d l o g i c := ’ 0 ’ ;

BEGIN

i f f a l l i n g e d g e (THREE STATE) theni f ( i s opened = ’1 ’ ) then−− c l o s e the current f i l ef i l e c l o s e (CCD FILE ) ;i s opened := ’ 0 ’ ;

end i f ;

−− open a new f i l e−− s t a r t a new exper i enceexp id := exp id + 1 ;

i f (CCD id = 1) thenccd name := ” a ” ;

e l s i f (CCD id = 2) thenccd name := ” b” ;

e l s i f (CCD id = 3) then


ccd name := ” c ” ;else

ccd name := ” d” ;end i f ;

i f ( exp id = 1) thenexp name := ”1” ;

e l s i f ( exp id = 2) thenexp name := ”2” ;





elseexp name := ”7” ;

end i f ;

−− opened the t e x t f i l ef i l e name := ”c :\MATLAB\from CCD\exp” & exp name & ccd name &

” . txt ” ;f i l e o p e n ( f s t a tu s , CCD FILE , f i l e name , READMODE) ;i s opened := ’ 1 ’ ;

e l s i f r i s i n g e d g e (SHD) then

i f (FROM FILE = ’0 ’ ) theni f (CCD id = 1 or CCD id = 3) then−− d i r e c t l i g h tdata := inc ( data ) ;

else−− mirrordata := dec ( data ) ;

end i f ;else

−− data from a f i l ei f f s t a t u s = open ok and i s opened = ’1 ’ then

i f not e n d f i l e (CCD FILE) then−− read datar e ad l i n e (CCD FILE , l i n e b u f ) ;read ( l i n e bu f , r e ad i n f o ) ;

A.4. BS62 BLOCKS : BS62 BLOCKS.VHD AND BS62 1BLOCK.VHD 79

data := s t d l o g i c v e c t o r ( to uns igned ( r ead in f o , 8 ) ) ;else−− c l o s e f i l ef i l e c l o s e (CCD FILE ) ;i s opened := ’ 0 ’ ;

end i f ;end i f ;

end i f ;VIDEO <= data ;

end i f ;END PROCESS;

−− put the analog s i g n a l from the CCD in the ADCADC Data : PROCESS( datac lk )BEGIN

i f r i s i n g e d g e ( datac lk ) thenD <= VIDEO;


END a r ch i ;

A.4 BS62 BLOCKS : BS62 blocks.vhd and BS62 1block.vhd

−− memory b l o c k BS62LV Emulation

l ibrary IEEE ;use IEEE . STD LOGIC 1164 . a l l ;use IEEE . numer ic std . a l l ;use work . user pkg . a l l ;

ENTITY BS62LV ISGENERIC ( ADDRESS WIDTH : i n t e g e r ) ;PORT (

A : IN s t d l o g i c v e c t o r ( (ADDRESS WIDTH − 1) downto 0 ) ;D : INOUT s t d l o g i c v e c t o r (7 downto 0 ) ;NCE : IN s t d l o g i c ;NOE : IN s t d l o g i c ;NWE : IN s t d l o g i c

) ;end BS62LV ;

ARCHITECTURE archi BS62LV OF BS62LV IS


TYPE mem type IS ARRAY(0 TO 2∗∗ADDRESS WIDTH − 1)OF s t d l o g i c v e c t o r (7 downto 0 ) ;

SIGNAL mem : mem type ;BEGIN

c on t r o l p r o c : PROCESS(A, D, NCE, NOE, NWE)

BEGIN

i f (NCE = ’0 ’ ) then

i f (NWE = ’0 ’ ) thenmem( t o i n t e g e r ( unsigned (A) ) ) <= D;D <= ”ZZZZZZZZ” ;

ELSIF (NOE = ’0 ’ ) thenD <= mem( t o i n t e g e r ( unsigned (A) ) ) ;

ELSED <= ”ZZZZZZZZ” ;

END IF ;else

D <= ”ZZZZZZZZ” ;end i f ;

END PROCESS;END archi BS62LV ;

−− memory b l o c k s BS62LV s imu la t i on

l ibrary IEEE ;use IEEE . STD LOGIC 1164 . a l l ;use IEEE . numer ic std . a l l ;use work . user pkg . a l l ;

ENTITY BS62LV BLOCKS ISGENERIC ( ADDRESS WIDTH : INTEGER) ;PORT (

A : IN s t d l o g i c v e c t o r ( (ADDRESS WIDTH − 1) downto 0 ) ;A D, B D , C D, D D : INOUT s t d l o g i c v e c t o r (7 downto 0 ) ;RAMCE : IN s t d l o g i c ;A NOE0, A NOE1 : IN s t d l o g i c ;B NOE0, B NOE1 : IN s t d l o g i c ;C NOE0, C NOE1 : IN s t d l o g i c ;D NOE0, D NOE1 : IN s t d l o g i c ;A NWE0, A NWE1 : IN s t d l o g i c ;B NWE0, B NWE1 : IN s t d l o g i c ;C NWE0, C NWE1 : IN s t d l o g i c ;

A.4. BS62 BLOCKS : BS62 BLOCKS.VHD AND BS62 1BLOCK.VHD 81

D NWE0, D NWE1 : IN s t d l o g i c

) ;END ENTITY BS62LV BLOCKS;

ARCHITECTURE SRAM OF BS62LV BLOCKS IS

COMPONENT BS62LV ISGENERIC ( ADDRESS WIDTH : INTEGER ) ;PORT (

A : IN s t d l o g i c v e c t o r ( (ADDRESS WIDTH − 1) downto 0 ) ;D : INOUT s t d l o g i c v e c t o r (7 downto 0 ) ;NCE: IN s t d l o g i c ;NOE: IN s t d l o g i c ;NWE: IN s t d l o g i c

) ;ENDCOMPONENT;

BEGIN

sram a0 : entity work .BS62LVGENERIC MAP( ADDRESS WIDTH => ADDRESS WIDTH)PORTMAP (

A => A,D => A D,NCE => RAMCE,NOE => A NOE0,NWE => A NWE0

) ;sram a1 : entity work .BS62LV

GENERIC MAP( ADDRESS WIDTH => ADDRESS WIDTH)PORTMAP (

A => A,D => A D,NCE => RAMCE,NOE => A NOE1,NWE => A NWE1

) ;sram b0 : entity work .BS62LV


A => A,D => B D,NCE => RAMCE,NOE => B NOE0,


NWE => B NWE0) ;

sram b1 : entity work .BS62LVGENERIC MAP( ADDRESS WIDTH => ADDRESS WIDTH)PORTMAP (

A => A,D => B D,NCE => RAMCE,NOE => B NOE1,NWE => B NWE1

) ;sram c0 : entity work .BS62LV


A => A,D => C D,NCE => RAMCE,NOE => C NOE0,NWE => C NWE0

) ;sram c1 : entity work .BS62LV


A => A,D => C D,NCE => RAMCE,NOE => C NOE1,NWE => C NWE1

) ;sram d0 : entity work .BS62LV


A => A,D => D D,NCE => RAMCE,NOE => D NOE0,NWE => D NWE0

) ;sram d1 : entity work .BS62LV


A => A,D => D D,NCE => RAMCE,NOE => D NOE1,

A.5. NET2270 : NET2270.VHD 83

NWE => D NWE1) ;

END SRAM;

A.5 net2270 : net2270.vhd

−− expor t USB datal ibrary IEEE ;use IEEE . STD LOGIC 1164 . a l l ;use IEEE . numer ic std . a l l ;use std . t e x t i o . a l l ;use work . user pkg . a l l ;

entity USB CONTROLLER i sport (

D :IN STD LOGIC VECTOR(7 downto 0 ) ;DACK :IN STD LOGIC;NCS :IN STD LOGIC;IOR :IN STD LOGIC;IOW :IN STD LOGIC;USB :OUT STD LOGIC VECTOR(31 downto 0)

) ;end USB CONTROLLER;

architecture NET2270 of USB CONTROLLER i s

signal buf : STD LOGIC VECTOR(31 downto 0 ) ;

signal i : s t d l o g i c v e c t o r (1 downto 0) := ”00” ;

BEGIN

p r o c bu f f e r : PROCESS( dack )

−− outputvariable f open , f c l o s e : STD LOGIC := ’ 0 ’ ;f i l e o u t f i l e : t ex t ;variable f s t a t u s : FILE OPEN STATUS ;variable l i n e b u f : l i n e ; −− b u f f e r l i n e

BEGINi f (NCS = ’0 ’ ) then


−− net2270 a c t i v a t e di f ( f open = ’0 ’ ) then−− open f i l ef i l e o p e n ( f s t a tu s , o u t f i l e , ”USB Output . txt ” , write mode ) ;f open := ’ 1 ’ ; −− f i l e opens

end i f ;

i f ( dack = ’1 ’ and r i s i n g e d g e ( dack ) ) theni <= inc ( i ) ;buf <= buf (23 downto 0) & d ;i f ( i = ”00” ) then

usb <= buf ;i f ( f open = ’1 ’ ) then

wr i t e ( l i n e bu f , t o i n t e g e r ( unsigned ( buf ) ) ) ;w r i t e l i n e ( o u t f i l e , l i n e b u f ) ;

end i f ;

end i f ;end i f ;

e l s i f ( f c l o s e = ’0 ’ and f open = ’1 ’ ) thenf i l e c l o s e ( o u t f i l e ) ;f c l o s e := ’ 1 ’ ;


end NET2270 ;

CCD Appendix AA.1 Introduction

The CCD and CMOS image sensors are both pixelated metal oxide semiconductor. Theyaccumulate signal charge in each pixel proportional to the local illumination intensity.

They have the same basic functions :

1. Optical collection of photons, i.e., a lens.

2. Wavelength discrimination of photos, i.e., a filter.

3. Detector for conversion of photons to electrons, i.e., a photodiode.

4. A method to readout the detectors., i.e., a CCD or CMOS.

A.2 CCD Sensors

Principle

A CCD sensor is generally composed of photodiodes having a vertical overflow drain(VOD), transfer gates, vertical CCD (V-CCD) registers, a horizontal CCD register andan output amplifier[18] (see img A.1).

Figure A.1: Diagram showing 768(H) x 490(V) element interline CCD image sensor andan equivalent unit cell circuit.

85

86 APPENDIX A. CCD APPENDIX

Each photodiode is connected to one half single stage of the vertical CCD register.The transfer gates are controlled by vertical transfer pulses or clocks. The vertical CCDregisters are driven by four-phase double clocking pulses φV 1, φV 2, φV 3 and φV 4. Thehorizontal register is driven by two phase pulses or clocks, φH1, φH2. Finally, the signalreadout is achieved by an amplifier.

Device operation

The device operates in an interlace scanning mode, based on NTFS format. φV 1 andφV 3, which drive the transfer gate and vertical CCD register, have three levels VL andVM and VH , while φV 2 and φV 4, which drive only the vertical CCD register, use twolevels VL and VM . Typical values for VL, VM and VH are -5, 3 and 12 V, respectively.

Readout

When exposure is complete, a CCD transfers each pixel’s charge packet sequentially toa common output (see fig. A.2).

Electron-to-VoltageConversion

Photon-to-ElectronConversion

Charge-Coupled DeviceImage Sensor

Figure A.2: CCD - Pixels transfer

A.3. CMOS SENSORS 87

A.3 CMOS Sensors

Principle

When exposure is complete, a CMOS imager (fig. A.3) converts charge to voltage at thepixel. The image sensor[7] consists of an array of pixels that are typically selected a rowat a time by row select logic. This can be either a shift register or a decoder. The pixelsare read out to vertical column busses that connect the selected row of pixels to a bankof analog signal processors.

GainEl

ectr

on-t

o-Vo

ltage

Conv

ersio

nPh

oton

-to-

Elec

tron

Conv

ersio

n

Column Amps

Column Mux

Row

Acc

ess

Figure A.3: A CMOS sensor

A.4 Comparison CCD & CMOS

The difference in readout techniques between CCD and CMOS has significantimplications[14] for sensor architecture, capabilities and limitations.

Eight attributes characterize image-sensor performance :

• Responsivity : The amount of signal the sensor delivers per unit of input opticalenergy. CMOS imagers are marginally superior to CCDs, in general, because gainelements are easier to place on a CMOS image sensor.

• Dynamic Range : The ratio of a pixel’s saturation level to its signal threshold. Itgives CCDs an advantage by about a factor of two in comparable circumstances.CCDs still enjoy significant noise advantages over CMOS imagers because of quietersensor substrates (less on-chip circuitry).

88 APPENDIX A. CCD APPENDIX

• Shuttering : The ability to start and stop exposure arbitrarily. CCDs can deliversuperior electronic shuttering, with little fill-factor compromise, even in small-pixelimage sensors.

• Speed : An area in which CMOS arguably has the advantage over CCDs becauseall camera functions can be placed on the image sensors.

• Windowing : One unique capability of CMOS technology is the ability to readouta portion of the image sensor. This allows elevated frame or line rates for smallROI. CCDs generally have limited abilities in windowing.

• Antiblooming : The ability to gracefully drain localized overexposure without com-promising the rest of the image in the sensor. CMOS generally has natural bloom-ing immunity. CCDs, on the other hand, require specific engineering to achievethis capabilities.

msc thesis - tu delftce-publications.et.tudelft.nl/publications/332_the... · 2012-07-19 · msc...

Documents