Embedded Systems Architecture

IR Remote Tic-Tac-ToeAnuwat SaetowJohn O’Donnell

Chris SmallwoodOlga Kardonik

May 14, 2011

Project Outline

• Goal: Develop a tic-tac-toe game using the iMX21 and TLL5000 platform capable of using any IR input device as a controller and output to VGA monitor

• Two Main Tasks– IR input (application code, driver, hardware)– VGA output (protocol, verilog)

Task Breakdown

• Anuwat Saetow, Olga Kardonik:– Learn VGA Protocol– Code graphics in Verilog– Test code for graphics generation & output

• John O’Donnell, Chris Smallwood:– Build & test IR Sensor– Develop driver code– Develop button sensing algorithm and game code

in user application

Project Diagram

Remote Control

Infra-Red Detector

TLL5000 monitor

VGA

user application:game code,

mode control,Calibration,

Signal decode

Port E[7]

FPGA:VGA Controller

ir_sensor.ko:GPIO ctrl reg,Sample PE7,Output FPGA

IR waves

Mezzanine Connector

IR Input

John O’DonnellChris Smallwood

Components of IR Input

• Hardware– Obtain all necessary hardware components to build IR circuit– Test hardware to ensure functionality

• Application code– Calibrates the remote– Runs the game– Decodes the button presses during the game

• Driver code– Sets the GPIO ctrl registers during module initialization– Samples PE[7] (bit used as input from IR circuit)– Writes display data to FPGA

Hardware Components

• iMX21• TLL5000 platform• RS 276-640 IR Detector• 7805 +5V Voltage Regulator• 1 x 3.3uF C• 3 x 1Mohm R• 1 x diode• 1 x 9V battery• IR remote

IR Sensor Schematic

1

3

2

1

IR DetectorVCC

GND

OUT

OUT GND IN

7805 5V Regulator

3.3uF

2Mohm

1Mohm

+9V

GND

To GPIO input:

RS 276-640 iMX21 PORTE[7](mezz board J5 pin 15)

IR Sensor Hardware

Waveform of one button push

Duration 37ms, minimum pulse width 400us, low preamble pulse 2.7ms

Start Sequence Zoom-in

Idle state is high; low preamble pulse = 2.7msPreamble pulse is 786 counts, so sample rate is 291kHz (3.44us)Min pulse width is 400us, so min sample count = 116

End of Button Sequence

When signal stays high for over (8 * start pulse), end detection.

Read Button Routine

Read GPIO

Statechange?

Data start? Data start?

COUNT+ COUNT=0 PULSE[n] = COUNTn+COUNT=0

COUNT=0Data start=1

COUNT>X?

YNY

N

Y

N

RETURN

Data start=0

iMX21 GPIO Port Logic

PORTE[7]-IR sensor

SSR: GPIO Input Register

User Application

Responsible for:• Initializing IR driver• Calibrating buttons• Determining button pushes after calibration (during game)• Running game code• Sending encoded display data through driver to FPGA

User Application Flow ChartProgram Start

Calibration

Game Start

X’s turn X win? Tie? O’s turn O

win?

X win O win

N

NNYYY

Calibration

• During calibration stage, each button is pressed and its pulse data stored in a 9x100 two-dimensional array calibration_button_archive[9][100]– 9 different buttons– Stores widths of up to 100 transitions per button push

• Unused elements set to 0

Example calibration_button_archive[][] array:

button\n 1 2 3 4 5 6 …

#1 785 114 98 240 120 126 …

#2 779 120 99 122 236 243 …

#3 780 240 117 90 112 239 …

… … … … … … … …

Determining Button Pushes• Used Sum-of-Absolute-Differences (SAD) algorithm to determine correct button

pushes during the game• Example:Current Button Data

Calibration_button_archive[][]

Deltas

SAD#1: 358 SAD#2: 36 SAD#3: 271

Button 775 120 103 115 219 239

#1 785 114 98 240 120 126#2 779 120 99 122 236 243#3 780 240 117 90 112 239

#1 10 6 5 125 99 113

#2 4 0 4 7 17 4

#3 5 120 14 25 107 0

IR Driver

• Responsible for:– Device registration (char)– Mapping to FPGA and GPIO control registers– Setting GPIO control registers

• Done in module initialization, no need to change during game

– Readl() Sample Status Register (SSR) to retrieve IR input• via ioctl()

– Writel() encoded display data to FPGA• Via ioctl()

Memory Mapping (GPIO)#define GPIOE_BASE 0x10015400#define GPIOE_MASK 0x00003F#define GPIOE_SIZE 0x40

/********** Perform Memory REMAP for GPIO ************/ if (check_mem_region(GPIOE_BASE, GPIOE_SIZE)) { printk(KERN_ERR "IRsensor: Unable to acquire GPIOE address.\n"); return -EBUSY; }

request_mem_region(GPIOE_BASE, GPIOE_SIZE, IRsensor_NAME); gpio_ptr = (volatile unsigned int *)__ioremap(GPIOE_BASE, GPIOE_SIZE, 0);

if (!gpio_ptr) { printk(KERN_ERR "IRsensor: Unable to map GPIOE.\n"); release_mem_region(GPIOE_BASE, GPIOE_SIZE);

return -EBUSY; }

GPIO Ports

• Six total GPIO ports, labeled A-F– We used port E because we found some pins on the mezzanine that

mapped to it

• Each GPIO port has its own set of 17 control registers– Only three control registers for port E needed for our project

GPIO Control Registers

• GPIO IN USE Register (GIUS)– Determines whether pin is being utilized as GPIO

or some other peripheral function• Data Direction Register (DDIR)– Determines whether each pin of port is

input/output• Pull-Up Enable Register (PUEN)– Determines whether each pin of port is pulled-up

or tri-stated when not driven internally or externally

Other GPIO Ctrl Regs (not used)

• Output Configuration Registers (OCR)– When pin is set as output, determines what driver

of pin is• Input Configuration Registers (ICONF)– Controls what data drives “A_out” & “B_out”

• Interrupt Configuration Registers (ICR)– Used to setup GPIO interrupts

VGA Output

Anuwat SaetowOlga Kardonik

VGA specifications

• VGA – Video Graphics Array. • VGA video signal contains 5 active signals: two digital signals for synchronization of the video - HSYNC

and VSYNC; three analog signals (0-0.7 v) to control the color – R, G, B. • Other colors are produced by changing the analog levels of R,

G, and B.

640x480 mode

The video signal must redraw the entire screen 60 times per second to provide for motion effect and to reduce flicker. This period is called refresh rate. 60 Hz refresh rate which used in 640x480 mode is corresponding to 40 ns per pixel or to 25 MHz pixel clock frequency.

1. VSYNC signal tells the monitor to start displaying a new image and the monitor starts in the upper left corner with pixel (0,0).2. HSYNC signal tells the monitor to refresh another row of 640 pixels.3. After 480 rows of pixels are refreshed with 480 HSYNC signals, a VSYNC resets the monitor to the upper left corner and the process continues.

Horizontal and Vertical Timing

Horizontal Parameter

Time, us pixels

Visible area 25.4 640

Front porch 0.6 16

Sync pulse width

3.8 96

Back porch 1.9 48

Whole line 31.8 800

Vertical Parameter

Time, ms lines

Visible area 15.253 480

Front porch 0.318 10

Sync pulse width

0.064 2

Back porch 1.049 33

Whole frame

16.683 525

Pixel clock frequency 25.175 MHz

Horizontal frequency (HSYNC) 31.46 kHz

Screen refresh (VSYNC) 60 Hz

Sync and Data Signals

A B C VIDEO DATA

Front porch Back porch

Sync pulse width

HSYNC and VSYNC signals have negative polarity in 640x480, 60 Hz resolution.

Data blank interval

Schematic of the “Output” part

FPGAHSYNC, VSYNC generator

Video data generator

ADV 7125 DAC

monitor

VGA

Clock divider

3x [7:0]

FPGA_CLK3 is used for the generation of clock signal for ADV7125 .Monitor is divided to 9 virtual fields to accommodate the game.

Digital to Analog Converter and 15-pin high-density D-sub VGA connector

Mapping between FPGA and the digital part of the DAC ADV7125

NET VGA_VSYNC LOC=N7; #VDAC_VSYNC NET VGA_HSYNC LOC=N3; #VDAC_HSYNC NET VGA_CLOCK LOC=R1; #VDAC_IO26NET VGA_SYNC_N LOC=V6; #VDAC_IO1 NET VGA_BLANK_N LOC=U7; #VDAC_I02 NET VGA_PSAVE_N LOC=R5; #VDAC_IO22

NET VGA_B[0] LOC=P8; #VDAC_IO27NET VGA_B[1] LOC=W1; #VDAC_IO0 NET VGA_B[2] LOC=V2; #VDAC_IO6 NET VGA_B[3] LOC=U2; #VDAC_IO11 NET VGA_B[4] LOC=U1; #VDAC_IO12NET VGA_B[5] LOC=T2; #VDAC_IO17NET VGA_B[6] LOC=T1; #VDAC_IO18NET VGA_B[7] LOC=R2; #VDAC_IO25

NET VGA_G[0] LOC=V5; #VDAC_IO3 NET VGA_G[1] LOC=U5; #VDAC_IO8 NET VGA_G[2] LOC=U6; #VDAC_IO7 NET VGA_G[3] LOC=T6; #VDAC_IO15NET VGA_G[4] LOC=R6; #VDAC_IO21NET VGA_G[5] LOC=R8; #VDAC_IO19NET VGA_G[6] LOC=T8; #VDAC_IO13NET VGA_G[7] LOC=T7; #VDAC_IO14

NET VGA_R[0] LOC=R3; #VDAC_IO24NET VGA_R[1] LOC=T5; #VDAC_IO16NET VGA_R[2] LOC=T4; #VDAC_IO23NET VGA_R[3] LOC=U3; #VDAC_IO10NET VGA_R[4] LOC=U4; #VDAC_IO9 NET VGA_R[5] LOC=V3; #VDAC_IO5 NET VGA_R[6] LOC=V4; #VDAC_IO4 NET VGA_R[7] LOC=R7; #VDAC_IO20

Testing the Output • To test the output part before it was integrated with the real input

part we wrote an interactive simulator of the input and the code of Tic-Tac-Toe game.

int check_win(){ int i; for (i = 0; i < 3; i++) { if((matrix[i*3] == matrix[1 + i*3]) && (matrix[1 + i*3] == matrix[2 + i*3]) && (matrix[1 + i*3] != 0))

return WIN;

if((matrix[0 + i] == matrix[3 + i]) && (matrix[3 + i] == matrix[6 + i]) && (matrix[0 + i] != 0))return WIN;

} if((matrix[0] == matrix[4]) && (matrix[4] == matrix[8]) && (matrix[0] != 0))

return WIN;

if((matrix[2] == matrix[4]) && (matrix[4] == matrix[6]) && (matrix[2] != 0))return WIN;

return NEXT;}

0 1 23 4 56 7 8

Encoding between User and FPGA

field9 field8 field7 field6 field5 field4 field3 field2 field1

17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

color color color color color color color color color

Mode of operation

21 20 19 18 bits

1 1 1 1 Grid, Calibration and Active mode

0 0 0 1 X or Red Win display

0 0 1 0 O or Blue Win display

0 0 1 1 Invalid input display

0 0 0 0 Blank display

user_application.c sends 22-bit data to the FPGA with the following mapping:

Field Data

0 0 black

0 1 red

1 0 blue

1 1 green

Progress in the game

We started from the simple boxes colored red and blue

Waveforms: HSYNC and Data

Note, the frequency on the picture corresponds to the mode specifications.

Waveforms: VSYNC and Data

Note, delta X on the picture corresponds to “front porch + pulse width + back porch” of the mode specifications.

Parameters in top.v for the VGA driver

// -----------------------------------------------------------------------------//VGA 640x480 @ 60Hz: V_refresh 31.46875kHz, Pix_freq 25.175MHz// -----------------------------------------------------------------------------param h_disp = 640; // visible displayparam h_front = 16; // front porchparam h_width = 96; // pulse widthparam h_back = 48; // back porchparam h_sum = 800; // total pulseparam v_disp = 480; // visible displayparam v_front = 10; // front porchparam v_width = 2; // pulse widthparam v_back = 33; // back porchparam v_sum = 525; // total pulse

Parameters in top.v for the game visualization

// -----------------------------------------------------------------------------// Grid, X, O// -----------------------------------------------------------------------------localparam v_grid1 = 160; localparam v_grid2 = 320;localparam h_grid1 = 213; localparam h_grid2 = 427; localparam grid_width = 8;localparam x_width = 12;localparam x_trim = 60;localparam o_radius_1 = 60**2;localparam o_radius_2 = 45**2;localparam center_1y = 73;localparam center_2y = 240;localparam center_3y = 407;localparam center_1x = 100;localparam center_2x = 320;localparam center_3x = 540;

Equations for Grid, X, and Oif (((vpix_count >= (v_grid1 - grid_width)) && (vpix_count <= (v_grid1 + grid_width))) || ((vpix_count >= (v_grid2 - grid_width)) && (vpix_count <= (v_grid2 + grid_width)))) begin

if ((((vpix_count < ((hpix_count - (center_1x - x_width)) + center_1y)) && (vpix_count > ((hpix_count - (center_1x + x_width)) + center_1y))) || ((vpix_count < (((center_1x + x_width)) - hpix_count + center_1y)) && (vpix_count > (((center_1x - x_width)) - hpix_count + center_1y)))) && ((vpix_count < (center_1y + x_trim)) &&(vpix_count > (center_1y - x_trim)) &&(hpix_count < (center_1x + x_trim)) && (hpix_count > (center_1x - x_trim)))) begin

if (((((center_1x - hpix_count)*(center_1x - hpix_count)) +((center_1y - vpix_count)*(center_1y - vpix_count))) < o_radius_1) && ((((center_1x - hpix_count)*(center_1x - hpix_count)) +((center_1y - vpix_count)*(center_1y - vpix_count))) > o_radius_2)) begin

Previously recorded demo

… btw, we had really big success in the lab. People around asked to try the game!

Summary• Problem Statement: Build a game that can use any IR remote control and VGA

monitor to demonstrate hardware/software codesign, using simple and robust input interfaces.

• Most consumer devices require dedicated controller and display devices, adding to product cost. This project was not scoped to improve on the state of the art in games.

• Project achieved the desired goal of implementing a simple game using the lab hardware and simple user interfaces.

• Lessons learned:o Reached limit of FPGA resources creating VGA graphics.o Lost time due to starting with SVGA standard, should have used onboard VGA test pattern first.o Needed to minimize code in read-button algorithm to maximize sampling resolution.

• References:o VGA Standard: Wikipedia & various web-based resources.o MC9328MX21RM User Manualo “Multi Infrared Remote Control Cloner”, Anuwat Saetow, EE464H Senior Design Project, Fall 2006