EngSci Adventure Game ECE532 Digital Systems Design Students: Simeng Ding

Li Liu

Chen Xing Zhao

Submitted to:

Prof. Paul Chow

TA. Vincent Mirian

Date of Submission:

April 9, 2012

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

2

Table of Contents 1.0 Overview ............................................................................................................................................... 3

1.1 Project Motivation ............................................................................................................................ 3

1.2 Background and Goals ...................................................................................................................... 3

1.3 Block Diagram ................................................................................................................................... 4

1.4 IP Description .................................................................................................................................... 4

2.0 Outcome ............................................................................................................................................... 6

2.1 Accomplished Original Goals ............................................................................................................. 6

2.2 Review of Work Schedule ................................................................................................................. 8

2.3 Final Product ..................................................................................................................................... 9

2.4 Future Improvement ....................................................................................................................... 10

3.0 Description of Blocks........................................................................................................................... 10

3.1 MicroBlaze Soft Processor (Software) ............................................................................................. 10

3.1.1 Overview .................................................................................................................................. 10

3.1.2 Execute Game Logic ................................................................................................................. 11

3.1.3 Control and Interact with Detector Module ............................................................................. 13

3.1.4 Interaction with tone_gen Module .......................................................................................... 14

3.1.5 Interaction with the clear screen module ................................................................................ 14

3.1.6 Interaction with the renderer module ..................................................................................... 14

3.2 Video_to_ram ................................................................................................................................. 14

3.3 TFT Controller ................................................................................................................................. 14

3.4 MPMC & DDR Memory ................................................................................................................... 14

3.5 PLB Bus Interface of Custom IP ....................................................................................................... 15

3.6 Detector Module ............................................................................................................................. 16

3.7 Audio Module ................................................................................................................................. 17

3.8 Renderer Module ............................................................................................................................ 17

3.9 Clear Screen Module ....................................................................................................................... 19

4.0 Description of Design Tree .................................................................................................................. 20

5.0 Tips and Tricks ..................................................................................................................................... 21

6.0 Conclusion........................................................................................................................................... 21

7.0 References .......................................................................................................................................... 22

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

3

1.0 Overview This section provides an overview of the project design.

1.1 Project Motivation

After spending 5 years in Engineering Science, we have grown a strong motivation to put in use

what EngSci has taught us. The idea of this project is to combine our knowledge in hardware,

software and game design with the spirit of Engineering Science. We would try to make EngSci

a culture instead of being a single program by making a product that is both playable and

featured with EngSci lifestyle. We hope that our product could capture the attention of many

EngSci Alumni to remind them the happy and painful time back then as well as grabbing

potential EngSci students attention.

This document provides a thorough understanding of the project “EngSci Adventure” in terms

of its functionalities and technical design. Many block diagrams other visualizations are

provided in this document for the purpose of simplification.

1.2 Background and Goals

For over 20 years, Super Mario Bros. was the best-selling video game of all time (before being

outsold by Nintendo's own Wii Sports in 2009).[1] In Super Mario Bros., the player controls

Mario (and in a two-player game, a second player controls Mario's brother Luigi) as he travels

through the Mushroom Kingdom in order to rescue Princess Toadstool from the antagonist

Bowser. The player moves from the left side of the screen to the right side in order to get to the

flag pole at the end of each level. The game world has coins scattered around it for Mario to

collect, and special bricks marked with a question mark ("?"), which when hit from below by

Mario, may reveal more coins or a special item.

Kinect, originally known by the code name Project Natal,[2] is a motion sensing input device by

Microsoft for the Xbox 360 video game console and Windows PCs. After selling a total of 8

million units in its first 60 days, the Kinect holds the Guinness World Record of being the

"fastest selling consumer electronics device".[3][4][5] Based around a webcam-style add-on

peripheral for the Xbox 360 console, it enables users to control and interact with the Xbox 360

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

4

without the need to touch a game controller, through a natural user interface using gestures

and spoken commands.[6]

The goal of this proposed project is to combine the game plot of Super Mario Bros and the

motion sensing feature of Kinect to create a new game “EngSci Adventure”.

1.3 Block Diagram

BRAM

DATA

BRAM

INSTR

MICROBLAZE

LMB LMB

PLB

URART

(DEBUGGING)

MEMORY (DDR)

MEMORY CONTROLLER

CLEAR SCREEN FRAME RENDERERVIDEO TO RAMMOTION

DETECTOR

VIDEO_OUT

(XPS_TFT)

PLB PLB PLB PLB PLB

CUSTOM LOGIC+

AUDIO CODEC

GPIO GPIO

VIDEO DECODER CHIP

COMPOSITE VIDEO

GPIO

SPEAKER

SVGA

LEGENG

CUSTOM IP

EXISTING IP

1.4 IP Description

Block Function Origin

IP Core Inside the FPGA

Video to ram Stores digital video signal into DDR-SDRAM through MPMC

Obtained from course discussion board

detector

Reads video frames from memory, processes the colour of each pixel to identify the location of centroid of the beacon, and writes the location into the memory

Group

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

5

Microblaze + dlmb and ilmb

Processor core for setting up video_to_ram IP and audio IP, storing game logic, and drawing the game

Xilinx IP

PLB_bus

6 PLB bus are used: 1: Used by Microblaze to communicate with IIC, GPIO, tft, and MPMC 2:Used by video to ram to save frames to memory 3: Used by detector to read pixels from and writing the result back to memory 4: Used by TFT to read pixels from memory for display 5: Used by renderer to read objects saved in memory and rendering them to frame buffer in memory 6: Used by clear_screen to do burst write to frame buffers in memory

Xilinx IP

Renderer Draw all the objects that need to be displayed on each frame according to a list of object type and location stored inside the memory

Group

TFT_controller

Reads a video frame from memory and sends to the monitor to be displayed

Xilinx IP

MPMC_controller

Multiport memory controller that regulates memory read/write operations

Xilinx IP

XGpio

Configures the tone_gen audio core, and adds a hardware resets to the game

Xilinx IP

Clear_screen Clear all the object on the screen before drawing a frame so that the objects for the next incoming frame can be drawn on empty screen

Group

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

6

tone_gen

Receives audio commands from Microblaze through 4 parallel GPIOs and generates sound

Past Project 2010: Human Pong

Devices outside the FPGA

DDR memory The memory stores the incoming video frame, output video frame, and beacon location

Given

Video Camera Captures the video and outputs to the video decode chip

Group

VGA Output video Given

Video Decode chip Converts the frame by frame composite video signal captured by the camera into digital signal. Digital signal is then fed to the video_to_ram IP block

Given

Speakers Output audio Given

2.0 Outcome This section describes the original goals and the actual outcome of the project as well as

explanation decision point

2.1 Accomplished Original Goals

The accomplished goals from the original goals list are highlighted

All primary goals have been completed.

1) Basic game play including rendering all graphics in colour, character moving, collision

with obstacles, applying obstacle effects, exams, keeping score.

2) Detection of movement by tracking markers.

3) Begin, and end game controls.

4) Achieve high frame rate (30fps).

Some details of the primary goals have been adjusted:

1) The winning requirement and mark awarded for answering the exam is modified, to an

appropriate difficulty level. There are now 60 marks in the level, 40 marks for correct

exam answer, 20 marks for 2nd best answer; total of 80 marks is needed to pass level.

2) The duration of the effects for bed and beer are adjusted to 3 seconds each. This

adjustment was made to improve the game play.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

7

3) The leaning motion has been modified to a step to left or right. During the design, we

felt that leaning and jumping maybe difficult for player to do at the same time. So a

move to left or right is a more user friendly control.

4) Clear screen is now done in a separate block from the renderer. When implementing

the renderer, the arbitration between read and write was complex. So instead of

introducing further complexity to the render block, we created a separate block just for

clearing the screen.

Most of the minor features are completed as well:

1) Background music is played when the game is running.

2) Timer added to the game to show time remaining in the level

3) Score shown on screen to keep track of the score as player plays game

4) Game pause/resume is implemented.

Some of the extra features and minor features were not implemented in the system by the time

this report is written.

1) Sound effect for jumping and collecting marks are not implemented. Implementation

requires a second audio channel. It can be done quite easily with some extra work. The

majority of the work on this effect is not related to software or hardware but about

artistic understanding. We have spent quite a lot of time deciding which music to play as

the background music and to transcribe the notes to machine codes. The addition of

sound effect for jumping and collecting marks would require a lot more time on issue

that is not related to the technical aspect of the project. Since the demo day is coming

up, we focused on debugging the existing system. If time permits, the sound effects will

be added. Currently we are still using existing music as background music. We would

aim for more EngSci featured music (e.g. write out own music and sound effect for

jumping) later on.

2) One stage of the game is currently implemented. Since the level is difficult enough that

no one so far has been able to pass it (which is similar to passing EngSci) there is no

need at the moment for a new level design. Moreover, our initial goal was not to design

something that is easy to pass since the more difficult it is, the more interested people

would be in passing it. The new levels can be added easily constructing and importing

the new level file. If after all the debugging and there is still time, we will add extra

levels to the game.

In conclusion, we have met all the primary design goals. Most minor and extra features that are

associated with game play are fully implemented and tested.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

8

2.2 Review of Work Schedule

The milestone was adjusted to focus on hardware first then the software. Initially, the proposed

milestone planned to work on the software and hardware side in parallel. After consulting with

the TA, TA suggested to get the hardware implementation working first, and then focus on the

software.

Throughout the course of the project, the milestone goals set for most of the weeks were met.

Initially, we were on track and meeting the all the milestones on time. During reading week, we

underestimated the difficulty of the renderer implementation. We were delayed on the

renderer implementation. After the reading week, we actively consulted the TA and revised the

design of the renderer block to make the render block functional. Once the render block is

functional, the clear screen and detector block were implemented with ease. This is because

the clear screen, detector and renderer block shared a lot of functionality in common.

In the end, we were able to catch up to the schedule and finish before the demo. We were able

to achieve it because when preparing the proposed milestone, buffer room was left in place for

correcting potential mistakes and overcoming road blocks. The blocks are developed

individually and tested in individual projects to speed up the debugging process. As well, due to

previous experience in programming games on other platform, the software implementation

went very well and finished faster than expected.

The table below provides a summary of the proposed and the actual milestones.

Date Proposed Weekly Milestones Actual Progress

Feb 1, 2012 Lab Test

Lab Test

Feb 8, 2012

VGA video output module completed, able to output coloured screen to monitor; Basic functions of memory controller (MPMC) modules completed and tested, able to read/write data from MicroBlaze processor.

VGA output, MPMC integrated with the system. MicroBlaze is capable of reading and writing data from and to memory via MPMC. VGA out can display the frame buffer in memory on screen.

Feb 15, 2012

Start working on render block. Load configuration files to RAM. Implement C program that passes an array of object to renderer. Incorporate tone_gen block to the design.

Configuration file is able to be loaded into the memory via XMD. Rendering can be done in software. Tone_gen integrated into the system. Simple notes can be played via Tone_gen. No progress was done for renderer block.

Feb 22, 2012

(Reading Week)

(Reading Week)

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

9

Feb 29, 2012 Finish renderer. Implement colour detector and video to ram to detect the marker on player’s body. Design level layout.

Detector is implemented to detect for a specific colour in a frame buffer. Render FSM designed. PLB is decided to be used as interface for renderer.

Mar 7, 2012

Design graphics for the game and put them in the format that renderer can render onto the screen. Implement game simulation step.

Video to ram implemented. Detector now can detect red pixels in live video feed. Render implemented, and is controlled by 2 GPIO. Software control renderer to render desired graphic on screen. Simulations of character movement, collision detection are implemented.

Mar 14, 2012

Implement majority of the game logic. The game is capable of running from beginning of level to the end with pre-programmed inputs

Renderer and detector block integrated into a single project. Implemented level loading and game play. The game scene is rendered according to level configuration. Decided to implement clear_screen in hardware.

Mar 21, 2012

Integrate software with hardware inputs. Implement exam, begin game, end game screen. Implement pause/resume and replay controls.

Implemented jump detection in software. Effects of obstacles are implemented.

Mar 28, 2012 Testing and debugging. Work on additional features if time permits

Implemented Clear_screen in hardware. Start, end, pause screens are impelemented. Timer, score are displayed on screen.

Apr 2-5, 2012

ECE Design Fair. All features should be implemented. Work on debugging and final report

Exam screen implemented. Background music implemented. Software code optimized to achieve high frame rate. (30fps)

Apr 9, 2012 Individual and group report due. Individual and group report due

Apr 11, 2012 Final demonstration Final demonstration

2.3 Final Product

The final product is quite successful. The game runs smoothly at high frame rate. This game is

excellent for exercise and education purpose. During the game play, there is a lot of jumping

needs to take place, which is quite tiring. As well, there is a question at the end of each level

that can be used to quiz player.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

10

2.4 Future Improvement

There are few improvements possible on the current system.

Right now there’s only one difficulty level. The game restart at the beginning of the level once

the end of the exam session is reached. Additional levels can be easily added by rearranging the

coordinates of the objects in the time frame.

The graphics of the game can be improved further. The size of all the alphabets are the same

and the variety of obstacles are limited. The graphics can be easily added by designing the

image in photoshop and then using the helper program to convert it into machine readable

format.

Another interesting feature worth exploring could be making this game multiplayer. Now after

we have made this game interactive, there is no reason why we should limit this game to single

player. Using two cameras or beacons of different color for different players, we can foresee

that it would be very interesting to see several character all racing towards the end of the game.

The two players need to be displayed on separate half of the screen. That would also make the

game much more challenging and playable.

Last but not least, there is right now only one audio channel added for playing background

music. Additional audio channels can be added for special sound effect when characters bump

into obstacles or to make the background music sounds fuller.

3.0 Description of Blocks All descriptions provided below are based on documentations of Xilinx EDK/ISE and IP cores of related blocks. For more information please refer to [3] and [4].

3.1 MicroBlaze Soft Processor (Software)

3.1.1 Overview

The MicroBlaze is a built-in soft processor that was created for use in the FPGA by Xilinx. The processor is automatically included when we start a project in EDK. It is capable of executing C code. The MicroBlaze is used to run the software implementation of the game. Its main roles are:

1) Execute game logic. 2) Control and interact with detector module. 3) Control and interact with renderer module. 4) Control and interact with clear screen module. 5) Control and interact with sound module.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

11

3.1.2 Execute Game Logic

The main purpose of the software is to execute the main game logic. The game logic is divided into the following subsections.

1) Level setup 2) Game simulation step

a. Obtain user input b. Play Music c. Simulate character action d. Collision detection with obstacles e. Add objects to render list f. Check for end game condition

3) Game start, replay, exam control

Level Setup Before the going into a new level, the level configuration, collision list and background music data are first loaded into the stack for faster access. The game play variables such as the score and timer are reset to default value. Game Simulation Step Before rendering every frame, the character location and action needs to be calculated. The software determines the music note to play, the objects to draw on the screen, effects to apply to character, and end game condition. First, the music note for background music is determined. The music information is stored in an array. Each element in the array is a Note struct that contains a value for key and a value for duration. The software keeps track of the current note being played with a counter. After sending the play command for each note, the counter is incremented. If the counter exceeds the max value, then the counter resets to 0 and replays the music. The software sends a play command only if the audio block is done playing the previous command. Then, user input is acquired to determine the character’s action. The character’s location is represented by 2 integer variables, specifying the x and y location. The x direction movement rate is a global variable that is added to the x location at every frame. The y coordinate is determined by the jump action. User input is used to determine if there is a jump. If a jump is detected, then the character enters a jump mode. The character’s location in next frame is determined by:

No Jumping: (X,Y) = (X0 + Vx, 480) Jumping: (X,Y) = (X0 + Vx, JumpTrajectory[i])

The JumpTrajectory array contains pre-calculate Y coordinate of the character’s jump trajectory. Once jumping is initiated, the values in JumpTrajectory are used to determine the locations of the character for the duration of the jump. In the event that when the input is invalid, the game pauses and resumes only if a jump is detected.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

12

After the character’s location is updated, the software determines the objects that will appear on the screen. A level configuration array is used to store the objects that appear in the level. The array is populated when levelsetup() is called. The content is a copy the level data in the memory. Each entry in the array stores location and the type of the object. The software goes through the level configuration array and determines which objects will appear on the screen. For each object that will appear on the screen, collision detection with the character is done. The character and the object collide if

1) Vertical distance between the center of character and objects is smaller than 70% of the sum of character’s and object’s height/2.

2) Horizontal distance between the center of character and objects is smaller than 70% of the sum of character’s and object’s width/2.

Collide if: dy < 0.7x(h0/2+h1/2) and dx < 0.7x(w0/2+w1/2)

If the character has collided with an object, the effect of the object is applied to the character by modifying the character’s speed, or incrementing the score. A counter is used to apply effects that last for more than 1 frame. The counter is set to the number of frame that the effect will last. The counter is decremented by 1 for each frame, and when the counter reaches 0, the effect is removed. All the objects that is determined to appear on the screen but have not collided with the character is then added to the render list for rendering. At the end of each game simulation step, the end game is checked. The timer is decremented and checked if there is still time left for the player. As well, the current location of the character is checked to see if the character has reached the end of the level. In order to speed up the software execution, several optimizations in software have been made. First, all the calculation are done using integers. This allows us to take advantage of using bit shifts as multiplication and division. All the multiplication and division are done in powers of 2 if possible.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

13

Secondly, when going through the level configuration array to determine which objects to display on screen, the software will start at the location that is closest to the character. This location is updated at each call to the simulation loop. By doing so, it is not necessary to go through the entire list before drawing every frame to determine objects to check for collision and render. Instead, only the few objects that is going to be in screen is checked for collision. Game Start, End, Exam Control The game start, end and exam screen are stored in the same format as level configuration. It is consisted of a list of objects (letters) and their location on the screen. Jumping is used to initiate the start of the game, and replay of the game. As well, jumping is used to confirm the answer when answering the exam.

3.1.3 Control and Interact with Detector Module

Detection The program uses the location of the marker at each frame to determine the action that the player. The module runs continuously and stores the following values in the memory.

1) Sum of the X coordinate of red pixels. (xsum) 2) Sum of the Y coordinate of red pixels. (ysum) 3) Total number of red pixels found. (pCount)

The software uses these 3 values to determine the centroid of the red marker by calculating : X_Center = xsum/pCount Y_Center = ysum/pCount

Motion Detection The fundamental control of the game is composed of jump and standing at the left/right of the screen. The pCount value is used to determine if the input is valid and the centroid of the marker is used to determine the actions. A valid input is defined as a human wearing the marker is present in the camera’s view. Since the marker is a block of red, a valid input must have an abundant amount of red pixels. From experience, we have determined that a valid input has a red pixels count of more than 3000. As well, the location of the centroid should be within the boarder of the screen. So the 20 < Y_Center < 460 and 20 < X_Center < 620. In order to determine if there is a jump, the current location of the marker is compared with the previous location. A jump is defined as an upward change in the marker’s location that is greater than 50 pixels. The left/right detection is determined by the horizontal location of the marker. If the marker is located on the right side of the camera’s view, then the character is then moved to the left side of the screen, and vice versa. The mirroring is done because the player’s location is mirrored on the screen.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

14

3.1.4 Interaction with tone_gen Module

When the game is running, the micro-blaze continuously passes the period and length information of each note to be played into the tone_gen block through GPIO. The specific Super Mario tone is played periodically until the game gets paused to reaches the end.

3.1.5 Interaction with the clear screen module

The Micro-blaze is connected to the clear screen module through two GPIO signal, one input and one output. The output to clear screen instructs the hardware block which video frame to do the clearing on. The input from clear screen notifies the micro-blaze that the clearing operation is done and ready for the new screen to be rendered.

3.1.6 Interaction with the renderer module

Similar to the clear screen module, the renderer block is connected to the micro-blaze through GPIO. The instruction sent from the micro-blaze informs the renderer which one of the two video frames to write to and which one of the two object-lists to render.

3.2 Video_to_ram

The video_to_ram IP block was obtained from the course discussion board. This block stores digital video signal into the memory, and in our project, memory location 0x40000000 - 0x401FFFFF with frame size of 1024 x 512, with valid pixel data frame size of 640 x 480.

3.3 TFT Controller

The XPS TFT controller is an IP block in the Xilinx IP library. It acts as the PLB master of one PLB bus that is connected to the MPMC. It reads the video data from a range of memory locations, and in our project, the base address is 0x41000000. It then converts these video data to VGA compatible data that is available to display on the monitor. It also acts as a slave to the MicroBlaze, the processor could change the base memory location that the TFT controller displays by writing to a slave register. This speeds up the debugging process as we can either display our game or the captured video by writing to a slave register in the XMD stub.

3.4 MPMC & DDR Memory

The MPMC (multi-port memory controller) is an IP block in the Xilinx IP library. It allows up to 8 buses to be connected to the same memory through different ports. In this project, we used a total of 6 ports in the MPMC:

Port 0: used by the Microblaze to write hardware configurations and read important slave registers Port 1: used by the video_to_ram IP block to write video frame to the memory. Port 2: used by the TFT controller to display the video frame to the monitor by reading from the memory. Port 3: used by the detector (custom IP) to read incoming video frame and colour bound configurations, and write detector location to the memory.

Port5: used by the renderer to write objects into the video frames in DDR.

Port6: used by the clear screen to clear all the video out frame to the background color

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

15

3.5 PLB Bus Interface of Custom IP

The custom IP uses the PLB bus to communicate with the MPMC. To ensure the correct functionality of the custom IP, it has to be able to do both write and read operations to the memory. Two separate finite state machines (FSM) are implemented: write FSM and read FSM. The state diagrams for both write and read FSM are shown below. The state diagrams were created with reference for PLB bus write and read timing diagrams shown in [7] and [8]. All states will return to the INIT state upon system reset, which is not shown for neatness reasons. Two FSM will not request to access the memory at the same time because of the register r_nw. A post-processing state is added after the read complete state to update the custom logic's internal registers.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

16

3.6 Detector Module

Since the CPU requires only three values (sum of x-coordinates, sum of y-coordinates, and number of points on the frame) at a slow refresh rate (maximum once per frame, or approximately 10 frames-per-second), the colour detection module can be implemented quite easily. The module basically scans through the frame data provided by video-to-ram module, and counts how many points fall within the colour range specified, while recording the x-coordinate and y-coordinate of the points. Specifically, the flow of the algorithm can be described as below: 1) Starting at the first pixel of the video-to-ram, initialize all register variables, which includes the 3 output values, as well as all the state-related variables;

2) For all pixels in the range of video-to-ram output addresses, determine first if the red value of the pixel is bigger than twice the values for both blue and green, then check if the red value is bigger than 48. If so, increment pixel_count, add the current x-coordinate value to the x_cord_sum and the current y-coordinate value to the y_cord_sum;

3) When it reaches the end of video-to-ram output range, the colour detection module outputs three data: x_cord_sum, y_cord_sum, and pixel_count to memory location 0x49210000, 0x49210004, and 0x49210008 respectively. The addresses were picked arbitrarily in the unused address range in the memory; 5) Return to the first pixel, loop back to 1).

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

17

Margin on valid pixel range to avoid noises When determining whether a specific pixel falls within the range of valid pixels from camera, we give some extra margin in addition to the 640-by-480 range. Specifically we count only the 630-by-470 block of pixels in the middle of each frame, throwing out 10 pixels along each screen edge. The reason was because we observed black pixels (0x0) and sometimes even random noises along the edges inside the frame. If the noise happens to fall within the colour detection range, the algorithm would yield incorrect results. Given that most of the time beacon will not be at the screen edges, we decide to put a safety margin of 10 pixels on each coordinates to improve the system performance. Colour replacement for debugging Our module also has hard-coded colour replacement feature for debugging purposes: instead of processing the pixels internally, we can configure the IP core to output all pixels read, with desired beacon colours being replace by an arbitrary value. Because in the module we use only single-beat read and write on PLB, the refresh rate can be significantly limited when performing such colour replacement. However by directing the video output module to display the frame after colour replacements, we can easily debug the system by seeing whether the desired the pixels are detected and replaced. This feature requires one addition write to PLB after each read, and therefore is turned off in our final product to increase the refresh rate of beacon location detection.

3.7 Audio Module

The audio module is ported directly from last year’s project: Human Pong [9]. Some modifications on software were made for better integration with our system. The module includes two main parts: controller, or the tune generator, and the AC’97 codec. An example of AC’97 codec with documentation of the chip used on board can be found at [10]. The controller is connected to MicroBlaze CPU through 4 parallel GPIOs for 1 audio channel, 4 signal lines for each channel (32-bit tone period and duration, 1-bit begin and done pins). When receiving data from the CPU, the controller averages (essentially, performs mixing of) data from the two channels and produces a square waveform according to the tone periods and durations given. The produced waveform is then fed into AC’97 codec to produce sound at the audio output of the development board. By using parallel GPIOs, driver implementation of the audio module becomes trivial: simply

sending data over GPIO with GPIO driver does the job. A wrapper function in C is written to

simplify the process of generating sound in software.

3.8 Renderer Module

The render module is used for drawing the necessary objects onto the empty frame. As the game advances, the positions and varieties of objects displayed on successive frames changes according to the current state of the character and the design of the game level in the overall time frame.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

18

To achieve this goal, the shape and color of all the objects are stored in DDR memory as 32-bit value s containing the offset from the lower level corner and color for all the pixels. Another series of coordinates are rewritten dynamically by the game logics in microblaze into a certain location in memory containing the type and coordinate of the lower left corner of all the objects in current frame. The renderer module then successively read through the list and draw the appropriate pixels on the screen. The detailed design for the data structure containing the object information is described below. A new object list is written to the same location in DDR at the beginning of each frame. Each list

is at most 512*4 bytes long and terminated by 0xFFFFFFFF (if the list only contains 1 object, it

will have 4 byte of data and then followed by 0xFFFFFFFF).

Each object is represented in a 32 bits format

0000 0000 00 00 0000 0000 0000 0000 0000

10 bits, X coordinate of the bottom left corner of sprite on the screen.

10 bits, Y coordinate of the bottom left corner of sprite on the screen.

12 bits for the location index of the pixel data (unsigned int loc = 0 – 4095). The start location of the pixel data = 0xA2000000 + loc*4000

The max value of the location bits should only be 255 (8 bits), skip drawing if the location bits is

greater than 255.

When drawing, each 32bits entry in the pixel data table has the format:

0000 0000 0000 0000 0000 0 000 000 0 0000

8 bits, X coordinate offset from the bottom left corner going right.

8 bits, Y coordinate offset from the bottom left corner going up.

RGB 565 format. The MSB 5 bits are Red value, middle 6 bits are Green value, and LSB 5 bits are Blue value.

When going through the pixel data table, there should be at most 64x64x4 = 16384 bytes (4096

32bit segments). Each table began with 0x****#### where **** represents the width of the

image and #### represents the height of the image. The table is terminated by 0xFFFFFFFF.

However, the renderer shall never read pass startAddress+16384. In other words, the renderer

shall at most read 16384 bytes per each object to be rendered.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

19

3.9 Clear Screen Module

Clear Screen module is used for clear the output video frame of all objects at the end of each frame display, so that the next frame can be drawn on an empty background. In the original plan, the clear screen function is done in software. When the idea is tested out, we observed noticeable delay between successive frames on the screen. To increase the frame rate, we plan to move the clear screen function into the renderer block. Yet after the initial design of the render logic, we realized that the complexity of the render logic is already reaching a formidable level without the clear screen added in. Incorporating the clear screen would make the block to difficult to test and debug. Therefore, in the end, we decided to create to separate block for the clear screen function. The block uses the PLB burst write to render the pixels in the video frame to the same value. GPIO signals from the micro blaze are used to arbitrate the writing to two different frame for double buffering.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

20

4.0 Description of Design Tree This section illustrates the content of every folder that we have submitted regarding our project.

A README file has also been included at the top level of our directory for the same purpose.

Folder Description

Adventure This folder contains XPS project Our custom pcores are stored in the directory \pcores Our C code is stored in the directory \code Our graphics and level design files are stored in the directory \Sprites

Key files include: 1. detector.v - contains the custom logic to identify the beacon location by the use of color bounds of pixels. --> \Adventure\pcores\detector_v1_00_a\hdl\verilog\detector.v 2. renderer.v – contains the custom logic to draw objects onto the frame --> \Adventure\pcores\renderer_v1_00_a\hdl\verilog\ renderer.v 3. clearscreen.v – contains the custom logic to clear all frame pixels --> H:\Adventure\pcores\clearscreen_v1_00_a\hdl\verilog \ clearscreen.v 4. tone_gen.v - contains the custom logic for outputting audio signal to speaker --> \Adventure\pcores\tone_gen_v1_00_a\hdl\verilog\ tone_gen.v 5. video_to_ram.v - send digital video signal into DDR-SDRAM through MPMC --> \Adventure\pcores\video_to_ram_v1_00_a\hdl\verilog\ video_to_ram.v 6. main.c – contains all the game logic and audio control --> \Adventure\code \ main.c 7. download.tcl – contains the tcl code for downloading the graphics, level

design and the program into RAM

--> Adventure\Sprites\ download.tcl

8. Graphics and level design files --> Adventure\Sprites\*.bin

Doc This folder includes all the documentations for our project Project Proposal: ECE532 Project Proposal.pdf Group Report: Group_report.pdf

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

21

5.0 Tips and Tricks One of the most helpful strategies that we have found out throughout the term is to share our

works and to keep everything synchronized in Dropbox. It is also very important to have version

control over time. There were also times that more than one person is editing the same file so

we kept each file specified by person’s name and the version number. It was especially helpful

when we needed to go back to our previous design and to redo some blocks and it was quite

easy to find out who’s responsible for the issue that way. As well, after each milestone, the

project is zipped and kept in a private cloud in case the working copy is corrupted or lost.

We also found out that the three helper programs that we’ve written to parse the image, to

parse the level configuration and to parse the input music really helped us to speed up the

revision process for level design and graphic/music selections.

6.0 Conclusion This document provides a comprehensive technical documentation to our group design ‘EngSci

Adventure’ for ECE532: Digital Systems Design. This project with a span of 3 months has been

very successful with all the major original goals accomplished. Further improvements can be

done upon request and an updated version of this document will be provided.

ECE 532: EngSci Adventure By Simeng Ding, Li Liu, Chen Xing Zhao

22

7.0 References [1] Rus, McLaughlin. "IGN Presents The History of Super Mario Bros.". IGN.

http://games.ign.com/articles/833/833615p2.html. Retrieved 2010-08-03.

[2] "E3 2010: Project Natal is "Kinect"". IGN. June 13, 2010.

http://xbox360.ign.com/articles/109/1096876p1.html. Retrieved March 18, 2011.

[3] Leigh Alexander (March 9, 2011). "Microsoft: Kinect Hits 10 Million Units, 10 Million Games". Gamasutra. http://www.gamasutra.com/view/news/33430/Microsoft_Kinect_Hits_10_Million_Units_10_Million_Games.php. Retrieved March 10, 2011.

[4] Tim Ingham (March 9, 2011). "Kinect cruises past 10m sales barrier". CVG.

http://www.computerandvideogames.com/292825/news/kinect-cruises-past-10m-sales-barrier/.

[5] "Kinect Confirmed As Fastest-Selling Consumer Electronics Device". Guinnessworldrecords.com. http://community.guinnessworldrecords.com/_Kinect-Confirmed-As-Fastest-Selling-Consumer-Electronics-Device/blog/3376939/7691.html. Retrieved March 10, 2011.

[6] ""Project Natal" 101". Microsoft. June 1, 2009. Archived from the original on June 1, 2009.

[7] Paul Chow (January 2012). ECE 532: Digitial System and Design (Online) http://www.eecg.toronto.edu/~pc/courses/532/2012/

[8] IBM Corp. (2007, May) IBM 128-Bit Processor Local Bus Architecture Specifications Version 4.7. [Online]. http://www-01.ibm.com/chips/techlib/techlib.nsf/techdocs/3BBB27E5BCC165BA87256A2B0064FFB4/$file/PlbBus_as_01_pub.pdf [9] Xilinx, Inc. (2010, December) Xilinx PLBV46 Master Burst (1.01a) Documentation. [Online]. http://www.xilinx.com/support/documentation/ip_documentation/plbv46_master_burst.pdf [10] Chris Langan, Kevin Lam, and Nancy Chong. (2011, January) ECE532 Digital Systems Design Past Projects. [Online]. http://www.eecg.toronto.edu/~pc/courses/532/2011/pastprojects.html [11] Xilinx, Inc.; Eric Gallimore; Nathaniel Smith; Mark Chang. (2006, April) AC'97 Audio Codec Controller for Digilent XUP-V2P. [Online]. http://embedded.olin.edu/xilinx_docs/projects/audio-v2p.php