vchat | audio based public chat terminal with automated login
DESCRIPTION
Vchat : Audio based public chat terminal with automated login using FPGA Spartan 6 boards Key Features: Hands-free 2-way Audio communication Automatic login with face detection Simple and user friendly interfaceTRANSCRIPT
VchatEE4214 Project Presentation
OUTLINE
• About the Project
• Design Process
• Demo
• Technical Details
• Future Improvements
• Learning Points
• Team and Task Distribution
• Q & A
Key Features
• Hands-free
• 2-way Audio communication
• Automatic login with facedetection
• Simple and user friendlyinterface
About the Project
Vchat :Audio based public chat terminalwith automated login
Applications
• Public Terminals
• Elderly care
• Hospital Use
• Company Network
• Virtual Window
System Design – Initial Concept
Challenges Faced:• USB Drivers
• Spartan 3 too weak• 8-bit colour
• Slow
Initial Design Concept• Interface two Spartan 3E
• 2-way Video
• USB Webcams
• Keyboard based chatting
• Automatic login
• Interface with USB webcam
System Design - Final
Final Design• Interface two Spartan 6• 2-way Audio• HDMI-in Camera• Automatic login• High resolution video• Hardware Accelerated
• Data manipulation, Logic separation
Demo
Technical Details
HDMI Microblaze 1
Microblaze 0
VDMA
AC97 Audio Codec
Ethernet Lite
Face Detected Audio ReceivedVideo In/Out
Audio In/Out
Data In/Out
- Face Detection/Tracking- Graphic User Interface
- Audio Processing- Communication
VMDA = Video Direct Memory Access
System Architecture
System Architecture
AXI = Advanced eXtensible Interface
Data Corruption:• Microblaze has to sample
a frame for face detection.• HDMI Read/Write speed
>> Microblaze.• HDMI In refreshes frame
at constant rate (60fps), and should not be stopped.
HDMI
Video In/Out (60fps)
Microblaze 1(Face Detection)
RAM
Frame
(5-10fps)
VDMA
HDMI & GUI
VMDA = Video Direct Memory Access
HDMI
Video In/Out (Fast)
RAM
Frame Frame
Frame Frame
Normal
Microblaze 1(Face Detection)
RAM
Frame
(Slow)
Frame
Frame Frame
Sampling
HDMI
Video In/Out (Fast)
HDMI & GUIFrame Sampling with Data Corruption Prevention
RAM128MB
Pixel Extraction
Load at Initialization
bmp
Microblaze 1
HDMI Frame
HDMI & GUI
Face DetectionOriginal Skin Color
Erode Noise
Fill Holes
Face Detected
Nguyen, T. T. (2012). Real-time Face Detection and Tracking. Master of Engineering Thesis, School of Electrical and Computer Engineering, Cornell University.
Result
Audio & Ethernet
AC97 Controller
Mic
Line Out
LM4550AC-97Audio Codec
Microblaze 0
Spartan-6
Ethernet lite
ConnectorMarvell
M88E1111Microblaze 0
Spartan-6
Hardware Software
EthernetEthernet Communication Scheme:• Send: Implicit naming, non-blocking (broadcast message)• Receive: Implicit naming, non-blocking (receive message from everyone), but
check for “valid” address and message.
Ethernet frame, Destination MAC Address = 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF
12
3 4
ACK: Node 1 Online
ACK: Node 1 Online
Transmitting
BufferAudio In
Ethernet Full
Microblaze 0
N samples
1 packet = N samples
Tx Data
44 kHz
44 kHz
Receiving
Buffer
Buffer
Buffer
…Audio Out
Ethernet
At least 1 empty
Microblaze 0
Rx DataFilter
Audio & Ethernet
𝐻 𝑧 =1
4(1 + 2𝑧−1 + 𝑧−2)
Audio Transmitting Receiving
Time
Receiving
Nondeterministic
Time
e.g. Buffer N = 2
Audio Cycle(44kHz)
Transmitting every N Audio Cycles
Transmitting
Deterministic
Audio & Ethernet
• Audio In/Out: Fixed rate at 44kHz. • It is more efficient to send multiple samples in 1 Ethernet packet.• Transmitting: deterministic, the processor know when the buffer is full to
broadcast data away → fixed rate and synchronous with audio.• Receiving: nondeterministic, incoming data can arrive at any time, not
synchronous with audio → multiple buffers. Moreover, data can come from multiple senders.
Audio & Ethernet
Schedule
Time
TimeData Arrive (Interrupt)
Audio Cycle
Audio InAudio Out Data Tx
Data Rx
Real Time Requirements
• Audio & Ethernet
• Audio in, Audio out, Ethernet Tx, and Ethernet Rx scheduling.
• Data buffering.
• Video
• Data corruption prevention.
• DDR2 RAM
• Shared memory access (hardware).
• Dual Microblaze
• Producer/Consumer.
Future Improvements
• Implementation of TCP/IP protocolfor scalable network.
• Face and voice recognition forautomated login with personalinformation
• Machine learning software.
• More powerful hardware.
• 2-way video capabilities
• Faster microprocessor (ARM + FPGA).
• HDL implementation of video codec.
Learning Outcomes
• Complete system implementation
• Application of concepts
• Data and logic separation forhardware optimized processes
• Parallel processing
• Dealing with time-critical IO andreal-time deadlines
• Team work is essential
Team and Task Distribution
Anh Tuan Nguyen A0074465Y System Design & Integration.
Dao Duc Huan A0030591N Face Detection Algorithm.
Huynh Van Vinh A0055979H Audio to Ethernet Communication.
Rahul Rajeev A0077497J Interfacing with peripherals:• HDMI (PLB Bus).• Audio Interfacing.• Keyboard Controller (Spartan3)
User Interface Design:• Image to Pixel-Array
Shambavi Krishnamurthi
A0077496L
Ziyi Wang A0091881X
Thank YouProf. Akash and Chin Hau
Q & A