rafi khan, mike levine, jacob metrick, and hari ganesan
TRANSCRIPT
![Page 1: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/1.jpg)
Soundulous
A better music visualizer
Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan
![Page 2: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/2.jpg)
Concept
Streaming music and visualizer to mobile device
Interactive visualizer with touches Sync to multiple phones Two user functions:
New: a user (master) creates a session Join: a user joins an existing session
![Page 3: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/3.jpg)
What Is It Good For?
Audience: teens, young adults, other bored people
Fun group activity Parties! Bar Mitzvahs! Weddings!
![Page 4: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/4.jpg)
Technical Challenges
Streaming images to each phone Combining touch events from
various phones Integrating touches into visualizer
quickly Network Options
TCP/HTTP UDP Some combination of both
![Page 5: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/5.jpg)
Technical Challenges
The Visualizer! Need to do audio parsing for volume,
frequency detection FFT, amplitude, etc.
Map touch events to changes in volume, or frequency, or something else
Make it fast Make it cool
![Page 6: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/6.jpg)
The Client Side
Listening to touch events Mixing events Differentiating between “touches”
and “swipes”
Rendering the bitmap images successfully
![Page 7: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/7.jpg)
The Server Side
Getting packets and transferring to input of visualizer
So far: one thread Getting data via UDP from client
![Page 8: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/8.jpg)
The Server Side
Using Processing (processing.org) Making visualizer based on inputs So we need three more threads:
One for doing the actual rendering One for receiving video data from
processing One for sending video data to client(s)
Total threads: 4! Wow!
![Page 9: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/9.jpg)
Workflow (creating session) Phone posts to /new Server responds with a port number
(eg. 10001) Server starts sending visualization
data (compressed as JPG) on 10001 Phone opens an HTTP connection on
10001, and renders the incoming JPG data
Whenever the phone detects a touch event, it sends the event via UDP
![Page 10: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/10.jpg)
Workflow (joining session)
Phone posts to /join with the port number
Server responds to confirm. Exact same workflow as with
creating!
![Page 11: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/11.jpg)
Demo
![Page 12: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/12.jpg)
Future Considerations
Making the full-scale app Limiting number of people/session Viewers/Enablers? Private/Public sessions?
More options Multiple visualizers Different touch functionalities
![Page 13: Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan](https://reader035.vdocuments.net/reader035/viewer/2022062422/56649f2f5503460f94c4a0cb/html5/thumbnails/13.jpg)
Thanks!