3d distributed music visualizertel-zur.net/teaching/weizmann/sc/2011b/nimrodtalmon.pdf · world...

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

3D Distributed Music Visualizer

Nimrod Talmon

Weizmann Institute of Science

2011

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Was ist das

• A music visualizer.

•

••

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Was ist das

• A music visualizer.

•

••

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Was ist das

• A music visualizer.

•

•

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Was ist das

• A music visualizer.

•

••

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Flow

• WAV file

• Music Analyzer

• World Generator

• Ray Tracer

• AVI file

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Flow

• WAV file

• Music Analyzer

• World Generator

• Ray Tracer

• AVI file

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Flow

• WAV file

• Music Analyzer

• World Generator

• Ray Tracer

• AVI file

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Flow

• WAV file

• Music Analyzer

• World Generator

• Ray Tracer

• AVI file

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Flow

• WAV file

• Music Analyzer

• World Generator

• Ray Tracer

• AVI file

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Flow

• WAV file

• Music Analyzer

• World Generator

• Ray Tracer

• AVI file

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Music Analyzer

• Map : Waveform→ {Array |Parameters}

• Volume• Amplitude Average

• Pitch• Frequencies• Maximum frequencies• DFT(W)

• Velocity• Derivative of other parameters

• Panning, Style, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Music Analyzer

• Map : Waveform→ {Array |Parameters}• Volume

• Amplitude Average

• Pitch• Frequencies• Maximum frequencies• DFT(W)

• Velocity• Derivative of other parameters

• Panning, Style, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Music Analyzer

• Map : Waveform→ {Array |Parameters}• Volume

• Amplitude Average

• Pitch• Frequencies• Maximum frequencies• DFT(W)

• Velocity• Derivative of other parameters

• Panning, Style, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Music Analyzer

• Map : Waveform→ {Array |Parameters}• Volume

• Amplitude Average

• Pitch• Frequencies• Maximum frequencies• DFT(W)

• Velocity• Derivative of other parameters

• Panning, Style, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Music Analyzer

• Map : Waveform→ {Array |Parameters}• Volume

• Amplitude Average

• Pitch• Frequencies• Maximum frequencies• DFT(W)

• Velocity• Derivative of other parameters

• Panning, Style, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Music Analyzer

• Map : Waveform→ {Array |Parameters}• Volume

• Amplitude Average

• Pitch• Frequencies• Maximum frequencies• DFT(W)

• Velocity• Derivative of other parameters

• Panning, Style, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

World Generator

• Map : Parameters → Objects

• Parameters• Music Analyzer Parameters

• Objects• 3D Circles (X, Y, Z, Radius, Color)

• Polygons, Camera movements, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

World Generator

• Map : Parameters → Objects

• Parameters• Music Analyzer Parameters

• Objects• 3D Circles (X, Y, Z, Radius, Color)

• Polygons, Camera movements, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

World Generator

• Map : Parameters → Objects

• Parameters• Music Analyzer Parameters

• Objects• 3D Circles (X, Y, Z, Radius, Color)

• Polygons, Camera movements, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

World Generator

• Map : Parameters → Objects

• Parameters• Music Analyzer Parameters

• Objects• 3D Circles (X, Y, Z, Radius, Color)

• Polygons, Camera movements, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

World Generator

• Map : Parameters → Objects

• Parameters• Music Analyzer Parameters

• Objects• 3D Circles (X, Y, Z, Radius, Color)

• Polygons, Camera movements, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Ray Tracer

• Projection : 3D → 2D

• (par)for x in X:

(par)for y in Y:

(par)for obj in Objects:

image(x,y) = closestCollision(x,y,obj)

• Shading, Reflections, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Ray Tracer

• Projection : 3D → 2D

• (par)for x in X:

(par)for y in Y:

(par)for obj in Objects:

image(x,y) = closestCollision(x,y,obj)

• Shading, Reflections, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Ray Tracer

• Projection : 3D → 2D

• (par)for x in X:

(par)for y in Y:

(par)for obj in Objects:

image(x,y) = closestCollision(x,y,obj)

• Shading, Reflections, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Ray Tracer

• Projection : 3D → 2D

• (par)for x in X:

(par)for y in Y:

(par)for obj in Objects:

image(x,y) = closestCollision(x,y,obj)

• Shading, Reflections, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Ray Tracer

• Projection : 3D → 2D

• (par)for x in X:

(par)for y in Y:

(par)for obj in Objects:

image(x,y) = closestCollision(x,y,obj)

• Shading, Reflections, etc...

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Movie Generator

• Movie = �{Images} (Concatenation of images)

••

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Movie Generator

• Movie = �{Images} (Concatenation of images)

•

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Movie Generator

• Movie = �{Images} (Concatenation of images)

••

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Computations

• DFT• ∀k ∈ [0 : n − 1] : FREQk =

∑AMPLITUDEne

− 2πiN kn

• Parallelize by time (over a moving window)• Parallelize the DFT by symmetries

• Ray Tracing

• ∀x , y , obj : COLLISION = −b±√b2−4ac2a

• Parallelize by X,Y• Parallelize by objects• Parallelize by concerns

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Computations

• DFT• ∀k ∈ [0 : n − 1] : FREQk =

∑AMPLITUDEne

− 2πiN kn

• Parallelize by time (over a moving window)

• Parallelize the DFT by symmetries

• Ray Tracing

• ∀x , y , obj : COLLISION = −b±√b2−4ac2a

• Parallelize by X,Y• Parallelize by objects• Parallelize by concerns

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Computations

• DFT• ∀k ∈ [0 : n − 1] : FREQk =

∑AMPLITUDEne

− 2πiN kn

• Parallelize by time (over a moving window)• Parallelize the DFT by symmetries

• Ray Tracing

• ∀x , y , obj : COLLISION = −b±√b2−4ac2a

• Parallelize by X,Y• Parallelize by objects• Parallelize by concerns

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Computations

• DFT• ∀k ∈ [0 : n − 1] : FREQk =

∑AMPLITUDEne

− 2πiN kn

• Parallelize by time (over a moving window)• Parallelize the DFT by symmetries

• Ray Tracing

• ∀x , y , obj : COLLISION = −b±√b2−4ac2a

• Parallelize by X,Y• Parallelize by objects• Parallelize by concerns

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Computations

• DFT• ∀k ∈ [0 : n − 1] : FREQk =

∑AMPLITUDEne

− 2πiN kn

• Parallelize by time (over a moving window)• Parallelize the DFT by symmetries

• Ray Tracing

• ∀x , y , obj : COLLISION = −b±√b2−4ac2a

• Parallelize by X,Y

• Parallelize by objects• Parallelize by concerns

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Computations

• DFT• ∀k ∈ [0 : n − 1] : FREQk =

∑AMPLITUDEne

− 2πiN kn

• Parallelize by time (over a moving window)• Parallelize the DFT by symmetries

• Ray Tracing

• ∀x , y , obj : COLLISION = −b±√b2−4ac2a

• Parallelize by X,Y• Parallelize by objects

• Parallelize by concerns

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Computations

• DFT• ∀k ∈ [0 : n − 1] : FREQk =

∑AMPLITUDEne

− 2πiN kn

• Parallelize by time (over a moving window)• Parallelize the DFT by symmetries

• Ray Tracing

• ∀x , y , obj : COLLISION = −b±√b2−4ac2a

• Parallelize by X,Y• Parallelize by objects• Parallelize by concerns

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Computations

• DFT• ∀k ∈ [0 : n − 1] : FREQk =

∑AMPLITUDEne

− 2πiN kn

• Parallelize by time (over a moving window)• Parallelize the DFT by symmetries

• Ray Tracing

• ∀x , y , obj : COLLISION = −b±√b2−4ac2a

• Parallelize by X,Y• Parallelize by objects• Parallelize by concerns

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Technologies

•

•• WAVE

• FFTW

• MULTIPROCESSING

• PIL

• TK

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Technologies

•

•

• WAVE

• FFTW

• MULTIPROCESSING

• PIL

• TK

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Technologies

•

•• WAVE

• FFTW

• MULTIPROCESSING

• PIL

• TK

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Technologies

•

•• WAVE

• FFTW

• MULTIPROCESSING

• PIL

• TK

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Technologies

•

•• WAVE

• FFTW

• MULTIPROCESSING

• PIL

• TK

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Technologies

•

•• WAVE

• FFTW

• MULTIPROCESSING

• PIL

• TK

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Technologies

•

•• WAVE

• FFTW

• MULTIPROCESSING

• PIL

• TK

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

Technologies

•

•• WAVE

• FFTW

• MULTIPROCESSING

• PIL

• TK

•

3DMV

NimrodTalmon

Introduction

Flow

Music Analyzer

World Generator

Ray Tracer

Movie Generator

Computations

Technologies

Questions

10X