audio unit graphs - dribin.org€¦ · audio unit graphs dave dribin – @ddribin black solid...
TRANSCRIPT
![Page 1: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/1.jpg)
Audio Unit Graphs
Dave Dribin – @ddribin
Black Solid
BitMaki Logomarks
1 Color: Deep Green
Design: MakaluMedia www.makalumedia.com • [email protected]
Full Color: Deep Green, Bright Green
Deep Green92 Cyan42 Magenta75 Yellow44 Black
Pantone 3435
#0c3124
Bright Green45 Cyan0 Magenta100 Yellow0 Black
Pantone 376
#8cc919
Sunday, May 2, 2010
![Page 2: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/2.jpg)
Playing Audio on iPhone OS
• Media Player Framework
• AV Foundation Framework
• OpenAL Library
• Audio Toolbox Framework
• Audio Unit Framework
Sunday, May 2, 2010
![Page 3: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/3.jpg)
Audio Architecture
OpenAL AudioToolbox
AVFoundation
AudioUnits
AudioHardware
Sunday, May 2, 2010
![Page 4: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/4.jpg)
Disadvantage of Audio Queues
• High Latency
• Cannot Access iPod Equalizer
• No way to get access to samples “as they play”
Sunday, May 2, 2010
![Page 5: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/5.jpg)
Audio Units
• Work only with uncompressed audio
• Work in constrained environment
Sunday, May 2, 2010
![Page 6: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/6.jpg)
What are Audio Units?
Sunday, May 2, 2010
![Page 7: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/7.jpg)
Audio Unit Pull Model
Sunday, May 2, 2010
![Page 8: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/8.jpg)
Core Audio Pull Model
iPod EQAudio Unit
ConverterAudio Unit
RemoteIOAudio Unit
Render Callback
Sunday, May 2, 2010
![Page 9: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/9.jpg)
Audio Unit Graph
iPod EQAudio Unit
ConverterAudio Unit
RemoteIOAudio Unit
Render Callback
Audio Unit Graph
Sunday, May 2, 2010
![Page 10: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/10.jpg)
Audio Unit Graph Data Types
• AUGraph
• AUNode
• AudioUnit
• AudioComponentDescription
typedef struct AudioComponentDescription { OSType componentType; OSType componentSubType; OSType componentManufacturer; UInt32 componentFlags; UInt32 componentFlagsMask;} AudioComponentDescription;
Sunday, May 2, 2010
![Page 11: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/11.jpg)
Audio Unit “Hello World”
• Play 440 Hz sound
• A440 Project on BitBucket:
• http://bitbucket.org/ddribin/a440
Sunday, May 2, 2010
![Page 12: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/12.jpg)
Demo
Sunday, May 2, 2010
![Page 13: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/13.jpg)
@interface A440AUGraph : NSObject <A440Player>{ AUGraph _graph; AUNode _outputNode; AUNode _converterNode; AudioStreamBasicDescription _dataFormat; A440SineWaveGenerator _sineWaveGenerator;}
- (BOOL)play:(NSError **)error;- (BOOL)stop:(NSError **)error;
@end
Sunday, May 2, 2010
![Page 14: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/14.jpg)
- (BOOL)play:(NSError **)error;{ NSAssert(_graph == NULL, @"Graph is already started");
OSStatus status; FAIL_ON_ERR(NewAUGraph(&_graph)); FAIL_ON_ERR([self addOutputNode]); FAIL_ON_ERR([self addConverterNode]); FAIL_ON_ERR(AUGraphConnectNodeInput(_graph, _converterNode, 0, _outputNode, 0)); FAIL_ON_ERR(AUGraphOpen(_graph)); [self setupDataFormat]; FAIL_ON_ERR([self setDataFormatOfConverterAudioUnit]); FAIL_ON_ERR([self setMaximumFramesPerSlice]); FAIL_ON_ERR([self setRenderCallbackOfConverterNode]); FAIL_ON_ERR(AUGraphInitialize(_graph)); A440SineWaveGeneratorInitWithFrequency(&_sineWaveGenerator, 440.0); FAIL_ON_ERR(AUGraphStart(_graph)); return YES; failed: // Error handling... return NO;}
Sunday, May 2, 2010
![Page 15: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/15.jpg)
- (OSStatus)addOutputNode;{ AudioComponentDescription description = { .componentType = kAudioUnitType_Output,#if TARGET_OS_IPHONE .componentSubType = kAudioUnitSubType_RemoteIO,#else .componentSubType = kAudioUnitSubType_DefaultOutput,#endif .componentManufacturer = kAudioUnitManufacturer_Apple, }; return AUGraphAddNode(_graph, &description, &_outputNode);}
Sunday, May 2, 2010
![Page 16: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/16.jpg)
- (OSStatus)addConverterNode;{ AudioComponentDescription description = { .componentType = kAudioUnitType_FormatConverter, .componentSubType = kAudioUnitSubType_AUConverter, .componentManufacturer = kAudioUnitManufacturer_Apple, }; return AUGraphAddNode(_graph, &description, &_converterNode);}
Sunday, May 2, 2010
![Page 17: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/17.jpg)
- (BOOL)play:(NSError **)error;{ NSAssert(_graph == NULL, @"Graph is already started");
OSStatus status; FAIL_ON_ERR(NewAUGraph(&_graph)); FAIL_ON_ERR([self addOutputNode]); FAIL_ON_ERR([self addConverterNode]); FAIL_ON_ERR(AUGraphConnectNodeInput(_graph, _converterNode, 0, _outputNode, 0)); FAIL_ON_ERR(AUGraphOpen(_graph)); [self setupDataFormat]; FAIL_ON_ERR([self setDataFormatOfConverterAudioUnit]); FAIL_ON_ERR([self setMaximumFramesPerSlice]); FAIL_ON_ERR([self setRenderCallbackOfConverterNode]); FAIL_ON_ERR(AUGraphInitialize(_graph)); A440SineWaveGeneratorInitWithFrequency(&_sineWaveGenerator, 440.0); FAIL_ON_ERR(AUGraphStart(_graph)); return YES; failed: // Error handling... return NO;}
Sunday, May 2, 2010
![Page 18: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/18.jpg)
- (OSStatus)setDataFormatOfConverterAudioUnit;{ AudioUnit converterAudioUnit; OSStatus status; status = AUGraphNodeInfo(_graph, _converterNode, NULL, &converterAudioUnit); if (status != noErr) { return status; } status = AudioUnitSetProperty(converterAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &_dataFormat, sizeof(_dataFormat)); return status;}
Sunday, May 2, 2010
![Page 19: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/19.jpg)
- (OSStatus)setMaximumFramesPerSlice;{
AudioUnit converterAudioUnit; OSStatus status; status = AUGraphNodeInfo(_graph, _converterNode, NULL, &converterAudioUnit); if (status != noErr) { return status; } UInt32 maxFramesPerSlice = 4096; status = AudioUnitSetProperty(converterAudioUnit, kAudioUnitProperty_MaximumFramesPerSlice, kAudioUnitScope_Global, 0, &maxFramesPerSlice, sizeof(maxFramesPerSlice)); return status;}
Technical Q&A QA1606
Sunday, May 2, 2010
![Page 20: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/20.jpg)
- (BOOL)play:(NSError **)error;{ NSAssert(_graph == NULL, @"Graph is already started");
OSStatus status; FAIL_ON_ERR(NewAUGraph(&_graph)); FAIL_ON_ERR([self addOutputNode]); FAIL_ON_ERR([self addConverterNode]); FAIL_ON_ERR(AUGraphConnectNodeInput(_graph, _converterNode, 0, _outputNode, 0)); FAIL_ON_ERR(AUGraphOpen(_graph)); [self setupDataFormat]; FAIL_ON_ERR([self setDataFormatOfConverterAudioUnit]); FAIL_ON_ERR([self setMaximumFramesPerSlice]); FAIL_ON_ERR([self setRenderCallbackOfConverterNode]); FAIL_ON_ERR(AUGraphInitialize(_graph)); A440SineWaveGeneratorInitWithFrequency(&_sineWaveGenerator, 440.0); FAIL_ON_ERR(AUGraphStart(_graph)); return YES; failed: // Error handling... return NO;}
Sunday, May 2, 2010
![Page 21: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/21.jpg)
static OSStatus MyRenderer( void * inRefCon, AudioUnitRenderActionFlags * ioActionFlags, const AudioTimeStamp * inTimeStamp, UInt32 inBusNumber, UInt32 inNumberFrames, AudioBufferList * ioData){ A440AUGraph * self = inRefCon; int16_t * sample = ioData->mBuffers[0].mData; UInt32 channelsPerFrame = self->_dataFormat.mChannelsPerFrame; for (UInt32 i = 0; i < inNumberFrames; i++) { FillFrame(self, sample); sample += channelsPerFrame; } ioData->mBuffers[0].mDataByteSize = inNumberFrames*self->_dataFormat.mBytesPerFrame; return noErr;}
Sunday, May 2, 2010
![Page 22: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/22.jpg)
AURenderCallback Dangers
• Don’t allocate any memory
• Don’t take any locks
• Don’t do any I/O
• Don’t waste time
• Don’t use any Objective-C?
Sunday, May 2, 2010
![Page 23: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/23.jpg)
AURenderCallback Solutions
• Lockless Ring Buffers
• CARingBuffer in “Public Utility”
• Lockless Buffer Queues
• Hope
Sunday, May 2, 2010
![Page 24: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/24.jpg)
Ring Buffer
Sunday, May 2, 2010
![Page 25: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/25.jpg)
Lockless Buffer Queue
• Dequeue buffers without locking
• Very carefully using OSAtomicCompareAndSwapPtrBarrier()
• Still need a way to signal render callback
• CFRunLoopSource
• CFRunLoopSignal
• CFRunLoopWakeup
Sunday, May 2, 2010
![Page 26: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/26.jpg)
Chip Player Sample
• Two variations:
• Everything in render callback
• Using lockless buffer queues
Sunday, May 2, 2010
![Page 27: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/27.jpg)
Sunday, May 2, 2010
![Page 28: Audio Unit Graphs - dribin.org€¦ · Audio Unit Graphs Dave Dribin – @ddribin Black Solid BitMaki Logomarks 1 Color: Deep Green Design: MakaluMedia • mike@makalumedia.com Full](https://reader030.vdocuments.net/reader030/viewer/2022040919/5e95884b8ce8b00b927066a8/html5/thumbnails/28.jpg)
MobileSynth
• Open Source synthesizer
• http://code.google.com/p/mobilesynth/
Sunday, May 2, 2010