porting after effects to the mercury gpu...
TRANSCRIPT
![Page 1: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/1.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Porting After Effects to the Mercury GPU EngineJamie Gjerde | Senior Computer Scientist
Scott Carver | Computer Scientist
![Page 2: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/2.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
How We Got Started
▪ Developer kitchen: third parties already using CUDA inside Effects
▪ Customers want more performance (native AE rendering on the GPU)
▪ Mercury GPU Engine: Premiere has full GPU rendering pipeline, many effects already ported
2
![Page 3: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/3.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Now….
3
• After Effects now supports GPU rendering in its effects pipeline
• Work is underway to port existing AE effects to GPU: >10 effects have been ported
![Page 4: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/4.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why GPU is Important for After Effects
▪ Significantly more compute/processing power for image processing tasks
▪ GPUs are improving at a faster rate than CPUs
▪ Doing more work on the GPU frees up CPU time
▪ In a mobile / non-desktop future, most compute power is GPU-based
4
![Page 5: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/5.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Initial Expectations / Requirements
▪ Faster renders
▪ Performance will scale with faster GPUs
▪ Leverage Mercury Engine and existing effect kernels
▪ Untie AE internals from CPU pixels, CPU implementations
▪ Visual parity is critical!
5
![Page 6: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/6.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Architectural challenges
▪ Many rendering tasks will be CPU for a long time.
▪ Render graph must support composite GPU and CPU rendering
▪ Must handle upload / readback transparently.
▪ CPU code cannot be replaced, must co-exist.
▪ Solutions?
▪ Render graph refactored to handle buffers independent of hardware: CPU, CUDA, OpenCL, Metal
▪ If an input buffer is located on one hardware, but rendering is happening on different hardware, buffer is transferred lazily.
6
![Page 7: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/7.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Architectural challenges: composite render graph
7
![Page 8: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/8.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Architectural challenges
▪ Adapt old rendering architecture
▪ Downside: Most code assumes CPU buffers, direct access, not built for pixel-level parallelization
▪ Downside: AE render graph itself is still basically single threaded / synchronous
▪ Upside: Render graph is very memory conservative (designed for 90’s memory limits)
▪ Upside: Single threaded render means simple GPU interaction, no async spaghetti
▪ Solutions?
▪ GPU and CPU paths share pre-render, frame dependency calculation, and setup code.
▪ GPU-specificity hidden from most components.
▪ Invoke GPU kernels only at lowest possible level.
8
![Page 9: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/9.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Architectural challenges
▪ Visual Parity is HARD
▪ Bit depth (8, 16, 32-float)
▪ Algorithm match between SSE implementation and OpenCL/CUDA/Metal code
▪ Sometimes, algo's need some rewrite on CPU as well.
▪ Even with effort: 8 and 16 won't always match due to rounding: float differences can accumulate.
▪ Solutions?
▪ Never mix GPU and CPU frames for an effect in a single comp.
▪ Downside: if we run out of memory for a frame, we can't transparently fall back to CPU
▪ On 8 and 16 bpc: clip to 0..1 after each GPU render node
▪ In some cases, clipping/rouning in kernel is also required
9
![Page 10: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/10.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Architectural challenges
▪ Memory
▪ AE frame sizes are large (30000 x 30000 x 32bpc maximum)
▪ Render sizes can vary from frame to frame, not predictable
▪ GPU’s cannot recover easily from out-of-memory (e.g. swap)
▪ Solutions?
▪ AE has existing fractional resolution workflows that can help.
▪ Tiled rendering is extremely complex problem for AE's render graph.
▪ However: some kernels could be tiled at a lower level for GPU renders.
10
![Page 11: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/11.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Native GPU Plug-in SDK
▪ Current internal effects are based on existing Premiere GPU SDK
▪ AE render graph handles allocation and CPU <> GPU transfers
▪ Effect receives GPU buffer via SDK call, with correct pixels and parameters for render.
▪ Future: Premiere SDK isn't rich enough to support all AE render features – expansion is a must for both internal effects and third parties.
▪ Internal effect development has illuminated areas where SDK needs iteration
▪ 3rd party developers need an SDK with richer API’s and tools to write efficient plugins for a variety of GPU platforms
11
![Page 12: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/12.jpg)
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 12
![Page 13: Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are](https://reader031.vdocuments.net/reader031/viewer/2022030505/5ab3ab897f8b9a284c8e8f20/html5/thumbnails/13.jpg)