GPU Renderfarm with Integrated Asset Management & Production System (AMPS)

Tackling two main challenges in CG movie production

Multi-plAtform Game Innovation Centre (MAGIC), Nanyang Technological University (NTU), Singapore

Presenter: Dr. Chen Quan

About MAGIC

• Established in Nanyang Technological University, Singapore on 1 Nov 2012

• Cross-disciplinary research with 26 professors, 33 researchers, 11 PhD

• To translate scientific ideas into technological products and services

• To increase the capability of Future Studios in media & entertainment

• Launched Future Studios Research Lab (FSR Lab) on 21 Jan 2014

2

How it all began… “The Boy And His Robot”

• Proposal: a 90-minute live-action Mecha film with fully GPU rendered special effects

• To demonstrate the possibility of creating high quality production by small teams and budget, with the help of

innovative technologies

3 Concept Art of The Boy And His Robot

Two main challenges in CG movie production

• How to manage rendering assets efficiently

▪ Large amount of digital assets ▪ Common outsourcing practice

• How to speed up CG rendering

4

▪ CPU renderfarm

• Substantial hardware setup

• High maintenance cost

• High power consumption

=> Not affordable for small studios :(

▪ GPU renderfarm

• Rendering algorithms can benefit from parallelization

• More cores

• Smaller size

• Lower cost

• AMPS: Asset Management & Production System

– Online resource management (e.g. archiving, retrieving, tracking, without location constraint)

– Can be adopted by other industries

• Integration of GPU renderfarm with AMPS

– Manage rendering assets efficiently

– Streamline the rendering pipeline (i.e. sending assets to the renderfarm online via web browser)

– Lower rendering cost and speed up rendering process

Our solutions

5

• Not flexible to satisfy varied needs from media production companies and studios

– e.g. Shotgun and TACTIC

• Lack of resource management features such as version control, workflow management,

flexible accessing/sharing rights management

– e.g. Dropbox, Box.net

• No linkage with rendering solutions

– Costs additional time and effort managing the rendering assets

Existing asset and workflow management solutions

6

CPU-based renderfarms

• GreenButton (RenderMan Pro Server, YafaRay,

LuxRender)

– External software to upload assets, monitor progress

and download results

– Does not have complete asset management features

Existing renderfarm services

7

GPU-based renderfarms

• Render Street, renderFlow, ultrarender (Cycles and/or

iRay renderers)

– Rendering jobs are done in company servers

• Sheep it! Render Farm (Cycles)

– Rendering job is distributed among clients/users

• OTOY (Octane)

– Utilizes NVIDIA GRID

Proposed workflow

8

3D scenes & other

data

Render request

3D scenes &

Render request

Rendered results

Rendered results &

other data

HP SL250s Gen 8 * 2 NVIDIA TESLA K20X * 6

Asset management component

9

Front-end

• User-friendly interface to

access and manipulate

assets

• In various ways: web-based

(internet browser), PC

client, mobile apps

Back-end

• Stores data (projects, assets,

users, workflows, etc.)

• Provides back-end functions

• A combination of web

services & MVC (Model-

View-Controller)

infrastructure

• Asset

– Media asset in production (e.g. images, videos, sound, 3d models, etc.)

– Version control support (i.e. revisions are kept for each asset)

– Can be shared to internal and/or external users

• User

– Each user has roles (that can be created/deleted/edited and assigned to different projects)

– Two types, real user or virtual user

• Real user : regular user, such as artist, director etc.

• Virtual user : interface between AMPS and external application/solution e.g. renderfarm

AMPS Components

10

3. Request asset info

5. Download assets request

Renderfarm component

11

Web browser

1. Rendering Request

AMPS

Thin App

8. Send job

2. Trigger & inputs

6. Assets

7. Assets

9. Assets

10. Job & Assets

12. Results

13. Results

14. Rendering results

11. Results

11. Results

11. Results

4. Asset info

Manager Node Rendering Nodes

Backburner Manager

10. Job & Assets

10. Job & Assets

Workflow of rendering

12

• Rendering specifications

Rendering experiment

13

• Rendering node specifications

• Manager node specifications

Resolution 1920 x 1080

Polygon count 752,608

Frame count 85

GPU RAM consumption

2,814 MB

Rendering • Path tracing • 1,250 samples per pixel • Maximum 3 bounces

OS Windows 7 Enterprise

CPU Intel Core i7 920 2.67 GHz & 2.67 GHz

RAM 6 GB

OS Windows Server 2012

Renderer OctaneRender Autodesk 3DS Max Plug-in ver. 1.18, hosted in Autodesk 3DS Max 2014

CPU Intel Xeon E5-2660 2.20 GHz & 2.19 GHz

RAM 64 GB

GPU 3x Tesla K20X

Hardware Configuration Rendering Time

(hr:min:sec)

1 node, with 1 K20X GPU 5:58:45

1 node, with 2 K20X GPUs 3:11:46

1 node, with 3 K20X GPUs 2:15:34

2 nodes, each with 1 K20X GPU 3:00:24

2 nodes, each with 2 K20X GPUs 1:36:37

2 nodes, each with 3 K20X GPUs 1:08:47

Rendering Experiment

14

• Speed-up for different numbers of GPUs

– Tested using one node

– Similar result for two nodes

Rendering experiment

15

• Speed-up for different numbers of nodes

– Each node with same numbers of GPUs

– Currently one frame can be handled by

only one GPU node

0

1

2

3

1 GPU 2 GPUs 3GPUs

1.87x (~2.0x)

2.65x (~3.0x)

0

1

2

3

1 node 2 nodes

2x

0

2

4

6 5.22x

1 node 1 GPU

2 nodes 3 GPUs per node

• Total acceleration

– 1 node with 1 GPU only vs.

2 nodes with 3 GPUs per node

• Conclusion

– Our solution can aid movie production in term of efficiency and saving rendering cost

– We can obtain rendering time acceleration that scales almost linearly with the number of GPU nodes

• Future Work

– AMPS plug-in for 3D authoring tool (e.g. 3DS Max) to submit rendering job directly

– Support for other 3D authoring tools (e.g. Maya and Blender) and rendering software

– Support for heterogeneous GPU renderfarm (rendering nodes with different OS and geographically

separated GPUs)

Conclusion and Future Work

16

IDM Futures Funding Initiative support

Tier-2 research funding support

GPUs Server nodes

3D models, test-bedding and feedbacks

OctaneRender

Acknowledgements

17

Thank you!

Contact: qchen@ntu.edu.sg

Website: magic.ntu.edu.sg