graphics and games - apple developer · 2017. 6. 7. · graphics and games. make expensive things...

169
#WWDC17 © 2017 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple. Michal Valient, GPU Software Engineer Richard Schreyer, GPU Software Engineer Introducing Metal 2 Session 601 Graphics and Games

Upload: others

Post on 26-Aug-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

#WWDC17

© 2017 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.

Michal Valient, GPU Software Engineer Richard Schreyer, GPU Software Engineer

•Introducing Metal 2 • Session 601

Graphics and Games

Page 2: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Make expensive things happen once

GPU in the driving seat

New experiences

Page 3: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

VR with Metal 2 Hall 3 Wednesday 10:00AM

Page 4: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Using Metal 2 for Compute Grand Ballroom A Thursday 4:10PM

Page 5: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Using Metal 2 for Compute Grand Ballroom A Thursday 4:10PM

Page 6: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Metal 2 Optimization and Debugging Executive Ballroom Thursday 3:10PM

Page 7: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Introducing Metal 2 Agenda

Page 8: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Introducing Metal 2 Agenda

•Argument Buffers

Page 9: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Introducing Metal 2 Agenda

•Argument Buffers•Raster Order Groups

Page 10: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Introducing Metal 2 Agenda

•Argument Buffers•Raster Order Groups•ProMotion Displays

Page 11: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Introducing Metal 2 Agenda

•Argument Buffers•Raster Order Groups•ProMotion Displays•Direct to Display

Page 12: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Introducing Metal 2 Agenda

•Argument Buffers•Raster Order Groups•ProMotion Displays•Direct to Display•Everything Else

Page 13: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

•Argument Buffers

Page 14: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Material Example

intensity : 0.3roughness : 0.6 surfaceTexture specularTexture sampler

Page 15: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Material Example

intensityroughness surfaceTexture specularTexture sampler

Page 16: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Traditional Argument Model

intensityroughness surfaceTexture specularTexture sampler

Page 17: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Traditional Argument Model

MTLBuffer

intensityroughness surfaceTexture specularTexture sampler

Page 18: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Traditional Argument Model

MTLBuffer

intensityroughness surfaceTexture specularTexture sampler

CPU time

API callssetBuffer

Page 19: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Traditional Argument Model

MTLBuffer

intensityroughness surfaceTexture specularTexture sampler

CPU time

API callssetBuffer setTexture

Page 20: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Traditional Argument Model

MTLBuffer

intensityroughness surfaceTexture specularTexture sampler

CPU time

API callssetBuffer setTexture setTexture

Page 21: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Traditional Argument Model

MTLBuffer

intensityroughness surfaceTexture specularTexture sampler

CPU time

API callssetBuffer setSamplersetTexture setTexture

Page 22: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Traditional Argument Model

MTLBuffer

intensityroughness surfaceTexture specularTexture sampler

CPU time

API callssetBuffer setSamplersetTexture setTexture draw

Page 23: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Traditional Argument Model

1 2 3 4

CPU time

API calls

intensityroughness surfaceTexture specularTexture sampler

MTLBuffer

Page 24: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Traditional Argument Model

MTLBuffer

intensityroughness surfaceTexture specularTexture sampler

CPU time

API calls1 2 3 4

Page 25: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Traditional Argument Model

MTLBuffer

intensityroughness surfaceTexture specularTexture sampler

CPU time

API calls

Object 1

1 2 3 4

Object 3

. . .

Object 2

Page 26: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers

MTLBuffer

intensityroughness surfaceTexture specularTexture sampler

Page 27: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers

MTLBuffer

intensityroughness surfaceTexture specularTexture sampler

Page 28: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers

CPU time

API calls

MTLBuffer

surfaceTexture specularTexture samplerintensityroughness

Page 29: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers

CPU time

API callssetBuffer

MTLBuffer

surfaceTexture specularTexture samplerintensityroughness

Page 30: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers

CPU time

API callssetBuffer draw

MTLBuffer

surfaceTexture specularTexture samplerintensityroughness

Page 31: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers

CPU time

API calls

MTLBuffer

surfaceTexture specularTexture samplerintensityroughness

Page 32: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers

CPU time

API calls

MTLBuffer

surfaceTexture specularTexture samplerintensityroughness

Page 33: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers

CPU time

API calls

Object 1

. . .

Object 7Object 4 Object 5 Object 6Object 3Object 2

MTLBuffer

surfaceTexture specularTexture samplerintensityroughness

Page 34: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Reduced CPU Overhead

0

1

2

3

4

5

6

7

8

2 Resources 8 Resources 16 Resources

Traditional modelArgument Buffers

Time (µs) Lower is better

iPhone 7

Page 35: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Reduced CPU Overhead

0

1

2

3

4

5

6

7

8

2 Resources 8 Resources 16 Resources

Traditional modelArgument Buffers

7.97

4.53

1.64

Time (µs) Lower is better

iPhone 7

Page 36: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Reduced CPU Overhead

0

1

2

3

4

5

6

7

8

2 Resources 8 Resources 16 Resources

Traditional modelArgument Buffers0.250.240.23

7.97

4.53

1.64

Time (µs) Lower is better

iPhone 7

Page 37: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers Benefits

Improve performance

Enable new use cases

Easy to use

Page 38: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers Shader example

struct Material { float roughness; float intensity; texture2d<float> surfaceTexture; texture2d<float> specularTexture; sampler textureSampler; };

kernel void my_kernel(constant Material &material [[buffer(0)]]) { ... }

Page 39: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers Shader example

struct Material { float roughness; float intensity; texture2d<float> surfaceTexture; texture2d<float> specularTexture; sampler textureSampler; };

kernel void my_kernel(constant Material &material [[buffer(0)]]) { ... }

Page 40: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Crowd rendering

Page 41: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Crowd rendering

MTLBuffer

specularTexture pantsTextureheightposition . . .. . .

Page 42: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Crowd rendering

MTLBuffer

character[0] specularTexture pantsTextureheightposition . . .. . .

GPU

CPU

Page 43: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Crowd rendering

MTLBuffer

character[0] specularTexture pantsTextureheightposition . . .. . .

character[1] specularTexture pantsTextureheightposition . . .. . .

character[2] specularTexture pantsTextureheightposition . . .. . .

. . .

GPU

CPU

Page 44: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Crowd rendering

MTLBuffer

character[0] specularTexture pantsTextureheightposition . . .. . .

character[1] specularTexture pantsTextureheightposition . . .. . .

character[2] specularTexture pantsTextureheightposition . . .. . .

. . .

GPU

CPUsetBuffer

Page 45: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Crowd rendering

MTLBuffer

character[0] specularTexture pantsTextureheightposition . . .. . .

character[1] specularTexture pantsTextureheightposition . . .. . .

character[2] specularTexture pantsTextureheightposition . . .. . .

. . .

GPU

CPUsetBuffer

drawIndexedPrimitives:... instanceCount:1000

Page 46: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Crowd rendering

MTLBuffer

character[0] specularTexture pantsTextureheightposition . . .. . .

character[1] specularTexture pantsTextureheightposition . . .. . .

character[2] specularTexture pantsTextureheightposition . . .. . .

. . .

GPU

CPU

Vertex Shader use per-character position

setBufferdrawIndexedPrimitives:...

instanceCount:1000

Page 47: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Crowd rendering

MTLBuffer

character[0] specularTexture pantsTextureheightposition . . .. . .

character[1] specularTexture pantsTextureheightposition . . .. . .

character[2] specularTexture pantsTextureheightposition . . .. . .

. . .

GPU

CPU

Fragment Shader use per-character textures

Vertex Shader use per-character position

setBufferdrawIndexedPrimitives:...

instanceCount:1000

Page 48: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Vertex shader

vertex VertexOutput instancedShader(uint id [[instance_id]], constant Character *crowd [[buffer(0)]]) { // Dynamically pick the character for given instance index constant Character &obj = crowd[id];

return transformCharacter(obj); }

Page 49: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Vertex shader

vertex VertexOutput instancedShader(uint id [[instance_id]], constant Character *crowd [[buffer(0)]]) { // Dynamically pick the character for given instance index constant Character &obj = crowd[id];

return transformCharacter(obj); }

Page 50: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Dynamic Indexing Vertex shader

vertex VertexOutput instancedShader(uint id [[instance_id]], constant Character *crowd [[buffer(0)]]) { // Dynamically pick the character for given instance index constant Character &obj = crowd[id];

return transformCharacter(obj); }

Page 51: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Set Resources on GPU Particle simulation

Page 52: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

MTLBuffer

particle[0] materialorientation . . .

particle[1] materialorientation . . .

particle[2] materialorientation . . .

. . .

Set Resources on GPU Particle simulation

Page 53: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

MTLBuffer

particle[0] materialorientation . . .

particle[1] materialorientation . . .

particle[2] materialorientation . . .

. . .

Set Resources on GPU Particle simulation

Simulation kernel

thread[0]

thread[1]

thread[2]

. . .

Page 54: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

MTLBuffer

material[0] rocks

material[1] moss

material[2] grass

. . .

MTLBuffer

particle[0] materialorientation . . .

particle[1] materialorientation . . .

particle[2] materialorientation . . .

. . .

Set Resources on GPU Particle simulation

Simulation kernel

thread[0]

thread[1]

thread[2]

. . .

rocks

moss

grass

Page 55: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

MTLBuffer

material[0] rocks

material[1] moss

material[2] grass

. . .

MTLBuffer

particle[0] materialorientation . . .

particle[1] materialorientation . . .

particle[2] materialorientation . . .

. . .

Set Resources on GPU Particle simulation

Simulation kernel

thread[0]

thread[1]

thread[2]

. . .

rocks

moss

grass

Page 56: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

MTLBuffer

material[0] rocks

material[1] moss

material[2] grass

. . .

MTLBuffer

particle[0] materialorientation . . .

particle[1] materialorientation . . .

particle[2] materialorientation . . .

. . .

Set Resources on GPU Particle simulation

Simulation kernel

thread[0]

thread[1]

thread[2]

. . .

rocksmoss

grass

Page 57: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

MTLBuffer

material[0] rocks

material[1] moss

material[2] grass

. . .

MTLBuffer

particle[0] materialorientation . . .

particle[1] materialorientation . . .

particle[2] materialorientation . . .

. . .

Set Resources on GPU Particle simulation

Simulation kernel

thread[0]

thread[1]

thread[2]

. . .

rocks

moss

grass

Page 58: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

MTLBuffer

material[0] rocks

material[1] moss

material[2] grass

. . .

MTLBuffer

particle[0] materialorientation . . .

particle[1] materialorientation . . .

particle[2] materialorientation . . .

. . .

Set Resources on GPU Particle simulation

Simulation kernel

thread[0]

thread[1]

thread[2]

. . .

rocks

moss

grass

Page 59: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Set Resources on GPU Shader example

struct Data { texture2d<float> tex; float value; };

kernel void copy(constant Data &src [[buffer(0)]], device Data &dst [[buffer(1)]]) { dst.value = 1.0f; //< Assign constants dst.tex = src.tex; //< Copy just a texture dst = src; //< Copy entire structure }

Page 60: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Set Resources on GPU Shader example

struct Data { texture2d<float> tex; float value; };

kernel void copy(constant Data &src [[buffer(0)]], device Data &dst [[buffer(1)]]) { dst.value = 1.0f; //< Assign constants dst.tex = src.tex; //< Copy just a texture dst = src; //< Copy entire structure }

Page 61: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Set Resources on GPU Shader example

struct Data { texture2d<float> tex; float value; };

kernel void copy(constant Data &src [[buffer(0)]], device Data &dst [[buffer(1)]]) { dst.value = 1.0f; //< Assign constants dst.tex = src.tex; //< Copy just a texture dst = src; //< Copy entire structure }

Page 62: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Set Resources on GPU Shader example

struct Data { texture2d<float> tex; float value; };

kernel void copy(constant Data &src [[buffer(0)]], device Data &dst [[buffer(1)]]) { dst.value = 1.0f; //< Assign constants dst.tex = src.tex; //< Copy just a texture dst = src; //< Copy entire structure }

Page 63: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Set Resources on GPU Shader example

struct Data { texture2d<float> tex; float value; };

kernel void copy(constant Data &src [[buffer(0)]], device Data &dst [[buffer(1)]]) { dst.value = 1.0f; //< Assign constants dst.tex = src.tex; //< Copy just a texture dst = src; //< Copy entire structure }

Page 64: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers can reference other Argument Buffers

Create and reuse complex object hierarchies

Multiple Indirections

struct Object { float4 position; device Material *material; ///< Many objects can point to the same material };

struct Tree { device Tree *children[2]; device Object *objects; ///< Array of objects in the node };

Page 65: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers can reference other Argument Buffers

Create and reuse complex object hierarchies

Multiple Indirections

struct Object { float4 position; device Material *material; ///< Many objects can point to the same material };

struct Tree { device Tree *children[2]; device Object *objects; ///< Array of objects in the node };

Page 66: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers can reference other Argument Buffers

Create and reuse complex object hierarchies

Multiple Indirections

struct Object { float4 position; device Material *material; ///< Many objects can point to the same material };

struct Tree { device Tree *children[2]; device Object *objects; ///< Array of objects in the node };

Page 67: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Support Tiers

CPU overhead reduction

Set resources on GPU

Multiple indirections

Arrays of Argument Buffers

Resources per draw call

Page 68: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Support Tiers

CPU overhead reduction

Set resources on GPU

Multiple indirections

Arrays of Argument Buffers

Resources per draw call

Tier 1

Yes

No

No

No

Unchanged

Page 69: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Support Tiers

CPU overhead reduction

Set resources on GPU

Multiple indirections

Arrays of Argument Buffers

Resources per draw call

Tier 1

Yes

No

No

No

Unchanged

Tier 2

Yes

Yes

Yes

Yes

500,000 textures and buffers

Page 70: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Argument Buffers Terrain rendering

Material changes with terrain

Trees placed by GPU

Particle materials generated by GPU

Available as sample code

Page 71: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 72: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 73: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 74: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 75: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 76: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 77: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 78: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 79: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

API Highlights

Argument Buffers are stored in plain MTLBuffer

Use MTLArgumentEncoder to fill Indirect Argument Buffers

Abstracts platform differences behind simple interface

Up to eight Argument Buffers per stage

Page 80: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Shader syntax struct Particle { texture2d<float> surface; float4 position; };

kernel void simulate(constant Particle &particle [[buffer(0)]]) { ... }

Page 81: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Shader syntax struct Particle { texture2d<float> surface; float4 position; };

kernel void simulate(constant Particle &particle [[buffer(0)]]) { ... }

Page 82: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Shader syntax struct Particle { texture2d<float> surface; float4 position; };

kernel void simulate(constant Particle &particle [[buffer(0)]]) { ... }

// Create encoder for first indirect argument buffer in Metal function 'simulate' let simulateFunction = library.makeFunction(name:"simulate")! let particleEncoder = simulateFunction.makeArgumentEncoder(bufferIndex: 0)

Page 83: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Shader syntax struct Particle { texture2d<float> surface; float4 position; };

kernel void simulate(constant Particle &particle [[buffer(0)]]) { ... }

// Create encoder for first indirect argument buffer in Metal function 'simulate' let simulateFunction = library.makeFunction(name:"simulate")! let particleEncoder = simulateFunction.makeArgumentEncoder(bufferIndex: 0)

Page 84: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Shader syntax struct Particle { texture2d<float> surface; float4 position; };

kernel void simulate(constant Particle &particle [[buffer(0)]]) { ... }

// Create encoder for first indirect argument buffer in Metal function 'simulate' let simulateFunction = library.makeFunction(name:"simulate")! let particleEncoder = simulateFunction.makeArgumentEncoder(bufferIndex: 0)

Page 85: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Shader syntax struct Particle { texture2d<float> surface; float4 position; };

kernel void simulate(constant Particle &particle [[buffer(0)]]) { ... }

// Create encoder for first indirect argument buffer in Metal function 'simulate' let simulateFunction = library.makeFunction(name:"simulate")! let particleEncoder = simulateFunction.makeArgumentEncoder(bufferIndex: 0)

// API calls to fill the indirect argument buffer particleEncoder.setTexture(mySurfaceTexture, at: 0) particleEncoder.constantData(at: 1).storeBytes(of: myPosition, as: float4.self)

Page 86: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Shader syntax struct Particle { texture2d<float> surface; float4 position; };

kernel void simulate(constant Particle &particle [[buffer(0)]]) { ... }

// Create encoder for first indirect argument buffer in Metal function 'simulate' let simulateFunction = library.makeFunction(name:"simulate")! let particleEncoder = simulateFunction.makeArgumentEncoder(bufferIndex: 0)

// API calls to fill the indirect argument buffer particleEncoder.setTexture(mySurfaceTexture, at: 0) particleEncoder.constantData(at: 1).storeBytes(of: myPosition, as: float4.self)

Page 87: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Shader syntax struct Particle { texture2d<float> surface; float4 position; };

kernel void simulate(constant Particle &particle [[buffer(0)]]) { ... }

// Create encoder for first indirect argument buffer in Metal function 'simulate' let simulateFunction = library.makeFunction(name:"simulate")! let particleEncoder = simulateFunction.makeArgumentEncoder(bufferIndex: 0)

// API calls to fill the indirect argument buffer particleEncoder.setTexture(mySurfaceTexture, at: 0) particleEncoder.constantData(at: 1).storeBytes(of: myPosition, as: float4.self)

Page 88: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Managing Resource Usage

Tell Metal what resources you plan to use

Use Metal Heaps for best performance

// Use for textures with sample access or buffers commandEncoder.use(myTextureHeap)

// Used for all render targets, views or read/write access to texture commandEncoder.use(myRenderTarget, usage: .write)

Page 89: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Managing Resource Usage

Tell Metal what resources you plan to use

Use Metal Heaps for best performance

// Use for textures with sample access or buffers commandEncoder.use(myTextureHeap)

// Used for all render targets, views or read/write access to texture commandEncoder.use(myRenderTarget, usage: .write)

Page 90: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Managing Resource Usage

Tell Metal what resources you plan to use

Use Metal Heaps for best performance

// Use for textures with sample access or buffers commandEncoder.use(myTextureHeap)

// Used for all render targets, views or read/write access to texture commandEncoder.use(myRenderTarget, usage: .write)

Page 91: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Best Practices

Organize based on usage pattern • Per-view vs. per-object vs. per-material • Dynamically changing vs. Static

Favor data locality

Use traditional model where appropriate

Page 92: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Richard Schreyer

•Raster Order Groups

Page 93: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Raster Order Groups

Ordered memory access from fragment shaders

Enables new rendering techniques • Order-independent transparency • Dual-layer GBuffers • Voxelization • Custom blending

Page 94: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Fragment Shaders with Blending

Fragment Shader Thread 1 Blend

Time

Page 95: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Fragment Shaders with Blending

Fragment Shader Thread 2 Blend

Fragment Shader Thread 1 Blend

Time

Page 96: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Fragment Shaders with Blending

Fragment Shader Thread 1 Blend

Time

WaitFragment Shader Thread 2 Blend

Page 97: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Fragment Shader Thread 1

Time

Fragment Shader Thread 2

Mid-Shader Memory Access

Write Memory

Read Memory

Page 98: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Fragment Shader Thread 1

Time

Fragment Shader Thread 2

Write Memory

Wait

With Raster Order Groups

Read Memory

Page 99: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Fragment Shader Thread 1

Time

Fragment Shader Thread 2

Write Memory

Wait

With Raster Order Groups

Read Memory

Page 100: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Blending manually to a pointer in memory fragment void BlendSomething( texture2d<float, access::read_write> framebuffer [[texture(0)]]) {

float4 newColor = …

// Non-atomic access to memory without synchronization float4 priorColor = framebuffer.read(framebufferPosition); float4 blended = custom_blend(newColor, priorColor); framebuffer.write(blended, framebufferPosition); }

Page 101: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Blending manually to a pointer in memory fragment void BlendSomething( texture2d<float, access::read_write> framebuffer [[texture(0)]]) {

float4 newColor = …

// Non-atomic access to memory without synchronization float4 priorColor = framebuffer.read(framebufferPosition); float4 blended = custom_blend(newColor, priorColor); framebuffer.write(blended, framebufferPosition); }

Page 102: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Blending manually to a pointer in memory fragment void BlendSomething( texture2d<float, access::read_write> framebuffer [[texture(0)]]) {

float4 newColor = …

// Non-atomic access to memory without synchronization float4 priorColor = framebuffer.read(framebufferPosition); float4 blended = custom_blend(newColor, priorColor); framebuffer.write(blended, framebufferPosition); }

Page 103: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

// Blending manually to a pointer in memory fragment void BlendSomething( texture2d<float, access::read_write> framebuffer [[texture(0), raster_order_group(0)]]) {

float4 newColor = …

// Hardware waits on first access to raster ordered memory float4 priorColor = framebuffer.read(framebufferPosition); float4 blended = custom_blend(newColor, priorColor); framebuffer.write(blended, framebufferPosition); }

Page 104: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Fragment Shader Thread 1

Time

Fragment Shader Thread 2

Write Memory

Wait Read Memory

With Raster Order Groups

Page 105: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Fragment Shader Thread 1

Time

Fragment Shader Thread 2

Write Memory

Wait Read Memory

With Raster Order Groups

Page 106: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Raster Order Groups Summary

Synchronization between overlapping fragment shader threads

Check for support with MTLDevice.rasterOrderGroupsSupported

Page 107: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

•ProMotion Displays

Page 108: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Without ProMotion 60 FPS

16.6 ms

GPU 31 2

Display 31 2

Page 109: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

With ProMotion 120 FPS

8.3ms

31

31

2

2

GPU

Display

Page 110: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Without ProMotion 48 FPS

33.3ms 16.6ms16.6ms

1

21

2

3

3

20.83ms 20.83ms20.83ms

GPU

Display

Page 111: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

With ProMotion 48 FPS

1

2

2

3

20.83ms

1

20.83ms20.83ms

3

20.83ms 20.83ms20.83ms

GPU

Display

Page 112: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Without ProMotion Dropped frame

33.3ms 16.6ms

16.6ms

1

21

2

3

3

21ms 16.6ms

16.6ms

GPU

Display

Page 113: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

With ProMotion Dropped frame

16.6ms

1

21

2

3

3

21ms 16.6ms

GPU

Display

Page 114: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

With ProMotion Dropped frame

16.6ms21ms 16.6ms

16.6ms

1

21

2

3

3

21ms 16.6ms

GPU

Display

Page 115: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Opting in to ProMotion

UIKit animations use ProMotion automatically

Metal views require opt-in with Info.plist key <key>CADisableMinimumFrameDuration</key> <true/>

Page 116: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Metal Presentation APIs Present immediately

present(drawable)

1

1GPU

Display

Page 117: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Metal Presentation APIs Present after minimum duration

1 2

33ms 33ms

GPU

Display

present(drawable, afterMinimumDuration: 1000.0 / 30.0)

1 (t1) 2 (t2)1 2

Page 118: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Metal Presentation APIs Present at specific time

t1 t2

1 2

GPU

Display

let time : CFTimeInterval = projectNextDisplayTime(); present(drawable, atTime: time)

21 2 (t2)1 (t1)

Page 119: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

let targetTime = // project when intend to display this drawable // render your scene into a command buffer for ‘targetTime’ let drawable = metalLayer.nextDrawable() commandBuffer.present(drawable, atTime: targetTime)

// after a frame or two…

let presentationDelay = drawable.presentedTime - targetTime // Examine presentationDelay and adjust future frame timing

Page 120: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

ProMotion Displays Summary

120 FPS

Reduced latency

Improved framerate consistency

Reduced stuttering from missed display deadlines

Page 121: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

•Direct to Display

Page 122: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Displaying Metal Content Two paths

GPU Composition

Direct to Display

Page 123: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Display

GPU Composition

Metal View

Other UIKit/AppKit Views

Other Application’s Windows

System Compositor

CompositionScale Colorspace Conversion

Page 124: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Direct to Display

Other UIKit/AppKit Views (occluded)

Other Application’s Windows (occluded)

System Compositor

CompositionScale Colorspace Conversion

Display

Full Screen Metal View

Page 125: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Direct to Display Requirements

Page 126: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Direct to Display Requirements

Opaque Layer

Page 127: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Direct to Display Requirements

Opaque Layer

No masking, rounded corners, and so on

Page 128: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Direct to Display Requirements

Opaque Layer

No masking, rounded corners, and so on

Full screen (or with “black bars” via an opaque black background color)

Page 129: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Direct to Display Requirements

Opaque Layer

No masking, rounded corners, and so on

Full screen (or with “black bars” via an opaque black background color)

Dimensions matching the display or smaller

Page 130: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Direct to Display Requirements

Opaque Layer

No masking, rounded corners, and so on

Full screen (or with “black bars” via an opaque black background color)

Dimensions matching the display or smaller

Color Space and Pixel Format compatible with the display

Page 131: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Color Space Metal Pixel Format P3 Display sRGB Display

sRGB bgra8Unorm_srgb Direct Direct

Colorspace Requirements

Page 132: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Color Space Metal Pixel Format P3 Display sRGB Display

sRGB bgra8Unorm_srgb Direct Direct

Display P3 (macOS) bgr10a2Unorm Direct GPU Composited

Colorspace Requirements

Page 133: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Color Space Metal Pixel Format P3 Display sRGB Display

sRGB bgra8Unorm_srgb Direct Direct

Display P3 (macOS) bgr10a2Unorm Direct GPU Composited

Extended sRGB (iOS) bgr10_xr_srgb Direct GPU Composited

Colorspace Requirements

Page 134: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Color Space Metal Pixel Format P3 Display sRGB Display

sRGB bgra8Unorm_srgb Direct Direct

Display P3 (macOS) bgr10a2Unorm Direct GPU Composited

Extended sRGB (iOS) bgr10_xr_srgb Direct GPU Composited

Linear Extended sRGB, or Display P3 rgba16Float GPU Composited GPU Composited

Colorspace Requirements

Page 135: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Detecting P3 Display Gamut

UIKit UITraitCollection.displayGamut == .P3

AppKit NSScreen.canRepresent(.p3)

Page 136: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 137: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 138: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve
Page 139: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Direct to Display Summary

Eliminate compositor usage of GPU

Useful for full-screen scenes

Supported on iOS, tvOS, and macOS

Use Metal System Trace to verify

Page 140: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

•Everything Else

Page 141: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Memory Usage Queries

New APIs to query memory usage per allocation MTLResource.allocatedSize MTLHeap.currentAllocatedSize

Query total GPU memory allocated by the device MTLDevice.currentAllocatedSize

Page 142: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

SIMDGroup-scoped Data Sharing

Share data across a SIMDGroup simd_broadcast(data, simd_lane_id) simd_shuffle(data, simd_lane_id) simd_shuffle_up(data, simd_lane_id) simd_shuffle_down(data, simd_lane_id) simd_shuffle_xor(data, simd_lane_id)

Page 143: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

SIMDGroup-scoped Data Sharing

Share data across a SIMDGroup simd_broadcast(data, simd_lane_id) simd_shuffle(data, simd_lane_id) simd_shuffle_up(data, simd_lane_id) simd_shuffle_down(data, simd_lane_id) simd_shuffle_xor(data, simd_lane_id)

Thread 0 Thread 1 Thread 2 Thread 3 Thread 4 Thread 5 … Thread 15

Thread 0 Thread 1 Thread 2 Thread 3 Thread 4 Thread 5 … Thread 15

Page 144: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Non-Uniform Threadgroup Sizes

0 1 2 3 4 5 6 7 8 9 10 11

1

2

3

4

5

6

7

Page 145: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Non-Uniform Threadgroup Sizes

0 1 2 3 4 5 6 7 8 9 10 11

1

2

3

4

5

6

7

4x4 Threadgroup

4x4 Threadgroup

4x4 Threadgroup

4x4 Threadgroup

4x4 Threadgroup

4x4 Threadgroup

Page 146: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Non-Uniform Threadgroup Sizes

0 1 2 3 4 5 6 7 8 9 10 11

1

2

3

4

5

6

7

4x4 Threadgroup

4x4 Threadgroup

4x4 Threadgroup

4x4 Threadgroup

4x4 Threadgroup

4x4 Threadgroup

Page 147: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

0 1 2 3 4 5 6 7 8 9 10 11

1

2

3

4

5

6

7

Non-Uniform Threadgroup Sizes

3x4 Threadgroup

3x3 Threadgroup

4x3 Threadgroup

4x3 Threadgroup

4x4 Threadgroup

4x4 Threadgroup

Page 148: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Viewport Arrays

Vertex Function selects which viewport

Useful for VR when combined with instancing

Height

2 x Width

Page 149: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Multisample Pattern Control

Select where within a pixel the MSAA Sample Patterns are located

Useful for custom anti-aliasing

1 Pixel

2

1

4

3

Page 150: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Multisample Pattern Control

Select where within a pixel the MSAA Sample Patterns are located

Useful for custom anti-aliasing

Toggle sample locations

1 Pixel

21

43

Page 151: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Resource Heaps

Now available on macOS

Control time of memory allocation

Fast reallocation and aliasing of resources

Group related resources for faster binding

Page 152: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Resource Heaps

Memory Allocation for B

Texture B

Memory Allocation for C

Texture C

Memory Allocation for A

Texture A

Page 153: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Resource Heaps

Heap

Memory Allocation for B

Texture B

Memory Allocation for C

Texture C

Memory Allocation for A

Texture A

Texture A Texture B Texture C

Page 154: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Resource Heaps

Heap

Texture A Texture B

Memory Allocation for B

Texture B

Memory Allocation for C

Texture C

Memory Allocation for A

Texture A

Texture C

Page 155: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Resource Heaps

Heap

Texture A Texture B

Memory Allocation for B

Texture B

Memory Allocation for A

Texture A

Page 156: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Resource Heaps

Heap

Texture A Texture B

Memory Allocation for B

Texture B

Memory Allocation for A

Texture A

Memory Allocation for D

Texture D

Texture D

Page 157: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Resource Heaps

Heap

Texture ATexture B

Texture D

Memory Allocation for B

Texture B

Memory Allocation for D

Texture D

Memory Allocation for A

Texture A

Page 158: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Other Features

Feature Summary

Page 159: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Other Features

Feature Summary

Linear Textures Create textures from a MTLBuffer without copying

Page 160: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Other Features

Feature Summary

Linear Textures Create textures from a MTLBuffer without copying

Function Constant for Argument Indexes Specialize bytecodes to change the binding index for shader arguments

Page 161: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Other Features

Feature Summary

Linear Textures Create textures from a MTLBuffer without copying

Function Constant for Argument Indexes Specialize bytecodes to change the binding index for shader arguments

Additional Vertex Array Formats Add some additional 1 component and 2 component vertex formats, and a BGRA8 vertex format

Page 162: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Other Features

Feature Summary

Linear Textures Create textures from a MTLBuffer without copying

Function Constant for Argument Indexes Specialize bytecodes to change the binding index for shader arguments

Additional Vertex Array Formats Add some additional 1 component and 2 component vertex formats, and a BGRA8 vertex format

IOSurface Textures Create MTLTextures from IOSurfaces on iOS

Page 163: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Other Features

Feature Summary

Linear Textures Create textures from a MTLBuffer without copying

Function Constant for Argument Indexes Specialize bytecodes to change the binding index for shader arguments

Additional Vertex Array Formats Add some additional 1 component and 2 component vertex formats, and a BGRA8 vertex format

IOSurface Textures Create MTLTextures from IOSurfaces on iOS

Dual Source Blending Additional blending modes with two source parameters

Page 164: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Summary

Argument Buffers

Raster Order Groups

ProMotion Displays

Direct to Display

Everything Else

Page 165: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

More Informationhttps://developer.apple.com/wwdc17/601

Page 166: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Related Sessions

VR with Metal 2 Hall 3 Wednesday 10:00AM

Metal 2 Optimization and Debugging Executive Ballroom Thursday 3:10PM

Using Metal 2 for Compute Grand Ballroom A Thursday 4:10PM

Page 167: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Previous Sessions

What's New in Metal, Part 1 WWDC 2016

Working with Wide Color WWDC 2016

Page 168: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve

Labs

Metal 2 Lab Technology Lab A Tues 3:10–6:10PM

VR with Metal 2 Lab Technology Lab A Wed 3:10–6:00PM

Metal 2 Lab Technology Lab F Fri 9:00AM–12:00PM

Page 169: Graphics and Games - Apple Developer · 2017. 6. 7. · Graphics and Games. Make expensive things happen once GPU in the driving seat ... iPhone 7. Argument Buffers Benefits Improve