Metropolis Light TransportProject User Guide

Kai HatchCS 6965

December 18, 2011

Background

Metropolis Light Transport is a global illumination method for ray tracing first developed by Veach and Guibas in 1997. MLT is based on Metropolis sampling from Metropolis et al., 1953. MLT's main strength is being able to explore local space in the scene, which allows for scenes that are difficult to sample to converge faster.

Metropolis Sampling(Note: out of all the papers I read about Metropolis sampling, the one from Pharr and Humphreys' book, Physically Based Rendering, was the easiest to understand. Most of the discussion below is from there.)

Suppose we have a function f that we want to sample. One way to sample the function is to create a sampling distribution proportional to f , then make a histogram of of samples taken from the distribution. We then scale this histogram to approximate f.

We first choose an initial sample, and every sample after this is made by mutating the sample before it to create a proposed candidate sample. This new candidate sample may be accepted or rejected. By allowing these samples to transition to a new sample state given certain rules, we eventually come up with a distribution proportional to f. In order to do this, we must be able to calculate the the probability density of these transitions. This is known as the tentative transition function.

One important property about these transitions is that the probability of going from state Xi → Xi+1 must be the same as the probability of Xi+1 → Xi. This is known as detailed balance.

Given a tentative transition function, we must be able to calculate if the new candidate sample is accepted or rejected. This equation is:

a ( X → X ' )=min (1, f ( X ' ) T ( X ' → X )f ( X ) T ( X → X ' ) )where f(x) is the value of the function at x, and T(x → x') is the transition function of a sample going from state X to X'. If the transition probabilities are the same, then the transition probability terms cancel out.

The Metropolis sampling algorithm in code is:X = Xofor i = i to n

X' = mutate(X)a = accept(X, X')

if (random() < a)X = X'

record(X)

Metropolis Sampling in Path TracingThe conversion of Metropolis to path tracing is fairly straightforward. The conversion is listed below:

Metropolis Sampling MLT in Path Tracingf The function we want to sample Color (pixel) values from our scene

Samples X, X'

x-values where we want to sample at Light path of rays through the scene

Mutation StrategiesThe greatest flexibility of MLT lies in the choice of mutation (transition) functions. The easiest type of mutation strategy is to choose a new pixel location and send a new ray through. This mutation type is symmetric in that the given a current pixel location, the probability of choosing a new pixel location is the same as the new pixel location picking the current location. This allows for the transition probabilities to cancel out of the acceptance equation. Using this type of mutation is also what is used, more or less, in normal path tracing.

However, this type of mutation still requires a lot of time and a lot of samples per pixel to reduce noise to acceptable levels. Another mutation strategy, and this is where MLT shines, is to use a mutation strategy that explores local space by perturbing existing light paths. In the original MLT paper, Veach and Guibas proposed using an exponential equation to do this:

x '=x i ± be−log(b /a)ζ

where ζ is a uniform random variable.

Scale Proportional DistributionAfter we have a histogram of samples, it then has to be scaled to approximate f. To do the scaling in path tracing, we divide the histogram color values by the average color of the scene image. The average can be found by normal path tracing at random pixel locations for a given amount, say 10,000 samples.

Overall Discussion of the Project Area

Additional Sources UsedThe first five assignments we did was a good introduction to ray tracing, but definitely not enough for Metropolis Light Transport. I used three main additional sources for my project:

• Metropolis Light Transport, Veach and Guibas. 1997.◦ This is the original MLT paper, and the first one I read. This paper had a ton of

technical information in it, and a lot of it went over my head. Though, if I re-read it now, I could probably go, “Ohh, so that's what they were talking about...” Because this paper was so detailed, this led me to my second source.

• A Practical Introduction to Metropolis Light Transport, Cline and Egbert. 2005.◦ This paper's goal was to make sense of the original Veach and Guibas paper to a

level where most anyone could understand. It really helped get the gist of Metropolis sampling, and actually gave code to get an example working using 2D images. They then showed how it could be applied to path tracing. When discussing mutation strategies however, they were pretty technical, too. And in the end they said, “In our implementation, we did this...”, but didn't actually show any helpful implementation ideas. It was a good paper to make sense of the first one, though.

• Physically Based Rendering, Pharr and Humphreys. 2010.◦ I found this book after looking around online for more information on MLT. This is the

second edition of the book and just barely came out. One of the new additions in this edition was a special chapter dedicated to MLT, and also had an implementation. This got me very excited, but I really could have used this book sooner. A lot of my project is based on their implementation, but also with some features from Kevin Suffern's book Ray Tracing from the Ground Up. This book has everything that a good ray tracer would want. I'm really excited to just read though this book, even though it's 1200 pages long. But the material I read so far was really, really good.

Time SpentI think this was the longest and hardest project I've ever worked on in all my schooling. I think a lot of it was trying to make sense of subtle C++ problems. Just for the last three weeks, I've put in 10-12 hours everyday almost. I didn't expect it to be this hard.

How Much I Got DoneEven with as much time I put into this project, because it had so much depth, I've just now got it working.

I worked on getting MLT working on just the Sponza scene because it seemed like a better candidate to try MLT on than the other two test scenes we had for assignments. I forgot that we had a repository of scenes to test with, but I think that's because even a tar, gzipped version of the scenes was still 900MB to download, and I forgot I could have grabbed it a the U.

I have a working “run_rt” version, and the TRaX version you can make out the Sponza scene, but it's hard to tell since the image is 128x128. It also takes a really, really long time to render.

Extensions• Specular Reflection : My project, as it stands, is a diffuse-only scene. If I had additional time,

I would have implemented a scene that possibly had some mirror reflection and transparencies. This would have allowed me to better test MLT use of local space.

• Specialized Scenes : Using additional scenes that are traditionally hard to path trace to find light sources would have been fun, too.

• Bidirectional path tracing : This project was hard enough to get it to work with path tracing. I would have been nice to get bidirectional path tracing running. The implementation in Pharr's book did include bidirectional path tracing, but I had to concentrate on getting normal path tracing working first, since that's all I know. There were papers on it I could have read, but it'll have to be for fun now.

• Area Lights : Since we did point lights for our assignments, it would have been more physically correct to use area lights. Suffern's book has a chapter, and I'm sure that PBRT (Physically Based Rendering) has one, too, but Suffern said that to get area lights working would be fairly involved. I was going to do area lights, but once I got it working with point lights, I decided that I didn't want to do a ton of HW into Christmas Break. It'd maybe be something I add on later. My first idea for area lights was to use the same coordinates of the light from TRaX, but make an emissive sphere centered at that point as the light.

• Timing Tests vs. Path Tracing : I would be cool to do timing tests because supposedly MLT should give much better results than normal path tracing in the same amount of computation time.

Implementation

PBRT Metropolis Light TransportPBRT chapter 13 has a section dedicated to Metropolis sampling, which is good to read before doing its implementation of MLT, chapter 17. Their implementation is made up of the following:

• Sample Structures ◦ struct PathSample

▪ This holds the BSDF sample values (uniform floats) at each path vertex.◦ struct LightingSample

▪ This is used for sampling the light, but since I used point lights, I didn't need this.

◦ struct MLTSample▪ This holds vectors of PathSamples and LightingSamples, as well as the pixel

location (x, y) coordinates, representing a path through the scene.

• Mutation Strategies ◦ LargeStep()

▪ This sets all PathSamples in a path to have new random uniform floats for the BSDF and the pixel location.

◦ SmallStep()▪ This uses the exponential equation above to mutate an existing MLTSample

(path through the scene). The uniform floats are perturbed stored in the BSDF samples and camera using this equation.

PBRT then uses functions called “GeneratePath()” and “PathL()” to figure out the path contributions for each vertex and the light. They spend 11 pages in the book describing these two functions, and their overcomplexity finally made me cut them out of my implementation after trying to get them to work forever. Instead I decided to use what we had been using for our previous homeworks in the while loop, and that worked the first time. I think it took me about two weeks of messing with “GeneratePath()” and “PathL()” to come to the decision to use our HW code.

PBRT also had really involved, complex classes for everything. I didn't need anything so advanced, so most of the utility classes I used were from our previous homeworks, such as Vector, Color, Ray, etc.

My ImplementationMy code uses the following classes:

• Utility Classes ◦ Color (RGB triple)◦ HitRecord (Ray-object intersections)◦ Point (X,Y,Z - point)◦ Ray◦ Util (TRaX memory access)◦ UtilMath (See TRaX Implementation)◦ Vector (X,Y,Z – direction)

• Geometric Shape Classes ◦ Box (BVH stuff)◦ TriangleKai (Scene objects)

• Scene Classes ◦ PinholeCamera◦ PointLight

The program runs in “main.cpp”. In the function “main()”, I have a MLTInfo struct that is passed by reference to each function to give member data they need, such as samples per pixel, scene width, height, etc. For HW5, I had the program encapsulated in a “RayTracer” that ran in it's “render()” method, but that gave me weird memory errors. The approach of setting program values inside a struct was much easier.

I have a probability variable for each pixel sample in the scene for how many are LargeStep() mutations and how many are SmallStep() mutations. Currently, I've found that the best images come out with 50% LargeStep(), and 50% SmallStep(). In the “run-rt” version, I have an allocated array of Color objects that are written into, then only at the end are they normalized and written into the TRaX frame buffer. The TRaX version does it in the frame buffer, on-the-fly.

Run-RT ImagesThese images are using LargeStep mutations only. One interesting thing is that it looks to be indirect illumination only, although shadows are present.

LargeStep mutations only; 20 mutations per pixel

LargeStep mutations only; 100 mutations per pixel

These next images are SmallStep mutations only. I really liked these ones. They gave the images a really soft, fuzzy touch. The higher sampled ones reminded me of Impressionist paintings.

SmallStep mutations only; 20mpp

SmallStep mutations only; 100 mpp

Here are images at 50% LargeStep, 50% SmallStep. These seem to result in the best quality out of all of them. Although, I did do 25% LargeStep, 75% SmallStep, but I only rendered it at 4mpp.

50% LargeStep, 50% SmallStep; 20mpp

50% LargeStep, 50% SmallStep; 100mpp

25% LargeStep, 75% SmallStep; 4mppIt would be interesting to see how this would compare to 50%-50% at higher quality.

TRaX ImplementationHere are the issues I ran into when converting “run_rt” to “simhwrt”:

• Dynamic Memory ◦ The PBRT implementation made heavy use of the STL vector class. I had to convert

everything to C-style arrays, but I ran into problems because I couldn't allocate anything on the heap like the STL vectors did, and I also ran into limits into what the stack frame would let me allocate. This severely limited the amount of memory I could use. As a result much of the high-quality sampling suffered. One example was the bootstrapping sample to find the initial sample. I think I ended being able to do 100 samples instead of 100,000 samples, like I had been doing in “run_rt”.

◦ In “run_rt”, I used a C-style array of Color objects to store pixel values, and only at the end normalize them and store them in the frame buffer. I was limited again by how much I could allocate on the stack. In the end, I had to have the frame buffer be read and written to while running. I'm not how this affected the image quality afterwards, since TRaX renderings took too long to see.

◦ The TRaX Programming Guidelines online say that dynamic memory isn't used for performance reasons. But if dynamic memory were available, and it were used judiciously, then it might help out a lot in some circumstances.

• Math functions ◦ The simulator wouldn't allow system library “cos()”, “sin()”, “expf()”, “pow()”, and “log()”. I

had to make my own approximation functions to simulate these. The cosine, sine, and the exponential functions were relatively straightforward using Taylor Series loops. Because these Taylor series use factorials and high-range ints, I was limited by how many terms I could put into the series. I was able to get to 8 for cosine and the exp(), and 7 for sine. My implementation used cosine and sine in the domain of [0, 2π], but having 7-8 terms only gave acceptable results up to π. Luckily, I noticed that using 7-8 terms gave good results from [-π, π]. So I had to do some domain shifts and flips, but I was able to remap the input values to give correct output values. Fortunately for me, my pow() usage converted into a square root, and the log() was able to be converted to a constant value. Here is an image I made while testing my cosine and sine functions for calculating reflection rays off the diffuse materials. Note the blue illumination from the background inside the dark areas of the pillars:

• Virtual Functions ◦ Almost all ray tracers make heavy use of inheritance. It was a bit of challenge to do it

without it, but it was do-able.• Random functions

◦ The PBRT implementation made use of the random generator seed, to redo certain random calculations to avoid having to store values. This is the problem I ran into when calculating the initial MLT sample. Since I couldn't re-seed the generator to give me back the same “random” values, I had to store them. This added to the memory problems I couldn't use on the stack.

◦ In “run_rt”, to get good coverage for LargeStep to all pixels, the indices for all the pixels were stored in an array and shuffled to give random access. This guaranteed that all pixels would be hit at least once. My implementation used “random_shuffle()” from the C++ library, but TRaX wouldn't allow me to use it.. PBRT had a function to do the shuffle, but they used random ints, and we only had random floats. I fudged it to return ints by calling a random float and casting it to an int. In the end, I couldn't use the pixel indices because I was limited by how much array space I could allocate on stack.

TRaX ImagesThis image took a long time to render in the simulator, on the order of hours, where “run_rt” would run in maybe a third that time on my AMD Phenom II X4 945. It is rendered using the default settings of the simulator (one-core), with a ray depth of 5, and 20mpp. Its output is listed in the project files on the web. I attempted to render the same scene with a full set of TRaX resources a couple of times, but the rendering time took too long. I'm confident that the image it would have output would have been a viewable once, since the only difference was that all TRaX resources were enabled. My first attempt was with a ray depth of 5 and 20mpp. It ran for 20 hours and hadn't finished. I tried again with a ray depth of 3, 15 mpp, and it ran 12 hours and hadn't finished either.

Simulator, default settings, one core; ray depth 5, 20mppYou can barely make out the Sponza scene.

There also seems to be a fair amount of blue in it.

Final NotesI have two versions of “main.cpp” in my project. The current “main.cpp” in the project is the simulator version. There is also “main.cpp.run_rt”. This file is the working “run_rt” code, that makes use of system library functions, etc.

I've also zipped up all images I was able to render and placed them in the project direction next to to this file.