Morphing

COSC450 Assignment 2

Due Monday 2nd May 2016, 5pm

This assignment is worth 20% of your final grade

Who is this?

1

1 Overview

Morphing is a common visual effect, where one image slowly changes into an-other. In this assignment you will implement a classic morphing algorithm byBeier and Neely [1]. In this method we start with two images we want to morphbetween. The user then draws lines on the images, which lie along correspond-ing features. For example, if morphing between two faces, you might draw linesdown each person’s nose, around their eyes, etc. An example of this is shownin Figure 1, for a morph between US Presidents George W. Bush and BarackObama.

Figure 1: Lines to control a morph between George W. Bush and Barack Obama.

To make an intermediate frame, the lines are first interpolated. A simplelinear interpolation of the endpoints will do. Suppose the line in the first imageruns from p to q, and the corresponding line in the second image is from p′

to q′. If we want to move some proportion, α, from image one to image two,then the interpolated line runs from (1 − α)p + αp′ to (1 − α)q + αq′. Notethat if α = 0 we get the original line in the first image, and if α = 1 we get thesecond line. Figure 2 shows the interpolation of the lines for the example fromFigure 1.

Beier and Neely’s paper explains how to use the positions of these lines towarp an image. Basically, the lines near a pixel in the warped image determinewhere in the original images its colour should be copied from. Two colours areprovided for each pixel in the warped image, one from each source image. Theseare interpolated in the same way as the lines’ locations.

The final result is the ability to produce an image that is an arbitrary mix ofthe two input images. This mix is controlled by α and by rendering a series offrames where α varies from 0 to 1 a smooth animation can be made. The controllines ensure that important features in the images line up correctly, preventing‘ghosting’ artefacts. An intermediate frame from our sample morph is shown inFigure 3.

2

Figure 2: Interpolating the control lines with α = 0.5. The lines for the warpedimage (middle) are half-way between their positions for George W. Bush (left)and Barack Obama (right).

Figure 3: Morph frames with (from left to right) α = 0, 0.5, and 1.

2 Assignment Requirements

For this assignment you must implement Beier and Neely’s morphing algorithm,experiment with a number of parameters, and write a report discussing whatyou have found. Code is provided to draw and edit lines on a pair of images. Askeleton solution to the assignment is also provided, which handles the readingof parameters from the command line and display of the results. You mustimplement the functions interpolateLines and warpImage in morpher.cpp.You may wish to add additional functions in order to structure your code.

The code provided runs on the lab machines, and requires the OpenCVlibraries, which are available in my public folder. Your code should not dependon any other libraries or utilities.

As well as your code you should provide input for a sample animation (twoinput images and a file describing the control lines). Your report should clearlyexplain the command line parameters needed to run this animation.

Your report should briefly discuss your implementation of the algorithm, but

3

should focus on an experimental evaluation. There are several parameters tothe algorithm, which are called a, b, and p in Beier and Neely’s presentation.You should design experiments to demonstrate the effects of changing theseparameters, show the results of those experiments, and discuss the meaning ofeach parameter.

3 Resources

Beier and Neely first presented their method at SIGGRAPH in 1992 [1]. Copiesof the original paper are available through the library’s ACM subscription, oronline, but they are scans from the paper proceedings so not of the highestquality. Thaddeus Beier has an HTML version, although some of the equationsand figures may be easier to read in the original paper. Both versions give fairlyclear descriptions and pseudo-code for the core method.

OpenCV is available on the lab machines in ~steven/Public/OpenCV. Theversion there is 2.4.9, and documentation is available at http://opencv.org/

documentation/opencv-2-4-9.html. Makefiles which link to this library areprovided for the editor and skeleton solution. Source code for these is in~steven/Public/COSC450, under the MorphEditor and Morpher subdirecto-ries. For example, to get a copy of the editor for the morphing program youcould use (from the command line)

cd /directory/where/you/put/cosc450/stuff

cp -R ~steven/Public/MorphEditor .

In order to run the resulting programs successfully, you will need to runthe included setPath.sh script. This script sets up the path to the OpenCVlibrary files, which need to be linked in at run-time. For example, to compileand run the editor from the command line, you would change to the appropriatedirectory and then enter the commands

make

source setPath.sh

./morphEditor bush.jpg obama.jpg presidents.txt

3.1 The Morph Editor

The morphEditor program takes as input (via the command line) two imagesand a file to record the lines in. For example:

morphEditor bush.jpg obama.jpg presidents.txt

The two images are loaded, and any lines in the file (if it exists) are drawn.The editor has a number of modes, which are set with key presses, and the

lines are edited with the mouse. Note that the key presses are only registeredwhen one of the two images is active. Typing into the terminal window doesnothing. The keyboard commands are:

4

Q Quit the program.

A Enter Add line mode. In this mode left clicking starts a new line, andclicking again finishes the line. New lines appear in the same location inboth images.

D Enter Delete line mode. In this mode you can remove lines by clicking onthem. The line you can delete by clicking is highlighted in red in bothimages.

E Enter Edit line mode. In this mode you can click and drag the end points ofthe lines. The line you are editing is highlighted in yellow in both images,but only the active line is changed.

S Save the current set of lines to the file. Note that the lines are also saved au-tomatically to morphEditorTemp.txt after each change is made to them.

R Revert the lines to the last saved version.

3.2 The Skeleton Solution

The skeleton solution takes a pair of images and a line file as commands, followedby four optional commands. The first is the step to take between images. If thisnumber is between 0 and 1 it is treated as the value of α as discussed above. Ifit is greater than 1, it is the number of frames over which to morph the images.If this value is not provided, it defaults to 0.5.

The next three are (in order) the parameters a, b, and p for the algorithm.Any that are not supplied default to 1.

Examples:

morpher bush.jpg obama.jpg presidents.txt

morpher bush.jpg obama.jpg presidents.txt 0.5

morpher bush.jpg obama.jpg presidents.txt 20

morpher bush.jpg obama.jpg presidents.txt 0.75 0.1 2 0.5

The first example should blend halfway between the two images with a = b =p = 1. The second produces 20 frames animating from one image to the otherwith the default parameters. The last example blends 25% of bush.jpg and75% of obama.jpg with parameters a = 0.1, b = 2, and p = 0.5. There is nochecking on the validity of any parameters.

The resulting warped images are displayed on screen and saved to files ofthe form warpX.XXX.png, where X.XXX is the value of α.

The implementation provided needs the functions interpolateLines andwarpImage implemented to work properly. As provided, interpolateLines

returns the first set of lines unchanged, and warpImage returns a copy of theinput image.

5

4 Experiments

Graphics and vision algorithms often have parameters which control their effects,and often take in images as input. This raises a number of questions: What aregood parameter values? How sensitive are these values? What is a ‘typical’ inputfor the algorithm? What are the edge cases? You need to conduct experimentsto answer these questions, and designing good experiments is important.

Parameter values are relatively easy to experiment with. Different parametersettings can be used and the results compared. For example, the parameters a,b, and p in the morphing algorithm are all numbers, and so you can test a rangeof values. Questions to keep in mind are:

• What is the useful range of values? You should test a range of valuesacross the range, and the boundary cases.

• What about values outside that range? Often 0 or negative values cancause problems.

• How sensitive is the parameter to changes in value? Suppose some param-eter can take values in the range [0, 1], and you find out that 0.2 is a goodvalue. What happens when you change it to 0.21, 0.3, or 0.5? Parametersthat give good results over a fairly wide range of values are often betterthan those that have to be set precisely.

• When there are multiple parameters, how do they interact? This risksa combinatorial explosion – if you have 3 parameters with 10 settingseach you have 103 possible combinations. Checking them all may not befeasible, and these are fairly small numbers of parameters/settings.

Numeric parameters are fairly easy (if time-consuming) to deal with, butother variations in input are not so easy to quantify. The morphing algorithmtakes in two images and a set of lines. What is a ‘typical’ image or set of lines?What are the edge cases? These are not easy questions to answer, but it is oftenconvenient to think about easy and hard cases. For example, the presidentialmorph in Figure 3 is a fairly easy case because the two images have very similarposes. Testing on easy, moderate, and hard cases can help.

When designing experiments you need to think about a few different things:

• What do you want to know about the algorithm?

• How can you design an experiment to test this?

• How many tests will you need to run, and how much work is that?

For example, one simple question could be ‘How does the parameter a affectthe results?’ A simple test would be to try different values of a and visuallycompare the results. You need to balance the range of values of a which youtest against the amount of time you want to take. Also, you may want to try afew different morphs to see how general the results are.

6

So, suppose you have 5 different values of a you want to use, and 3 differentmorphs. That’s 15 times you need to run the program, which is probably OK,but then you need to worry about b, and p, and maybe some other factors. Yourapidly end up wanting to run lots of experiments, and doing this manually istedious and error-prone, so you may want to write short scripts to automatethis. Bash scripts or languages like Python are commonly used for this.

You also need to think about how to analyse and present your results. Youcould run your program with every combination of 20 different values for each ofof a, b, and p, and use 5 different pairs of images, giving 5×20×20×20 = 40, 000morphs. How do you analyse those to find out some useful information? Howdo you present those results in a short report? If you can’t do either of thosethings, it probably isn’t worth running all those experiments, so planning whatyou want to do is important.

5 Marking Scheme

• 20% for your code. As usual I’ll be looking for clarity and correctness.I should be able to easily understand how your code works, and how itrelates to the algorithm presented in the paper. While some marks maybe earned for efficient implementations, this is not required and a clear,direct implementation of the core algorithms can earn full marks.

• 20% for example animation results. I’ll be looking for how well you havedefined the control lines for the animation, and how visually pleasing theresult is. For very high marks a more challenging morph than the exampleabove would be expected.

• 20% for general report quality and a discussion of your implementation ofthe algorithm. This includes clarity of presentation, and some explanationof how your implementation works.

• 20% for experimental design, including the choice of test cases to use, anexplanation of how you ran your experiments, and how you evaluate theresults. I do not expect an exhaustive set of experiments (that would betoo much work), but you should discuss limitations of your results andhow you might extend them given more time.

• 20% for discussion of the results of the experiments. This includes howyou present the results, the conclusions you draw from those results, andhow well the two support each other.

Late assignment submissions will be penalised at the rate of 10% per workingday. Extensions to the deadline may be granted where appropriate, but shouldbe sought well in advance. The usual university regulations relating to academicintegrity apply (http://www.otago.ac.nz/study/academicintegrity/), andany work you submit must be your own, or be clearly attributed to the originalauthor.

7

6 Deliverables

You should email me (mailto:steven@cs.otago.ac.nz)the following material bythe deadline:

• The source code and makefiles for your implementation. This should buildand run on the lab machines under OS X.

• A sample morph, consisting of two images and a text file describing thecontrol lines. Your report should include clear instructions for runningthis morph.

• Your report, preferably in PDF format. I would expect that reports wouldbe about 6-8 pages, but this will vary and figures and illustrations maymake your report longer.

If you are sending .zip files from outside of the department, they may becaught by the university’s spam filters. The filters aren’t too smart though, soreplacing the extension with .piz or .txt stops this. If you do this then let meknow in the body of the email what you have done.

Acknowledgements

The portrait of George W. Bush is in the public domain. The portrait of BarackObama from the Obama-Biden Transition Project is used under the CreativeAttribution Commons 3.0 (CC-BY-3.0) License.

References

[1] T. Beier and S. Neely, Feature-Based Image Metamorphosis, ComputerGraphics 26(2), 35–42, 1992

8