By,

FensaMerry Saj

LBSITW

OVERVIEW Introduction

RelatedWork

System Setup

Reconstruction Approach

Results

Conclusion

References

10/5/2013 2

INTRODUCTION Many computer graphics applications require realistic3D models of human bodies.

Depth cameras such as Microsoft Kinects are able tocapture depth and image data at video rate.

Kinect is compact, low-price and as easy to use as avideo camera.

10/5/2013 3

10/5/2013 4

RELATED WORK

Scanning devices based on structured light or laserscan can capture human body with much high quality,but is very expensive (about $240,000).

Two main approaches employed in depth cameratechnology are:-

based on the time-of-flight principle, measuringtime delay between transmissions of a light pulse(about$8,000).

based on light coding; projecting a known infraredpattern onto the scene and determining depth based onthe pattern’s deformation.

10/5/2013 5

RELATED WORK(contd…)A most popular one based on light coding is the MicrosoftKinect Sensor which is at a price of only $150.

3 Main Types:-

Registration without a template.

This method requires high quality scan data & needs smallchanges in temporal coherence.

Registration with a template.

This method needs a relative accurate template & then usesthe template to fit each scan.

10/5/2013 6

RELATED WORK(contd…)Registration with a semi-template.

Rough template, such as the skeleton model ofarticulated object, can be utilized.

The first type requires high quality input data & iscomputationally expensive; the second one needs anaccurate template which is hard to fulfil in manyapplications.

Here, this system uses the third type.

10/5/2013 7

RELATED WORK( contd…)

The idea of creating a graph of pairwise alignmentsbetween scans.

First, pairwise rigid alignment is computed in thegeometric level.

Global error distribution then operates on an upper level,where errors are measured in terms of the relative rotationsand translations of pairwise alignments.

The graph methods can simultaneously minimize theerrors of all views rapidly and do not need all scan inmemory.

10/5/2013 8

SYSTEM SETUP Two kinects are used to capture the upper and thelower part of a human body respectively, withoutoverlapping region, from one direction.

A third kinect is used to capture the middle part of thehuman body from the opposite direction.

The distance between two sets of Kinects is about 2meters.

A turntable is put in between them.

10/5/2013 9

The setup of our system

10/5/2013 10

RECONSTRUCTION APPROACH

Denote Di={Mi,Ii},i=1,….n as the captured data, n is thenumber of captured frames, Mi is the merged mesh & Ii isthe corresponding image of the i-th frame respectively.

First, a rough template is constructed.

The template is used to deform the geometry of successiveframes pair wisely.

Global registration is performed to distribute errors in thedeformation space.

Finally, reconstructed model is generated using Poissonreconstruction method.

10/5/2013 11

An accurate template is unavailable.

We construct an estimated body shape as the templatemesh T1 from the first frame.

It is impossible to use this template to register eachframe by geometry fitting but it can track the pairwisedeformation of successive frames.

T1={v1^k,k=1…K; K is the number of nodes of T1(typically 50-60).

10/5/2013 12

Suppose Mi, i=1…n forming a cycle. fi,j denotes theregistration that can deform mesh Mi to register withmesh Mj.

To find the pairwise registration f1,2,f2,3,…fn-1,n,fn,1.

Deformation Model:

Suppose we have two meshes Mi & Mj, and templatemesh at frame i is Ti, then

10/5/2013 13

Pairwise registration:

For successive frames Mi and Mi+1,corresponding featurepoints are obtained by optical flow in the correspondingimages.

Projection to the first frame:

n-1 pair wise deformation is required to recover all therelative position of all frames.(refer fig.6)

The desired pairwise deformation f̂1,2,f̂2,3,…f̂n-1,n,f̂n,1should meet the following conditions:

1.It is cyclic consistent.

2.The original pairwise deformation is relatively correct, sominimize the weighted square error of the new and olddeformation.

10/5/2013 14

Overview of our reconstruction algorithm

10/5/2013 15

10/5/2013 16

Different 3D full human models generated by the system

10/5/2013 17

RESULTS Global non-rigid registration gives better result thanglobal rigid alignment.

Since the color image and depth information arecaptured simultaneously and calibrated, the colorinformation of deformed mesh is generatedautomatically.

Virtual try on.

Personalized avatar-video games,online shopping ,human computer interaction etc.

10/5/2013 18

Realistic virtual try on experience based on the reconstructed model.(Left)the try on results;(right)the corresponding meshes.

10/5/2013 19

Personalized avatar generated by our system.The motion of the human body is driven by a given skeleton motion sequence

10/5/2013 20

CONCLUSION The proposed method can deal with non-rigidalignment and complex occlusions.

The two stage registration algorithm is efficient and ofmemory efficiency.

The system can generate convincing 3D human bodiesat a much low price.

It has good potential for home oriented VRapplications.

10/5/2013 21

REFERENCES [1]Jing Tong; Jin Zhou; Ligang Liu; Zhigeng Pan; HaoYan, ”Scanning 3D Full Human Bodies UsingKinects”,Visualization and Computer Graphics, IEEETransactions on, vol.18, no.4, April 2012.

[2]Srivishnu Satyavolu,Gerd Bruder,PeteWillemsen,Frank Stenicke,”Analysis of IR-basedvirtual reality tracking using multiple Kinects”,2012IEEE Virtual Reality,2012.

10/5/2013 22

10/5/2013 23

10/5/2013 24