thesis reddy
Post on 19-Feb-2018
232 Views
Preview:
TRANSCRIPT
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 1/54
A Non-obtrusive Head Mounted A Non-obtrusive Head Mounted
Face Capture SystemFace Capture System
Thesis Committee:Thesis Committee:Dr. George C. Stockman (MainDr. George C. Stockman (Main
Advisor) Advisor)
Dr. Frank Biocca (Co-Advisor)Dr. Frank Biocca (Co-Advisor)
Dr. Charles OwenDr. Charles Owen
Dr. annick !olland ("#ternalDr. annick !olland ("#ternalFac$ltFac$lt
Chandan K. Reddy Chandan K. Reddy
Masters Thesis !e"enseMasters Thesis !e"ense
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 2/54
Modes o" CommunicationModes o" Communication
&e#t onl% - e.g. Mail' "lectronic Mail&e#t onl% - e.g. Mail' "lectronic Mail
oice onl% e.g. &ele*hone oice onl% e.g. &ele*hone
+C camera ,ased conerencing e.g.+C camera ,ased conerencing e.g.
e, came, cam
M$lti-$ser &eleconerencingM$lti-$ser &eleconerencing
&eleconerencing thro$gh irt$al&eleconerencing thro$gh irt$al
"nvironments"nvironments
A$gmented !ealit% Based A$gmented !ealit% Based
&eleconerencing&eleconerencing
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 3/54
#rob$em !e%nition#rob$em !e%nition
Face Ca*t$re S%stem ( FCS )
irt$al iew S%nthesis
De*th "#traction and /D FaceModeling
0ead Mo$nted +ro1ection Dis*la%s
/D &ele-immersive "nvironments
0igh Bandwidth 2etwork Connections
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 4/54
Thesis ContributionsThesis Contributions
Com*lete hardware set$* or the FCS.Com*lete hardware set$* or the FCS.
Camera-mirror *arameter estimation or theCamera-mirror *arameter estimation or the
o*timal con3g$ration o the FCS.o*timal con3g$ration o the FCS.
Generation o 4$alit% rontal videos rom two sideGeneration o 4$alit% rontal videos rom two side
videos videos
!econstr$ction o te#t$re ma**ed /D ace model!econstr$ction o te#t$re ma**ed /D ace model
rom two side viewsrom two side views "val$ation mechanisms or the generated rontal"val$ation mechanisms or the generated rontal
views. views.
C
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 5/54
&'istin( Face Capture&' st n( Face CaptureSystemsSystems
Advantages 5 Advantages 5 Freedom or 0ead MovementsFreedom or 0ead Movements
Draw,acks 5Draw,acks 5 O,str$ction o the $ser6s Field o viewO,str$ction o the $ser6s Field o view
Main A**lications 5Main A**lications 5 Character Animation and Mo,ileCharacter Animation and Mo,ile
environmentsenvironments
CourteCourte
sy :sy :
FaceCap)d - aFaceCap)d - aproductproduct
"rom Standard"rom Standard!eviation!eviation
*ptica$ Face Trac+er , a*ptica$ Face Trac+er , aproduct "rom Adaptiveproduct "rom Adaptive
*ptics*ptics
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 6/54
&'istin( Face Capture&'istin( Face CaptureSystemsSystems
Advantages 5 Advantages 5 2o ,$rden or the $ser2o ,$rden or the $ser
Draw,acks 5Draw,acks 5 0ighl% e4$i**ed environments and0ighl% e4$i**ed environments and
restricted head motionrestricted head motion
Main A**lications 5Main A**lications 5 &eleconerencing and&eleconerencing and
CourteCourte
sy:sy:
Nationa$ te$e-immersionNationa$ te$e-immersion
nitiativenitiative
Sea o" CamerasSea o" Cameras/NC Chappe$ Hi$$0/NC Chappe$ Hi$$0
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 7/54
#roposed Face Capture#roposed Face CaptureSystemSystem
2ovel Face Ca*t$re S%stem that is ,eing2ovel Face Ca*t$re S%stem that is ,eing
develo*ed.develo*ed.
&wo Cameras ca*t$re the corres*onding&wo Cameras ca*t$re the corres*onding
(F. Biocca and . +. !olland' 7&ele*ortal ace-to-ace s%stem8'
+atent Filed' 9:::.)
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 8/54
Advanta(es Advanta(es
;ser6s 3eld o view is $no,str$cted;ser6s 3eld o view is $no,str$cted +orta,le and eas% to $se+orta,le and eas% to $se Gives ver% acc$rate and 4$alit% aceGives ver% acc$rate and 4$alit% ace
imagesimages Can *rocess in real-timeCan *rocess in real-time Sim*le and $ser-riendl% s%stemSim*le and $ser-riendl% s%stem
Static with res*ect to h$man headStatic with res*ect to h$man head Fli**ing the mirror cameras view theFli**ing the mirror cameras view the
$ser6s view*oint$ser6s view*oint
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 9/54
App$ications App$ications
Mo,ile "nvironmentsMo,ile "nvironments
Colla,orative ork Colla,orative ork
M$lti-$ser &eleconerencingM$lti-$ser &eleconerencing Medical AreasMedical Areas
Distance <earningDistance <earning
Gaming and "ntertainment ind$str%Gaming and "ntertainment ind$str% OthersOthers
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 10/54
System !esi(nSystem !esi(n
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 11/54
&1uipment Re1uired&1uipment Re1uired
0ardware0ardware 9 li*stick cameras9 li*stick cameras
9 lenses with ocal length9 lenses with ocal length
=9mm=9mm9 mirrors with =.> inch9 mirrors with =.> inchdiameterdiameter
9 Matro# Meteor ??9 Matro# Meteor ??standard cardsstandard cards
<ighting e4$i*ment<ighting e4$i*ment
GA to 2&SC Converter GA to 2&SC Converter
A +ro1ector A +ro1ector A Micro*hone A Micro*hone
SotwareSotware 2etwork 2etwork M?< <?&" @.:M?< <?&" @.:
is$al St$dio .: is$al St$dio .:
Ado,e +remiere .: Ado,e +remiere .:
So$nd !ecorderSo$nd !ecorder
?nternet 9?nternet 9
2AC /::: M+"G "ncoder2AC /::: M+"G "ncoder
2AC ::: M"G Decoder2AC ::: M"G Decoder
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 12/54
*ptica$ 2ayout*ptica$ 2ayout
&hree Com*onents to ,e considered&hree Com*onents to ,e considered CameraCamera
MirrorMirror
0$man Face0$man Face
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 13/54
Speci%cation #arametersSpeci%cation #arameters
CameraCamera Sensing area5 /.9 mm 9. mm (8).Sensing area5 /.9 mm 9. mm (8). +i#el Dimensions5 ?mage sensed is o+i#el Dimensions5 ?mage sensed is o
dimensions @E *i#els. Digitied imagedimensions @E *i#els. Digitied image
sie is /9: 9: d$e to restrictions o thesie is /9: 9: d$e to restrictions o the!AM sie.!AM sie.
Focal <ength(Fc)5 =9 mm (C< =9;M).Focal <ength(Fc)5 =9 mm (C< =9;M). Field o iew (FO)5 =>.9Field o iew (FO)5 =>.9 :: ==. ==. ::..
Diameter (Dc)5 =9mmDiameter (Dc)5 =9mm Fn$m,er (2c)5 = -achieve ma#im$m lightness.Fn$m,er (2c)5 = -achieve ma#im$m lightness. Minim$m orking Distance (MD)- 9:: mm.Minim$m orking Distance (MD)- 9:: mm. De*th o Field (DOF)5 to ,e estimatedDe*th o Field (DOF)5 to ,e estimated
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 14/54
Speci%cation #arameters Contd.0Speci%cation #arameters Contd.0
MirrorMirror Diameter (Dm) I Fn$m,er (2m)Diameter (Dm) I Fn$m,er (2m) Focal <ength (m)Focal <ength (m) Magni3cation actor (Mm)Magni3cation actor (Mm)
!adi$s o c$rvat$re (!m)!adi$s o c$rvat$re (!m) 0$man Face0$man Face
0eight o the ace to ,e ca*t$red (0J 9>:mm)0eight o the ace to ,e ca*t$red (0J 9>:mm) idth o the ace to ,e ca*t$red (J =@> mm)idth o the ace to ,e ca*t$red (J =@> mm)
DistancesDistances Distance ,etween the camera and the mirror.Distance ,etween the camera and the mirror.
(D(DcmcmJ=>:mm)J=>:mm)
Distance ,etween the mirror and the ace. (DDistance ,etween the mirror and the ace. (Dmf mf
J9::mm)J9::mm)
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 15/54
Customi3ation o" Cameras andCustomi3ation o" Cameras andMirrorsMirrors
OK-the-shel camerasOK-the-shel cameras C$stomiing camera lens is a tedio$s task C$stomiing camera lens is a tedio$s task &rade-oK has to ,e made ,etween the 3eld o view&rade-oK has to ,e made ,etween the 3eld o view
and the de*th o 3eldand the de*th o 3eld
Son% DC <S= with =9mm lens is s$ita,le or o$rSon% DC <S= with =9mm lens is s$ita,le or o$ra**licationa**lication
C$stom designed mirrorsC$stom designed mirrors A *lano-conve# lens with :mm diameter is coated A *lano-conve# lens with :mm diameter is coated
with ,lack on the *lanar side.with ,lack on the *lanar side. &he radi$s o c$rvat$re o the conve# s$race is&he radi$s o c$rvat$re o the conve# s$race is
=>>.: mm.=>>.: mm. &he thickness at the center o the lens is > mm.&he thickness at the center o the lens is > mm. &he thickness at the edge is /.@ mm.&he thickness at the edge is /.@ mm.
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 16/54
4$oc+ dia(ram o" the system4$oc+ dia(ram o" the system
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 17/54
&'perimenta$ setup&'perimenta$ setup
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 18/54
5irtua$ 5ideo Synthesis 5irtua$ 5ideo Synthesis
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 19/54
#rob$em Statement#rob$em Statement
6eneratin( virtua$ "ronta$ vie7 "rom t7o side6eneratin( virtua$ "ronta$ vie7 "rom t7o side
vie7s vie7s
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 20/54
!ata processin(!ata processin(
&wo s%nchronied videos are ca*t$red in real-&wo s%nchronied videos are ca*t$red in real-time (/: ramesIsec) sim$ltaneo$sl%.time (/: ramesIsec) sim$ltaneo$sl%.
For eKective ca*t$ring and *rocessing' theFor eKective ca*t$ring and *rocessing' thedata is stored in $ncom*ressed ormat.data is stored in $ncom*ressed ormat.
Machine S*eci3cations (<orelei LMachine S*eci3cations (<orelei Lmetla,.cse.ms$.ed$)5metla,.cse.ms$.ed$)5 +enti$m ??? *rocessor+enti$m ??? *rocessor +rocessor s*eed5 @ M0+rocessor s*eed5 @ M0 !AM Sie5 /E MB!AM Sie5 /E MB 0ard Disk write S*eed (*ractical)5 MBIs0ard Disk write S*eed (*ractical)5 MBIs
M?<-<?&" is con3g$red to $se =>: MB o !AMM?<-<?&" is con3g$red to $se =>: MB o !AM
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 21/54
!ata processin( Contd.0!ata processin( Contd.0
Sie o = second video /: N /9: N 9: N/Sie o = second video /: N /9: N 9: N/
.> MB .> MB ;sing =>: MB !AM' onl% =: seconds video;sing =>: MB !AM' onl% =: seconds video
rom two cameras can ,e ca*t$redrom two cameras can ,e ca*t$red h% does the *rocessing have to ,e oinePh% does the *rocessing have to ,e oineP
Cali,ration *roced$re is not a$tomaticCali,ration *roced$re is not a$tomatic Disk writing s*eed m$st ,e at least = MBIS.Disk writing s*eed m$st ,e at least = MBIS.
&o ca*t$re 9 videos o : N E: resol$tion' the&o ca*t$re 9 videos o : N E: resol$tion' theDisk writing s*eed m$st ,e at least > MBIS PPPDisk writing s*eed m$st ,e at least > MBIS PPP
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 22/54
Structured 2i(htStructured 2i(httechni1uetechni1ue
A s4$are grid in the A s4$are grid in the
rontal view a**ears as arontal view a**ears as a
4$adrilateral (with c$rved4$adrilateral (with c$rved
edges) in the real sideedges) in the real side
+ro1ecting a grid on+ro1ecting a grid on
the rontal view othe rontal view o
the acethe ace
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 23/54
Co$or 4a$ancin(Co$or 4a$ancin(
Hardware based approach White balancing of the cameras
Why this is more robust ? – why not softwarebased ? There is no change in the input camera
Better handling of varying lighting conditions
No pre - knowledge of the skin color is reuired
No additional overhead
!ts enough if both cameras are color balanced relativel
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 24/54
Left Calibration Left Calibration
Face Image Face Image
Transformation TablesTransformation Tables
Projector Projector
Right Calibration Right Calibration
Face Image Face Image
*8-$ine Ca$ibration Sta(e*8-$ine Ca$ibration Sta(e
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 25/54
TransformationTransformation
TablesTables
Right Right
Face Image Face Image
Left Left
Face Image Face Image
Right Warped Right Warped
Face ImageFace Image
Left Warped Left Warped
Face ImageFace Image
Mosaiced Face ImageMosaiced Face Image
*perationa$ Sta(e*perationa$ Sta(e
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 26/54
5irtua$ video synthesis 5irtua$ video synthesisCa$ibration phase0Ca$ibration phase0
5irtua$ video synthesis5irtua$ video synthesis
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 27/54
5irtua$ video synthesis 5irtua$ video synthesiscontd.0contd.0
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 28/54
5irtua$ Fronta$ 5ideo 5irtua$ Fronta$ 5ideo
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 29/54
Comparison o" the Fronta$ 5ie7sComparison o" the Fronta$ 5ie7s
First ro7 , 5irtua$ "ronta$ vie7sFirst ro7 , 5irtua$ "ronta$ vie7s
Second ro7 , *ri(ina$ "ronta$ vie7sSecond ro7 , *ri(ina$ "ronta$ vie7s
id h i i
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 30/54
5ideo Synchroni3ation &ye 5ideo Synchroni3ation &yeb$in+in(0b$in+in(0
First ro7 , 5irtua$ "ronta$ vie7sFirst ro7 , 5irtua$ "ronta$ vie7s
Second ro7 , *ri(ina$ "ronta$ vie7sSecond ro7 , *ri(ina$ "ronta$ vie7s
h h d
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 31/54
Face !ata throu(h HeadFace !ata throu(h HeadMounted SystemMounted System
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 32/54
)! Face Mode$)! Face Mode$
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 33/54
Coordinate SystemsCoordinate Systems
There are "ve coordinate systems in our application
World #oordinate $ystem %W#$&
'ace #oordinate $ystem %'#$& (eft #amera #oordinate system %(##$&
)ight #amera #oordinate system %)##$&
*ro+ector #oordinate $ystem %*#$&
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 34/54
Camera Ca$ibrationCamera Ca$ibration
Conversion rom /D world coordinates to 9DConversion rom /D world coordinates to 9Dcamera coordinates - +ers*ective &ransormationcamera coordinates - +ers*ective &ransormation
ModelModel
ss << ++rr
ss << ++cc
ss
CC==== CC=9=9 CC=/=/
CC==
CC9=9= CC9999 CC9/9/
CC99
CC/=/= CC/9/9 CC//// ==
9 9
ss ++ % %
ss ++
==
ss ++##
"liminating the scale actor"liminating the scale actor
$$ 1 1 (c (c==== c c/=/= $$ 1 1) #) # 1 1 Q (cQ (c=9=9 c c/9/9 $$ 1 1) %) % 1 1 Q (cQ (c=/=/ c c////
$$ 1 1) ) 1 1 Q cQ c==
v v 1 1 (c (c9=9= c c/=/= v1) #1 Q (c v1) #1 Q (c9999 c c/9/9 v1) %1 Q (c v1) %1 Q (c9/9/
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 35/54
Ca$ibration sphereCa$ibration sphere
A s*here can ,e $sed or Cali,ration A s*here can ,e $sed or Cali,ration
Cali,ration *oints on the s*here areCali,ration *oints on the s*here are
chosen in s$ch a wa% that thechosen in s$ch a wa% that the
Aim$thal angle is varied in ste*s o Aim$thal angle is varied in ste*s o
>>oo
+olar angle is varied in ste*s o /:+olar angle is varied in ste*s o /:oo
&he location o these cali,ration&he location o these cali,ration
*oints is known in the /D coordinate*oints is known in the /D coordinate
S%stem with res*ect to the origin oS%stem with res*ect to the origin o
the s*herethe s*here &he origin o the s*here de3nes the&he origin o the s*here de3nes the
origin o the orld Coordinateorigin o the orld Coordinate
S%stemS%stem
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 36/54
#roector Ca$ibration#roector Ca$ibration
Similar to Camera Cali,rationSimilar to Camera Cali,ration 9D image coordinates can not ,e o,tained9D image coordinates can not ,e o,tained
directl% rom a 9D image.directl% rom a 9D image.
A 7Blank ?mage8 is *ro1ected onto the s*here A 7Blank ?mage8 is *ro1ected onto the s*here
&he 9D coordinates o the cali,ration *oints&he 9D coordinates o the cali,ration *ointson the *ro1ected image are notedon the *ro1ected image are noted
More *oints can ,e seen rom the *ro1ector6sMore *oints can ,e seen rom the *ro1ector6s*oint o view some *oints are common to*oint o view some *oints are common to,oth camera views,oth camera views
!es$lts a**ear to have slightl% more errors!es$lts a**ear to have slightl% more errorswhen com*ared to the camera cali,rationwhen com*ared to the camera cali,ration
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 37/54
)! Face Mode$ Construction)! Face Mode$ Construction
h%Ph%P &o o,tain diKerent views o the ace&o o,tain diKerent views o the ace
&o generate the stereo *air to view it in&o generate the stereo *air to view it in
the 0M+Dthe 0M+D
Ste*s re4$iredSte*s re4$ired
Com*$tation o /D <ocationsCom*$tation o /D <ocations C$stomiation o /D ModelC$stomiation o /D Model
&e#t$re Ma**ing&e#t$re Ma**ing
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 38/54
Computation o" )!Computation o" )!
pointspoints /d *oint estimation $sing stereo/d *oint estimation $sing stereo
Stereo ,etween two cameras is not *ossi,leStereo ,etween two cameras is not *ossi,le,eca$se o the occl$sion ,% the acial,eca$se o the occl$sion ,% the acialeat$reseat$res
0ence two stereo *air com*$tations0ence two stereo *air com*$tations <et camera and *ro1ector<et camera and *ro1ector !ight camera and *ro1ector!ight camera and *ro1ector
;sing stereo' com*$te /D *oints o;sing stereo' com*$te /D *oints o*rominent acial eat$re *oints in FCS*rominent acial eat$re *oints in FCS
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 39/54
)! 6eneric Face Mode$)! 6eneric Face Mode$
A generic ace model with /> vertices and A generic ace model with /> vertices andE=E trianglesE=E triangles
<et5 ront view and !ight5 side view<et5 ront view and !ight5 side view
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 40/54
Te'ture Mapped )! FaceTe'ture Mapped )! Face
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 41/54
&va$uation&va$uation
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 42/54
&va$uation Schemes&va$uation Schemes
"val$ation o acial e#*ressions and is not"val$ation o acial e#*ressions and is notst$died e#tensivel% in literat$rest$died e#tensivel% in literat$re
"val$ation can ,e done or acial alignment'"val$ation can ,e done or acial alignment'
ace recognition or static imagesace recognition or static images
<i* and e%e movements in a d%namic event<i* and e%e movements in a d%namic event
+erce*t$al 4$alit% 0ow are the moods+erce*t$al 4$alit% 0ow are the moods
conve%edPconve%edP
&wo t%*es o eval$ation&wo t%*es o eval$ation O,1ective eval$ationO,1ective eval$ation
S$,1ective eval$ationS$,1ective eval$ation
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 43/54
*bective &va$uation*bective &va$uation
&heoretical "val$ation&heoretical "val$ation 2o h$man eed,ack re4$ired2o h$man eed,ack re4$ired &his eval$ation can give $s a meas$re&his eval$ation can give $s a meas$re
o o Face recognitionFace recognition Face alignmentFace alignment Facial movementsFacial movements
Methods a**liedMethods a**lied 2ormalied cross correlation2ormalied cross correlation "$clidean distance meas$res"$clidean distance meas$res
& $ ti & $ ti
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 44/54
&va$uation ma(es&va$uation ma(es
; "rames 7ere considered "or obective; "rames 7ere considered "or obectiveeva$uationeva$uation
First ro7 , virtua$ "ronta$ vie7sFirst ro7 , virtua$ "ronta$ vie7s
Second ro7 , ori(ina$ "ronta$ vie7sSecond ro7 , ori(ina$ "ronta$ vie7s
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 45/54
Norma$i3ed Cross-Norma$i3ed Cross-
Corre$ationCorre$ation
!egions considered or normalied cross-!egions considered or normalied cross-correlationcorrelation
( <et5 !eal image !ight5 irt$al image)( <et5 !eal image !ight5 irt$al image)
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 46/54
Norma$i3ed Cross-Norma$i3ed Cross-
Corre$ationCorre$ation
<et ,e the virt$al image and ! ,e the real image<et ,e the virt$al image and ! ,e the real image <et w ,e the width and h ,e the height o the images<et w ,e the width and h ,e the height o the images
&he 2ormalied Cross-correlation ,etween the two images and ! is given ,%&he 2ormalied Cross-correlation ,etween the two images and ! is given ,%
wherewhere
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 47/54
Norma$i3ed Cross-Norma$i3ed Cross-
Corre$ationCorre$ation ideo ideo
FramFrameses
<et<ete%ee%e
!ight!ighte%ee%e
Mo$tMo$thh
"%es"%esQQ
Mo$tMo$t
hh
Com*leCom*lete acete ace
FramFrame=e=
:.EE:.EE :.E@:.E@ :./:./ :.E:.E :.E:.E
FramFrame9e9
:.:. :.@9:.@9 :.E>:.E> :.@E:.@E :.E>:.E>
FramFrame/e/
:.:. :.@:.@ :.9:.9 :.@E:.@E :.E:.E
& $id !i t& c$idean !istance
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 48/54
&uc$idean !istance&uc$idean !istancemeasuresmeasures
"$clidean distance ,etween two *oints i and 1 is given ,%"$clidean distance ,etween two *oints i and 1 is given ,%
<et !i1 ,e the e$clidean distance ,etween two *oints i and 1 in<et !i1 ,e the e$clidean distance ,etween two *oints i and 1 in
the real imagethe real image
<et i1 ,e the e$clidean distance ,etween two *oints i and 1 in<et i1 ,e the e$clidean distance ,etween two *oints i and 1 in
the virt$al imagethe virt$al image
!i 9 < Ri - 5i <!i 9 < Ri - 5i <
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 49/54
&uc$idean !istance&uc$idean !istance
measuresmeasuresramesrames DDa a DD, , DDc c DDcgcg DDdgdg DDegeg "rror"rror
FrameFrame==
9.:9.:::
:.E:.E::
.=.=>>
/./.
9.9.>>
/./.
9.E:9.E:
FrameFrame99
:.>:.>
/.:/.:::
:.@:.@
..==
:.:.//
:.E:.E::
=.@=.@
FrameFrame
//
=.E=.E
EE
/.E/.E
.9.9
././
9.9.
EE
=.E=.E
//
/.=/.=
FrameFrame
=.:=.:
9.9.@@
9.=9.=::
././//
/.:/.:==
.:.:EE
/.//./
FrameFrame =.=. 9.99.9 >.>>.> .. =.9=.9 =.=. 9.99.9
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 50/54
Subective &va$uationSubective &va$uation
"val$ates the h$man *erce*tion"val$ates the h$man *erce*tion Meas$rement o 4$alit% o a talking aceMeas$rement o 4$alit% o a talking ace Factors that might aKectFactors that might aKect
R$alit% o the videoR$alit% o the video Facial movements and e#*ressionsFacial movements and e#*ressions S%nchroniation o the two halves o the aceS%nchroniation o the two halves o the ace Color and &e#t$re o the aceColor and &e#t$re o the ace R$alit% o a$dioR$alit% o a$dio S%nchroniation o a$dioS%nchroniation o a$dio
A *reliminar% st$d% has ,een made to A *reliminar% st$d% has ,een made toassess the 4$alit% o the generated videosassess the 4$alit% o the generated videos
C $ iC $ i dd F tF t
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 51/54
Conc$usionConc$usion andand FutureFuture =or+ =or+
5irtua$
Fronta$ ma(e
Te'ture Mapped)! Face Mode$
5irtua$
Fronta$ 5ideo
)! Facia$ Animation
Conc$usion Future =or+
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 52/54
Summary Summary
Design and im*lementation o a novelDesign and im*lementation o a novel
Face Ca*t$re S%stemFace Ca*t$re S%stem
Generation o virt$al rontal view romGeneration o virt$al rontal view rom
two side views in a video se4$encetwo side views in a video se4$ence "#traction o de*th inormation $sing"#traction o de*th inormation $sing
stereo methodstereo method
&e#t$re ma**ed /D ace model&e#t$re ma**ed /D ace modelgenerationgeneration
"val$ation o virt$al rontal videos"val$ation o virt$al rontal videos
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 53/54
Future =or+ Future =or+
Online *rocessing in real-timeOnline *rocessing in real-time
A$tomatic cali,ration A$tomatic cali,ration
/D acial animation/D acial animation S$,1ective "val$ation o the virt$al rontal videosS$,1ective "val$ation o the virt$al rontal videos
Data com*ression while *rocessing andData com*ression while *rocessing and
transmissiontransmission
C$stomiation o camera lensesC$stomiation o camera lenses ?ntegration with a 0ead Mo$nted +ro1ection?ntegration with a 0ead Mo$nted +ro1ection
Dis*la%Dis*la%
7/23/2019 Thesis Reddy
http://slidepdf.com/reader/full/thesis-reddy 54/54
Than+ >ouThan+ >ou
Do$,ts'Do$,ts'
R$eriesR$eries
S$ggestionsS$ggestions
top related