Challenges in Perception for Learning, Cognition and Control
Approaches
DariusBurschkaMachineVisionPerceptionGroup(MVP)TechnischeUniversitätMünchen,Germany
DariusBurschka–MachineVisionandPerceptionGroup(TUM)
Structure of a CNN for Robotics Applications
Rawsensorinformation(depthimagefromanRGB-Dsensor)istakenasinputtoconnectwiththerobotcontrolcommanddirectly
Idealcase:learnthecouplingbetweenperceptionandcontrolcommandsfromdemonstrations
DariusBurschka–MachineVisionandPerceptionGroup(TUM)
Curse of Complexity in Direct ProcessingIntheory,onecouldusealltheextractedfeatureswithaclassifiersuchasasoftmaxclassifier,butthiscanbecomputationallychallenging.Forinstanceimagesofsize96x96pixelswith400learnedfeaturesover8x8inputs.Eachconvolutionresultsinanoutputofsize(96−8+1)∗(96−8+1)=7921,andsincewehave400features,thisresultsinavectorof892∗400=3,168,400featuresperexample.Learningaclassifierwithinputshaving3+millionfeaturescanbeunwieldy,andcanalsobepronetoover-fitting.
DariusBurschka–MachineVisionandPerceptionGroup(TUM)
Deep-Learning for Perception in Robotics
Whatdoallthesetaskshaveincommon?
Source:L.Tai,“Deep-LearninginMobileRobotics…”
DariusBurschka–MachineVisionandPerceptionGroup(TUM)
Learning Approaches vs. Conventional Tools
Imagesource:Berkley
Grouping/Segmentation,Labeling,Identification
becausedifferencesinthepropertiesofpixelstoentireimageareconsideredduetothestructureofCNNs
Problem:notclear,whattheprocessingisbasedon!
?
Hand-designedfeatureshelptoestimatemetricdistancesbetweenthemusingcalibrationparameters
DariusBurschka–MachineVisionandPerceptionGroup(TUM)
Deep-Learning for Control in Robotics
Source:L.Tai,“Deep-LearninginMobileRobotics…”
DariusBurschka–MachineVisionandPerceptionGroup(TUM)
Can we avoid Metric Information in Representation of Environments?
DariusBurschka–MachineVisionandPerceptionGroup(TUM)
Fast Uncalibrated Monocular Estimation of Independent Motion Components
Representation of the Environment in Collision Space
DariusBurschka–MachineVisionandPerceptionGroup(TUM)
Examples of non-metric Control (RoboMobil)
DariusBurschka–MachineVisionandPerceptionGroup(TUM)
Examples of non-metric Control (RoboMobil)
DariusBurschka–MachineVisionandPerceptionGroup(TUM)
Conclusions
• Imagelabelingandimageretrieval(indexing)representalargeapplicationfieldforperceptionandimageprocessing(Google,Apple,etc.),butroboticrequiresoftenametricmappingfromthesensorontocontrolvalues• Workonalternativesforcouplingbetweenthesensorandtheactuatorshouldbeoneoftheperceptiongoals• Currentlyuseddatarepresentationsneedtobere-definedforlearningapplications.
