omnidirectional vision for mobile robotspsfmr.univpm.it/slide/menegatti.pdffrom an optical point of...
TRANSCRIPT
-
Omnidirectional Vision for Omnidirectional Vision for Mobile RobotsMobile Robots
Emanuele MenegattiEmanuele Menegatti
IASIAS--Lab Lab Intelligent Autonomous Intelligent Autonomous
Systems LaboratorySystems Laboratory
University of PaduUniversity of Paduaa
ITALYITALY
-
E. Menegatti - Omnidirectional vision 2
HistoryHistory
Animals Animals PaintingsPaintingsPanoramasPanoramasPhotographyPhotographyComputer VisionComputer VisionRobot VisionRobot Vision
ChapterChapter 1:1:
-
3
Animal Vision Animal Vision -- 112 Millions species2 Millions speciesFew hundred distinct organisational plansFew hundred distinct organisational plans
BUT BUT only two basic types of only two basic types of imaging eyes of wide use:imaging eyes of wide use:
••Single lens camera like eyesSingle lens camera like eyes
••MultiMulti--lens compound eyeslens compound eyesFrom R. Dawkins: Climbing Mount Improbable. Norton, 1996
-
E. Menegatti - Omnidirectional vision 4
AnimalAnimal Vision Vision -- 22Q: Q: Why are perspective systems insufficient and why is field of Why are perspective systems insufficient and why is field of view important?view important?A: A: Perspective systems are one imaging modality of many, we Perspective systems are one imaging modality of many, we are interested in sensors better suited to specific tasks. Sensoare interested in sensors better suited to specific tasks. Sensor r modality should enter into design of computer vision systemsmodality should enter into design of computer vision systems
For example, perhaps for flight For example, perhaps for flight wide fieldwide field--ofof--view sensorsview sensors are are appropriate, and in general useful appropriate, and in general useful for mobile robots.for mobile robots.
[Slide from C. Geyersee Reference Section]
-
E. Menegatti - Omnidirectional vision 5
OmnivisionOmnivision in Animalsin Animals
500 million years ago500 million years agoTrilobitesTrilobitesDiurnal InsectsDiurnal InsectsNocturnal InsectsNocturnal InsectsCrustaceousCrustaceousGigantocyprisGigantocypris(not (not omnivisionomnivision))
-
E. Menegatti - Omnidirectional vision 6
OmnivisionOmnivision in Paintingsin Paintings
““The wedding of Giovanni The wedding of Giovanni ArnolfiniArnolfini” J. Van ” J. Van EyckEyck 13901390--14411441
Witch MirrorsWitch Mirrors
-
E. Menegatti - Omnidirectional vision 7
OmnivisionOmnivision in Paintingsin Paintings
““The praetor and his wife” Q. The praetor and his wife” Q. MetsysMetsys 14661466--15301530Witch MirrorsWitch Mirrors
-
E. Menegatti - Omnidirectional vision 8
Distortion in PaintingsDistortion in Paintings
AnamorphosisAnamorphosis
-
E. Menegatti - Omnidirectional vision 9
Distortion in PaintingsDistortion in Paintings
AnamorphosisAnamorphosis
-
E. Menegatti - Omnidirectional vision 10
PanoramasPanoramas
Patented by Patented by Robert BarkerRobert Barker -- 17871787
(meaning of the XIX century)(meaning of the XIX century)
-
E. Menegatti - Omnidirectional vision 11
PhotographyPhotography
Edinburgh Edinburgh -- Courtesy of Courtesy of EdVecEdVec
MaunaMauna--Kea ObservatoryKea Observatory
-
E. Menegatti - Omnidirectional vision 12
Omnidirectional Cameras Omnidirectional Cameras -- 11
CompoundCompound--eye eye camera camera
(from (from Univ. of Maryland, College Park. )
Panoramic cameras Panoramic cameras (from Apple)(from Apple)
Omnidirectional Omnidirectional camerascameras
(from (from University of Picardie - France)
-
E. Menegatti - Omnidirectional vision 13
Omnidirectional Cameras Omnidirectional Cameras -- 22Omnidirectional sensors come in many varieties, but by Omnidirectional sensors come in many varieties, but by definition must have a wide fielddefinition must have a wide field--ofof--view. view. From an optical point of view:From an optical point of view:
~180~180ºº FOVFOV
wide FOV wide FOV dioptricdioptriccameras (e.g. fisheye)cameras (e.g. fisheye)
~360~360ºº FOVFOV
polydioptricpolydioptric cameras (e.g. cameras (e.g. multiple overlapping cameras)multiple overlapping cameras)
>180º FOV
catadioptriccatadioptric cameras (e.g. cameras (e.g. cameras and mirror systems)cameras and mirror systems)
[Slide adapted from C. Geyersee Reference Section]
-
14
CompoundCompound--eye cameraseye cameras
The The RingcamRingcam at at Microsoft ResearchMicrosoft Research
ViewplusViewplusSoftpiaSoftpia Japan & Gifu UniversityJapan & Gifu University
Pros:- High resolutionper viewing angle
Cons:- Bandwidth- Multiple cameras
calibrating and synchronizing;
- expensive
LadybugLadybugPointGreyPointGrey
http://www.ptgrey.com/products/pands.html
-
15
CompoundCompound--eye cameraseye cameras (inward)(inward)
Virtualised RealityVirtualised RealityCMUCMU
SMART eMotion (Spin-off Univ. of Padua)
now BTS Bioengineering
-
E. Menegatti - Omnidirectional vision 16
Panoramic camerasPanoramic cameras
CirkutCirkut CameraCamera
Panning cameraPanning camera Swing lensSwing lens
RoundshotRoundshot
Pros:- High resolutionper viewing angle
Cons:- slow acquisition;- No dynamic scene- expensive
-
17
Omnidirectional cameras Omnidirectional cameras -- 11
PALPALPanoramic Annular lensPanoramic Annular lens
CatadioprticCatadioprticSensorSensor Two folded mirrorTwo folded mirror
sensorsensorCons:- Blindspot- Low resolution
Pros:- Single image
-
18
Omnidirectional cameras Omnidirectional cameras -- 22
Cons:- Low resolution at periphery
Pros:- Single image
[Slide adapted from T. Pajdlasee Reference Section]
-
E. Menegatti - Omnidirectional vision 19
Q: What kind of sensor should one use?Q: What kind of sensor should one use?A: Depends on your application.A: Depends on your application.1.1. If you are primarily concerned with:If you are primarily concerned with:
–– resolutionresolution –– surveillance (coverage)surveillance (coverage)
and can afford the bandwidth & expense,and can afford the bandwidth & expense,you might stick with you might stick with polydioptricpolydioptric solutionssolutions
2.2. If you are concerned withIf you are concerned with–– bandwidth bandwidth --mobile robots mobile robots ––servoingservoing, SFM, SFM
investigate investigate catadioptriccatadioptric or single wideor single wideFOV FOV dioptricdioptric solutionssolutions
Confused?Confused?Confused?Confused?Confused?Confused?Confused?
[Slide adapted from C. Geyersee Reference Section]
http://www.cis.upenn.edu/~kostas/omnigrasp.htmlhttp://www.fxpal.xerox.com/smartspaces/flycam/flycam_home.htm
-
E. Menegatti - Omnidirectional vision 20
Other myths and hesitationOther myths and hesitation--11Myth:Myth: CatadioptricCatadioptric images are by necessity highly distorted.images are by necessity highly distorted.Truth:Truth:
Actually no; parabolic mirrors induce no distortion Actually no; parabolic mirrors induce no distortion (perpendicular to the viewing direction).(perpendicular to the viewing direction).With Hyperbolic mirrors is possible to reconstruct With Hyperbolic mirrors is possible to reconstruct perspective imagesperspective images
Myth:Myth: Omnidirectional cameras are more complicated than Omnidirectional cameras are more complicated than perspective cameras, and harder to do SFM with.perspective cameras, and harder to do SFM with.
Truth:Truth: Actually no; parabolic mirrors are easy to model, Actually no; parabolic mirrors are easy to model, calibrate and do SFM with.calibrate and do SFM with.
[Slide adapted from C. Geyersee Reference Section]
-
E. Menegatti - Omnidirectional vision 21
Other myths and hesitationOther myths and hesitation--22Truth:Truth: Omnidirectional systems have lower resolutionOmnidirectional systems have lower resolution
(because they collect light from a larger FOV and (because they collect light from a larger FOV and capture it with an image sensor with the same size of capture it with an image sensor with the same size of perspective cameras)perspective cameras)
Tradeoff:Tradeoff:Balance resolution and field of view for your needsBalance resolution and field of view for your needsEspecially exploiting the flexibility of Especially exploiting the flexibility of catadioptriccatadioptric systems systems in which by in which by chosingchosing the curvature of the mirror profile the curvature of the mirror profile you can chose the resolution in different portion of the you can chose the resolution in different portion of the imageimage
[Slide adapted from C. Geyersee Reference Section]
-
E. Menegatti - Omnidirectional vision 22
CatadioptricCatadioptric CameraCamera
Composed of:Composed of:Standard CameraStandard CameraConvex MirrorConvex MirrorSupportSupport–– transparent cylindertransparent cylinder–– Lateral barLateral bar
-
E. Menegatti - Omnidirectional vision 23
An omnidirectional camera viewAn omnidirectional camera view
Panoramic Cylinder Panoramic Cylinder ((fromfrom Centre for Machine Perception, Praha)
OmnidirectionalOmnidirectionalimageimage
-
E. Menegatti - Omnidirectional vision 24
Omnidirectional camerasOmnidirectional cameras
AdvantagesAdvantagesWide vision fieldWide vision field
OneOne--shot imageshot image
High speedHigh speedVertical LinesVertical LinesRotational InvarianceRotational Invariance
Customisable field of viewCustomisable field of view
Customisable resolutionCustomisable resolution
DisadvantagesDisadvantagesLow ResolutionLow ResolutionDistortionsDistortions
Low readabilityLow readability
-
E. Menegatti - Omnidirectional vision 25
Omnidirectional Mirror DesignOmnidirectional Mirror DesignChapter 2:Chapter 2:
The aim is to create a custom mapping between the The aim is to create a custom mapping between the world coordinates and the image coordinates.world coordinates and the image coordinates.
Families of Mirrors Families of Mirrors Geometrical Solid FiguresGeometrical Solid FiguresCustom profileCustom profile
Algorithm for mirror designAlgorithm for mirror designx
y P
-
26
Families of Mirrors Families of Mirrors -- 11
Geometrical Solid Figures Geometrical Solid Figures (from H. Ishiguro)(from H. Ishiguro)
-
27
The mirror we designed...The mirror we designed...
Three parts:Three parts:Measurement MirrorMeasurement MirrorMarker MirrorMarker MirrorProximity MirrorProximity Mirror
Mirror ProfileMirror ProfileThe task determines the
mirror profile
E. Menegatti, E. Pagello, et al.Designing an omnidirectional vision system for a goal keeper robotRoboCup-2001: Robot Soccer World Cup V. (Springer 2001)
-
28
Examples of IASExamples of IAS--Lab MirrorsLab Mirrors
FraunhoferFraunhofer Institute’s MirrorInstitute’s MirrorRoboCup MirrorRoboCup Mirror
Smallest MirrorSmallest MirrorNow, these mirrors are sold by our SpinNow, these mirrors are sold by our Spin--off company off company IT+RoboticsIT+Robotics
-
E. Menegatti - Omnidirectional vision 29
IT+Robotics IT+Robotics srlsrlIT+Robotics is a SpinIT+Robotics is a Spin--Off Company of the Off Company of the University of PaduaUniversity of Padua
Its business are:Its business are:••Omnidirectional vision systems (HW & SW)Omnidirectional vision systems (HW & SW)••Intelligent VideoIntelligent Video--surveillance Systemssurveillance Systems••SmallSmall--scale Humanoid Robotsscale Humanoid Robots
Link: Link: http://www.it-robotics.it/
-
E. Menegatti - Omni Vision 30
Map matchingMap matchingImageImage--based localizationbased localizationObservation of Optical Flow Observation of Optical Flow BiomimeticBiomimetic BehavioursBehavioursIntegration of OmniIntegration of Omni--vision with other sensors:vision with other sensors:–– SonarSonar–– Laser range finderLaser range finder
Outdoor NavigationOutdoor NavigationSLAM (Simultaneous Localization And Mapping)SLAM (Simultaneous Localization And Mapping)Environment reconstruction & 3D mappingEnvironment reconstruction & 3D mappingMiscellaneaMiscellanea
Chapter Chapter 3:3:
OmniOmni--Vision for Mobile RobotsVision for Mobile Robots
-
E. Menegatti - Omni Vision 31
Navigation/Localization TricksNavigation/Localization Tricks
Invariance of AzimuthInvariance of AzimuthRotational InvarianceRotational InvarianceVertical Lines mapped in radial linesVertical Lines mapped in radial linesCircumferential continuityCircumferential continuityPeriodicity of the imagePeriodicity of the imageRobustness to occlusionRobustness to occlusion
-
E. Menegatti - Omnidirectional vision 32
Invariance of AzimuthInvariance of Azimuth
The azimuth of the object is maintained by the sensor
-
E. Menegatti - Omnidirectional vision 33
Rotational InvarianceRotational Invariance
Initial Position Image Counter-Rotated Robot Rotated by 90°
P2P2
P4P4 P3P3
P5P5
P1P1
-
E. Menegatti - Omni Vision 34
Vertical Lines Vertical Lines radial linesradial lines
Original Image Edge Detection+
Hough Transform
New Design to stretchvertical lines
-
E. Menegatti - Omnidirectional vision 35
Continuity & PeriodicityContinuity & Periodicity
Original Image
Panoramic Cylinder
Fourier Transform
-
E. Menegatti - Omnidirectional vision 36
Robustness to occlusionRobustness to occlusionThanks to the wide FOV, usually occluding objects do not change Thanks to the wide FOV, usually occluding objects do not change much the imagemuch the imageSeveral similarity measures have been proved to be robust to Several similarity measures have been proved to be robust to occlusionocclusion
Extreme case presented by Extreme case presented by JoganJogan & & LeonardisLeonardis
Matjaž Jogan, Aleš Leonardis“Robust localization using an “Robust localization using an
omnidirectional appearanceomnidirectional appearance--basedbasedsubspace model of environment”subspace model of environment”
Robotics and Autonomous Systems 45 (2003) 51–72
-
E. Menegatti - Omnidirectional vision 37
ApplicationsApplications
Map matchingMap matchingImageImage--based localizationbased localizationObservation of Optical Flow Observation of Optical Flow BiomimeticBiomimetic BehavioursBehavioursIntegration of OmniIntegration of Omni--vision with other sensors:vision with other sensors:–– SonarSonar–– Laser range finderLaser range finder
Outdoor NavigationOutdoor NavigationSLAM (Simultaneous Localization And Mapping)SLAM (Simultaneous Localization And Mapping)
Environment reconstruction & 3D mappingEnvironment reconstruction & 3D mappingMiscellaneaMiscellanea
-
E. Menegatti - Omnidirectional vision 38
Map matching Map matching -- 11
YagiYagi used the vertical used the vertical edges of the objects to edges of the objects to find position of the robot find position of the robot on a mapon a mapEdges trackingEdges tracking
Y. Yagi, Y. Nishizawa, M. Yachida,MapMap--Based Navigation for A Mobile Robot with Based Navigation for A Mobile Robot with OmnidirectionalOmnidirectional Image SensorImage Sensor COPIS,COPIS,
IEEE Trans. Robotics and Automation,pp.634-648,Vol.11,No.5,1995.10
-
E. Menegatti - Omnidirectional vision 39
Map matching Map matching -- 22
Menegatti et al. used the Menegatti et al. used the Chromatic Transitions of Chromatic Transitions of Interest to perform scan Interest to perform scan matchingmatchingMonteMonte--Carlo Localization Carlo Localization AlgorithmAlgorithmAlmost the same Almost the same approach used with Laser approach used with Laser range Findersrange Finders
E. Menegatti, A. E. Menegatti, A. PrettoPretto, A. , A. ScarpaScarpa, E. Pagello, E. PagelloOmnidirectional vision scan matching for robot localization in dOmnidirectional vision scan matching for robot localization in dynamic environments ynamic environments
IEEE Transactions on Robotics, Vol. 22,IEEE Transactions on Robotics, Vol. 22, Iss. 3 June 2006 pages 523Iss. 3 June 2006 pages 523-- 535 535
-
E. Menegatti - Omnidirectional vision 40
ImageImage--based navigation based navigation -- 11Ishiguro and Menegatti:Ishiguro and Menegatti:–– FFT magnitude for positionFFT magnitude for position–– FFT phase for headingFFT phase for heading–– SelfSelf--organization of the memoryorganization of the memory–– ImageImage--based Localisationbased Localisation–– Hierarchical LocalizationHierarchical Localization–– ImageImage--Based Monte Carlo LocalisationBased Monte Carlo Localisation
Emanuele Menegatti, M. Zoccarato, E. Pagello, H.Ishiguro, ````ImageImage--based Montebased Monte--Carlo Localisation with Carlo Localisation with Omnidirectional images'' Omnidirectional images''
Robotics and Autonomous Systems, Elsevier - 2004
Emanuele Menegatti, Takashi Maeda, Hiroshi Ishiguro, ````Hierarchycal ImageHierarchycal Image--based Memory for Robot Navigationbased Memory for Robot Navigation,'' ,''
Robotics and Autonomous Systems, Elsevier - 2004
-
E. Menegatti - Omnidirectional vision 41
ImageImage--based navigation based navigation -- 22KröseKröse et al:et al:–– Used Principal Component Analysis to Used Principal Component Analysis to
extract linear featureextract linear feature–– Dataset described in term of Dataset described in term of
eigenimageseigenimages–– Probabilistic localizationProbabilistic localization
B. Kröse, N. Vlassis, R. Bunschoten, and Y. Motomura. “A probabilistic model for “A probabilistic model for appareanceappareance--based based
robot localization”robot localization”Image and Vision Comp, vol. 19(6):pp. 381–391, April
2001..
-
E. Menegatti - Omnidirectional vision 42
ImageImage--based navigation based navigation -- 33Gross et al:Gross et al:–– Used slices of the Used slices of the
panoramic cylinderpanoramic cylinder–– Slices confronted via Slices confronted via
colour histogramscolour histograms–– Hybrid map: Hybrid map:
topological map topological map aumentedaumented with with metric information metric information
T. Wilhelm, H.T. Wilhelm, H.--J. J. BöhmeBöhme, and H., and H.--M. Gross. M. Gross. ““A multiA multi--modal system for tracking and analyzing faces on a mobile robot”modal system for tracking and analyzing faces on a mobile robot”
Robotics and Autonomous Systems, 48:31Robotics and Autonomous Systems, 48:31––40, August 2004.40, August 2004.
-
E. Menegatti - Omnidirectional vision 43
Observation of Optical FlowObservation of Optical Flow
Hiroshi Ishiguro, Kenji Ueda and Saburo Tsuji, ``Omnidirectional Visual Information``Omnidirectional Visual Information forfor NavigatingNavigating a Mobile Robota Mobile Robot'', '', IEEE Int. Conf.
on Robotics and Automation (ICRA-93), pp. 799-804, 1993.
Ishiguro used:Ishiguro used:–– Foci of Expansion (FOE) to Foci of Expansion (FOE) to
estimate relative positionsestimate relative positions–– No encoder infoNo encoder info
Svoboda used:Svoboda used:–– Optical flow to discriminate Optical flow to discriminate
translation and rotationstranslation and rotations
Tomáš Svoboda, Tomáš Pajdla, and Václav Hlavác.“Motion estimation using central panoramic cameras”
IEEE Int. Conf. on Intelligent Vehicles, 1998.
-
E. Menegatti - Omnidirectional vision 44
BiomimeticBiomimetic BehavioursBehavioursArgyros, A.A.; Tsakiris, D.P.; Groyer, C.
Biomimetic centering behavior Robotics & Automation Magazine, IEEE Publication Date: Dec. 2004 . Vol.11,
Iss. 4 pp.21- 30
M.V. Srinivasan. A new class of mirrors for wide-angle imaging.
Proceedings, IEEE Workshop on Omnidirectional Vision and Camera Networks. Madison, Wisconsin, USA., June 2003.
G.L. Barrows, J.S. Chahl and M.V. Srinivasan (2003) Biomimetic visual sensing and flight control. The Aeronautical Journal, London: The Royal Aeronautical
Society, vol, 107, No. 1069, pp. 159-168.
-
E. Menegatti - Omnidirectional vision 45
Integration with other sensorsIntegration with other sensors
Shin-Chieh Wei, Yasushi Yagi and Masahiko Yachida,“On“On--line Map Building Based On Ultrasonic and Image Sensor, 1996 line Map Building Based On Ultrasonic and Image Sensor, 1996 IEEE Int. Conf.
on Robotics and Automation(ICRA-98) 1998
YagiYagi used:used:–– Sonar to detect free spaceSonar to detect free space–– Fused the sonar, edge, Fused the sonar, edge,
colour information in a colour information in a occupancy grid occupancy grid
ClerentinClerentin used:used:–– Laser to find rangeLaser to find range–– Fused laser and edgesFused laser and edges
A. Clerentin, L. Delahoche, C. Pegard, E. Brassart"A localization method based on two A localization method based on two omnidirectionalomnidirectional perception systems cooperationperception systems cooperation “
ICRA'2000, San Francisco, April 2000.
-
E. Menegatti - Omnidirectional vision 46
Outdoor Navigation Outdoor Navigation -- 11
OmnidirectionalOmnidirectional Vision for Vision for Road Following with NN:Road Following with NN:–– Road classification Road classification –– Steering angleSteering angle
ZZ.. Zhu, SZhu, S. . Yang, GYang, G. . XuXu, X, X..Lin, Lin, DingjiDingji Shi Shi "Fast road classification and orientation estimation using omni"Fast road classification and orientation estimation using omni--view images and view images and
neural networks," neural networks," IEEE Transaction on Image Processing, Vol. 7, No.8, August 1998, pp. 1182-1197.
-
E. Menegatti - Omnidirectional vision 47
Outdoor Navigation Outdoor Navigation -- 22
Paul Paul BlaerBlaer and Peter Allenand Peter Allen“Topological Mobile Robot Localization Using Fast Vision Techniq“Topological Mobile Robot Localization Using Fast Vision Techniques”ues”
Proceedings of the 2002 IEEE International Conference on RoboticProceedings of the 2002 IEEE International Conference on Robotics & Automation s & Automation 20022002
ImageImage--based navigation:based navigation:–– Topological navigationTopological navigation–– Histogram matchingHistogram matching–– Different localisation Different localisation
accuraciesaccuracies
-
E. Menegatti - Omnidirectional vision 48
Outdoor Navigation Outdoor Navigation -- 33
JoséJosé--Joel GonzalezJoel Gonzalez--BarbosaBarbosa and Simon and Simon LacroixLacroixRover localization in natural environments by indexing panoramicRover localization in natural environments by indexing panoramic imagesimages
Proceedings of the 2002 IEEE International Conference on RoboticProceedings of the 2002 IEEE International Conference on Robotics & s & Automation 2002Automation 2002
ImageImage--based navigation:based navigation:–– Dimension reduction with Dimension reduction with
PCAPCA–– Histogram matchingHistogram matching
-
E. Menegatti - Omnidirectional vision 49
SLAMSLAMMichael Michael KaessKaess and Frank and Frank DellaertDellaert, ,
Visual SLAM with a MultiVisual SLAM with a Multi--Camera RigCamera Rig, , Georgia Tech Technical Report GITGeorgia Tech Technical Report GIT--GVUGVU--
0606--06, 200606, 2006
Thomas Thomas LemaireLemaire, Simon , Simon LacroixLacroix. . Long Term SLAM with panoramic vision. Long Term SLAM with panoramic vision.
Submitted to Journal of Fields RoboticsSubmitted to Journal of Fields Roboticsspecial issue on special issue on
"SLAM in the Fields"."SLAM in the Fields".
-
E. Menegatti - Omnidirectional vision 50
Environment ReconstructionEnvironment Reconstruction
H.BaksteinH.Bakstein, , T.PajdlaT.Pajdla. Rendering Novel Views from a Set . Rendering Novel Views from a Set of Omnidirectional Mosaic Images. Workshop onof Omnidirectional Mosaic Images. Workshop on
Omnidirectional Vision and Camera Networks 2003, CD Omnidirectional Vision and Camera Networks 2003, CD ROM, IEEE June 2003.ROM, IEEE June 2003.
C. Geyer, K. C. Geyer, K. DaniilidisDaniilidis,,Mirrors in Motion: Mirrors in Motion: EpipolarEpipolar geometry and motion geometry and motion
estimation, estimation, c. Inter. Conf. on Computer Vision, October, 2003, c. Inter. Conf. on Computer Vision, October, 2003,
Nice, France.Nice, France.
-
E. Menegatti - Omnidirectional vision 51
Omnidirectional Distributed Vision System (ODVS)
Requirements:•Robots’ only sensor: omnidirectional vision•No use of external computer•Every robot shares its measures•Every robot fuses all measures received by teammates•Measures can refer to different instants in time
The aim of the ODVS: to track multiple moving objects in highly dynamic environments by sharing the information gathered by every single robot
E. Pagello, A. E. Pagello, A. D’AngeloD’Angelo, E. Menegatti , E. Menegatti Cooperation Issues and Distributed Sensing for MultiCooperation Issues and Distributed Sensing for Multi--Robot Systems Robot Systems IEEE Proceedings of IEEE (in press due October 2006)IEEE Proceedings of IEEE (in press due October 2006)
-
E. Menegatti - Omnidirectional vision 52
OneOne StaticStatic Vision Vision AgentAgent ((omnidirectionalomnidirectional camera)camera)
FiveFive StaticStatic AcusticAcustic AgentsAgents ((steerablesteerable microphonemicrophonearraysarrays))
OneOne Mobile Vision Mobile Vision AgentAgent (robot (robot withwith omnidirectionalomnidirectionalcamera)camera)
=
=
Experiment Layout
Audio-Video Surveillance System with Mobile Robot
E. Menegatti, M. E. Menegatti, M. CavasinCavasin, E. Mumolo, M. , E. Mumolo, M. NolichNolich, E. Pagello , E. Pagello CombiningCombining Audio and Video Audio and Video SurveillanceSurveillance withwith a Mobile Robot a Mobile Robot InternationalInternational Journal on Journal on ArtificialArtificial Intelligence Intelligence ToolsTools (in press)(in press)
=
-
E. Menegatti - Omnidirectional vision 53
The End!The End!
Thanks for your attention!
My publications and more information can be found in my web page:
http://www.dei.unipd.it/~emg
-
E. Menegatti - Omnidirectional vision 54
ReferencesReferencesWWW:The page of omnidirectional visionhttp://www.cis.upenn.edu/~kostas/omni.html
ICCV03 Course on Omnidirectional Visionhttp://www.cis.upenn.edu/~kostas/omni/iccv03.html
Book:R. Benosman & S.B. Kang (Eds.)Panoramic VisionSpringer 2001
http://www.cis.upenn.edu/~kostas/omni.htmlhttp://www.cis.upenn.edu/~kostas/omni/iccv03.html
-
E. Menegatti - Omnidirectional vision 55
ReferencesReferencesSpecial issues:K. Daniilidis& N. PapanikolopoulosThe Big PictureIEEE Robotics & Automation Magazine Dec. 2004
Hiroshi Ishiguro and Ryad BenosmanSpecial issue on omnidirectional vision and its applications Machine Vision and Applications (2003) Vol 14
Yasushi Yagi and Katsushi IkeuchiSpecial Issue on Omni-Directional Research in JapanInternational Journal of Computer Vision Vol. 58, Num. 3, Springer July 2004
Peter Sturm, Tomas Svoboda and Seth TellerSpecial issue on Omnidirectional Vision and Camera NetworksComputer Vision and Image Understanding Vol. 103, Iss. 3, Sept. 2006
Omnidirectional Vision for Mobile RobotsHistoryAnimal Vision - 1Animal Vision - 2Omnivision in AnimalsOmnivision in PaintingsOmnivision in PaintingsDistortion in PaintingsDistortion in PaintingsPanoramasPhotographyOmnidirectional Cameras - 1Omnidirectional Cameras - 2Compound-eye camerasCompound-eye cameras (inward)Panoramic camerasOmnidirectional cameras - 1Omnidirectional cameras - 2Confused?Other myths and hesitation-1Other myths and hesitation-2Catadioptric CameraAn omnidirectional camera viewOmnidirectional camerasOmnidirectional Mirror DesignFamilies of Mirrors - 1The mirror we designed...Examples of IAS-Lab MirrorsIT+Robotics srlOmni-Vision for Mobile RobotsNavigation/Localization TricksInvariance of AzimuthRotational InvarianceVertical Lines radial linesContinuity & PeriodicityRobustness to occlusionApplicationsMap matching - 1Map matching - 2Image-based navigation - 1Image-based navigation - 2Image-based navigation - 3Observation of Optical FlowBiomimetic BehavioursIntegration with other sensorsOutdoor Navigation - 1Outdoor Navigation - 2Outdoor Navigation - 3SLAMEnvironment ReconstructionThe End!ReferencesReferences