Survey and Intervention HROV

P. Ridao, M. Carreras, D. Ribas, N. Gracias, R. Garcia

Resumen

Traditional underwater archaeology is performedvia SCUBA diving but is constrained by the prac-tical depth that a diver can work (normally lim-ited to 50 meters) and the time that can bespent underwater. New technologies, like mannedsubmersibles, remotely operated vehicles (ROVs)and quite recently autonomous underwater vehi-cles (AUVs) allow archaeologists to survey deeper,dramatically increasing the number of underwa-ter sites reachable for archaeological study. Whiletechnology plays a significant part in this work, itmust be combined with archaeologists? researchmethodology so that archaeology in deep waterconforms to the required standards. In this paperwe discuss how HROVs may become the perfecttool to assist archaeologist in their work.

1 Introduction

From the last few years of deep-water archaeologyit has been clearly demonstrated that archaeolo-gists can benefit from new underwater technologybut their requirements pose new and sometimesfundamental problems for engineers [2]. Mannedsubmersibles have the unique advantage of allow-ing scientists to physically reach the deep and per-form systematic observation on sites of particularinterest as well as recovering artifacts [14]. How-ever, because of their operational costs and com-plexity, their availability is very limited. A bigleap towards deep-water archaeology was given bythe use of ROVs. With continuous power supplyfrom the deployment ship, they can support a widerange of sensors, uninterrupted operations and canrecover artifacts with heavy but precise manipu-lators [1, 5, 12]. ROVs are generally powered andcontrolled from a tether cable. While this pro-vides unlimited autonomy in terms of power andtime, it limits the range of the ROV and as thedepth operations increase so increases the deploy-ment and usage costs (e.g. bigger support ship).Although limited in power, AUVs alleviate someof these drawbacks such as avoiding the tetheringrange limit, thus increasing the efficiency of ar-chaeological UUVs over the last decade, for both

commercial and scientific applications. Early usesof AUVs in marine archaeology were reported in[8]. In 2004, Meo [9] discussed the applicationsof new technologies in the study of underwaterarcheological sites, emphasizing the advantages ofAUVs in such scenarios. More practically, Foleyet al. [6] provided a detailed description of thearchaeological study of two 2000-year-old ships.Due to the depth of the sites (70m) the use ofdivers was not practical. As an alternative, theresearch team successfully used AUV systems tomap and study the sites. An extensive reviewof the AUVs usage and deepwater archaeologicaltechniques can be found in [2], [?]. The experienceof our team applying AUV technology to marinearchaeology is based on our participation in thesurvey of two shipwrecks described in the follow-ing subsections.

Figura 1: GIRONA AUV deployed to survey LaLune Shipwreck (toulon-France).

1.1 La Lune XVII century shipwreck

La Lune Shipwreck is a XVII century wreck lyingin 90m of water, of the coast of Toulon in France.Accidentally discovered by IFREMER in 1993 thewreck was assessed soon after its discovery by theFrench DRASSM and considered to be one of thebest preserved in the world. During August 2012,GIRONA500 AUV [11] (Fig. 1) was deployed to

Actas de las XXXVI Jornadas de Automática, 2 - 4 de septiembre de 2015. Bilbao ISBN 978-84-15914-12-9 © 2015 Comité Español de Automática de la IFAC (CEA-IFAC) 830

Figura 2: Fig. 2. Photomosaic of a XVII centuryshipwreck La Lune build during in August 2012using GIRONA500 AUV

Figura 3: 2.5D multimodal map of la Lune Ship-wrek consisting on a photomosaic rendered overthe microbathymetry of the shipwreck.

Figura 4: 3D reconstruction of the anchor andsome of the canons.

create a preliminary optical cartography of the site[7], to serve as the base map for posterior archae-ological intervention by DRASSM. The data werecollected in two consecutive dives of one hour each.Different data products were produced: 1) a high-resolution (1 mm x pixel resolution) 2D photomo-saic of the seafloor (Fig. 2), 2) a 2.5D bathyme4rytextured with the photomosaic (Fig. 3) and 3) 3Dreconstructions of objects of interest (Fig. 4).

Figura 5: GIRONA AUV deployed to survey Capdel Vol Shipwreck.

Figura 6: (Top) High resolution (1 mm/pix)Photo-Mosaic. (Bottom) Sonar mosaic.

1.2 Cap de Vol 10 BC shipwreck

The Cap del Vol Shipwreck cruise (Fig. 5) tookplace in August the 30th and the 31th at Port de laSelva (Costa Brava-Spain) on board the THETISship of the Catalan Centre of Underwater Archae-ology (ACdPC). This was a joint effort betweenthe ACdPC and UdG teams to evaluate the useof advanced underwater robotics technologies forsubmerged cultural heritage applications. Duringthe cruise the GIRONA500 AUV systematically

Actas de las XXXVI Jornadas de Automática, 2 - 4 de septiembre de 2015. Bilbao ISBN 978-84-15914-12-9 © 2015 Comité Español de Automática de la IFAC (CEA-IFAC) 831

targeted the shipwreck. Four dives were carriedout during a two days cruise, three of them inautonomous mode and one operated as a ROV.During the two first missions the robot gatheredoptical data using a high resolution stereo pairdesigned and developed at CIRS-VICOROB. The3rd mission acquired optical and acoustic data.The 3rd mission was teleoperated, with the mainaim of exploring the shipwreck at very low speedwhile gathering forward looking sonar imagery. Itdemonstrated us the potential of the HROV con-cept for cultural heritage applications. In thiscase, 2 data product were produced: 1) a High res-olution (1 mm/pix) Photo-Mosaic (Fig. 6 - Top)and 2) a Sonar mosaic (Fig. 6 - Bottom) obtainedthrough the registration of sector images gatheredwith ARIS multibeam forward looking sonar. Theacoustic imagery was gathered while manually pi-loting the surge and sway DOF while automati-cally keeping heading and altitude (3 m) to ensurea consistent shadow effect across the scene.

Based on this experience, we advocate for theuse of a light weight HROV system easily de-ployable from small ships, like those commonlyused by marine archaeologists. The HROV is re-configurable to operate as an AUV to performautonomous opto/acoustic shipwreck surveys, asthose described above, and as a smart ROV toperform site intervention and light excavation. Inthis paper we further develop these concepts.

2 Technical Description

This section describes the proposed lightweightsurvey and intervention HROV (section 2.1) aswell as how we think it can be applied to marinearchaeology (section 2.2).

2.1 HROV System

Conceptually a HROV is an AUV system whichmay optionally be tethered, with a thin cable forcommunications only, to be operated as a ROV.Its capability to work as an AUV, as well as anROV, makes it a very attractive design. More-over, it perfectly fits our concept of facing thesurvey phase autonomously and the interventionphase semi-autonomously under the supervision ofan human operator, taking profit of the broadbandcommunications to provide a high definition sen-sory feedback of the operational environment. Bytargeting semi-autonomous intervention, we ex-pect to be able to achieve a system smarter thanpure teleoperated ROV style manipulation. Theproposed HROV will be based on the GIRONA500 platform complemented with a thin broad-band communications Tether, a TMS, and an elec-

trical robot manipulator.

2.2 Operational Concept

We foresee three different modes of operationwhich are described in the next sections.

2.2.1 Discovery of new sites:

Archaeologists look for new sites in places wherethey have an historical evidence of a shipwreck or,alternatively, in areas where fishermen or divershave detected evidences of ship remains. Nor-mally, they dive in the place seeking wreck ar-tifacts. UUVs can contribute to the location ofnew wrecks, by significantly extending the bottomtime as well as the coverage of the explorations.Conventional Side Scan Sonar (SSS) and Multi-beam bathymetry mapping, as well as optical im-agery can be collected during the search. More-over, after the dives, the imagery can be auto-matically processed by an offline classifier looking,for instance, for ceramic remains. This will pointout candidate areas to carry out a 100% coverageopto/acoustic survey.

2.2.2 Site Survey

When the site coordinates are known, the box tobe surveyed can be estimated and a survey trajec-tory obtained. Next, the HROV will be deployedin tether-less mode. The robot will dive and fol-low the pre-programed path at a constant altitude(¡5m) while gathering data with the cameras andsonars. At the end of the mission the robot will berecovered and the data downloaded. After down-loading the panoramic camera imagery, a topolog-ical panoramic map can be made available to thearchaeologists, allowing for a virtual tour of theshipwreck, similarly to google street view. In themeantime, an offline mapping tool can be used toset-up accurate photo or sonar mosaics producinga high resolution (less than 1 mm/pixel in the caseof optical and 10mm/pixel when acoustic) maps ofthe site. These maps will be used to plan accu-rately the next day excavations by the archaeol-ogist divers if the operation takes place shallowwater. It is worth noting that we do not plan toreplace the archaeologist divers by the robot op-eration in shallow waters, but provide them witha tool which will allow them to devote their timeonly to the tasks that may not be automated.

2.2.3 Site Intervention

The HROV will operate in tether mode. An HMIwill be used to provide the archaeologist with anAugmented Reality view of the operation site. Itwill render in real-time the Photo/Sonar Mosaic

Actas de las XXXVI Jornadas de Automática, 2 - 4 de septiembre de 2015. Bilbao ISBN 978-84-15914-12-9 © 2015 Comité Español de Automática de la IFAC (CEA-IFAC) 832

together with an out-of-body representation of theHROV. The archaeologist will be able to guidethe vehicle or, alternatively, the end effector. Inthe last case, the system will automatically decidewhether to move the arm, the vehicle or jointlymove both to achieve the desired motion. Therobot will be able to perform station keeping, andthe panoramic camera will allow the archaeologistto visually explore the 360o using the VR head-set. Two main intervention tasks are foreseen: 1)grasping objects and 2) performing light excava-tion of selected objects. Using a force torque sen-sor, the archaeologist will be able to perceive theapplied force, allowing for a careful manipulationof the objects. For the excavation, it will be pos-sible to mount the air lift in the arm, as well as tomount it in a fixed position while using the armto help in the cleaning of the area by smoothlymoving the water, and hence the sand, around theobject being excavated.

2.3 Key Challenges

To achieve the functionality described in the tech-nical description, several fundamental challengeshave to be faced as described hereafter.

2.3.1 Immersive presence

Archaeologist are used to dive to do interventiondirectly over the site. Hence, one of the challengesis how to bring them virtually to deep sites whichare inaccessible to them. Although conventionalROVs provide remote views over HD screens, newtechnologies exist which may potentially improvethe experience to immerse the archaeologist in thesite. For instance, we are using hemisphericalcameras during autonomous surveys to provide of-fline maps similar to those of ”google street view”underwater, allowing the archaeologist to freelynavigate between the omnidirectional views usingan immersive head-mounted display (HMD). An-other challenge is to provide this immersion dur-ing the intervention. In this case it is necessaryto track the operator’s head attitude to estimateits vantage point in order to extract the necessarypictures from the hemispherical camera, transportthem in realtime through the tether, and renderthem on the HMD. In this case latency should bekept very small to reduce the motion sickness.

2.3.2 Semi-autonomous Intervention

Besides allowing the users to teleoperate theHROV using a joystick for the vehicle and a hap-tic for the arm, more advanced end effector (EE)guidance methods would be pursued. For in-stance, it would be allowed to ask the HROV toautonomously navigate to a certain waypoint, or

the EE to adopt a certain pose. It should also benecessary to teleroperate the EE directly, allow-ing the system to decide if the vehicle, the arm, orboth have to be moved to achieve the goal. It willbe necessary to establish constrains, avoiding forinstance vehicle collisions with the ship remains onthe seafloor. More advanced manipulation skillsbased on the perception of the environment andthe joint motion planning of the HROV and thearm have to be studied in order to allow the sys-tem to navigate within the site while performinglight intervention task programmed at high level.

2.3.3 Automated Classification

Underwater imagery, both optical and acoustic,has become a widespread tool for seafloor studies,owing to the rapid advancement in imaging elec-tronics. However, the advancements in the analy-sis have not kept pace with the improvements inimage acquisition systems. For the vast majorityof optical surveys, a human analyst must still in-spect every frame to extract relevant information.A clear challenge lies in the capacity to automatethis process. This is further pressing for the caseof optical imagery where the higher spatial resolu-tion, when compared to other sensing modalities,translates to very large amounts of data. Progresshas been attained by several research groups in au-tomated classification of underwater imagery us-ing texture information, mostly on the habitatmapping realm ([10, 3, 15], but no single tech-nique is yet widely accepted as robust. A mid-term research challenge lies on the use differentsensors (such as SSS, microbathymetry from bothMBES and from high resolution optical stereo,and magnetometers), in order to exploit their com-plementarity in providing discriminative cues forthe classification. An example of the impact of au-tomated classification on HROV-assisted archae-ology, would be the ability to detect and character-ize the artifact distribution in large-area surveysaround areas of prior archaeology evidence, even-tually leading to the discovery of new sites. Forexisting sites, it would also be valuable in charac-terizing the mobility of small artifacts and the im-pact of natural or human disturbances over time.

2.3.4 Multimodal 2D/3D Mapping

How to combine data coming from different sen-sor modalities into a single and consistent modelof the environment is one of the main challenges.Advancing towards the co-registration of several2D/3D maps, potentially from surveys in differ-ent periods of time is also necessary.

Actas de las XXXVI Jornadas de Automática, 2 - 4 de septiembre de 2015. Bilbao ISBN 978-84-15914-12-9 © 2015 Comité Español de Automática de la IFAC (CEA-IFAC) 833

2.3.5 Benchmarking

Marine archaeologists are very exigent about theresolution and accuracy of the maps of the sub-merged sites they build. After years of work, theyhave come up with a methodology for mappingmanually with high accuracy and resolution theshipwrecks they survey. It is necessary to de-velop metrics to assess the accuracy of their cur-rent methods and to compare them with the mapsbuilds automatically with the help of robots. Atthe same time it is necessary to develop bench-marks to assess the intervention tasks, definingmetrics in order to quantify the progress.

3 The Future

In this section, future foreseen technological ad-vances that may have an impact on the conceptof the survey and intervention HROV system arediscussed:

• Wireless Operation: Getting rid of the thintether, using an opto/acoustic communica-tions was recently demonstrated at short dis-tance [4] For the next years, the advances inunderwater wireless communications technol-ogy, are expected to increase this distance,as well as the communication bandwidth, en-abling online reconfiguration from AUV toROV and vice versa.

• Multi-vehicle Operation: With the evolutionof modems high speed communications atshort distances (electromagnetic or optical,for instance), will be possible, to exchangeconsiderably larger volumes of informationamong vehicles. This will foster new oper-ational concepts base on tight cooperationamong vehicles. For instance, we can think onusing an small AUV as a flying camera pro-viding alternatives views which are not pos-sible from the HROV cameras due to occlu-sions.

• Intervention using semantic information: Ad-vanced 3D imaging methods (stereo, laserscanners) will allow gathering 3D point cloudsof the site of interest. Next, prior modelsof the expected objects (amphorae, for in-stance....) will be used for object recogni-tion to enable semantic mapping, providinga more elaborated world model for manipula-tion planning. Semantic maps pave the wayto scene understanding, where the feasibilityof accomplishing a task may be evaluated onthe fly, as well as the risk for the robot.

Agradecimientos

This work was supported by the Spanishproject DPI2014-57746-C3-3-R (MERBOTS-ARCHROV)

Referencias

[1] Ballard RD, McCann AM, Yoerger D, Whit-comb L, Mindell D, Oleson J, et al., (2000)The discovery of ancient history in the deepsea using advanced deep submergence tech-nology, Deep-Sea Research, Part I: Oceano-graphic Research Papers. Vol 47, pp. 1591-1620.

[2] B. Bingham, B. Foley, H. Singh, R. Camilli,K. Dellaporta, R. Eustice, A. Mallios, D.Mindell, C. Roman, and D. Sakellariou.Robotic tools for deep water archaeology:Surveying an ancient shipwreck with anautonomous underwater vehicle. Journal ofField Robotics 27.6 (2010): 702-717.

[3] Bewley, M.S., B. Douillard, N. Nourani-Vatani, A.L. Friedman, O. Pizarro, and S.B.Williams. 2012. Automated species detection:An experimental approach to kelp. The Aus-tralasian Conference on Robotics and Au-tomation (ACRA)

[4] Farr, N.; Bowen, A.; Ware, J.; Pontbriand,C.; Tivey, M., ”An integrated, underwateroptical /acoustic communications system,”OCEANS 2010 IEEE - Sydney , vol., no.,pp.1,6, 24-27 May 2010.

[5] B. Foley and D. Mindell, ?Precision surveyand archaeological methodology in deep wa-ter,? ENALIA: The Journal of the HellenicInstitute of Marine Archaeology, vol. VI, pp.49?56, 2002.

[6] Foley, B. P.; DellaPorta, K.; Sakellarious,D.; Bingham, B. S.; Camilli, R.; Eustice, R.M.; Evagelistis, D.; Ferrini, V. L.; Katsaros,K.; Kourkoumelis, D.; Mallios, A.; Micha,P.; Mindell, D. A.; Roman, C.; Singh, H.;Switzer, D. S.; Theodoulou, T. (2009). ”The2005 Chios Ancient Shipwreck Survey: NewMethods for Underwater Archaeology.” Hes-peria 78(2): 269-305.

[7] N. Gracias, P. Ridao, R. Garcia, J. Escartin,M. L?Hour, F. Cibecchini, R. Campos, M.Carreras, D. Ribas, N. Palomeras, L. Magi,A. Palomer, T. Nicosevici, R. Prados, R.Hegedus, L. Neumann, F. de Filippo andA. Mallios. Mapping the Moon: Using alightweight AUV to survey the site of the17th Century ship ?La Lune?. Proceedings ofthe Oceans MTS/IEEE OCEANS conference,Bergen, Norway, June 2013

Actas de las XXXVI Jornadas de Automática, 2 - 4 de septiembre de 2015. Bilbao ISBN 978-84-15914-12-9 © 2015 Comité Español de Automática de la IFAC (CEA-IFAC) 834

[8] D. Mindell and B. Bingham, New archaeolog-ical uses of autonomous underwater vehicles,in MTS/IEEE OCEANS Conference and Ex-hibition, vol. 1, pp. 555?558, 2001.

[9] G. Meo, The HMI of an experimental un-derwater vehicle for archeological survey, in-spection and remote touring of importantsubmarine sites and finds, in MTTS/IEEETECHNO-OCEANS Conference?04, vol. 2,pp. 818?821, 2004.

[10] Pizarro, O., P. Rigby, M. Johnson-Roberson,S. Williams, and J. Colquhoun. 2008. To-wards image based marine habitat classifica-tion. OCEANS?08 MTS/IEEE, Quebec

[11] Ribas, D., Palomeras, N.,Ridao, P., Car-reras, M.: Girona500 AUV, from survey tointervention. IEEE/ASME Transactions onMechatronics (Focused Section on MarineMechatronic Systems) 17(1) (2012)

[12] C. Roman, G. Inglis, and J. Rutter, Applica-tion of structured light imaging for high reso-lution mapping of underwater archaeologicalsites, in OCEANS 2010 IEEE - Sydney, 2010.

[13] C. Roman and R. Mather, Autonomousunderwater vehicles as tools for deep-submergence archaeology, Proceedings of theInstitution of Mechanical Engineers, Part M:Journal of Engineering for the Maritime En-vironment, vol. 224, pp. 327?340, 2010.

[14] D.Sakellariou, P. Georgiou, A. Mallios, V.Kapsimalis, D. Kourkoumelis, P. Micha, T.Theodoulou and K. Dellaporta. Searching forAncient Shipwrecks in the Aegean Sea: theDiscovery of Chios and Kythnos HellenisticWrecks with the Use of Marine Geological-Geophysical Methods. International Journalof Nautical Archaeology, vol. 36, no. 2. 2007.

[15] A. Shihavuddin, N. Gracias, R. Garcia, A.Gleason and B. Gintert, Image-Based CoralReef Classification and Thematic Mapping.Remote Sensing, 5(4), pp. 1809-1841. 2013.ISSN 2072-4292. doi:10.3390/rs5041809

Actas de las XXXVI Jornadas de Automática, 2 - 4 de septiembre de 2015. Bilbao ISBN 978-84-15914-12-9 © 2015 Comité Español de Automática de la IFAC (CEA-IFAC) 835