rfcs project - telerescuer

TeleRescuer Project RFC-CT-2014-00002

TELERESCUER SYSTEM FOR VIRTUAL TELEPORTATION OF RESCUER FOR INSPECTING COAL MINE

AREAS AFFECTED BY CATASTROPHIC EVENTS

T e l e R e s c u e r

P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

2

SCHEDULE

1. Introduction

2. Work Packages and Bar Chart

3. Budget information

4. Deliverables

5. Detailed identification of needs, formulating requirements (WP1)

6. Research into the UV (WP 2)

7. Research into virtual teleportation technology… (WP3)

8. Dissemination of results (WP5)

9. Management and Coordination (WP6)

10. Conclusions


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

3

INTRODUCTION GENERAL IDEA

TeleRescuer is an innovative system for inspecting coal mine roadways, especially those affected by catastrophes such as fire, explosion of methane or coal dust, and the others.

The system allows virtual teleportation of a mining rescuer to those areas of a coal mine, in which he could not remain due to hazards for life or health


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

4

INTRODUCTION MAIN PARTS


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

5

WORK PACKAGES AND BAR CHART


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

6

WORK PACKAGES AND BAR CHART WP1-WP3

Added to initial bar chart in official ammendment no 1 (accepted by EC)


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

7

WORK PACKAGES AND BAR CHART WP4-WP6


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

8

DELIVERABLES ACHIEVED DELIVERABLES Deliverable number

Deliverable name Foreseen finalisation date

Real finalisation date

Form Location

D1.1. Formal specification of requirements

M6 31.12.2014 Written document

Appendix 2 to the first annual report

S1.1. A decision as to whether the system has to be ATEX-ready/ATEX-certified

M6 31.12.2014 Written document (minutes from the meeting)


D3.1 A report on rescuers’ knowledge acquisition and representation

M6 31.12.2014 Written document


D3.3 A report on the simulations of the operations of rescuers’ in a hazardous area of the coal mine with augmented reality elements

M18 Written document

Appendix 2 to the mid-term report

D5.1 A TeleRescuer project official webpage

M3 30.09.2014 Website Appendix 5 to the first annual report and internet (www.telerescuer. Polsl.pl)

D5.3 Articles about the obtained results in scientific and industry journals

M12-M36 - Published papers

List of papers presented in section 2 (Project overview table)

D5.4 A scientific seminar presenting the theoretical results achieved to-date

M18 23.09.2015 Meeting -

D6.1 A Consortium Agreement M3 07.07.2014 Written document


D6.2 A first annual report M9 (initial deadline: M15)

31.03.2015 Written document

CIRCABC

D6.3 Mid-term financial and technical reports

M21 Written document

CIRCABC


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

9

DELIVERABLES FUTURE DELIVERABLES

Deliverable number

Deliverable name Foreseen finalisation date

D2.1. A prototype of a mechatronic platform of the UV M24

D2.2. A prototype of a communication system and a sensory system M27

D2.3. A prototype of a control system M27

D2.4. A prototype system for building maps M27

D2.5 A method and a prototype system for the autonomous operation of the UV in a known environment

M27

D3.2 A prototype of an effective human-machine interface for virtual teleportation M19

D3.4 A prototype of the training simulator M30

D4.1 A report on tests of the system and its components M36

D5.2 A brochure about the TeleRescuer project (as an electronic pdf file and in print) M33

D5.5 A promotional seminar intended for the potential recipients of the project’s results M35

D6.4 A second annual report M27

D6.5 Final technical and financial reports M36+9


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

10

DETAILED IDENTIFICATION OF NEEDS, FORMULATING REQUIREMENTS (WP1)

The identification of the needs has been carried out in close collaboration with the Central Mining Rescue Station (CMRS). Results of those activities have been summarized, reported and converted into formal requirements.

Requirements have been specified for each subsystem of the UV, including:

• Robot platform: Platform’s mobility, dimensions, weight, protection level, operating time and range; Required external mechanical equipment and it’s parameters;

• Communication system: Optical fiber communication; Wireless communication with motes;

• Sensory system: Internal sensors: orientation sensors, robot state sensors, protection sensors; External sensors: gas sensors (CH4, CO, CO2, O2), air velocity senor and also

requirements on their placement; Cameras: quantity , type and optimal placement;

• Control system with autonomous operation: Autonomous operation modes and conditions for autonomous operation;


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

11

RESEARCH INTO THE UV (WP2) MECHATRONIC SUBSYSTEM OF THE UV (T2.1)

Mechatronic subsystem is composed of: Base robotic platform:

• High mobility tracked platform with independent tracks’ flippers

• Designed to meet ATEX standards requirements

• Capable of being used in various missions due to different configurations of external equipment

External equipment:

• Cylinder arm with sensors, cameras and lights

• Laser scanner • Mote dispenser • Fiber unwinder


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

12

Managed 2 microcontrollers:

I. 1st C: Arduino Uno + Arduino Ethernet Shield Connections:

• SPI: Voltage sensor (fuel gauge) to control batteries;

• SPI: Ethernet shield; • I2C: Inertial Measurement Unit (IMU) - MPU-

6050; • UART RS-485:

II. 2nd C: PIC Microcontroller series 30GP or 33MC from Microchip Sensor control module (outside the cylinder) managing gas sensors: • Methane (min. 0 ÷ 20% V/V) • Mass flow (0 ÷ 20 m/s) • O2 (0 ÷ 25%) • CO (0 ÷ 10000 ppm) • CO2 (0 ÷ 5% vol) • Humidity/Temperature (0÷100% /-20÷+60ºC)

• Free pins: 4 PWM pins (illumination), 4 I/O digital pins (relay, …)

Arduino Uno + Eth shield

MPU-6050

SENSORS

SENSORY SYSTEM (T2.2)

RESEARCH INTO THE UV (WP2)


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

13

Constraints: • Reduced space for all the camera electronics (130 x 110 x 160 mm box) • Need to work under fast Ethernet protocol (Not Giga because of the possible danger caused by a laser: IEC-

68070-28) • Ability to change online the video bandwidth output of each camera • Real time low latency video compression • On-off capabilities for each camera • PoE with the lower power consumption as possible • At least one thermal sensor for smoke • Illumination will be outside the safety box so it must be ATEX ready

Visible light cameras: NC353L Elphel model

2 fish-eye (180º): at front and at rear • Lens • 5Mpix color sensor board model 0353-00-17 • IP fast Ethernet camera main board 10353

1 stereoscopic camera (2 lenses) • Additional Synchronization board for the stereo rig model 10359

Thermal Camera: FLIR AX8 With converter to Ethernet cable

Illumination: Adaro Tecnologia M1 enclosure Own electronics (PWM, omnidirectional LEDs)

VISION SYSTEM (T2.2)



P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

14

Spool of Sedi-Ati

Possible Secondary Switches

WIRED COMMUNICATION SYSTEM (T2.2)


Initial Possibility: • Own solution; • A simple unwinder (with an Optical Fiber Rotatory Joint – OFRJ of 2 channels) + secondary system to

wind the optical fiber once the robot is outside;

Problems: • OFRJ: Much expensive; • Dimensions of a reel with strong optical cable;

Solution: Spool for unmanned ground vehicles with not much strong cable;

100Base-FX or 1000Base-FX? • 1000Base-FX is not easy to atexize; • 1000Base-FX normally uses lasers: difficult to satisfy the regulation IEC-68070-28; • 100Mbps is enough for cameras with an acceptable resolution;

3 switches needed: 1) Primary station (in the safe place):

• 1 Optical Fiber port (to the robot); • 1 Ethernet port: ATEX Ethernet (80 m máx.) for the first mote; • 1 port for the operator;

2) Main switch (in the chassis): 4 ports (PC board, 1st mote, secondary switch, optical (for the O.F.)); 3) Secondary Switch (in the cylinder):

• Small (112 mm x 155 mm x 134 mm OR 106 mm x 160 mm x 134 mm); • Managed; • Able to bear high temperatures; • PoE ports (4 at least for cameras); • 8 ports (4 cameras, 1 C, 1 primary switch, 1 relay, 1 free);


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

15

Requirements: • Able to work in hazardous

environment; • Size constraints; • Range distance; • Robust;

Technical requirements: • Ad-hoc network: just forwarding

elements, no routers, no Aps; • Self-organizing network: Same LAN,

pre-configured IP addresses;

Technologies: 1) UWB-Decawave: Data rate (6.8 Mb/s

máx. Non effective!!! ; Max 10m ). Not optimized for communications;

2) IEEE 802.11 (Atexized Raspberry Pi);

ATEX-ready Batteries: • Lithium iron phosphate batteries

(LiFePO4)

WIRELESS COMMUNICATION SYSTEM (T2.2)


First prototype


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

16

RESEARCH INTO THE UV (WP2) CONTROL SYSTEM (T2.3)

Mote #n

Chassis – inner space

4x RoboteQ FBL 2360- Track + Track Arm motors- Absolute sensor tilt of arms (connected to RoboteQ AI)- 2 x IRC + Hall (connected to RoboteQ)

PC Board

Vision (2D/3D)- 1 IR, 2x for stereo (common Eth.channel), 2 x fish = 5 cameras with 4 IP interface

UC3M

Cylinder Control System - Communication with external Sensor Control System (RS485) and MCS (Ethernet)- Cameras power ON/OFF- Lighting – front and rear- Voltage of power supply measuring, - IMU – Tilting x,y,z,..- for info about head tilt – artificial horizon for operator

UC3M

Ethernet Switch3D Laser scanner

12V, 20W

end

cap

26+2 TemperatureSensors (DALLAS)

Mote #02

Mote #01

(CH4, CO, CO2, O2, temperature outer, humidity,.....

Sensor Control System

CPU

CH4, CO, CO2, O2, temperature outer, humidity,.....

Metalic Ethernet (4 wire)

Optical serial communication (2 fibres)

Metalic CAN bus (2 wire)

Metalic serial communication + power(2 + 2 wires)

Metalic 1wire bus (3 wire)

Optical Fibre Interface

USB/CAN interface

USB/1Wire Interface

Main IMU

Autonomy Lock

Button

Central Stop

Button

Metalic USB (4 wire)

end

cap

end

cap

end

cap IMU Configuration

USB Reserve

RS485 Reserve

BatteryManagementSystem

CAN

CAN

DI ENABLE

USB (2xRS232)

USB

USB

USB

RS485

USB

ETH

ETH ETH ETH

1WI

Optical Interface

Optical Ethernet (1 - 2 fibres)

ETH

ETHETHETH

Optic Fibre ReelUC3M

ETH

ETH

RS48

5

RS23

2/RS

485

DC/DC converter

Ethernet switch

Main Switch

Flameproof enclosure bushing

ETHETH

UC3M

UC3M

SUT

SUT

Optical Fibre Interface +

ENABLE control (VŠB)

UC3M

SUT

SUT

SUT

SUT

VSB

VSB

SUT

SUT SUT

UC3MFinal Assebling + cabling: SUT, VSB

UC3M

SUT, UC3M

MOTE Relaser

SUT

Wireless comunication

module (UC3M)

RoboteQSensor arm Tilting, – Sens.armCylinder, Methane arm 3 axes= 1 drive, 1 motor,2x solenoid (1 clutch, 1 brake)2 x Absolute sensors – yes SUT

VPwrCtrl

(ENABLE)

RS232

Methane sensor on methane arm

Main Control System and its Connectivity with All Subsystems


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

17

RESEARCH INTO THE UV (WP2) METHODOLOGIES FOR BUILDING MAPS (T2.4)

Laser Range Finder (Sick LMS111) + Positioning unit; Methane sensor SC-CH4 with ATEX switches off 3D LRF,

when methane concentration exceed the limit; A few methods were programmed for improvement of

visualization and coloring output with additional information;

Prototype tested in Królowa Luiza Coal Mine (October 2015): no failures were encountered;


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

18

RESEARCH INTO THE UV (WP2) AUTONOMOUS OPERATION OF THE UV (T2.5)

• Localization in a (not) known environment, • Path planning in a known environment, • Movement realization according to a

planned path in a constantly changing environment

Main challenges

Growing uncertainty over time

Reduced uncertainty thanks to map matching

Based on R. Siegwart (ETH Zurich)

When autonomy will be used?


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

19

RESEARCH INTO THE UV (WP2) AUTONOMOUS OPERATION OF THE UV (T2.5)

Tests in simulated environments Tests of inertial navigation

Robot localization based on 3D scans

TRohn/vrep_scanner_telerescuer_1.avi

KSienkiewicz/IMU_tests_using_3Dprinter.mp4


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

20

RESEARCH INTO VIRTUAL TELEPORTATION TECHNOLOGY… (WP3) OPERATOR STATION

In the operator station the following technolgies are used:

3 monitors (144 Hz)

3D technology

Nvidia 3D vision


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

21

RESEARCH INTO VIRTUAL TELEPORTATION TECHNOLOGY… (WP3) OPERATOR STATION - SOFTWARE STRUCTURE


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

22

DISSEMINATION OF RESULTS (WP5) RESULTS ACHIEVED (1/2)

The official TeleRescuer logo;

TeleRescuer website: www.telerescuer.eu (D5.1);

Template for multimedia presentations and official documents;


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

23

DISSEMINATION OF RESULTS (WP5) RESULTS ACHIEVED (2/2)

A brochure on the main project’s findings - with the use of augmented reality techniques (D5.2);

A scientific seminar presenting the theoretical results (D5.4): Seminar on 23th of September, 2015 (Gliwice,

Poland); Seminar on 9th March, 2016 (Gliwice, Poland); Webinars/teleconferences (in average one per

month);

Scientific papers: 10 publications) (D5.3);


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

24

DISSEMINATION OF RESULTS (WP5) PUBLICATIONS IN 2014-2015 (D5.3)

1. W. Moczulski, K. Cyran, P. Novak, A. Rodriguez, M. Januszka, TeleRescuer - a concept of a system for teleimmersion of a rescuer to areas of coal mines affected by catastrophes. Abstract, 28 November 2014 (full paper will be published in: Proc. of the Institute of Vehicles, Warsaw University of Technology in 2015).

2. Smart Autonomous Mobile Systems for Exploring the Unknown. Poster presented at the 1st PERASPERA Workshop, Noordwijkerhout, The Netherlands, 11-12 February 2015.

3. P. Novák, J. Babjak, T. Kot, W. Moczulski, Control System of the Mobile Robot TELERESCUER. Proc. of the Optirob Conference, Bucharest, Romania, 27-30 June, 2015.

4. D. Myszor, W. Moczulski, K. Cyran: Innowacyjny interfejs ratownika umożliwiający wirtualną teleportację w celu inspekcji wyrobiska kopalni dotkniętego katastrofą (in Polish). Proc. of the 42nd Symposium on Technical Diagnostics (Abstracts), Silesian University of Technology, Faculty of Transport, Wisła, Poland, 02-06 March, 2015.

5. Novák P., Babjak J., Moczulski W. Control System of the Mobile Robot TELERESCUER. Applied Mechanics and Materials. 2015, vol. 772, pp. 466-470, doi : 10. 4028, ISSN : 1662-7482.

6. Novák P., Babjak J., Kot T., Olivka P. Exploration Mobile Robot for Coal Mines. In Modelling and Simulation for Autonomous Systems. International Workshop, MESAS 2015, Prague, Czech Republic, April 29-30, 2015, 209-215, ISBN 978-3-319-22383-4.

7. Kot T., Novák P., Babjak J. Virtual Operator Station for Teleoperated Mobile Robots. In Modelling and Simulation for Autonomous Systems. International Workshop, MESAS 2015, Prague, Czech Republic, April 29-30, 2015, 144-153, ISBN 978-3-319-22383-4.

8. Timofiejczuk A., Adamczyk M., Bagiński M., Golicz P., Wymagania dla robotów uczestniczących w akcjach ratowniczych w podziemnych kopalniach węgla kamiennego. Mechanizacja, automatyzacja i robotyzacja w górnictwie. Monografia. Krzysztof Krauze (Red.). Centrum Badań i Dozoru Górnictwa Podziemnego w Lędzinach, Katedra Maszyn Górniczych, Przeróbczych i Transportowych AGH w Krakowie. Lędziny: Centrum Badań i Dozoru Górnictwa Podziemnego, 2015, s. 59-65

9. Timofiejczuk A., Adamczyk M., Mura G., Nocoń M., Moczulski W., Układ mobilny specjalistycznego robota do inspekcji wyrobisk kopalnianych dotkniętych katastrofą. Mechanizacja, automatyzacja i robotyzacja w górnictwie. Monografia. Krzysztof Krauze (Red.). Centrum Badań i Dozoru Górnictwa Podziemnego w Lędzinach, Katedra Maszyn Górniczych, Przeróbczych i Transportowych AGH w Krakowie. Lędziny: Centrum Badań i Dozoru Górnictwa Podziemnego, 2015, s. 66-74

10. Mura G., Adamczyk M., Nocoń M.. Numerical simulation of mobility of miners rescue robot. 13th Conference on Dynamical Systems Theory and Applications. DSTA 2015, Łódź, December 7-10, 2015, Poland. Eds. J. Awrejcewicz, M. Kaźmierczak, P. Olejnik, J. Mrozowski. Łódź: Wydaw. Politechniki Łódzkiej, 2015, s. 222


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

25

MANAGEMENT AND COORDINATION (WP6)

Collaboration with the use of knowledge management system: MOBIROB platform on the base of OpenKM;

At least 20 technical meetings since the start of the project: including videoconferences and live meetings;

4 official meetings: kick-off meeting (Gliwice, July 2014), first annual meeting (Madrid, March 2015), half-year meeting in 2015 (Ostrava, July 2015), mid-term meeting (Gliwice, January 2016);

All actions in TeleRescuer project related to WP6 have been strongly supported by the Project Management Centre at the Silesian University of Technology


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

26

CONCLUSIONS PROBLEMS ENCOUNTERED AND CORRECTIVE ACTIONS

Due to financial issues at AITEMIN, this partner had to leave the consortium, with effect on the 31st of March 2015.

The withdrawal of AITEMIN was organized in an orderly manner.

A suitable replacement partner, with capacity to carry out the work initially allocated to AITEMIN was proposed, and accepted by the consortium.

AITEMIN will cooperate with the new partner in order to achieve a seamless transition.

Instead of AITEMIN the consortium included two new beneficiaries: UC3M and KOPEX.

KOPEX represents mining experience and knowledge necessary for completing the project.


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

27

CONCLUSIONS

All tasks in the framework of the TeleRescuer project are carried out in accordance with the schedule and budget: All tasks from WP1 have been

finished;

Tasks from WP2, WP5 and WP6 have been started and are still in progress.;

Tasks from WP4 are planned in the future;

All planned objectives, deliverables and milestones have been achieved.

Since the aproval of the amendment has taken quite a long time, the Consortium will apply for the extension of the project.


P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2

28

CONCLUSIONS FUTURE WORK

Regarding to WP2: Mechatronic subsystem of the UV; Communication system and sensory system; Control system; System for map building; System responsible for autonomous operation of UV in a known

environment; Regarding to WP3:

Software and hardware of the human-machine interface (HMI) for virtual teleportation technology;

Software and hardware for training simulator;

Regarding to WP4: Plan of validation tests.

In the next one year period the following results will be achieved:

TeleRescuer Project RFC-CT-2014-00002

Silesian University of Technology Coordinator

TeleRescuer Project Office

D.Sc. (habil) Anna TIMOFIEJCZUK Silesian University of Technology

Faculty of Mechanical Engineering

Vice Dean for Organisation and Development

Project Coordinator

Akademicka Street 2A 44-100 Gliwice

Konarskiego Street 18a

44-100 Gliwice Phone: +48 32 237 24 26

Fax: +48 32 237 13 60 e-mail: [email protected]

www.telerescuer.eu

THANK YOU FOR ATTENTION