research and development of drone and roadmap to evolution

Nonami, K.

Review:

Research and Development of Drone and Roadmap to EvolutionKenzo Nonami

Autonomous Control Systems Laboratory Ltd.WBG Marive West 32F, 2-6-1 Nakase, Mihama-ku, Chiba-city, Chiba 261-7132, Japan

E-mail: [email protected][Received March 22, 2018; accepted May 8, 2018]

The history of drone dates far back. Human in thosedays must have had a yearn for the sky. This articlefirst gives an overview of the history of research anddevelopment of drone including fixed wing and rotarywing, followed by, in particular, the history of devel-opment of rotary wing drone with the following fourperiods: before 1990 when radio control is at its dawnperiod; 1990 to 2010 when drone is at its dawn period;2010 to 2015 when hobby-use drone is at its spread pe-riod; and 2016 and after when industrial-use drone isat its dawn period to growth period. In addition, thisarticle presents the Japanese government’s roadmap,followed by a five-stage class of drone in terms of flightlevel and autonomy. In particular, autonomy will bediscussed from the points of view of guidance, navi-gation, and control. Then with reference to an idealstate of drone in future, the article will discuss impor-tance of the guidance system by fault tolerant controland supervisor control with implementation of the AItechnology and the like.

Keywords: drone, R&D history, evolution, roadmap, au-tonomy

1. Introduction

A drone is a flying robot that is not operated by hu-man but capable of autonomous flight controlled by thecomputer, and also referred to as a small unmanned aerialvehicle, UAV (Unmanned Aerial Vehicle), and UAS (Un-manned Aerial System). So, a radio controlled drone thatis wirelessly operated by a human is also included in thedrone because it requires the gyrofeedback system (an-gular velocity feedback control). The drone was initiallydeveloped for a military purpose, had its test flight be-fore the World War II, and then was fully deployed in theIraq War and the Afghan War from around 2000. On theother hand, with the evolution of mobile phones and smartphones, sensors and microprocessors became super small,MEMS, and high performance, which led about 20 yearsago to small, electric powered multi-rotor helicopters thatweighed a few hundred grams to few kilo grams and werecapable of autonomous flight. These drones ranged widefrom hobby-use to industrial-use, and were said to even

Fig. 1. A bamboo-copter.

bring the industrial revolution in the sky. In terms of tech-nical degree of completion, these drones had many prob-lems in safety, reliability, and durability and were still inthe stage of dawn period, which has limited their mainpurpose of use to hobby. However, utilization in industryhas started to gradually increase since around 2016. Thisarticle will present the history of research and develop-ment of drone so far and a roadmap of future evolution ofit.

2. History of Research and Development ofUnmanned Aerial Vehicles

The history of aerial vehicle dates far back. Humanin those days must have had a yearn for the sky. Wewill first give an overview of the history of unmannedaerial vehicles, followed by the history of developmentof mainly small rotary wing unmanned aerial vehicle, socalled drone, divided into the following four periods, be-fore 1990, 1990 to 2010, 2010 to 2016, and 2016 and af-ter.

It had known that lifting power is generated by rotatinga propeller attached to a bar as in Fig. 1, and there wasa rotor craft toy that appeared in the 4th century BC [a].Japan had a bamboo-copter [b] in the Nara Period. Thisis a basic principle of the helicopter.

A realistic design for human’s dream of flying in thesky was made by Leonardo da Vinci in 1483, the end ofthe 15th Century. Da Vinci left a sketch as in Fig. 2, whichis referred to as an air screw of Da Vinci. His notes and

322 Journal of Robotics and Mechatronics Vol.30 No.3, 2018

https://doi.org/10.20965/jrm.2018.p0322

© Fuji Technology Press Ltd. Creative Commons CC BY-ND: This is an Open Access article distributed under the terms of the Creative Commons Attribution-NoDerivatives 4.0 International License (http://creativecommons.org/licenses/by-nd/4.0/).

http://creativecommons.org/licenses/by-nd/4.0/

Research and Development of Drone and Roadmap to Evolution

Fig. 2. The air screw of Da Vinci.

Fig. 3. The world’s first wirelessly operated unmannedaerial vehicle.

drawings had an accurate description on how the deviceacted. Similar to the modern helicopters, the air screwis designed to compress the air so as to get flight, powerwas generated by four men getting on the central platformthat rotated the crank so as to rotate the shaft. Da Vincibelieved that sufficient rotation would successfully makethe device float above the ground, but in reality it is said tobe impossible to fly due to restriction of weight [a]. Theair screw is said to be the origin of the modern helicopters.

In 1917, Elmer Sperry started development of wire-lessly controlled “Hewitt-Sperry Automatic Airplane” incollaboration with an inventor and wireless engineer Pe-ter Hewitt. This automatic airplane was an airplane with-out pilot, and stabilized by the gyroscope technology ofSperry [c]. This is referred to as a full-scale wirelessly op-erated unmanned aerial vehicle presented in Fig. 3. Afterthat, wirelessly operated unmanned aerial vehicles weredeveloped in the Britain and in the U.S., Queen Bee inabout 1935 as presented in Fig. 4 and Target drone in 1937as presented in Fig. 5, respectively. These were used astarget planes of naval force and air force. It is said thatthis is the birth of the word “drone,” which is a kind of

Fig. 4. Queen Bee, developed by Britain.

Fig. 5. Target drone, developed by the U.S.

English word play by the U.S. in rivalry with the queenbee of the Britain [c].

Aerial vehicles are classified mainly into fixed wing air-crafts and rotary wing aircrafts. What is referred to as adrone today is small unmanned aerial vehicles that includethe both, but we will now focus on electric powered ro-tary wing aircrafts, so called multi-rotor helicopters (mul-ticopters), many of which are used for civilian purposesas industrial-use unmanned aerial vehicles.

3. Dawn Period of Remotely Operated Drone(Before 1990): Start-Up Period of RadioControl Business

In France, the Breguet Brothers developed in 1907 thefirst manned helicopter under the instruction of CharlesRichet [d]. As presented in Fig. 6, the first Breguet-Richetgyroplane was referred to as a quadcopter, which is alsoreferred to as the first multicopter [1].

The first full-scale manned helicopter, FA-61 heli-copter [e] was produced in Germany in 1936 presentedin Fig. 7, and Sikorsky’s VS-300 [a] was produced byVought-Sikorsky in the U.S. in 1939 presented in Fig. 8.FA-61 helicopter had a twin rotor and VS-300 had a singlerotor and a tail rotor, and these helicopters are the same asmanned helicopters of today. Since then, Sikorsky heli-copters became the mainstream of the world.

On the other hand, a hobby-use wirelessly operated

Journal of Robotics and Mechatronics Vol.30 No.3, 2018 323

Nonami, K.

Fig. 6. The first Breguet-Richet gyroplane.

Fig. 7. FA-61 helicopter, made in Germany.

Fig. 8. Sikorsky’s VS-300.

radio control helicopter with BellHiller swash plate wasborn by Dieter Schluter of West Germany in about1968 [f]. This was the world’s first of the hobby-use radiocontrol. Figs. 9 and 10 present the world’s first hobby-useRC helicopter. In those days, the flight time was about10 minutes, which achieved the level of model radio con-trol helicopters of today. In Japan, RC helicopters weredeveloped in the late 1970s and these hobby-use modelradio control helicopters were going to be mass producedaround the world in the early 1980s. Since these were ra-dio control helicopters, their performance was determineddepending on the operation skills of the human. At thesame time, however, embedded system technology, smallsensors, early micro-computers, and communication de-vices appeared, and some the early level of Avionics sys-tem, which was the auto pilot later, started to be imple-mented, leading to the dramatic growth period of dronefrom 1990s to 2000s.

Fig. 9. RC helicopter of Dieter Schluter (during wirelessoperation flight).

Fig. 10. RC helicopter of Dieter Schluter (landed).

4. Dawn Period of Autonomous Flying Drone(1990 to 2010): Growth Period of MainlyR&D Driven by Universities and ResearchInstitutions

Highly reliable UAV (Unmanned Aerial Vehicle) ap-peared in 1990s, and research and development of it be-came active in universities and other academic fields. InGeorgia Institute of Technology, the first internationalflying robot competition was held in 1991 [2]. Thiswas started as a university event but gradually expandedworldwide into the field of small unmanned aerial vehi-cle among government, industry and academia. Most ofthe studies focused on issues of higher performance hard-ware platforms, software systems, aerodynamic models,and autonomous flight control, and these state-of-the-arttechnologies were competed. In addition, DARPA (De-fense Advanced Research Project Agency) set up in 1997a project of MAV (Micro Air Vehicle), which led devel-opment of small drone with the diameter not exceeding15 cm at an annual research funding of 35 million USD(about 4000 million JPY) [3]. However, this project wastoo small in scale to be put into practice and was termi-



Fig. 11. A UAVs view of drone R&D of establishment period in the Nonami Laboratory, Chiba University, and AutonomousControl Systems Laboratory Ltd.

nated in 2001. During this period, international flyingrobot competitions were held in many countries.

Research and development of small unmanned aerialvehicle, in particular rotary wing drone, was actively car-ried out throughout the world. Main teams of them are asfollows [4].

1) Baykar Machine Inc. – Malazgirt Mini UnmannedHelicopters;

2) Beihang University – FH Series UAV Helicopters;3) Carnegie Mellon University – Yamaha-R50-Based;

UAV Helicopters;4) Chiba University – Sky Surveyor, Mini Surveyor;5) Codarra Advanced Systems – AVATAR;6) Draganfly Innovations Inc. – Draganflyer Series

Multiple Rotor UAVs;7) Epson Tokyo R&D – Micro Flying Robot;8) ETH Zurich – AkroHeli, PIXHAWK and muFly;9) Georgia Institute of Technology – Gtmax;

10) High Eye Aerial Service – He and X2F Series UAVHelicopters;

11) Honeywell – Duct-Fan-Based UAV iSTAR;12) Israel Aerospace Industries – Naval Rotary UAV;13) Linkoping University – WITAS Helicopters;14) Massachusetts Institute of Technology –

Draganflyer-Based Quadrotor UAV;15) Nanjing University of Aeronautics and Astronautics

– Yujingling Quadrotor UAV;

16) NASA Ames Research Center – Yamaha-Rmax-Based UAV Helicopters;

17) National University of Singapore – HeLion, She-Lion, BabyLion;

18) Neural Robotics Inc. – Guncopter;19) SAAB Aerosystems – Skeldar V-150 VTOL UAV;20) Shanghai Jiao Tong University – Sky-Explorer;21) Schiebel – Schiebel UAV Helicopters;22) Sikorsky Aircraft Inc. – Cypher and Cypher II;23) Technische Universitat Berlin – Marvin Series UAV;24) University of California at Berkeley – Ursa Major

and Ursa Maxima;25) University of New South Wales – MAVstar;26) University of Southern California – AVATAR;27) University of Waterloo – Custom Ducted fan UAV;28) US Naval Research Laboratories – Dragon Warrior;29) Yamaha Inc. – R50 and Rmax UAV Helicopters.

We will now explain about the drone R&D in the Non-ami Laboratory, Chiba University, which is 4) in the list.As presented in Fig. 11, the laboratory started in 1998to research single rotor and succeeded in 2001 in fullyautonomous control of a small unmanned helicopter ofweight of about 10 kg for the first time in Japan. Afterthat, the laboratory worked on autonomous control of var-ious aircrafts, converted into autonomous control of mul-ticopters after 2006, and completed in 2011 a domesticminisurveyor MS-06. They completed in 2015 a mass


Nonami, K.

produced aircraft MS-06LA as presented at the bottomright of Fig. 11. On the basis of these outcomes, they es-tablished in 2013 Autonomous Control Systems Labora-tory Ltd., a venture company launched by Chiba Univer-sity. In fact, many of the world leading venture companiesof drone of today were born from the 30 main drone re-search team of the world in 1990 to 2010, which werepresented earlier. Most of the venture companies were setup by then undergraduate students and graduate studentswho studied autonomous flight in research laboratoriesof the universities mentioned above and foreign studentswho stayed there for a few years and studies autonomouscontrol technology as their research subject.

During this period, many researchers paid attention todrone and released many research papers. Some of themdid not satisfy with simulation and started to produce theirown multicopters to test a compatible algorithm, in par-ticular attitude control algorithm. Although an algorithmwas designed with ease, it was not easy for the researcherto produce an actual multicopter. Even if a few multi-copters were designed, they were often not high in relia-bility. Some of the researchers built a test environmentsuch as a real-time indoor autonomous flight test envi-ronment using an existing highly reliable commerciallyavailable quadcopter and optical motion capture system.Under such an environment, the team of Vijay Kumar inUniversity of Pennsylvania [5] and Massachusetts Insti-tute of Technology [6] flew many micro UAVs so that theyverified advanced, complicated tasks.

With development of MEMS (Micro-Electro-Mechanical System), a super light-weight inertiameasurement unit (IMU) was born. A MEMS sensorwas commercially available but a low-cost MEMSIMU generates a great noise. So, the research of thefield started to pay attention to the method to removenoise in the attitude measurement of an MEMS IMU:digital filter design and implementation technology. Inaddition, designing a small multicopter requires notonly an algorithm but also a micro-computer that iscapable of executing the algorithm. In this stage, theprocessing speed of micro-computers such as SCM(Single Chip Microcomputer) and DSP (Digital SignalProcessor) was significantly improved. This improved thepossibility of designing a small multicopter. Universityresearchers built their own mathematical models, andbuilt and designed their original control algorithms in atheoretical manner. In addition, some pioneers started tomake their own original multicopters [6–8]. The initialconcept of quadcopter was only for military purpose,but they opened a way to consumers through the RC toymarket. In the early 1990s, a quadcopter of hand-heldsize referred to as Keyence Gyrosaucer [g] presented inFig. 12 was sold in Japan. It was the original aircraftsfor hobby use. This mini quadcopter, designed for anindoor use, included a polystyrene foam airframe andpropeller, and used two gyroscopes for “attitude controland slewing control.” It was capable of flight of aboutthree minutes per charge.

Microdrones GmbH [1] was set up in October 2005.

Fig. 12. Gyrosaucer, the original of hobby-use.

Fig. 13. MD4-200 of Microdrones Co.

Their first product MD4-200, presented in Fig. 13, wasreleased in April 2006. More than 250 products were soldin a short period of time. In 2010, MD4-1000 was re-leased. These products made a big success in the com-mercial market. In October 2006, a large community ofMikroKopter autopilot was set up. They released an open-source autopilot MikroKopter. Semi-autonomous flightalso made possible. In 2007, Ascending TechnologiesGmbH was established in Germany [h]. Ascending Tech-nologies was a leading Germany start-up that had con-trol software and hardware technologies, selling Falcon, aunique octo-rotor for aerial photographing, and X-3D-BL,a platform for research, as presented in Figs. 14 and 15,respectively, but became a subsidiary of Intel in 2016. Incooperation with the perceptual computing team of Intel,they are engaged in development of a UAV technologythat helps the drone to recognize its surrounding environ-ment. In 2008, Draganflyer released Draganflyer X6 [i].It was characterized by carbon fiber structure, folding air-frame, autopilot technology, a variety of payloads, anda unique hand-held controller. Similarly, they sold con-



Fig. 14. Falcon of Ascending Technologies Co.

Fig. 15. X-3D-BL of Ascending Technologies Co.

Fig. 16. Draganflyer IV of Draganflyer Co.

sumer multicopters such as Draganflyer IV of Fig. 16 andX-UFO without camera. They had a problem of not easyoperation caused by a small payload judged for the size ofthe airframe. That was because a GPS (Global Position-ing System) receiver was not mounted. What was moreimportant was that there was no smart phone coming outinto the world, thereby failing to acquire general users asthe customer.

In 2004, Parrot Company [1] launched a project re-ferred to as “AR.” The “AR drone” was intended for a

Fig. 17. AR drone of Parrot Co.

micro UAV for the mass market of video game and homeentertainment. In 2010, they released the AR drone ofFig. 17. This project was provided with technical sup-ports from navigation and control designers of MINESParis-Tech as well as from engineers of Parrot Co. [9].Its inexpensive price and excellent flight control made theAR drone come under the spotlight at once, making themulticopter market lively.

In this period, Germany, having 4 or 5 leading com-panies, was predominant but in the end, the AR droneof Parrot Co. of France took the world by storm. Therewere various reasons for this but tradeoff between flightperformance and price, purpose, and timing seemed to becritical.

5. Spread Period of Hobby-Use Drone (2010 to2016): Explosive Spread and Maturity Pe-riod Focusing Mainly on Hobby Use

Coming into the spread period of hobby-use drone,emergence of many open-source autopilots rapidly low-ered the hurdle at once for beginners to produce multi-copters. In particular, it became possible that flight con-trol technology for the most difficult autonomous flightwas implemented with ease free from the need to get in-volved in elaborative software and algorithm.

With the evolution of drone, there have been a varietyof flight controllers coming out in the world, but they areconverging little by little. The largest power is the flightcontroller (FC), a product of DJI, a Chinese drone man-ufacturer that takes up an overwhelming share in droneplane. This is a closed, vertically integrated product forwhich everything from hardware to software is controlledby DJI. The latest model FC is a product called A3. And,another power is open-source-oriented flight controllers.On the basis of know-how of the U.S. drone manufacturer3D Robotics and the Swiss Federal Institute of Technol-ogy in Zurich (ETH Zurich), there are various productssuch as APM, PX4, and Pixhawk, which use the dronecode as illustrated in Fig. 18 [j]. Recently, the effort of us-ing Linux OS has been advanced because it is more flexi-ble and open than the micro-computer, and there is a flightcontroller called NAVIO+, which runs on Erlebrain andRaspBerry Pi. Linux is also used in the Bebop Series ofParrot Co. and Solo of 3D Robotics Co. The DronecodeProject was set up as a non-profitable organization by theU.S. Linux Foundation in October 2014, but broken up in


Nonami, K.

Fig. 18. A main flow of open-source and closed-source FCs.

August 2016 under the influence of poor performance of3D Robotics and the like. Full details are not known, butthey still have a certain influence as ever.

iPhone was announced by Steve Jobs at MacWorldExpo in January 9, 2007, the day when it dramaticallychanged not only the history of telephone but also thepeople’s lifestyle. Evolution has been drastic ever since,in particular, evolution of MEMS sensor has acceleratedthe development of drone. iPhone 4, released in 2010,is provided with the full 9DoF motion sensing such as atriaxial accelerometer, a triaxial gyroscope, and a triax-ial electronic compass. With this progress of smart phonetechnology, size, cost, and power consumption have beenreduced. Smart devices including pinhole cameras are ca-pable of receiving and controlling video signals via WiFiwithin a specific distance, and thus they have evolved toget high resolution that is equivalent to single-lens reflexcameras. From the requests of the smart devices, the vol-ume energy density of lithium-ion battery has been dra-matically increasing. In addition, the intense competi-tion in the mobile phone market strongly demanded forall the mounted components to be small to the utmostlimit, thorough power saving, and highly functional andlow price for differentiation of the product. Among oth-ers, requirements conditions for being small and powersaving are totally strict, and the know-hows and outcomesof technology innovation realized here have significantlycontributed to the commercialization of various state-of-the-art devices to be mounted onto the drone. Ultimately,we have recently seen even a product that transforms asmart phone itself into a drone.

The AR drone [1] presented in Fig. 17 made a verygreat success in the toy market. Its technology and con-cept were very advanced. In the beginning, a downwardcamera was used to measure an optical flow and two ultra-sonic range finder were used to measure the altitude. Onthe basis of them, the speed can be obtained by using anestimation algorithm. Since the AR drone is used indoorand easy to operate, the usability was largely improved.Secondly, it was a light-weight, indoor aircraft made ofstyrene foam. Thirdly, the AR drone can be controlledwith a smart phone or a tablet using display of point ofview of the drone-mounted camera. In addition, it had aresearch potential by providing a software development

Fig. 19. PHANTOM-4 Pro [k] of DJI.

kit (SDK) so that the researchers can develop their ownapplications. As a result, the product rapidly spread as aresearch platform. Some companies were inspired by thesuccess of the AR drone and converted their strategy intothe hobby-use multicopter. DJI was established in 2006and their business was not necessarily good. However,they released in the end of 2012 an all-in-one quadcopter“Phantom.” They have been updating PHANTOM everyyear up to PHANTOM-4Pro of Fig. 19 at present [k].

In this stage, multicopter research tended to be moreautonomous and cooperative. In June 2013, RaffaelloD’Andrea of ETH Zurich gave a speech about “athleticmachines” in TED Global 2013. In order to demonstratethe excellent flight performance of the drone, he presenteda drone throwing a ball and another drone catching it, adrone engaged in an acrobatic flight, and flight demon-stration of a quadcopter controlled by Kinect. In June2015, Nature gave in a special section related to machineintelligence an article related to “future of science tech-nology and small scale autonomous unmanned aerial ve-hicle.” This review paper summarizes issues in design andproduction, measurement and control, and future researchtrend in the field of small drone.

Similar to the AR drone, “Phantom” of DJI can be con-trolled with ease, and its position can be retained by us-ing a GPS and an altitude sensor. In comparison with theAR drone, Phantom has a certain amount of pay load, andwind resistance. In particular, Phantom is attached with asport camera, and its enthusiasts shared videos via SNS.This facilitated more people to buy Phantom.

Mr. Chris Anderson, who was the editor-in-chief ofWired, also joined in 3D Robotics as the CEO in 2011.Under support from 3D Robotics, volunteers in the worldimmediately created a world class, universal flight codefor APM [l]. APM is led by Mr. Randy Mackay. In July2012, PX4 team, led by Mr. Lorenz Meier of ETH Zurich,released a PX4 autopilot platform that has hardware that isimmediately available from 3D Robotics. In August 2013,PX4 project and 3D Robotics released Pixhawk. Pixhawk



Fig. 20. Forecast of domestic market scale of the entire drone business.

is an advanced open autopilot for fixed wing aerial ve-hicles, ground rovers, and amphibian cars. Pixhawk hasimproved usability and reliability, and is designed to pro-vide a safety function that is incomparable to conventionalsolutions. In December 2013, Amazon released a video,expecting that within five years they were going to pro-vide “Prime Air,” which was made up of a quadcopter,that delivers a small package to your home in only half anhour. This idea shook the world.

6. Dawn Period to Growth Period ofIndustrial-Use Drone (2016 to Future):Start of “Industrial Revolution in the Sky”

The history of development of multicopter gives us thefollowing lesson. Gyrosaucer, X-UFO, and Draganflyerhave failed to come under the spotlight so much. On theother hand, as smart devices come out with appearanceof iPhone, smart multicopters such as the AR drone trig-gered the drone boom. Then, Phantom of DJI, which farexceeds the AR drone in terms of performance, acceler-ated the rapid spread throughout the world in a short pe-riod of time with the SNS boom as a background. Thisindicates that social background and technology level ofthe time of release of product are important touchstones.

Dronecode seemed to be a powerful strategy that tookthe drone industry by storm, but the fact is that the DJIstrategy has worked although Dronecode has a certain in-fluence. This suggests that the world is asking for an all-in-one, solution-type drone rather than a self-buildabledrone. It means people are waiting for appearance of atruly fully autonomous drone that achieves mission solu-tion not requiring the user to do anything at all.

It is undoubted that in future, drone will evolve fromhobby-use drone to industrial-use drone. For hobby-use

drone, an all-in-one drone with camera most satisfies thecustomer needs. For industrial-use drone, on the otherhand, customization is required depending on the purposeof use, and hence a simple, all-in-one type is incapable ofsatisfying the customer needs. In other words, customiza-tion is required so that the drone manufacturer and thedrone user in cooperation can reach the solution uniqueto the industry. For this reason, completion of solution-type drone requires a few trials and errors such as conceptvalidation, and thus it takes a certain period of time. It isnot until this that the business development gets started.This is a large difference between the hobby-use droneand the industrial-use one, where the hobby-use is appro-priate for mass production meanwhile the industrial-use isappropriate for small quantity, large variety production. Inthe infrastructure inspection of i-construction, 700 thou-sand bridges need to be inspected every five years, withno two bridges being identical. Accordingly, even thoughthe industrial-use drones have a common platform, thereare differences in the way of mounting a camera and theway of inspection. Then, in the case of infrastructure in-spection, processing of an enormous amount of obtaineddata, so called big data, will become very important infuture, which is said to bring a big business opportunity.

Figure 20 presents a future market forecast releasedby a think tank company Seed Planning [m]. The figurepresents future rise of the aircraft market of drone manu-facturers, the drone flight services that undertake inspec-tion, and the data storage and analysis market of obtainedbig data. As known from Fig. 20, it is forecasted that thedata storage and analysis market will achieve the high-est growth. In fact, it is said that the essence or the truevalue of the “industrial revolution in the sky” lies not inthe drone itself but in enormous big data such as inspec-tion and growth survey that are newly generated by uti-lizing the drone, convenience by which an item can bedelivered very soon after it was ordered in spite of lack of


Nonami, K.

Fig. 21. Flight levels of small unmanned aerial vehicles (drones) [n].

labor force, lowered cost due to labor cost left over by ahuman not intervening, the eco system that can promotethe environmentally-friendly low-carbon society thanks tobattery drive, and so on.

7. Roadmap for the “Industrial Revolution inthe Sky” Set by the Japanese Governmentand Advanced Autonomous Flight

The “Roadmap for the Application and Technology De-velopment of UAVs in Japan [n]” (April 28, 2016, ThePublic-Private Sector Conference on Improving the Envi-ronment for UAVs), which was set by the Japanese gov-ernment, defines the flight level of small unmanned aerialvehicles (drones) as in Fig. 21. It defines the flight levelinto four stages, where Level 1 is a radio control level,Level 2 is an autonomous flight drone with visual line ofsight (VLOS), Level 3 is an autonomous flight with be-yond visual line of sight (BVLOS) without any observerin an less-populated area, and Level 4 is an autonomousflight with beyond visual line of sight (BVLOS) in a pop-ulated area. It is anticipated that Level 3 is achieved inabout 2018 and Level 4 in about 2020s.

8. Correlation Between Flight Level and Au-tonomy (Safety) Class of Drone and Ulti-mate Autonomous Flight

The autopilot (AP) is an integrated system of hardwareand software of guidance (G), navigation (N), and con-trol (C) by which the drone is capable of carrying outa range of flight from a programed flight such as a ba-sic waypoint flight to an advanced autonomous flight, forexample, a flight while avoiding the obstacle and carry-ing out the trajectory plan in real time by itself. Fig. 22presents the difference between AP and FC. AP containsFC, i.e., AP is a broader concept, which also comprehendsthe work of skilled pilot of the manned aerial vehicle. In

Fig. 22. Difference between AP and FC and guidance, nav-igation, and flight control [10, 11].

the manned aerial vehicle, a skilled pilot carries out obsta-cle recognition and decision making, in other words, guid-ance, meanwhile the unmanned aerial vehicle is pilot-freeand hence that role needs to be played by the on-boardcomputer and the ground support system. On the otherhand, the flight controller (FC) is an integrated system ofhardware and software that carries out flight while keep-ing the unmanned aerial vehicle in a stable state in accor-dance with a given flight trajectory. In the case of flightof a commercially available hobby-use quadcopter, AP isimplemented with only the lower order structure, whereAP = NS + FC. In this case, FC is continuously calcu-lating a command for controlling the rotational speed ofmotor based on input of pilot while keeping the plane at-titude in a stable state.

The degrees of autonomy of drone in which Level 3and Level 4 of Fig. 21 are achieved are given in Fig. 23.Fig. 23 presents the autonomy of drone, which is almost asynonym of safety, classified into five stages from Class Ato Class E, with the concept, the guidance level, the nav-igation level, the control level, and the scenario on an as-sumption of logistics, all of which are detailed for each ofthe five stages.

Class E is a level at which the operation skill by hu-man is put to the test in a radio control operation drone.



Class Concept Guidance(intelligence and decision

making)

Navigation(measurement and estimation)

Control Scenarioand so on)

A

accurate

B

Fault tolerance / diagnosis during drone

reconstruction to make

C

Regarding programed

D

(autonomous board micro-computer

EAll decision making All measurement and recognition All control (control

including attitude control) are carried out

2020s

2019

2017

2015

2013

Fig. 23. Autonomy (safety) class of drone and future roadmap (Class A∼E).

Class D is a class in which an autonomous flight is madepossible as a so-called waypoint flight, i.e., a programedflight with everything from take-off to landing is deter-mined in the trajectory plan made in advance by human onan assumption that the GPS radio wave can be received.The guidance is all judged by a skilled person. It is a classin which everything is processed by the on-board CPU,automatically notifying communication failure, compassabnormality, remaining battery level, and so on. Most ofthe industry drones commercially available now can bejudged as Class D.

Class C is for drones capable of autonomous flight evenunder a non GPS environment. It takes various methodssuch as image processing using a camera, laser, lidar, totalstation, sounds, and radio wave. Various drone abnormal-ity notifications and the like are similar to those of Class Dand the presence and absence of mission continuation isjudged by a human. The world’s state-of-the-art dronesas of 2017 are thought to be in Class C, close to Class Bthough.

Class B is for advanced drones like flying robotsthat will appear in around 2019. They are defined asdrones (flying robot) that will never crash, which will au-tonomously deploy the parachute or the like and make acrash landing before crashing if an abnormality occurs.To do this, the abnormality diagnosis algorithm is con-stantly activated during flight and, if the health condition

of the flying robot is different from the normal condition,the cause of the abnormality is identified and whether ornot the mission is continued is autonomously judged. So,the guidance basically depends on the autonomy of thedrone (flying robot) side. SAA (Sense and Avoid) is alsorealized in this Class B. SAA is associated with discoveryand immediate avoidance of an obstacle present forwardin flight and trajectory re-planning on a real time basis.

Class A is an ideal form of flying robot, which can becalled a bioinspired flight, i.e., flying like a bird. GPSradio wave is not necessary any more. It carries out high-speed image processing from images taken by the cameraor the like that is mounted on the flying robot. It thus car-ries out self-positioning estimation. The flying robot it-self is capable of recognizing where it is flying currently.The flying robot has an ability of reaching the destinationthat is even 10 km away or farther with a landmark onthe ground without using GPS radio wave. It is a classin which of course the flying robot may receive a supportof UTM (UAV Traffic Management System) where nec-essary and is capable of safe flight with perceiving in ad-vance a flying robot abnormality while carrying out FTA(Fault Tree Analysis), which is a fault analysis duringflight. In this stage, the learning effect of artificial intelli-gence (AI) can also be utilized, where the more the flyingrobot flies, the more intelligent the autonomous flight be-comes. This is supposed to be realized in 2020s.


Nonami, K.

Fig. 24. Correlation diagram between flight level of drone and autonomy (safety).

Figure 24 presents correlation between the flight levelset by the government of Japan as in Fig. 21 and the evo-lution level of drone as in Fig. 23. The figure gives aconcept of what degree of drone autonomy (safety) canpermit what degree of flight level. The beyond visual lineof sight (BVLOS) flight for long distance without any ob-server of Level 3 has to be the middle of Class C or higher,and a certain extent of capability of abnormality diagno-sis of the drone is desirable. If it is preferably evolved toClass B, SAA function is implemented, where abnormal-ity diagnosis can almost autonomously respond, and theflying robot has the function of autonomously detectingthe abnormality depending on the result of the abnormal-ity diagnosis and activating a safety device to prevent theflying robot from crashing. Therefore, it can be judged asa level that has no problem in autonomous flight in less-populated areas.

The autonomous flight in populated areas of Level 4has to be in Class A. In particular, it is capable of imme-diately recognizing change in three-dimensional environ-ment such as weather, radio wave, and magnetic field, andis fully provided with the guidance abilities such as FTAanalysis and crisis management capacity. In that sense, itis an unmanned aerial vehicle close to the manned aerialvehicle in terms of safety design, which is expected to sig-nificantly reduce the probability of accidents in populatedareas.

9. Autonomous Control Technology Requiredin Autonomy (Safety) Class B of Drone

Fault of multi-rotor helicopter during autonomousflight can be roughly divided into four: the first is com-munication system fault related to uplink and downlinkbetween the ground and the drone; the second is sensorsystem fault related to navigation such as in-IMU sensors

and barometer, GPS receiver, INS-related, and vision; thethird is control system fault mainly in the micro-computerboard that carries out control calculation and peripheraldevices; and the fourth is multicopter propulsion systemfault mainly in the drive system. These faults can be han-dled in general by employing a redundant system. How-ever, if it is impossible to employ a redundant system dueto various restrictions, the fault tolerant control, which ispresented below, is effective. In particular, it is difficultin general to realize employing a redundant propulsionsystem from a point of view of size, weight, and cost.So, we introduce the autonomous control technology thattargets at fault tolerant control of propulsion system ofthe multi-rotor helicopter. The propulsion system of themulti-rotor helicopter is made up of a propeller, a motor,a motor driver, and a battery. All of damages and faults ofthese components are propulsion system fault.

Figure 25 presents a fault tolerant control system [12]against propulsion system fault. Its basic idea is as fol-lows: the computer has a physical model that simulatesan actual system, and as in Fig. 25; a control input is in-put to these actual model and physical model; outputs areobtained from the both; and a difference between them isobtained. If the difference is within a permissive range,abnormality is not present. If it exceeds the permissiverange, it is judged that abnormality is present and an in-verse problem called fault system analysis called FTA(Fault Tree Analysis) up to what is the abnormality issolved. In Fig. 25, above the dashed line denotes softwareimplemented in the supervisor, where the abnormality di-agnosis algorithm carries out FTA analysis at the fastestspeed using the physical model. If a fault occurs, faultinformation is transmitted to a re-building section, thecontrol structure is switched to the optimal controller andcontrol parameter, and thus the flight is continued. Eventhough the control structure is momentarily switched, it isstill a switch in a finite time. In the case of a hexa-rotor



Fig. 25. Fault tolerant control system (fault tolerant control).

drive system fault, one ESC, drive motor, and propellersystem that was fault is stopped, and the control structureis momentarily changed. It is also important the controllerrobustness for the multicopter behavior not to greatly fluc-tuate in a period of time from when the control structurewas changed to when the control is started by the five mo-tors. For this reason, the sliding mode control [13], whichis a non-linear control that is capable of exerting a robustcontrol performance, is applied, so that one motor is mo-mentarily stopped, the control structure is changed, andthus the drone attitude is stabilized.

Next, we will discuss methods to constantly optimizethe controller by adapting an environment change duringflight. One of the methods is the self-tuning control. Onan assumption of a home delivery drone, after delivering aparcel, the drone weight will become light, which accord-ingly causes the center of gravity of the drone to be movedand the drone inertia main axis to fluctuate. In particular,in the case where no measure has been taken, the dronewill fly in a state where the controller is deviated from theoptimal state, such as a sharp rise of the drone, reductionin response speed, and generation of steady-state devia-tion. In the worst case, the control system becomes in-stable, resulting in accidents such as crash. Now, we willdiscuss a method to prevent the control performance frombecoming poor by appropriately adjusting the controllerparameter with the self-tuning technique of the adaptivecontrol theory even if the drone weight is momentarilyfluctuates.

The self-tuning is one of the control techniques thatassumes an unknown parameter included in the controltarget. A general block diagram of the self-tuning is pre-sented in Fig. 26 [12]. At the time of beginning of control,the controller is designed with the control target regardedas known, and the unknown parameter is estimated in se-quence and reflected into the controller. This configuresa controller using the unknown parameter that has beenonline identified in the end.

Figure 27 presents a slow-motion picture of a behav-ior when an actual drone ACSL-PF1 developed by Au-

Fig. 26. Block diagram of self-tuning [12].

Fig. 27. Drone behavior when it dropped 2 liters of water [12].

tonomous Control Systems Laboratory Ltd. (ACSL) withthe algorithm being implemented drops two liters of watermomentarily (about 0.1 seconds). It has successfully re-duced the drone rise to about 5 cm without rapidly risingin spite of mass change of 2 kg.

10. Ideal State of Drone Autonomous Controlin Near Future

As described with reference to Figs. 22 and 23, theguidance (G: Guidance), the navigation (N: Navigation),and the control system (C: Control) play an important rolein order to carry out an autonomous flight while quicklyrecognizing an obstacle in front of the drone along theflight, autonomously carrying out crash avoidance, rec-


Nonami, K.

Fig. 28. Ideal state of autonomous control in near future.

ognizing a complicated environment, and self-generatingthe flight route. These three elements (GNC) are the coretechnology for a fully autonomous control flight and willbe rapidly evolved in future as a brain of autonomousflight. In particular, as seen in logistics and home deliverydrones, when the level of autonomous control flight is ad-vanced to be beyond visual line of sight (BVLOS) flightand long distance flight, the guidance, the navigation, andthe control system will determine the performance in acrucial manner.

The guidance system in the UAVs plays a similar roleto the cerebrum of human, i.e., in charge of recognition,intelligence, and determination as presented in Fig. 28. Itcarries out so-called real time route generation, i.e., au-tonomous flight with determining a target trajectory realtime while detecting an obstacle and avoiding crash evenin a complicated unknown environment. In the case wherean abnormality occurs in the drone as a flying robot, it isdetermined whether or not flight can be continued and,if it is difficult, the flying robot returns to the groundwhile searching for a safe place. Such mission is includedand it hence corresponds to high-level autonomous flightthat requires an advanced, momentary determination. Ina manned aerial vehicle, it is an advanced technology thatis carried out by the pilot. However, in a UAV, the com-puter needs to do everything: recognition of the three-dimensional space that change in sequence; and momen-tary determination of the flight route and the altitude. Inthe current state, flight is carried out with no or little guid-ance. In this sense, the most important, urgent technolog-ical issue of drone is to implement a guidance function.The guidance function has the following two sections, asection to be implemented by the drone itself and a sectionto be carried out by the ground support system such as theUTM. Most of the functions of the guidance, i.e., recog-nition, intelligence, and determination, are expected to berealized by applying AI in future. Then, a total systemwill be achieved in which flying robots are brought into anetwork and connected also with the ground support sys-tem, and a manned aerial vehicle and an unmanned aerial

vehicle recognize each other when flying.Most of the current UAVs capable of autonomous flight

have realized a basic autonomous flight referred to as thewaypoint flight by two systems of the navigation systemand the control system without the guidance system.

Where the guidance corresponds to the cerebrum of hu-man cerebrum, the navigation and the control correspondto the small brain of human cerebellum, which controlsthe equilibrium sense and the motor function. The ad-vanced navigation system redundantly includes laser, ul-trasonic sensors, infrared ray sensors, single and stereocameras, 3D cameras, vision chips, and so on, carries outmapping and obstacle detection, and improves the accu-racy of localization as self-positioning estimation.

Regarding the drone autonomy (safety) presented inFig. 23, an example of method to realize Class A ofthe bioinspired flight is thought to be the structure asin Fig. 29. Fig. 29 is a chart in which Fig. 22 is de-scribed in detail, which presents the contents of the threeelements of the guidance (G), the navigation (N), andthe control (C). The supervisor corresponds to G, whichdetermines whether it is GPS/INS navigation or visualSLAM navigation, changes the structure of the controlsystem as necessary while carrying out an exact environ-ment recognition and momentary determination regardingevery event encountered during flight, and perfectly car-ries out the mission while generating a target trajectory inreal time. The fault tree analysis (FTA) estimates what isgoing on from the difference between the real-time iden-tification model and the ideal model. The abnormalitydiagnosis of the flying robot is carried out thus by obtain-ing a difference between the flight model system identi-fication and the ideal model during flight and by deter-mining whether the difference falls within the permissiverange. All of them are the role of the supervisor, i.e., theguidance G. Regarding crisis management, the encounterwith crisis is learned by AI in advance, and whether ornot to carry out the mission is determined with matchingthe degree of danger. Troubles during flight include var-ious events, how to send off an alert signal at the time ofthese abnormalities has been learned in advance by suffi-cient AI learning. Unless there is a special abnormality,the flying robot gets to the destination while highly accu-rately recognizing a three-dimensional environment usingthe vision sensor of the navigation. It is expected that theflying robot encounters a sudden change of weather anda gust during the flight, but for each time, with the con-trol system structure being variable, it flies to the destina-tion with the top priority given to the efficiency in normaltimes meanwhile with the top priority given to the abso-lute stability in times of unexpected disturbance and soon.

11. Conclusions

The drones of Class A of Fig. 23, which become next-generation industrial-use drones and might be called asflying robot in next generation, are required to have re-



Fig. 29. Realization of autonomy Class A by supervisor (guidance system).

liability, durability, and safety that allow them to repeata daily flight of about eight hours for about five years,needless to mention an obvious procedure of daily regularinspection and components replacement. The flying robotof near future is an advanced intellectual flying machineand flying robot but still remains a machine. So, the fly-ing robot, which flies very low where the weather tendsto change drastically, has to have a function capable of re-spond to abnormal events during flight. In other words,in the case of an abnormal situation caused by the fly-ing robot itself, an abnormal situation caused by weatherchange, and an abnormal situation caused by another in-evitable force, the next-generation industrial-use flyingrobot has to be a “drone that never crashes” that carries outa crash landing at a safe place to land that is searched bythe flying robot itself before crashing. It corresponds to atleast “Class B” of the autonomy class. On an assumptionof flight over populated areas and flight over urban areasat Level 4, which will be realized in 2020s, it is essentialto be an intelligent flying robot that detects an abnormal-ity, searches for a safe place to land by itself and slowlyfalls while there is a remaining power, and waits for therescue to come. In that sense, such an industrial-use fly-ing robot has not yet come out in the world. However, theflying robot evolves so fast that in 2020s the flying robotwill have the autonomy of Class A, and the era in whichSF movies come true will undoubtedly come where hun-dreds or thousands of logistics drones fly over urban areasin an orderly manner.

References:[1] Q. Quen, “Introduction to Multicopter Design and Control,”

Springer, 2017.[2] E. N. Johnson and D. P. Schrage, “The Georgia tech unmanned

aerial research vehicle: GTMax,” Proc. of AIAA Guidance, Nav-igation, and Control Conf., Austin, 2003.

[3] R. J. Wood, “The first takeoff of a biologically-inspired at-scalerobotic insect,” IEEE Trans. on Robotics, Vol.24, No.2, pp. 341-347, 2008.

[4] G. Cai, B. M. Chen, and T. H. Lee, “Unmanned Rotorcraft Sys-tems,” Springer, 2011.

[5] N. Michael, D. Mellinger, Q. Lindsey et al., “The grasp multi-ple micro-uav testbed,” IEEE Robot Autom. Mag., Vol.17, No.3,pp. 56-65, 2010.

[6] T. Hamel, R. Mahony, and A. Chriette, “Visual servo trajectorytracking for a four rotor VTOL aerial vehicle,” Proc. of IEEE Int.Conf. on Robotics and Automation, Washington, DC, pp. 2781-2786, 2002.

[7] E. Altug, “Vision based control of unmanned aerial vehicles withapplications to an autonomous four rotor helicopter, quadcopter,”Dissertation, University of Pennsylvania, 2003.

[8] S. Bouabdallah, P. Murrieri, and R. Siegwart, “Design and con-trol of an indoor micro quadcopter,” Proc. of IEEE Int. Conf. onRobotics and Automation, New Orleans, USA, pp. 4393-4398,2004.

[9] P. J. Bristeau, F. Callou, D. Vissire et al., “The navigation and con-trol technology inside the AR. Drone micro UAV,” 18th IFAC WorldCongress, Milano, Italy, pp. 1477-1484, 2011.

[10] K. Nonami, “Exact Nonlinear Model, Uncoupled LinearizationModel and Model Based Control of Unmanned Aerial Vehicles,” J.of the Society of Instrument and Control Engineers, Vol.56, No.1,pp. 3-9, Jan. 2017.

[11] K. Nonami, F. Kendoul, W. Wan, S. Suzuki, and D. Nakazawa, “Au-tonomous Flying Robots,” Springer, 2010.

[12] K. Nonami and S. Inoue, “Research on sub-committee of aerialrobots of ImPACT/TRC,” J. of the Robotics Society of Japan,Vol.35, No.10, pp. 700-706, 2017.

[13] K. Nonami and H. Tian, “Sliding Mode Control,” Corona Publish-ing, 1994.


Nonami, K.

Supporting Online Materials:[a] http://www.aerospaceweb.org/design/helicopter/history.shtml

[Accessed June 6, 2018][b] https://ja.wikipedia.org/wiki/%E7%AB%B9%E3%81%A8%E3%82

%93%E3%81%BC [Accessed June 6, 2018][c] https://understandingempire.wordpress.com/2-0-a-brief-history-of-

u-s-drones/ [Accessed June 6, 2018][d] http://www.aviastar.org/helicopters eng/breguet gyro.php

[Accessed June 6, 2018][e] http://www.defensemedianetwork.com/stories/nazi-helicopters-

henrich-fockes-fa-61/ [Accessed June 6, 2018][f] http://www.vrhc.co.uk/Heli%20History/1970/1970%20-

%20Heli%20History.htm [Accessed June 6, 2018][g] http://dronemedia.jp/gyrosaucer-keyence/ [Accessed June 6, 2018][h] Ascending Technology Company.

http://www.asctec.de/en/ascending-technologies/company/[Accessed June 6, 2018]

[i] http://www.draganfly.com/story [Accessed June 6, 2018][j] http://dronemedia.jp/open-source-drone-code/

[Accessed June 6, 2018][k] http://www.dji.com/jp/phantom-4-pro [Accessed June 6, 2018][l] Linux Foundation and leading technology companies launch open

source Dronecode Project.https://www.linuxfoundation.jp/press-release/linux-foundation-and-leading-technology-companies-launch-open-source-dronecode-project/ [Accessed June 6, 2018]

[m] https://www.borg.media/seedplanning-drone-2016-05-18/[Accessed June 6, 2018]

[n] http://www.kantei.go.jp/jp/singi/kogatamujinki/pdf/shiryou6.pdf[Accessed June 6, 2018]

Name:Kenzo Nonami

Affiliation:Autonomous Control Systems Laboratory Ltd.

Address:WBG Marive West 32F, 2-6-1 Nakase, Mihama-ku, Chiba-city, Chiba261-7132, JapanBrief Biographical History:1994- Professor, Chiba University2008- Vice President, Chiba University2012- Chairman, Mini-Surveyor Consortium (Chairman, Japan DroneConsortium from 2017)2013- CEO, Autonomous Control Systems Laboratory Ltd.2017- Emeritus Professor, Chiba UniversityMain Works:• Modeling and control of drones, UAVs• Autonomous control• Robotics, mechatronics and controlMembership in Academic Societies:• The Japanese Society of Mechanical Engineers (JSME)• The Robotic Society of Japan (RSJ)• The Society of Instrument and Control Engineers (SICE)• The Institute of Electrical and Electronics Engineers (IEEE)• Science Council of Japan (SCJ)


Powered by TCPDF (www.tcpdf.org)

http://www.tcpdf.org

research and development of drone and roadmap to evolution

Documents