automatic braking and lane-keeping design for …...automatic lateral control assistance system....

23rd ITS World Congress, Melbourne, Australia, 10–14 October 2016

1

Paper number AP-TP0380

Automatic Braking and Lane-Keeping Design for Driver

Assistance System

Ping-Ming Hsu, Po-Kai Tseng, Jin-Yan Hsu, Shih-Chieh Huang, and Tzu-Sung Wu* Automotive Research and Testing Center, Taiwan

Abstract A driver assistance system has the function of combining with autonomous emergency braking (AEB) system and lane-keeping is presented in this paper. The lane keeping system is designed to assist driver not to cross the lane when the driver is distracted. This is a class of automatic lateral control assistance system. Another kind of automatic longitudinal control assistance system is designed to prevent the collision with the obstacle in front of vehicle, named AEB. However, to integrate both lateral and longitudinal automatic control system into one driving assistance system involves with adequate control strategy so that the action of these two automatic control systems do not contradict each other. This paper presents the system architecture for lane keeping and AEB respectively. Moreover, the driver assistance system is integrated these two functions is also discussed. The experimental results show the performance of longitudinal and lateral dynamic control and the domination between two active control systems is logical.

Keywords: Lane Keeping Assistant, Autonomous Emergency Braking (AEB) System, Driver Assistance System, Digital Signal Processor (DSP).

Introduction The National Highway Traffic Safety Administration (NHTSA) has defined the formal classification of automatic driving system in five different levels [6]. Level 0 means the vehicle is without any driving assistance system and driver controls the vehicle completely at all time. Level 1 is function specific automation. The vehicle is equipped with some automatic control systems that provide partial active safety control individually. In the definition of level 2, the vehicle has combined at least two automation functions and these functions are needed to be operated in unison. Level 3 and 4 are defined in highly and fully automation. The difference between level 3 and 4 is driver could cede the control of all safety-critical functions partially or not in trip. Nowadays, most of luxurious commercial vehicles have at least one driver assistance system and these systems control vehicular attitude individually. In other words, most of the commercial vehicles are satisfy the NHTSA

Paper title < Automatic Braking and Lane-Keeping Design for Driver Assistance System >

2

automation level 1. In future, more and more car manufactures would like to develop high automation level system and have announced that the driver assistance systems are high probability to become standard equipment in new styling production. In this paper, the AEB and lane keeping functions are integrated into one automatic driver assistance system. The lane keeping system, a camera is fixed on the wind shield to recognize the lane marking in front of vehicle [1,4]. By using the relationship between the vehicle moving trajectory and the lane right boundary, system can control the vehicle to maneuver in the desired lane. For AEB [3,5,7] is designed to prevent the collision in low and middle speed operation condition. A millimetre wave radar sensing message is compared to the visual classification results to affirm the obstacle in front of vehicle. If the time to collision is not long enough for driver to react, the system will brake vehicle automatically. Finally, a control decision method is designed to integrate these two functions adequately. This paper is organized as follows. In section 2, the automatic control technologies in vehicle longitudinal and lateral dynamics are described and the integrated control strategy is also depicted. Section 3 shows the experimental results of AEB, lane keeping, and integrated function. And the last is conclusions.

Longitudinal and Lateral Driver Assistance System Lane Keeping Assistance System

The function of lane keeping is to assist driver not to cross lane boundary. The hardware of our lane keep assistance system consists of camera and steering wheel torque sensor. The camera module

is fixed on wind shield to detect the lane marking 5 and 10 meters far from the head of vehicle simultaneously. At the same time, a point 10 meters ahead of vehicle is predicted and the time to touch the right lane boundary is estimated as the time-to-lane crossing (TTLC), named TTLC1. Similarly, TTLC of the point 5 meters ahead of vehicle is also estimated as TTLC2. According to TTLC1 and TTLC2, an estimated trajectory is formulated to fit the curvature of right lane boundary. If the lane marking is painted in solid line on both sides, it is easy to calculate the lane center; if the lane marking is marked on right side only, our system can estimate the curvature according TTLC1 and TTLC2 and the controlled trajectory is thus determined. The control concept and lane-keeping control structure are shown in Figs. 1 and 2, respectively. Lane keeping assistance system is designed to be operated automatically since the speed is greater than 10km/hr.


3

Figure 1 – Lane keeping control method

Figure 2 – Lane keeping control structure

Autonomous Emergency Braking System In AEB system, the signals of millimetre wave radar, camera, inertial measurement unit are integrated in an embedded processor. Fig. 3 is the hardware structure of AEB system. The millimetre wave radar can detect the obstacle in front of vehicle up to 80 meters. By using Kalman filter method [2], the noise and disturbances of measurement signal of millimetre wave radar can be reduced. However, any physical substance can be found via millimetre wave radar, including road surface, railing, etc. It is possible to cause damage to driver if the control decision system adopts the incorrect message. In order to recognize the real obstacles in front of vehicle, the visual classification technology is accessed to correct the measurement signals. If the obstacle is distinguished into car or people, our control decision unit will adopt the measurement message; otherwise the message will be ignored. Since the obstacles are affirmed, the anti-collision algorithm design is based on time-to-collision (TTC). The estimation of TTC is relative to the related distance and speed of obstacles in front of vehicle. If vehicle speed is under 80 km/hr, AEB system is defined to be operated automatically. Generally, if TTC is not long enough for driver to brake the vehicle, our AEB system will brake vehicle automatically.

Figure 3 – AEB system structure


4

Complex Function Driver Assistance System According to the definition of NHTSA, a level 2 automation vehicle has combined at least two automation functions and these functions are needed to be operated in unison. In our proposed system, if vehicle speed is less than 10km/hr, the AEB function is turned on automatically only. If vehicle speed is greater than 80 km/hr, the automatic lane-keeping function is awaked only. Otherwise, AEB and lane-keeping functions are operated in the same time. In general case, if AEB function is triggered, lane-keeping function will be disabled, and vise versa. But if these two functions are enabled at the same moment, vehicle brake has high priority over the integration system. Based on the above analysis,

the operation speed range for AEB, lane-keeping, and integrated of AEB and lane-keeping assistance system and the structure of integrated AEB and lane-keeping are shown in Table 1 and Fig. 4.

Table 1 – The operation speed range for AEB, lane-keeping, and integrated of AEB and lane-keeping assistance system

AEB

Lane-keeping assistance system

Integrated of AEB and lane-keeping assistance system

0-10 km/hr 10-80 km/hr

Greater than 80

Figure 4 – Integrated AEB and lane-keeping structure

Empirical Results In this section, the performance of proposed control strategy is carried out through automatic driver assistance systems. Two experiments were conducted: one for the lane keeping system, and the other for the AEB. The vehicle as shown in Fig. 5(a), equipped with these two active safety mechanisms, in Fig. 5 was utilized with the objective of verifying the proposed control strategy. There


5

are two sensors utilized with this test car: one is a millimeter wave radar with 77GHz as shown in Fig. 5(b) and the other is a camera as shown in Fig. 5(c).

(a)

(b)

(c)

Figure 5 – Experimental vehicle, (a) the vehicle, (b) radar, and (c) camera. Performance analysis of Lane Keeping Assistance System The lane keeping assistance system is composed of camera, electrical power steering (EPS), and steering wheel torque sensor. The vehicle speed is also captured for steering control. The camera is used to detect lane marking and sends the lane marking to digital signal processor for computing lateral displacement. The DSP sends the displacement to ECU. When ECU received the displacement, ECU computes the steering angle. Figs 6-8 show the experimental results of lane keeping function. In the lane keeping function experiments, we turn on lane keeping function on a straight lane with 3.75m width lane marking when the vehicle speed reaches 60kph and record data, including command steering angle, feedback steering angle, EPS status, vehicle trajectory, steering torque, and steering angular rate. In Fig. 6, the upper subfigure shows the results of command steering angle(red line) and feedback steering angle (blue line) and the bottom subfigure shows the EPS status. The bottom subfigure shows that the EPS is keeping working in this experiment. From the upper subfigure of Fig. 6, we can see that the command steering angle is controlled within [2, -3] and the EPS steering response time is less than 0.3 s. The maximum steering command jump is 3 degree and the steering command change rate is around 0.5s. In Fig. 7, the upper subfigure shows the results of 5m and 10m trajectories in front of vehicle and the bottom subfigure shows the distance error between 5m and 10m, where the thick blue line

represents the lane markings. From Fig. 7, we can see that the vehicle is controlled within ±1m when lane keeping function works and the distance error between 5m and 10m in front of vehicle is within

±0.2m. That means lane keeping function can control vehicle smoothly In Fig. 8, the upper subfigure shows the torque from EPS and the bottom subfigure shows the

angular rate of EPS. The results show the EPS torque is controlled in ±0.2Nm when lane keeping function works. The average of angular rate is controlled in 3(deg/s). This results show the proposed lane keeping function would not result in high torque and high angular rate.


6

Figure 6 – Steering angle command and feedback

Figure 7 – Vehicle trajectory


7

Figure 8 – Steering wheel torque and angular rate Performance analysis of AEB system The considered system was verified in the scenario, named as car-to-car rear moving (CCRm) provided by Euro NCAP. Referring to [8] defines the specification of CCRm and summarized in Table 2 [8]. As listed in this table, testing speed must be within [30, 80] (km/hr). The host car with the specified speed was moving toward a front target, namely Euro NCAP vehicle target (EVT) (see Fig. 9 [8]), in the current scenario at the initial position that is at least 200 m away from the target. This EVT was simultaneously dragged with the speed 20 km/hr in the same moving direction of the host vehicle. Notice that the lane keeping system was activated before the AEB system.

Figure 9 – Euro NCAP vehicle target


8

Table 2- Specification of CCRm scenario

Experimental results are as shown in Fig. 10, in which Fig. 10(a) illustrates the time to collision (TTC) at the moment of activating sound warning with respect to each relative speed, and Fig. 10(b) displays the TTC at the moment of activating braking function corresponding to different relative speed. Notice that the relative speed here is defined as the host vehicle speed with respect to that of the target. Firstly, experimental results in Fig. 10(a) reveals that the faster the relative speed suffered, the more warning TTC the system desired. It is obvious that the warning TTC is linearly positive to the relative speed in Fig. 10(a). However, there is a warning TTC smaller than others with respect to the range [40, 50] (km/hr). This was caused by that the front target was sensed later in that case than others. Secondly, verification results in Fig. 10(b) shows that the braking TTC is linearly positive to the relative speed. The more relative speed one suffered, the more TTC the system desired to activate braking function. Numerical results are summarized in Table 3. Successful rate of collision avoidance is 29/30=96.67% based on data in this table. The minimal distance in cases that corresponding to ideal relative speed 50/60 (km/hr) are obviously longer than others because the minimal acceleration in these cases with ideal relative speed 50/60 (km/hr) were quite smaller than others, under a concern that the more relative speed brings the more braking TTC. Moreover, the host car bumped into the moving target in second case due to that the EVT was sensed not in time. The braking function could not be activated immediately.

Table 3- Numerical results of AEB system in CCRm scenario

Ideal relative speed (km/hr)

Real relative speed (km/hr)

Minimal relative distance (m)

Minimal acceleration (m/s^2)

10 4.7 0.62 -5.2

5.1 0.88 -5.1

3.6 0 N/A

6.8 1.64 -5.1

6 1.09 -4.9

20 18 1.58 -5.1

17.8 1.37 -5

15.7 1.1 -5.1

14.5 0.76 -5.6

15.3 1.34 -5.2


9

30 27.1 2.57 -5.6

26.9 2.69 -5.6

25.1 2.74 -5.6

24.6 2.13 -5.2

27.2 3.53 -5.7

40 35.1 1.8 -5.8

36.1 2.14 -6

35 2.55 -5.8

36.7 1.56 -5.7

35.5 1.82 -5.7

50 45.1 4.99 -5.9

46.9 4.37 -6.2

44.5 4.54 -6.1

44.4 3.9 -6.1

46.4 4.99 -6.1

60 52.1 6.7 -6.5

55.9 9.46 -6.5

59 10.57 -7.3

56.3 8.84 -6.4

56.1 9.3 -7.2

0 10 20 30 40 50 600

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Relative speed (km/hr)

TTC

(s)

TFCW V.S. Relative Speed

(a)


10

0 10 20 30 40 50 600

0.5

1

1.5

2

2.5

Relative speed (km/hr)

TTC

(s)

Tbraking V.S. Relative Speed

(b)

Figure 10 – Experimental results of AEB in CCRm scenario, (a) warning TTC corresponding to different relative speed, and (b) braking TTC with respect to different relative speed

Failure Mode of Integration of Lane Keeping Assistance and AEB System The lane keeping assistance and AEB have been installed on the vehicle. In this section, we validate the performance of the integration of lane keeping assistance and AEB. There are three experiments in this section to validate the ability of keeping assistance disable. We define some criteria to test the ability of keeping assistance disable. The keeping assistance disable definitions include longitude acceleration disable, torque disable, and longitude deceleration disable. The validation results are shown in Figs. 11-13. Particularly, the validation of integration of lane keeping assistance and AEB is shown in the experiment of longitude deceleration disable. In Fig. 11, we turn on the lane keeping function and let driver to turn steering wheel to intervene lane keeping function. The red line and blue line represents EPS state and torque, respectively. From

Fig. 11, we can see that the EPS will be disable when the steering wheel torque is greater than ±2Nm. This function is used to avoid suffer from emergency status. Driver can turn steering wheel to shut down lane keeping assistance. In Fig. 12, we turn on the lane keeping function and let driver to increase vehicle velocity. The red line and blue line represents EPS state and acceleration, respectively. From Fig. 12, we can see that the EPS will be disabled when the acceleration is greater than 1 m/s2. This function is used to avoid driver increasing velocity urgently when lane keeping function works. In Fig. 13, we turn on the lane keeping function and AEB at the same time on the experimental vehicle. The red line, green line and blue line represent the EPS state, AEB enable, and deceleration, respectively. We set a static vehicle on a straight lane with 3.75m width at 200m in the front of experimental vehicle and then drive the experimental vehicle to approach static vehicle. The speed of experimental vehicle is kept 30kph. When experimental vehicle starts and lane marking is detected,


11

the lane keeping assistance is engaged. When experimental vehicle approaches the static vehicle, the AEB function calculates the TTC to determine whether engage braking. When AEB is enable to trigger brake and the deceleration is greater than -3 m/s2, the lane keeping assistance is disable. This disable function is used to integrate lane keeping assistance and AEB such that lane keeping assistance can be disabled when brake is engaged urgently.

Figure 11 – Experimental results of lane keeping assistance disable via torque

Figure 12 – Experimental results of lane keeping assistance disable via acceleration


12

Figure 13 – Experimental results of lane keeping assistance and AEB disable via deceleration

Conclusions In this paper, a complex function driver assistance system consists of AEB and lane-keeping is proposed. This driver assistance combines two automation functions and makes the functions operation in unison. The AEB system is operated under the speed of 80 km/hr and lane-keeping assistance system is worked upper 10 km/hr. Our proposed system integrates both functions concertedly since the vehicle speed is in the range of 10 to 80 km/hr. The experimental results show the performance of AEB and lane keeping systems, respectively. The AEB system can detect the obstacles within 80 meters in front of vehicle and make the vehicle to avoid collision with obstacles. The lane keeping assistance system can control the vehicle to manoeuvre in the lane and the vehicle speed is up to 100 km/hr. Finally, the performance of integrated system is evident our design control decision is able to unite two automatic system very well.

References

1. Benine-Neto, A., Scalzi, S., and Mannar, S. (2011). Vehicle lane keeping control based on piecewise affine regions. In Proceedings on 14th Intelligent Transportation System, pp. 907-912.

2. Chen, B., Wu, Y., and Hsieh, F. (2010). Estimation of engine rotational dynamics using Kalman filter based on a kinematic model. IEEE Transactions on Vehicular Technology, vol. 59, no. 8, pp. 3728-3735.

3. Coelingh, E., Eidehall, A., and Bengtsson, M. (2013). Collision warning with full auto brake and pedestrian detection - a practical example of AEB. In Proceedings 13th Intelligent Transportation System, pp. 155-160.


13

4. Hsiao, T. and Tomizuka, M. (2005). Sensor fault detection in vehicle lateral control system via switching Kalman filtering. In Proceedings of the 2005 American Control Conference, pp. 5009-5014.

5. Sugimoto, Y. and Sauer, C. (2005). Effectiveness estimation method for advanced driver assistance system and its application to collision mitigation brake system. In Proceedings 19th ESV Conference; Washington D.C.

6. Wanger, J., Baker, T., Goodin, G., and Maddox, J. (2014). Automatic vehicles: policy implications scoping study, Report 600451-00029-1, Texas A&M Transportation Institute.

7. Westenberger, A., Muntzinger, M., Gabb, M., Fritzsche, M., and Dietmayer, K. (2013). Time-to-collision estimation in automotive multi-sensor fusion with delayed measurements. DOI: 10.1007/978-3-319-00476-1_2.

8. Euro NCAP. (2013). Test protocol—AEB systems. pp. 1-31.

automatic braking and lane-keeping design for …...automatic lateral control assistance system....

Documents