VISION BASED SINGLE-SHOT REAL-TIME EGO-

LANE ESTIMATION AND VEHICLE DETECTION FOR

FORWARD COLLISION WARNING SYSTEM

BY

MUHAMMAD ANWAR ALHAQ A MATIN

A thesis submitted in fulfilment of the requirement for the

degree of Master of Science (Mechatronics Engineering)

Kulliyyah of Engineering

International Islamic University Malaysia

OCTOBER 2019

ii

ABSTRACT

Vision-based Forward Collision Warning System (FCWS) is a promising assist feature

in a car to alleviate road accidents and make roads safer. In practice, it is exceptionally

hard to accurately and efficiently develop algorithm for FCWS application due to the

complexity of steps involved in FCWS. For FCWS application, multiple steps are

involved namely vehicle detection, target vehicle verification, time-to-collision (TTC).

These involve an elaborated pipeline for the FCWS application using classical computer

vision methods which limits the robustness of the overall system and limits the

scalability of the algorithm. Advancement in deep neural network (DNN) has shown

unprecedented performance for the task of vision-based object detection which opens

up the possibility to be explored as an effective perceptive tool for automotive

application. In this thesis a DNN based single-shot vehicle detection and ego-lane

estimation architecture is presented. This architecture allows simultaneous detection of

vehicles and estimation of ego-lanes in a single-shot. SSD-MobileNetv2 architecture

were used as a backbone network to achieve this. Traffic ego-lanes in this thesis were

defined in two ways; first as a second-degree polynomial and second as semantic

regression points. We collected and labelled 59,068 images of ego-lane datasets and

trained the feature extractor architecture MobileNetv2 to estimate where the ego-lanes

are. Once the feature extractor is trained for ego-lane estimation the meta-architecture

single-shot detector (SSD) was then trained to detect vehicles. This thesis had

demonstrated that this method achieves real-time performance with test results of 88%

total precision on CULane dataset and 91% on our own dataset for ego-lane estimation.

Moreover, we achieve 63.7% mAP for vehicle detection on our own dataset. The

proposed architecture shows that elaborate pipeline of multiple steps to develop

algorithm for FCWS application is eliminated. The proposed method achieves real-time

at 60fps performance on standard PC running on Nvidia GTX1080 proving its potential

to run on embedded device for Forward Collision Warning System.

iii

خلاصة البحث

تعتبر أنظمة تحذير الاصطدام الأمامي للمركبات بالمستشعرات المرئية من التقنيات الواعدة التي ستساعد في جعل الطرقات أكثر أمانا. من الناحية العملية، انه لمن الصعب للغاية تطوير الخوارزميات التي سيعتمد عليها هذه

ية. الخطوات هي ا معقدة للغاالأنظمة نظرا لأن الخطوات التي تنطوي عليهثانيا: التحقق من المركبة المستهدفة ،ثالثا: الوقت المركبة،أولا: كشف ذاتها،. ولأن كل خطوة من هذه الخطوات معقدة بحد الاصطدامالمتبقي قبل

على نطاق قابلة للاستخدام وخوارزميةيحد من تطوير نظام متكامل فهذه الحاسب المرئية بتقنيات الانظمة ذه كننا تصميم هبالرغم من أننا يم واسع

كبير بالمؤثرات يعيبها أنها تتأثر بشكل فإن هذه التقنيات التقليدية، في الآونة الاخيرة، التقنيات محدودة.يجعل استخدامات هذه مما الخارجية

يث ح العميقة الخوارزميات المعتمدة على الشبكات العصبيةحدث تطور في هذه المرئية. الأشياء استخدام هذه الشبكات في الكشف عن من الممكن

التطورات جعلت من الممكن استخدامها كبديل للمستشعرات التقليدية. في هذا البحث، تطرح موضوع الشبكات العصبية العميقة المعتمدة على رصد

. ego-lanes المركبات من لقطة واحدة ومعمارية تقديرد المركبات وقراءة مسارات القيادة في آن واحد. تسمح برص ريةالمعما هذه

استخدمت كشبكة رئيسية للوصول SSD +MobileNetv2 معماريةبطريقتين: الاولى ego-lanes . يمكن ان تعرّف مصطلحه النتائجلهذ

الثانية والثانية تعرف بأنها نقاط انحدار الدرجةتعرّف بأنها متعدد الحدود من من مجموعة ٥٩٬٠٦٨ي هذا البحث تم تجميع وتصنيف الدلالي. ف

وايضا تم تدريب معمارية مستخرج الميزات لتخمين ego-lane بيانات. بعد اتمام تدريب مستخرج الميزات يتم تدريب ego-lane مكان

(single-shot multibox detector (SSD رصد من اجلوتحقق الاداء في الوقت المركبات. هذا البحث قد أثبت أن هذه الطريقة تعمل

% من الدقة الكلية في مجموع ٨٨تبار بنسبة الحقيقي كما اظهرت نتائج الاخمجموع % في ٩١و CULane بياناتفقد توصلنا ل ،إضافة. ego lane estimation بياناتنا٦٣.٧% mAP لرصد المركبات في داخل البيانات الخاصة بنا. هذه

إطار في الثانية في جهاز ٦٠ وقت الحقيقيأداء في ال حققتالطريقة ايضا قابلية استخدام مما يثبت Nvidia GTX1080 حاسوب متوسط تعمل على

هذه الطريقة في الأجهزة المدمجة.

iv

APPROVAL PAGE

I certify that I have supervised and read this study and that in my opinion, it conforms

to acceptable standards of scholarly presentation and is fully adequate, in scope and

quality, as a thesis for the degree of Master of Science (Mechatronics Engineering)

…………………………………..

Hasan Firdaus bin Mohd Zaki Supervisor

…………………………………..

Zulkifli bin Zainal Abidin

Co-Supervisor

…………………………………..

Yasir Mohd. Mustafah

Co-Supervisor

I certify that I have read this study and that in my opinion it conforms to acceptable

standards of scholarly presentation and is fully adequate, in scope and quality, as a thesis

for the degree of Master of Science (Mechatronics Engineering)

…………………………………..

Malik Arman Bin Morshidi

Internal Examiner

…………………………………..

Siti Anom Ahmad

External Examiner

This thesis was submitted to the Department of Mechatronics Engineering and is

accepted as a fulfilment of the requirement for the degree of Master of Science

(Mechatronics Engineering)

…………………………………..

Syamsul Bahrin Abdul Hamid

Head, Department of

Mechatronics Engineering

v

This thesis was submitted to the Kulliyyah of Engineering and is accepted as a

fulfilment of the requirement for the degree of Master of Science (Mechatronics

Engineering)

…………………………………..

Ahmad Faris Ismail

Dean, Kulliyyah of Engineering

vi

DECLARATION

I hereby declare that this thesis is the result of my own investigations, except where

otherwise stated. I also declare that it has not been previously or concurrently submitted

as a whole for any other degrees at IIUM or other institutions.

Muhammad Anwar Alhaq A Matin

Signature ........................................................... Date .........................................

vii

INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA

DECLARATION OF COPYRIGHT AND AFFIRMATION OF

FAIR USE OF UNPUBLISHED RESEARCH

VISION BASED SINGLE-SHOT REAL-TIME VEHICLE

DETECTION AND EGO-LANE ESTIMATION FOR FORWARD

COLLISION WARNING SYSTEM

I declare that the copyright holders of this thesis are jointly owned by the student

and IIUM.

Copyright © 2019 Muhammad Anwar Alhaq A Matin and International Islamic University

Malaysia. All rights reserved.

No part of this unpublished research may be reproduced, stored in a retrieval system,

or transmitted, in any form or by any means, electronic, mechanical, photocopying,

recording or otherwise without prior written permission of the copyright holder

except as provided below

1. Any material contained in or derived from this unpublished research

may be used by others in their writing with due acknowledgement.

2. IIUM or its library will have the right to make and transmit copies (print

or electronic) for institutional and academic purposes.

3. The IIUM library will have the right to make, store in a retrieved system

and supply copies of this unpublished research if requested by other

universities and research libraries.

By signing this form, I acknowledged that I have read and understand the IIUM

Intellectual Property Right and Commercialization policy.

Affirmed by Muhammad Anwar Alhaq A Matin

……..…………………….. ………………………..

Signature Date

viii

ACKNOWLEDGEMENTS

Firstly, it is my utmost pleasure to dedicate this work to my dear parents and my family,

who granted me the gift of their unwavering belief in my ability to accomplish this goal:

thank you for your support and patience.

I wish to express my appreciation and thanks to those who provided their time,

effort and support for this project. Thank you to my co-supervisor Asst. Prof Dr. Zulkifli

Zainal Abidin and Assoc. Prof. Dr. Yasir Mohd. Mustafah for their support and

guidance. I would also like to extend my gratitude to my lab mate Mr. Syarifuddin

Ahmad Fakhri for his never-failing support in helping me in this project.

Finally, a special thanks to Asst. Prof Dr. Hasan Firdaus bin Mohd. Zaki as

supervisor for his continuous support, encouragement and leadership, and for that, I will

be forever grateful

This thesis project is supported by project collaboration of International Islamic

University Malaysia (IIUM) with Collaborative Research in Engineering, Science &

Technology Centre (CREST) and Delloyd R&D Sdn. Bhd. (Grant ID P11C2-17 &

SP17-029-0291)

ix

TABLE OF CONTENTS

Abstract ................................................................................................................ii

Abstract in Arabic ……………………………………………………………………………iii

Approval Page ......................................................................................................iv Declaration ...........................................................................................................vi

Copyrights ............................................................................................................vii Acknowledgements ..............................................................................................viii

Table of Contents .................................................................................................ix List of Tables........................................................................................................xi

List of Figures ......................................................................................................xii List of Abbreviations ............................................................................................xiv

CHAPTER ONE: INTRODUCTION ................................................................1

1.1 Overview .............................................................................................1 1.2 Statement of the Problem .....................................................................1

1.3 Research Objectives .............................................................................3 1.4 Research Methodology ........................................................................3

1.5 Research Scope ....................................................................................6 1.6 Report Organization .............................................................................6

CHAPTER TWO: LITERATURE REVIEW ...................................................8

2.1 Introduction .........................................................................................8 2.2 Object Detection using Traditional Approach .......................................11

2.3 Deep Learning Based Methods ............................................................12 2.4 Deep Learning Based Feature Extractor ...............................................16

2.5 Deep Learning Based Object Detection ................................................19 2.6 Comparison of classical method vs DNN Method ................................22

2.7 Collision Judgement.............................................................................24 2.7.1 Ego-lane identification ................................................................24

2.7.2 Collision risk identification .........................................................25 2.8 Chapter Summary ................................................................................25

CHAPTER THREE: SYSTEM DESIGN ..........................................................27

3.1 Introduction .........................................................................................27 3.2 Design Development Flowchart ...........................................................27

3.3 System Description ..............................................................................28 3.3.1 Deep Neural Net architecture ......................................................29

3.3.1.1 Ego-lane estimations .......................................................30 3.3.1.2 Vehicle detection ............................................................34

3.3.2 Dataset........................................................................................35 3.3.2.1 Ego-lane prediction dataset .............................................37

3.3.2.2 Vehicle detection dataset ................................................45 3.3.2.3 Overall dataset statistics ..................................................45

3.4 Performance Metric .............................................................................47 3.5 Chapter Summary ................................................................................48

x

CHAPTER FOUR: RESULTS AND DISCUSSION .........................................49

4.1 Introduction .........................................................................................49 4.2 DNN Based Ego-lane Estimation Model Results ..................................49

4.3 Vehicle Detection Training Results ......................................................52 4.4 Real-time Performance Result ..............................................................54

4.5 Implementation for FCWS Application Result .....................................55 4.6 Summary .............................................................................................56

CHAPTER FIVE: CONCLUSION AND RECOMMENDATIONS ................57

5.1 Conclusion ...........................................................................................57 5.2 Contributions .......................................................................................58

5.3 Limitation and Recommendation .........................................................59

REFERENCES ...................................................................................................61

xi

LIST OF TABLES

Table 2.1 Summary of Top-1 accuracy of some popular feature-

extractor CNN

17

Table 2.2 Summary of classical methods vs DNN methods

23

Table 3.1 Statistics of dataset for training ego-lane estimation

architecture

46

Table 3.2 Statistics of dataset for training vehicle detection

architecture

47

Table 4.1 Training loss for 2nd degree polynomial spline based ego-

lane models

50

Table 4.2 Test accuracy for 2nd degree polynomial spline based ego-

lane models

51

Table 4.3 Training loss for point regression based ego-lane models

51

Table 4.4 Test accuracy for point regression based ego-lane models

52

Table 4.5 Model detection accuracy comparison on CULane test

dataset

52

Table 4.6 Total loss of vehicle detection model

53

Table 4.7 Performance comparison of vehicle detection model

53

Table 4.8 Devices specifications to be used to run real-time test

54

Table 4.9 Real time performance on various platform 55

xii

LIST OF FIGURES

Figure 1.1 Flow chart of thesis project

5

Figure 2.1 Features in ADAS. “Increasing reliance on ADAS despite

limitations – Telematicswire ,” 2018

9

Figure 2.2 Illustration of general pipeline for Convolutional Neural

Network (Guo, Y. et al., 2016)

14

Figure 2.3 Shows how IoU are calculated

20

Figure 3.1 Flowchart of system development process

28

Figure 3.2 Overall single-shot vehicle detection and ego-lane

estimation pipeline

29

Figure 3.3 Ego-lane architecture with 2nd degree polynomial output

31

Figure 3.4 Given a resized to 224x224 image point regression model

should output estimation of x-points coordinate of the

fixed y-points.

32

Figure 3.5 Training point regression architecture will require dataset

labelled based on the resized image

33

Figure 3.6 Overall single-shot vehicle detection and ego-lane

estimation using MobileNetv2

34

Figure 3.7 CULane detection with mask labelling. (“CULane

dataset,” 2018)

36

Figure 3.8 Dashcam used to record and collect Malaysian dataset.

“Award of the Administrator KIPO – Good design,” 2016

36

Figure 3.9 Yellow points are the x-y coordinates manually labelled

for the curve lane. The red line is the line that are the

generated outcome of fitting the x-y coordinates to a 2nd

degree polynomial.

38

Figure 3.10 Shows how flipping the x-y axis allows us to get the

curve shape similar to that of an actual curved traffic lane

39

Figure 3.11 (a) Flipped labelled lane (b)The un-flipped labelled

lane

40

xiii

Figure 3.12 Output after labelling

41

Figure 3.13 Original CULane .txt file that contains the lane type

labels

41

Figure 3.14 Original CULane .txt file that contains the lane

coordinates labels

41

Figure 3.15 Samples of data with ego-lane a) 0 1 1 1 lane type b) 0 1

1 0 lane type c) 1 1 1 1 lane type and d) 1 1 1 0 lane type

42

Figure 3.16 Labelling tool to label our own ego-lanes. Yellow

drawings are the manually labelled points on the road.

Red lines are the outcome after fitting it to a polynomial

equation. Green lines are the saved outcome from the

saved labels

43

Figure 3.17 Visualization of ego-lane labels

43

Figure 3.18 Six float numbers corresponding to coefficient values of

two 2nd degree polynomials as labels for two ego-lane

44

Figure 3.19 Variations from the augmentation

44

Figure 3.20 Labelling vehicles using open-source labelling tool

labelimg

45

Figure 3.21 Samples from the augmented dataset for vehicle detection

labels

45

Figure 3.22 Evaluation output showing indicating the estimation lines

that falls on the ground truth lane and those that are not

47

Figure 4.1 Sample output from integrated single-shot vehicle

detection and ego-lane estimation model

56

xiv

LIST OF ABBREVIATIONS

ADAS – Advanced Driver Assistance System

AP – Average Precision

CNN – Convolutional Neural Network

CPU – Central processing unit

DNN – Deep Neural Network

FC – Fully connected layer

FCWS – Forward Collision Warning System

FPS – Frame per second

GPU – Graphical Processing Unit

IoU – Intersection of Union

LDWS – Lane Departure Warning System

MAE – Mean Absolute Error

mAP – Mean Average Precision

MSE – Mean Squared Error

SSD – Single Shot Multibox Detector

TTC – Time to collision

1

CHAPTER ONE

INTRODUCTION

1.1 OVERVIEW

Lack of attention by drivers is identified as the cause for 80% of driver related accidents

(Cui, Liu, Li, & Jia, 2010). With recent advances in technology many applications in

Advanced Driver Assistance Systems (ADAS) are implemented in cars to assist drivers

to ensure safety. With recent advances in Deep Neural Networks (DNN) achieving

highest accuracy in object detections this has become key area of computer vision

especially for ADAS to enable safer roads. Breakthrough in DNN brought opportunities

to bring detection models and to be implemented in vision based ADAS application for

secondary safety or crash protection technologies to deliver large life savings. Among

the potential benefits of high accuracy object detections in ADAS is collision avoidance

systems such as forward collision warning system, reverse collision warning system,

adaptive cruise control and emergency brake assist. Although many of these systems

have been developed using high end sensors, a breakthrough on vision-based detection

using DNN with high accuracy detection has opened opportunities for cheaper

replacement possibilities without compromising performance; moreover,

breakthroughs in DNN offers the possibility to realize the making of autonomous

vehicle.

1.2 STATEMENT OF THE PROBLEM

Developing an elaborate vision-based forward collision warning system (FCWS)

involves the fusion of multiple methods. These methods include: a method for vehicle

detection, a method of target vehicle tracking, and a method of Time to Contact

2

Calculation (TTC). All methods mentioned will be recurring every cycle to first detect

and then finally give decision if a target vehicle is indeed headed to collide with the host

vehicle. Each method has its own specific needs, some of which are required to be fast

and robust for the other methods to be effective.

Consider the problem of vehicle detection method. While many detection methods

prove to show high accuracy detection but noise variance for scenes on road is so high,

we need to make sure our detection model is robust enough to overcome noises and

understand the model’s limitations. Aside from being robust with variance of real road

scenarios, consideration must be made to ensure only the ones that have a suitable

accuracy/speed trade-off is chosen for real-time FCWS application. For FCWS to be

real-time specific detection speed requirement needs to be met before passing the

detections to tracking and TTC methods. This can be evaluated by benchmarking on

current state of the art DNN based model to see how well our current model performs.

Target vehicle tracking and TTC are dependent on each other. Consider a frame

with multiple vehicle detections, while real road scenario shows diverse condition, a

host car approaching a target car on a straight path should be treated differently from a

host car approaching a target car on a curved path. Moreover, cars that are on different

position of different lane should be treated differently. Problem with vision-based

multiple object detection model is that it only gives you the bounding box coordinates

of the object it is detecting. This does not give you the information if the object that is

detected in the previous frame is the same object detected in the next frame. For vision-

based FCWS to work, the algorithm needs to understand what detected cars are more

likely to be in a course to collision to the user’s car by keeping track of its velocity and

its position relative to the host car.

3

This require methods of tracking detected cars and identifying current host car’s

positioning and the target car’s positioning on the road. Defining the host-car’s ego-lane

allows the algorithm to define its driving course on the road. Tracking detected cars

allow the algorithm to predict how likely are they to collide with the host-car by keeping

track of their behavior. Finally, all methods integrated must ensure real-time

performance to achieve real time single shot FCWS.

1.3 RESEARCH OBJECTIVES

The research aims to achieve the following objectives:

1. To investigate and train deep learning based model to perform task of ego-lane

estimation.

2. To develop and integrate vehicle detection and ego-lane estimation so it

achieves simultaneous single-shot output on a PC system.

3. To evaluate the integrated architecture of the single-shot vehicle detection and

ego-lane estimation architecture for accuracy and real-time performance.

1.4 RESEARCH METHODOLOGY

The following methodology will be adopted to achieve the objectives of the project.

1. Extensive literature review on Forward Collision Warning workflow and its

components. Research will be done by collecting information from various

sources such as books, online journals and conference papers.

2. Data collection and train different DNN detection model for vehicle detection.

This is performed to test the detection accuracy and the model’s effectiveness

for FCW application on real road-scene dataset.

4

3. Design and development of experimental set up to calculate the system’s

performance at per-frame. This will be done to measure and record the cost of

performance with variant workflow in the algorithm.

4. Evaluating the performance of the detection algorithm in a simulated

environment. This will be performed on a PC setup on an offline video testing

to test effectiveness of the algorithm.

5. Develop a working FCWS and evaluate the effectiveness of the algorithm on a

real road scenario in a car. Test if the algorithm performs as such compared

with the simulated version.

6. Model testing and validation. Experiments will be conducted to validate the

proposed model.

Figure 1.1 shows the flowchart of the research methodology to be adopted in this

research:

5

Figure 1.1 Flow chart of the thesis project

6

1.5 RESEARCH SCOPE

The project aims to focus on developing a vision-based FCWS for ADAS application.

The scope of the project aims at developing a single-shot DNN-based architecture that

is capable to output vehicle detections as well as to output ego-lane estimation for

FCWS application on a single architecture that performs real-time of not less than 24

frames per second. The FCWS is limited to only vision-based input system. Algorithm

testing will be done offline as well as online to test its real-time performance.

1.6 REPORT ORGANIZATION

This research proposal is divided into several chapters.

Chapter 1: Introduction

This chapter discusses the overview of the project which includes research objectives,

problem statements and methodology.

Chapter 2: Literature Review

This chapter will review the literature of the general workflow of FCWS. The review

will cover the DNN based detection suitable for ADAS application as well as other

methods that makes a FCWS. This review will help us extract the important concepts

and help us get the general concept to finally develop an architecture of our own.

Chapter 3: System design

This chapter discusses the design to develop the DNN model for single-shot vehicle

detection and ego-lane estimation. Moreover, this chapter includes discussion on how

we evaluate methods and benchmark the algorithm.

Chapter 4: Results and discussion

7

Results of the individual sub-systems are compared and discussed in this chapter. This

includes all training performance and detections results. Moreover, performance of

algorithm speed will also be discussed here in the chapter.

Chapter 5: Conclusion

This chapter summarizes what was achieved in this project. Moreover, this chapter

discusses the limitations and recommendations for the thesis project.

8

CHAPTER TWO

LITERATURE REVIEW

2.1 INTRODUCTION

Driver’s safety is always a top priority issue for car manufacturers to ensure safe roads

and reduce accidents. ADAS system are rapidly becoming commonplace in a new car

market and this is due to increasing number pressure posed by car safety rating body

such as New Car Assessment Programme (NCAP) and National Highway Traffic Safety

Administration (NHTSA). These bodies set safety standard for new cars to raise safety

standards across the automotive industry. These safety standards pushes car

manufacturers to pursue researches to incorporate technologies that will make their cars

safer and thus makes roads safer for everyone. The ubiquity of camera technologies in

everyday life are pushing manufacturers to push research towards vision-based ADAS

to allow smarter and cheaper alternative to high end sensors. Moreover, a breakthrough

on vision-based detection using DNN with high accuracy detection has opened

opportunities for cheaper replacement possibilities without compromising performance;

moreover, breakthroughs in DNN offers the possibility to realize the making of

autonomous vehicle.

Figure 2.1 shows the different types of ADAS system in a car. Individual ADAS

system are made specifically to run specific task and can run stand alone. Wide variety

of equipment are installed on car to assist in passive ADAS and active ADAS. All which

will assist to protect us from the human factor and human error that cause most traffic

accidents (Ziebinski, Cupek, Grzechca, & Chruszczyk, 2017). Therefore, these systems

9

are equipped to assist drivers in driving and are designed to increase car safety and more

generally road safety.

For object detection, most existing systems still use traditional computer vision

approaches such as Cascaded Haar-like feature detection (Viola & Jones, 2001), HOG-

SVM (Dalal & Triggs, 2005) and Hough transform based line detection (Ballard, 1981).

However, such models are susceptible to environmental noises such as adverse lighting

condition, viewpoint changes, etc. This is due to the fact that these traditional methods

depend on lower level features that is based mostly but not strictly on edges, corners,

symmetries and histograms gradients.

DNN based methods have achieved unprecedented performance in solving

several computer vision problems involving image classification and object detection

which could become a key enabler for highly accurate and robust ADAS application.

The key benefits of DNN as opposed to traditional methods for classification and

detection is because they can extract appropriate features for the tasks. Following the

Figure 2.1 Features in ADAS. “Increasing reliance on ADAS despite limitations –

Telematicswire ,” 2018

10

breakthrough of AlexNet (Krizhevsky, Sutskever, & Hinton, 2017) architecture that led

to the popularity that shows DNN performs better than their traditional approach;

increasing number research focused on using DNN for object detection tasks. This led

to numerous successful DNN object detection architecture with high performances.

DNN based object detection architectures such as R-CNN (Girshick, Donahue,

Darrell, & Malik, 2014), SSD (Liu, et al., 2016), and YOLO (Redmon, Divvala,

Girshick, & Farhadi, 2016) are extension of classification based on DNN. The end goal

is concerned with giving object localization that involves drawing bounding box around

one or multiple objects of multiple classes. The extension architecture for the object

detection are called meta-architecture. Meta-architecture is usually built on top of the

pre-trained classification model, which is sometimes called feature extractor

architecture. This brings an end-to-end solution for object detection.

However, developing forward collision warning system (FCWS) requires

vehicle detection model and ego-lane estimation model which are two separate tasks

which is not practical for real applications in embedded ADAS. The core bottleneck in

such methods is that DNN was trained for each task and combined in a late fusion

manner. Moreover, although DNN constitutes highest performing models for vehicle

detection and ego-lane estimation, it is also known for data hungriness and

computational complexity. Therefore, we propose a unified DNN based single-shot

vehicle detection and ego-lane estimation architecture which allows both tasks to be

performed simultaneously in a single shot.

Breakthrough in DNN brought opportunities to bring detection models to be

implemented in vision-based ADAS application for secondary safety or crash protection

technologies to deliver large life savings. Among the potential benefits of high accuracy

object detections in ADAS is collision avoidance systems such as FCWS, reverse