a fog computing infrastructure for autonomous driving in
TRANSCRIPT
![Page 1: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/1.jpg)
A Fog Computing Infrastructure for
Autonomous Driving in Urban Environments
Modena Automotive Smart Area (MASA)
Marko Bertogna
University of Modena, Italy
http://hipert.unimore.it/
![Page 2: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/2.jpg)
Research on High-Performance Real-Time Systems
~30 people
– 4 faculties
– 10 post docs
– 7 PhD students
– 3 tech/admin
– Multiple students…
Ongoing EU projects:
Past EU projects:
Industrial collaborations:
2
HiPeRT Lab
>3MEuro funding
http://hipert.unimore.it/
Just started:
PRYSTINE,
SECREDAS
Copyright of the University of Modena - HiPeRT Lab
![Page 3: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/3.jpg)
The Motor Valley
3Copyright of the University of Modena - HiPeRT Lab
![Page 4: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/4.jpg)
Copyright of the University of Modena - HiPeRT Lab 4
![Page 5: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/5.jpg)
Copyright of the University of Modena - HiPeRT Lab 5
![Page 6: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/6.jpg)
What is MASA?MASA is a living lab created to develop, test and
validate automated and connected vehicles.
MASA MAIN ASSETS:
Simulator Proving ground Public road
Advanced
training
Research & product
development
Support for
experimental test
implementation
Public-private partnership that offers OEMs,
Tier 1 and Tier 2 a complete environment to
develop and validate solutions for Cooperative
Connected Automated Mobility
![Page 7: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/7.jpg)
Modena racetrack
Copyright of the University of Modena - HiPeRT Lab 7
![Page 8: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/8.jpg)
Copyright of the University of Modena - HiPeRT Lab 8
Urban Area
![Page 9: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/9.jpg)
▶ 4sq km urban area with a wide variety of real scenarios
▶ Roundabouts, crossroads, R/L turns, parking areas
▶ Dedicated road side equipment installed (Cameras, fog
nodes, Fiber, parking sensor…)
▶ Test new applications in real-life situations
▶ smart traffic lights, parking and environment sensors,
LTE dedicated antennas, cameras integrated on the
edge computing platform
Which dedicated services are available?MASA Public Road activities
Urban area
![Page 10: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/10.jpg)
EU project CLASS:Edge Cloud Computation: A Highly Distributed Software Architecture for Big Data AnalyticS
Funded by H2020-RIA ICT-16-2017 GA n°780622
Duration: 36 month (2018-2020)
Budget: 3.900.803
European Funding: 100%
Fog computing infrastructure for Urban Autonomous driving: a public-private partnership within Modena Automotive Smart Area (MASA)
• Accurate awareness of road users and obstacles in real-time
• Distributed traffic monitoring and enforcement in metropolitan areas
• Enabling technology for advanced AD applications in urban settings
V2I obstacle detection, Coordinated intersection crossing, Dynamic traffic signalling, Green routes forpublic vehicles, Smart parking: free lot detection and valet parking
![Page 11: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/11.jpg)
Real-time fog computing system
• Hundreds of smart cameras installed
• Cameras are locally connected to a
high-performance embedded board
• Edge nodes elaborate video streams detecting road users in real-time
• Elaborated information is sent in V2I
to vehicles for enhanced perception to L3/L4 autonomous driving
• Edge nodes are connected to local
servers receiving data at block level (a.k.a. Fog nodes)
• Fog nodes are fiber connected to
main control center
Control Room
Copyright of the University of Modena - HiPeRT Lab 11
![Page 12: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/12.jpg)
Car as a sensor
Copyright of the University of Modena - HiPeRT Lab 12
![Page 13: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/13.jpg)
Precise Localization and Mapping
Copyright of the University of Modena - HiPeRT Lab 13
![Page 14: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/14.jpg)
Real-time Urban Awareness
In-vehicle sensors
Infrastructure sensors Real-time detection Low-latency V2X communication
Autonomous vehiclesTraffic enforcementPublic authoritiesData analytics
14Copyright of the University of Modena - HiPeRT Lab
![Page 15: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/15.jpg)
Predictable communication infrastructure- 4G/LTE 5G
Low-latency V2X Infrastructure
Vehicle-to-vehicle
V2V
Vehicle-to-infrastructure
V2I
eNB
S-PGW PCRF
Athonet 4G+ Mobile Core
MME Local HSS
Local apps
eNB
IP
N e t w o r kApplication
layer
Copyright of the University of Modena - HiPeRT Lab 15
![Page 16: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/16.jpg)
Multiple infrastructure cameras detect road users and obstacles
– vehicles, buses, trucks, pedestrians, bicycles, etc.
A consistent representation of RU’s is sent to L3/L4 vehicles (V2I) in real-time
Edge-side: Real-time road user detection
To cloud infrastructure
Camera-to-car < 100ms !
• V2I sensing system for harsh
urban environment• Redundant and robust perception
• Safer obstacle detectionCopyright of the University of Modena - HiPeRT Lab 16
Distributed infrastructure for
real-time urban awareness
![Page 17: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/17.jpg)
Distributed intelligence system for real-time urban awareness
Smart city map updated every second
Tight reaction time to urban hazards
Traffic/parking monitoring & enforcement
Emergency vehicle routing
Cloud side: Advanced control center
Copyright of the University of Modena - HiPeRT Lab 17
![Page 18: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/18.jpg)
Highly valuable dataset for smart urban mobility enforcement and understanding
Highly detailed and accurate mobility data
Traffic/mobility analysis and prediction
Critical scenario identification
Road user behaviour understanding and prediction
– Improved algorithms for autonomous driving
Data anonymization for GDPR compliance
– Implemented at source/edge level
Big data analytics stack developed with IBM, UNIMORE, ATOS, Barcelona
Supercomputing Center
Copyright of the University of Modena - HiPeRT Lab 18
Use cases – Data analytics
![Page 19: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/19.jpg)
MASA Real-Time Smart Cameras
![Page 20: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/20.jpg)
![Page 21: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/21.jpg)
MASA platform
Copyright of the University of Modena - HiPeRT Lab 21
![Page 22: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/22.jpg)
CLASS Software infrastructure
Data Analytics Platform
Programming Models
OpenWhisk
Invoker
COMPSs + DataClay
Connectors
RotterdamCaaS API
Kubernetes
Docker
Linux
Edge
Co
mp
ute
Co
nti
nu
um
Copyright of the University of Modena - HiPeRT Lab 22
![Page 23: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/23.jpg)
Traditional Smart City flow: high latency
IP cameras: >200ms
YOLO v3 with tensor cores on
TitanV: 14ms
Private 4G network:
16ms latency
Metadata
Copyright of the University of Modena - HiPeRT Lab 23
![Page 24: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/24.jpg)
CLASS Smart City flow: under 100 ms
Smart cameras
YOLO v2 with
TensorRT on TX2: 56ms
Private 4G
network:16ms latency
<10ms
Train a
PredictiveModel of the
city using FogNodes and City
Cloud
Metadata
Copyright of the University of Modena - HiPeRT Lab 24
![Page 25: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/25.jpg)
Example from yesterday’s feed
Copyright of the University of Modena - HiPeRT Lab 25
![Page 26: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/26.jpg)
Small datagram using UDP
Sender ID: 1 Byte
Timestamp: 1 Byte (consider 1/200 of a second as an incremental modular
value)
# objects that will be sent: 1 Byte
– Latitude: 4 Bytes floating point precision
– Longitude: 4 Bytes floating point precision, should be enough for our needs
– Speed: 1 Byte (unsigned) 0 to 128 KM/h with 0,5 KM/h increment
– Orientation: 1 Byte (unsigned) 0 to 360 with 1,40625 degrees increment
– Category: half Byte with other half for parity
For 50 objects to send datagram of 553 Bytes = about half a KB
26
Data exchange
Copyright of the University of Modena - HiPeRT Lab
![Page 27: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/27.jpg)
Hybrid edge/fog DNN
Same layers
Edge result
Fog/Cloud result
Copyright of the University of Modena - HiPeRT Lab 27
![Page 28: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/28.jpg)
Open source simulator for transportation
Simulates large scale artificial agents, with daily transportation routines and other behavioural factors.
Easy interfaced to mapping systems (e.g. OpenStreet Maps)
Useful to simulate MASA traffic flow, V2X interactions and emerging behaviour before the actual real world deployment
Copyright of the University of Modena - HiPeRT Lab 28
MATSim: Multi-Agent Transport simulator
Misbehaving drivers modelled using ISTAT (Italian National Institute of Statistics) data– Car Accident frequency
– Average driver behaviours in traffic situations– Traffic flow recovery time after accidents
MASA historical traffic analyses, forecasts and driver interactions by interfacing MATSim to Visum– Extract drivers’ transportation routines as aggregate data provided by the city council
![Page 29: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/29.jpg)
900 Agents, represented as “independent traffic flows”
Each agent has its own routine and behavior
One way streets, traffic lights, yields, parking spaces as in the MASA
Contingencies:
○ Accidents
○ Misbehaving drivers
○ Emergency vehicles
Copyright of the University of Modena - HiPeRT Lab 29
MATSim on MASA
![Page 30: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/30.jpg)
Not only cars..
Copyright of the University of Modena - HiPeRT Lab 30
![Page 31: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/31.jpg)
Confidential, all rights reserved
![Page 32: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/32.jpg)
Confidential, all rights reserved
Control Unit: NVIDIA Jetson Xavier
Chassis: 6 motor wheels (independent)
able to climb over curbs
Speed: up to 6 km/h in pedestrian areas
up to 20 km/h in bike lane
Range: > 20 km
Load: Max Volume 50x50x45 cm
Max load 50 kg
Autonomous driving: localization and mapping, object detection, obstacle avoidance,
human overtaking in case of issues
Connectivity: Smart city data, destination/route request, Smart buildings
systems
LIFETOUCH MOVEO: Characteristics
![Page 33: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/33.jpg)
![Page 35: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/35.jpg)
Research Education
Competition
![Page 36: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/36.jpg)
WP3, MODENA
Chassis Design Sensor Integration
Software System Architecture Cloud-Based Simulation Tool
GPU accelerated
libraries
Vehicle and
environment
models in GazeboROSbag dataScenario Sim
![Page 37: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/37.jpg)
F1/10 racing platform
![Page 38: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/38.jpg)
With simple IKEA style instructions at http://f1tenth.org
![Page 39: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/39.jpg)
Fastware
Approximately $2,700
![Page 40: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/40.jpg)
2 times per year
Co-located with major conferences around the world
Each time more challenging!
PAST: CPSWeek @Porto, POR, ESweek @Turin, IT
NEXT: CPSWeek @Montreal, CAN (April '19)
![Page 41: A Fog Computing Infrastructure for Autonomous Driving in](https://reader033.vdocuments.net/reader033/viewer/2022050403/626ff82cf6a9bd1dbf5fae24/html5/thumbnails/41.jpg)
Get Involved !