autonomous driving with the mipi camera and … driving with the mipi camera and sensor interfaces...

Hezi Saar

April 6, 2017

Linley Autonomous Hardware Conference

Autonomous Driving with the MIPI Camera and Sensor Interfaces

© 2016 Synopsys, Inc. 2

Abstract

• Advanced Driver Assistance Systems (ADAS) SoCs for self-driving cars incorporate numerous interfaces for functions such as camera, radar, lidar and sensors. It is vital for such interfaces to meet the new stringent automotive standards, and offer the low power and high performance requirements that designers are looking for. The MIPI interfaces for camera (CSI-2) and sensors (I3C) are playing an essential role in enabling ADAS SoCs for autonomous driving. This presentation describes how these MIPI specifications are implemented in automotive SoCs and why.


Agenda

Implementation of MIPI interfaces in applications beyond mobile

MIPI camera and sensor specifications for automotive applications

Meeting automotive reliability requirements

Summary


MIPI Specifications in New ApplicationsAutomotive, IoT / Wearables, Virtual / Augmented Reality


Safety-Critical ADAS ApplicationsRequire ISO 26262 Functional Safety Compliance and ASIL Certification

Electronics Failures Can Have Hazardous Impact

≠


Advanced Driver Assistance Systems (ADAS)

• Passive Driver Assistance Systems– Back-up, surround view camera

– Distance alert system

A Rapidly Evolving Technology: Car Mirrors, Multiview Systems

• Active Driver Assistance Systems– Back-up camera with identification & braking

– Collision avoidance


Example: Automotive Surround ViewUsing MIPI CSI-2 Image Sensors & DSI Display

Rear Camera

Display

CAN Interface

MPU Proprietary, LVDS or Ethernet Switch

DRAM Flash Memory

Power SupplyFront Camera

Module

Left Camera Module

Right Camera Module

Rear Camera Module

Other Camera Module

Vbat

LVDS or Ethernet Link

MIPI CSI-2 Image Sensors

MIPI DSI Display

Front Camera

Left Camera

Right Camera

Other Camera Module


Automotive Applications Require Different SoC Architectures

High-End ADAS Infotainment MCU

• LPDDR4, Ethernet AVB, MIPI, HDMI, PCIe, SATA, ADC

• Embedded Vision• Security• Sensor Fusion• Requires Functional Safety

• USB, LPDDR4, Ethernet AVB, MIPI, HDMI, PCIe, SATA, ADC, UFS, eMMC

• Real-time Multimedia• Security• Sensor Fusion

• IP: Ethernet 10/100/1000, ADC, I/F peripherals• Medium Density NVM

ARC HS

ARC HSARC HS


MIPI CSI-2 and D-PHY Specifications


MIPI CSI-2 Over D-PHY Overview

CSI-2 Device

DPH Y

CSI-2 Host

DPHY

Clk+

Clk-

L0+

L0-

Clk+

Clk-

L0+

L0-L1+

L1-

L1+

L1-

Frame Buffer

CSI-2 Transmitter

PacketBuilder

LaneDistribution

CCI SlaveSCL

SDA

SCL

SDA

CCI Master

CSI-2 Packet

CSI-2 Packet

D-PHYHS Burst

D-PHYHS Burst

CSI-2 Receiver

PacketDecoder

LaneMerger

CSI-2 Packet

CSI-2 Packet

Frame Buffer

VC CRCD

T WC ECC

Payload byte size Data CRC processing

ECC protecting the header

Data Format Definition

Virtual Channel Identification

Packet Builder


MIPI CSI-2 Packets

Data PFPHFS FEData PFPH

KEY:SoT – Start of Transmission EoT – End of Transmission LPS – Low Power StatePH – Packet Header PF – Packet Footer + Filler (if applicable)FS – Frame Start FE – Frame EndLS – Line Start LE – Line End

Frame Start Packet

Frame End Packet

First Packet of Data

Last Packet of Data

VVALID

HVALID

DVALID

SoT LPSEoT SoT EoTLPSEoT SoTEoT SoT

Short packets used for frame synchronization

Image data

Low power states between

image lines

Short packets used for frame synchronization


MIPI CSI-2 v2.0 Feature Enhancements for Automotive Applications • RAW-16 and RAW-20 color depth

– improves intra-scene High Dynamic Range (HDR) and Signal to Noise Ratio (SNR) to bring “advanced vision” capabilities to autonomous vehicles and systems

• Latency Reduction and Transport Efficiency (LRTE) feature – supports increased image sensor aggregation without adding to system cost;

facilitate real-time perception, processing and decision-making; and optimized transport to reduce the number of wires, toggle rate and power consumption

• Differential Pulse Code Modulation (DPCM) 12-10-12 compression– reduces bandwidth while delivering superior SNR images devoid of compression

artifacts for mission-critical vision applications

• Scrambling to reduce Power Spectral Density (PSD) emissions– minimize radio interference and allow further reach for longer channels

• Expanded number of virtual channels from 4 to 32 – accommodate the proliferation of image sensors with multiple data types and

support multi-exposure and multi-range sensor fusion for Advanced Driver Assistance Systems (ADAS) applications such as enhanced collision avoidance Image courtesy of the MIPI Alliance


Virtual Channels

Virtual Channel 0 – Line 0Virtual Channel 0 – Line 1Virtual Channel 0 – Line 2Virtual Channel 0 – Line 3Virtual Channel 0 – Line 4

Virtual Channel 0 – Line N

Virtual Channel 1 – Line 0Virtual Channel 1 – Line 1Virtual Channel 1 – Line 2Virtual Channel 1 – Line 3Virtual Channel 1 – Line 4

Virtual Channel 1 – Line M

Line 0Line 1Line 2Line 3Line 4

Line N

Line 0Line 1Line 2Line 3Line 4

Line M


D-PHY Architecture

• Synchronous forwarded DDR clock link architecture

• One clock and multiple data lanes configuration

• Static/dynamic de-skew supported through calibration

• No encoding overhead• Low-power and high-speed modes• Primarily targeting camera and display• Spread spectrum clocking supported for

EMI/EMC considerations• Large ecosystem, proven in millions of

phones and cars

The Popular Physical Layer Used for CSI-2 and DSI Specifications

Two Data Lane Configuration


MIPI I3C Specification Standardizing Sensor Interface


Why I3C?

• Today Smartphones typically have 10-15+ sensors– Which require 12-18+ pins

• Different Sensors have different requirements – Fingerprint vs Compass

• Typical approach is to connect sensors using a mix of I2C and SPI – I2C for lower data rates and SPI for higher

data ratesMultiple side band signals– For interrupts, chip selects, power

management

The Challenge of Integrating Multiple Sensors with Different Requirements

This will increase the package size and add complexity which translates into additional costs

So many I/Os

required!!

Image courtesy of the MIPI Alliance

No Standard driver for these fragmented interfaces


Why I3C?

It takes the goodness of I2C – Two-wire, Simple

It takes the goodness of SPI – Low Power and Speed

Adds features such as– In-band interrupt/command support– Dynamic addressing– Advanced power management– High data rates

While maintaining support for legacy I2C sensors

– Evolutionary, not revolutionary

You Can do Everything with Two IOs!

I/Osreduced to just two!!



I3C Enables Efficient System ArchitecturesExample: Sensor Hub

Power Reduction, More Efficient System, Faster Data Transfer


MIPI CSI-2, D-PHY & MIPI I3C

• Supports advancements in imaging for new applications: Health, Convenience, Security, Lifestyle, Efficiency

• Camera Controller Interface (CCI) and Always-ON advancement considerations using I2C and future MIPI I3C

Sensor

Sensor

Sensor

Primary Camera Module

Image Sensor

Lens Actuators/Controllers

SDASCL

Secondary Camera Module

Image Sensor/SoC

Pixels

Pixels

Host Processor/ISP


Summary


MIPI Specs for Multimedia, Storage, Sensor & Wireless Connectivity in Automotive Applications

• Real time video & data network

• Gateways• Telematics• V2V• V2I• Security

Vehicle Networks & V2X

• Navigation• Audio/video• Entertainment

Infotainment

• Instrument clusters• Voice recognition• Hi-def displays• Surround view

Driver Information

• Parking assist• Lane departure warning

& Lane keep aid• Pedestrian detection &

correction• Automatic emergency

braking

Driver Assistance



Meet quality levels required for automotive applications

Accelerate ISO 26262 functional safety assessments to help ensure designers reach target ASIL levels

Reduce risk & development time for AEC-Q100 qualification of SoCs

Key Requirements of Automotive-Grade IPReduce Risk and Accelerate Qualification for Automotive SoCs

Reliability

Functional Safety

Quality

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRw&url=http://www.allisonpr.com/blog/verification-nation-one-checkmark-to-rule-them-all/&ei=WHclVbeFDZe0oQSd8oHABA&bvm=bv.90237346,d.cGU&psig=AFQjCNHD-MO7m96WIumNhhfLQMd8x7Yzkg&ust=1428604819958407




ASIL Ready ISO 26262 Certified MIPI IP

• MIPI CSI-2, D-PHY and I3C protocols

– Leveraged for mobile and beyond – IoT, automotive, AR/VR

• Enables new set of applications in automotive, AR/VR, IoT markets

– Lowers integration risk for application processors, bridge ICs and multimedia co-processors

• Future proof IP supporting variety of speeds, proven in silicon

– Reduces cost and power for multiple instantiations

– Testability features enable low cost manufacturing

Single-Vendor Solution, Production-Proven, Interoperable

Industry’s first MIPI I3C Demo

High-End Processor

Image Signal

Processing

CSI-2 Host Controller

D-PHY

CSI-2 Host Controller

D-PHY

DSI Device Controller

D-PHYCSI-2 Device

Controller

D-PHY

DSI Host Controller

D-PHY

I3C

Synopsys Provides Complete Camera, Display & Sensor Interface IP Solutions

Thank You

autonomous driving with the mipi camera and … driving with the mipi camera and sensor interfaces...

Documents