ConniroTM Solution Brief

ConniroTM

Connected solutions with enhanced intelligence, personalization, integration and connectivity

2 ｜ January 2020

ConniroTM Solution Brief

As the mobility environment evolves, the ways in which in-vehicle

user experiences are provided are also changing. Whereas drivers,

traditionally, had to be aware of and take control over the entire

environment inside the vehicle, in the future, an intelligent assistant

in a car will offer a variety of customized services in an integrated

manner, taking driver and passenger preferences into account.

Overview

LG Connected Solutions - ConniroTM

ConniroTM enables carmakers to provide a superior driving experience that is safe and convenient as well

as fun through a variety of interactions between vehicles and users, so that they can prepare for the era

of autonomous driving and innovation of mobility services based on connectivity technology.

ConniroTM provides a differentiated driving experience for the driver and passengers with integrated

solutions specially designed for the delivery of intelligent, personalized, integrated, and connected

experiences to respond flexibly to changes in the mobility environment.

Current Challenges Facing the Automotive Evolution

Technological advances in automotive industry introduce many challenges for carmakers:

● As autonomous driving has become the mainstream throughout all automotive sectors, the burden of driving

is reducing, and the demands on in-vehicle infotainment are increasing.

● Drivers and passengers seek to have more personalized experience from autonomous driving. Vehicle intelligence

is an essential means to power personalized and predictive services.

● UX/UI providing intuitive and natural interactions using multimodal input will play an important role in delivering

contextual safety and more convenient mobility experiences.

● An increased number of sensors in vehicles enables the collection of data within the vehicle relating to the

driver and passengers. How to turn the data into valuable functions and services optimized to the users will

be one key differentiating factor.

● Customer demand for better connectivity and high-quality connected services is increasing rapidly. Delivering new

types of connected services and seamless experience based on the connected services will be more important

than ever.

2 ｜ January 2020

ConniroTM Solution Brief

January 2020 ｜ 3

ConniroTM Solution Brief

ConniroTM provides a wide range of market leading connected

solutions for diverse requirements. It has already been

successfully applied and commercialized in many global car

models and the enhanced technical perfection allows carmakers

to reduce HMI development time and maintenance costs.

All Solutions of Conniro

TM

Driver Monitoring System for

driving safety by monitoring the driver’s

condition

Interior Monitoring System for

convenience and personalization by

monitoring a vehicle interior

Augmented Reality for intuitive

navigation and driver safety alerts

by augmenting the real world

Virtual Personal Assistants

for user interactions by combining

various HMIs and intelligent services

January 2020 ｜ 3

ConniroTM Solution Brief

4 ｜ January 2020

ConniroTM Solution Brief

ConniroTM AR (Augmented Reality) provides an intuitive user experience for navigation and driver safety alerts

by augmenting the real world with computer-generated visual information. The solution works with a variety of

vehicle sensors, including ADAS, front-facing cameras, and GPS. Driving-related information and safety warnings

are then displayed in the driver’s line of sight as augmented (3D) and non-augmented (2D) images.

● Support for various display such as HUD, AVN/CID, and mobile

● AR core engine with support for vehicle dynamic compensation, real-time eye position matching and time

warping by Unscented Kalman filter

● Sensor fusion which merges information received from in-vehicle sensors and predicts the future state of all

objects around the car to compensate message latency

● Advanced HMI including 3D carpet navigation guidance, ADAS object, and lane highlight, turn-by-turn overlay, pin hovering

for the destination, and POI highlight

● SDK with support for multiple HMI frameworks (CGI Studio, Kanzi, QT & Adobe After Effects), AR engine emulator, and

data simulator

Highlights Benefits

Key Features

● Applies the industry’s top-performing AR technology

● Provides intuitive navigation guide and Point of Interest

(POI) highlight

● Ensures active safety with various sensors

● Enables extended user experience with advanced POI

service integration

● Maximum flexibility and reusability of AR design

● Minimum use of CPU and memory resources

● Best performance through various optimization experiences

● Easy-to-update additional UX features depending on the

availability of extra vehicle sensors

Input/Output of AR Engine Use Cases

Navigation Guidance

Building POI Highlight

ADAS Warning

AR Parking Service

AR Engine

AVN/CID

ADAS Information

VehicleInformation

Sensor Data

NavigationInformation

HD MapInformation

HUD Mobile

Augmented Reality

January 2020 ｜ 5

ConniroTM Solution Brief

ConniroTM DMS (Driver Monitoring System) identifies drivers and monitors the driver’s condition. The solution ensures

driving safety and maximizes convenience and personalization of the driving experience.

● Tracking driver’s head gesture, eye-gaze, and eyelid

● Detecting if the driver is distracted, drowsy, asleep, or speaking

● Anti-spoofing for face identification and autopilot mode

● Detecting blockages in the field-of-view of a camera

● Grid detection for camera calibration

Key Features

Highlights Benefits

● Driver state monitoring by tracking the driver’s facial

landmarks and eye-gaze for driving safety

● Driver identification and anti-spoofing achieved by deep

neural network-based features

● Ground Truth System that evaluates the performance on

various camera position and driving condition

● Lightweight tracking algorithm and applications on

embedded system

● Accurate performance evaluation and validation via in-vehicle

Ground Truth System

● Experienced experts support along the whole development

process - from algorithm adaptation and software

integration on a target device to application development

Personalization /

Convenience

- Anti-spoofing- Driver Identification- Area of Interest- Head Gesture

Safety

- Distraction Detection- Drowsiness Detection- Eyes on the Road- Yawn and Speaking

Detection

Camera Status

- Camera Blockage- Out-of-focus- Steering Wheel

Detection

Applications

Service Interface

Function Manager / Scheduler

Service

Engine

Ground Truth SystemCore Algorithms

Head TrackingHardware

- Head / Gaze tracking GT sys.

Software- Evaluation sys.- Labeling tool

Eyelid/Gaze Tracking

Input DMS Offering

IR Camera Configuration (Features, Camera)RGB Camera

DMS Software Architecture

Driver Monitoring System

6 ｜ January 2020

ConniroTM Solution Brief

ConniroTM IMS (Interior Monitoring System) monitors a vehicle interior in real-time. The solution determines the

situation inside the vehicle and provides the driver and passengers with services for safety, convenience, and

personalization.

Interior Monitoring System

● AI-based detection and decision algorithm embedding

- To detect, track and classify the occupants and objects in the cabin

- To decide child presence, left-over items and occupant status (behavior)

● Flexible configuration

- IMS stand-alone (AI algorithm is embedded)

- Client-server (Al algorithm runs on a server)

- Head unit consolidation

Key Features

Highlights Benefits

● Object detection and occupant behavior analysis for

convenience services

● Occupant identification and analyzing their behavior

patterns for personalized services

● Flexible and highly scalable solution for various needs - range

from stand-alone installations to head unit consolidation

● Support for a video conference call in and out the car with

RGB-IR sensor

IMS Algorithm Flow

Detection Decision Control

Lost Items

Boarding Status

Face ID / Emotion

Child Presence

Body Pose Decision(Sleeping, lean-back, etc.)

Behavior Analysis

Personalization

Driver Safety

Sleeping Status

Gesture

Video conferencing

Object Tracking /Classification

Human Tracking /Classification

Head / Hand Tracking(Point of Interest) Audio / Video Haptic

Feedback

Vehicle Components(Airbag, AVN, …)

ConniroTM IMS is compatible with 2MP RGB-IR camera with Vertical Cavity Surface Emitting Laser (VCSEL)• RGB and IR Images output simultaneously with wide-angle (130 degrees)• About 70% less power consumption than legacy LEDs

January 2020 ｜ 7

ConniroTM Solution Brief

ConniroTM VPA (Virtual Personal Assistants) helps the customer achieve maximum comfort and enjoy all connected

services and functions of vehicle by natural and intelligent interaction, providing valuable personalized services.

● Intelligent interaction to complete a task based on more clear user intent by interpreting various multimodal

inputs

● Connection to appropriate assistant based on user intent for consistent voice service experience

● Proactive and reactive suggestions for achieving maximum comfort and enjoying all the connected services and

vehicle functions

● Cloud assistant platform for seamless connected service experience (Microsoft)

● SDK package with prebuilt voice service & multimodal device adaptor on Linux with QT, ready-to-run sample

applications, documentation for HMI design

Virtual Personal Assistants

Key Features

Highlights Benefits

● Multi-modal interaction with the seamless and natural

combination of interactions such as GUI, touch, gestures,

and eye gazing based on voice

● Multi-assistants with a combination of different AI assistant

services

● Multi-Intelligence with proactive and reactive suggestions

provided for tailored information and entertainment

services

● White-labeled personal assistant with new multi-modal HMI

- Seamless integration into legacy systems

- Flexibility and scalability for extending new UX/UI

● Valuable personalized connected services

- Intelligence engine for data monetization

- Service arbitrator for providing valuable services

External assistants Connected services

OEM GUI Framework / Application

Dialog Design (OEM, 3rd party, …)

Multi-modal interaction Multi-assistants Multi-intelligence

Application Framework Adaptor

System Layer

VPA Software Architecture

VPAOffering

Intent interpretation Domain classification Learning / recommendation

Dialog management OEM/External assistant Rule engine

Hybrid context management Behavioral log management

Cloud Assistant Platform

chatbot

Touch Voice Eye gazing Gesture Car Data

QNA MAKER

LG Electronics, Inc. is a global leader and technology innovator in home

appliances, consumer electronics, mobile communications and vehicle

components. LG Electronics has been providing total solutions for vehicles based

on the experiences in delivering the best electronic devices to customers. From

next-generation vehicle parts such as OLED, wireless charger and battery for

electric cars to convergence systems like infotainment, telematics and digital

cluster, LG developed a global approach for an integrated customer experience.

Contact Us

For further information on ConniroTM,

please contact

swsolutions@lge.com

Copyright ⓒ 2020, LG Electronics. All rights reserved. No express or implied warranties are provided for herein. All specifications are subject to change and any expected

future products, features or functionality will be provided on an if and when available basis. The product names and logos mentioned in this document are trademarks

or registered trademarks of their respective owners.

Please visit the LG Software Solutions website swsolutions.lge.com and

get more information about ConniroTM. The website offers useful resources

you need to get started with ConniroTM, along with other solutions for the

automotive industry.

Learn More