innovation of social value through big data - pldt...

Innovation of social value through

Big Data NEC analysis technologies for discovering hidden value

March 2015

Yoshiki Seo

Big Data Strategy Division, NEC

© NEC Corporation 2014

NEC Vision “Innovative Social Infrastructure”

Traffic solutions Energy solutions Natural-resource solutions Financial solutions Medical solutions

ICT

ICT ICT

ICT

ICT

ICT

ICT that organically integrates individual infrastructure to be

the foundation of a future society that creates efficiency and

fairness in all regions and communities

Airport

Undersea Towers

Retailers Bank

Distribution

bases Trains

Companies

Seaport Post office

Broadcaster

Important

facilities

Hospital

Fire dept.

Municipal

offices

Dam/Water

Roads

Tele-

communications

Factories

Change Human Life through “Data Driven Approach”

Collection of large-scale data

Analysis and prediction

Solution of social issues

• Invariant analysis

• Heterogeneous mixture learning

• Facial image analysis

• Behavior analysis

• Textual entailment

recognition

• Surveillance cameras

• Smart devices

• Diverse sensors

SDN : Software-Defined Networking

・Network virtualization

・Cyber security

Diverse sensors and human interface technologies

High-performance/high-

reliability core IT technologies

Next-generation network technologies

CLOUD BIG

DATA

SDN

* Rated as No. 1 among organizations participating in an evaluation task organized by the U.S. National Institute of Standards and Technology (NIST)

Unique

Unique

Unique

No.1

No.1

*

*

From the seafloor

to outer space

World’s first SDN switches

Essential to future

information system

Innovation of Social Infrastructure via ICT

Leveraging information captured by our unique and highly competitive ICT assets

to become a social value innovator

• Accumulated data

https://www.intra.nec.co.jp/istorage/sites/www.intra.nec.co.jp.istorage/files/webfm/picture/a/n3780_6s.gif


NEC’s Big Picture of Smart Water Solution

Integration

Control

Prediction / Diagnosis

Detection

Demand Prediction

Water Purification

Management

Energy Power

Saving Operation

Deterioration Analysis Monitoring usage at home

Remote valve

control

Predictive diagnosis

Maintenance

Quick repair

Monitoring Sensor

of water leakage

Life extension of water pipe

Monitoring of Social Media

I’m smoking 420*!

Cities can be safeguarded by preventing terrorism and accidents through

the analysis of video and audio data captured using surveillance cameras, as well as integrated analysis of textual data (e.g. twitter) and sensor data

Example of APAC Safer Cities Trial

Detection of abnormalities in communities using video surveillance. Discovery of calls for demonstrations via

twitter through cyber information monitoring. → Integrated analytics engine used to extrapolate the likelihood and

objectives of illegal demonstrations and alert the authorities.

Example

Video surveillance (Facial image analysis)

BlogsTwitterBlogsTwitter

Com

ple

x E

vent P

rocessin

g

Event detection

(Acoustic analysis)

Detection of other events

(Image analysis of

fingerprints etc.)

Cyber

information monitoring

(Recognizing

textual entailment) RISK Events

Disasters Hazardous materials

Fights Crime

Demonstrations

*420: Slang term for cannabis



Big Data Analytics


The Process of Delivering Social Benefits from Big Data

Real world

Sensing Actuation/

Optimization Social

benefits

Anticipation/predictions;

decisions

Cyberspace

Remote sensing

Vibration

sensors

Mobile

sensors

Open data

Social sensing

Face

authentication

Language

analysis

Voice

recognition Information

systems

Human sensing

Co

llec

ting

A

cc

um

ula

ting

Au

tom

atic

eva

lua

tion

s

Eva

lua

tion

as

sis

tan

ce

Heterogeneous-

Mixture

Learning

Textual-

Entailment

Recognition

Simulation

Technologies

Automatic

control

Robotics

Automatic

operation

Quality-

improvement

measures

Resource

planning

Digital

signage

Incentives

Analytics

Invariant

Analyzer

RAPID Machine

Learning


Overview of NEC’s Big Data Offerings

Creating solution menu for each domain based on advanced field-proven use cases and

consolidate in a structured manner including Platform

サービス提供

, Isilon

NEC Big Data Solutions

オペレーション高度化 / 最適化

Operation Advancement/Optimization

情報管理の強化、犯罪・不正の検知

Enhancing Information Governance, Detection of Crime/Fraud

製品 / サービス価値向上・改善

Product/Service Improvement in value

顧客獲得・維持、販売促進

Acquiring & keeping customers, Promotion

Platform

SDN

SDN Product

（UNIVERGE PF Series, etc.）

IaaS

NEC Cloud IaaS

Professional Service

Solution Menu

- Monitoring & Predicting Failures in Plant

- ICT in Agriculture - M2M for

Manufacturing

- Product Demand Prediction/Automatic Ordering

- Energy Demand Prediction - Demand Prediction of Repair

Parts - Quality Analysis - Demand Prediction (SAS)

- Human Resources Matching - Facial Recognition

Technology - Customer

Analysis/Campaign Management

- EBM Analysis - Web Access Analysis

Analytics Technology

SAS MicroStrategy Dr.Sum Oracle BI Business Objects MS SQL BI SAS 等 ISV 製品

Invariant Analysis

Rapid Machine Learning

Heterogeneous Mixture Learning

Text Inference

Recognition

Server Storage: iStorage

Network Operation & Maintenance

Real time Event

Processing （Oracle ＣＥ

Ｐ）

Scale-out ＤＢ（ＩＲ

Ｓ）

Parallel Distributed Processing

（Ｈａｄｏｏｐ）

In-memory ＤＢ

（SAP HANA DataBooster）

Ultra High SpeedＤＷＨ

（Data Platform for Analytics ）

ＥＴＬ（PowerCenter

DataCoordinator）

Parallel Integrated ＤＢ（Oracle

Exadata Oracle DB SL）

M2M Platform（CONNEXIVE

）

Security （Anonymization

Technology, Anonymized Calculation）

Memory DB （TAM）

Big Data Discovery

Program

Big Data Education Program

Platform Planning Service

- Enhancement of Information Governance

- Medical/Healthcare - Vehicle Traffic Control - Big Data Archive

Optimization


１．小売業における需要予測 Value 1

Detection of abnormalities in facilities

Invariant Analysis

* Collection and analysis of large volumes of sensor data to detect when

operations are “different than normal”

Detection of abnormalities in facilities

Social infrastructure

(Bridges, expressways, etc.)

IT systems, data centers, telecom

networks Power plants

Manufacturing plants (Assembly, chemical, etc.)

Automobiles, trains,

aircraft, ships, etc.

Detection of signs of failure and/or abnormalities in domains in

which failures could have a high economic and/or social impact

Sensor data Operation logs

Provision of safety and security solutions that contribute

to society

Visualization of “normal”

operations

[Invariant model]

Detection of signs indicating that

operations are “different than

normal”

[Real-time failure detection]

Mechanical and automatic

visualization of all relationships

between each sensor data

Comprehensive viewing of all

relationships enables

abnormalities to be detected

Detection of abnormalities by comparing past data and

real-time data

Invariant Analysis Technology

Detection of

abnormalities

Effectiveness of “Invariant analysis technology” for Large-scale Plants confirmed through field trials at

Shimane Nuclear Power Station

Chugoku Electric Power Co., Inc.

No. of sensors per power plant: 3,500

100 of data from each sensor in a second

Collaboration between experts in power

plant operations working for the customer

and NEC’s own analysts.

Discovery of signs indicating that operations

are “different than normal” from correlations

between 3,500 x 3,499 sets of sensors

Advanced Analysis

Real-Time

On-site know-how



Demand forecasting

Heterogeneous Mixture Learning


Heterogeneous-Mixture Learning Technology

Sunday

Monday

Tuesday

Saturday

⁞

Sunny

Cloudy

Snowy

Condition Ａ＝Ｙ

Condition Ａ＝Ｎ

Condition Ｂ＝Ｙ

Condition Ｂ＝Ｎ

Sunday

Monday

Saturday

⁞

Day

Night

Day

Night

Day

Night

Conventional method: categorizing patterns manually

Trying to

categorize

data by day

Trying to

categorize data

even more

minutely

Trying to

categorize data

by weather

Heterogeneous

mixture of data

Hard to find accurate

patterns if data types

mixed

○

○

○

○

⁞

Manual categorization methods limited

to trial and error and inaccurate for

categorization

⇒ High-precision prediction difficult

○

Efficient trial-and-error process

achieved through automatic

categorization preventing

inaccurate prediction

⇒ High-precision prediction

achieved

○

⁞

NEC’s new technology: heterogeneous-mixture learning technology

Automatically categorizing data patterns into classes


Fresh products

Disposal loss problems in the retail industry

�Á�ï�

ú�À

Due to short shelf-lives

and high frequency of

orders, losses due to

disposal of unsold items

significantly affect costs.

Food retail

Appropriate product demand forecasting is strongly required.

Calendar attributes Weather changes

Human judgment based on intuition and experience has limitations due to the large number and

complexity of products.

Changes according

to day of the week

Page 15

Disposal loss problems in the retail industry

Use of heterogeneous mixture learning technology to predict

demand for goods to be delivered after three days

From verification result calculations, a 30% reduction in losses was achieved,

compared to previous purchases of cream puffs

Verification

Analysis example

Discovery of a negative correlation between cream puff sales

trends and the minimum temperature

Cream puff sales fall when the minimum temperature rises

0 Negative correlation

Correlation between cream puff sales and explanatory factors

Positive

correlation

Minimum temperature

Same category sales

Same product sales


Customer’s Problem

Existing prediction model can not handle dynamic changes in the market, and

Error Ratio is significantly high. Low Profit & High Operational Cost

Conditio

n &

Para

mete

rs

Input

Pric

e

Pre

dic

tion

Dete

rmin

e

Tra

de-in

Pric

e

Oth

er

Consid

era

tions

Investig

ate

Pro

ducts

Current Customer’s Price Prediction for Trade-in CertainProducts

Auction Price Prediction using Heterogeneous Mixture Learning

NEC is now proposing a new Price prediction system using Heterogeneous mixture learning technology.


Churn Retention Cycle

collect

design

predict

act

evaluate

optimize

understand

implement

customer’s churn

customer segmentation

statistical analysis

predictive modeling

KPI

campaign/promotion

Plan/bundle/package

loyalty program

churn propensity

resource prediction/allocation

campaign policy

KPI review

update prediction model

call operators’ performance

Call/Text/Chat/IVR


Big Data Drives Churn Management

Call Detail Record

Users’ Profile

Billing History

Churners’ History

Campaign

Promotion

Price Plan

Loyalty Program

Call Trace

HSS/HLR

Performance

Counter

inbound

outbound

IVR

Chat Churn

Management


Fair price forecasting

Deterioration forecasting

Product demand forecasting

Power demand

forecasting

Forecasting of members’

purchasing trends

Applications for heterogeneous mixture learning technology

Enables the realization of demand forecasting, adjustments

and various forecasting solutions for society:

Page 20


Information Governance

Textual Entailment

Recognition

Textual Entailment Technology

Based on an understanding of the total sentence meaning rather than single words, more

sophisticated analysis and use become possible independently of differences in expressions

×

×

○

○

○

○

In emails, sentences, and daily reports, expressions that can create suspicions of dishonest transactions

(bid-rigging, etc.) are automatically monitored, and a warning is sent to the writer.

Adjusting the sales price with

other companies in the same

industry

Expect cooperation of other

companies in the same industry

regarding the price decision

Adjust other companies

and price

To adjust the sales price within the

company, while adjusting the

prices of similar products of

other companies

Problematic expressions (semantic)

that need to be extracted Sentences

recognized

Correct

understanding

Control needed

Incorrect

Understanding

Control

unnecessary

(leak)

Incorrect

Understanding

Control

necessary

(False alarm)

Correct

understanding

Control needed

Correct

understanding

Control needed

Correct

understanding

Control unnecessary

Correct

understanding

Control needed

Correct

understanding

Control needed

Correct

understanding

Control

unnecessary

(Conventional)

Word based

recognition result

NEC

Text inference

recognition result

Ex.

Differences between conventional word-based recognition and NEC textual inference

recognition

NEC textual inference

recognition Takes into consideration the importance of words and their semantic correspondences as well as the sentence structure involving

the subject, predicate, etc. It recognizes relations between these two types of meanings in a sentence.

Page 22

Information Governance Usecase

Document

Score

Doc A 0.9

Doc Ｂ 0.7

Doc Ｃ 0.55

Doc Ｄ 0.48

Doc Ｅ 0.33

Has

required

information

Writers do not always have full knowledge of

document management standards and thus are

not fully competent to judge on their own

Conventional

method (Human judgment)

NEC method

(Human +

machine

judgment)

Decision on need for information control by management

Document author alert

Normal

processing

▌Because the number of documents is so large,

it is difficult for a few document managers to

check all of them.

Problem

1

Emails

Sales

reports

Information

control

required

Information

control

not

required

Has

required

information

Only all suspicious documents are

checked,

and efficient, strict management is

achieved

Encourages

verification with

writer, reduces

judgment mistakes.

Effect

2

Effect

1

Busines

s

law

violation

Informatio

n

leakage

Documen

t

detection

Confirmation by authorizing organization

Emails

Sales

reports

Automatic decision whether required

management information is included in the text Solution

Problem

2

Public risk detection

Expansion of applications to compliance enhancement

and customer voice analysis

Compliance enhancement

Areas of application for textual inference recognition technology

Customer voice analysis for call centers



Matching solution

RAPID Machine Learning

RAPID Machine Learning Original

Technology Deep Learning Engine optimized by NEC

achieving high speed and light processing

High Speed and Light Precious analysis

Global No.1 Decreasing error rate

Analyzing unstructured data such as images and texts!

competitor RAPID

accuracy 97.24 % 97.29 %

Processing Time (sec)

833 4

Memory 2200 MByte

32 MByte

Increase processing speed while

keeping accuracy and small

memory

RAPID can deal with big data easily and improve accuracy

Collaboration Filter

RAPID

parameters 9000 210000

Error rate 29% 14%

Human Resource Matching Solution

Registration ＲＡＰＩＤ Machine Learning

Job seekers (Students, seniors, career

changers, workers from overseas)

Placement agencies Companies

seeking employees

Job seekers

Data

Blogs

Information

on job

listings

NEC independently developed RAPID machine learning from deep learning technology

Entry

Companies suited

to each job seeker

Top class personnel

that match corporate needs

Page 27

Matching of employers

and people Customer behavior analysis

Areas of application for RAPID machine learning

Image monitoring

of urban areas※ Tourism matching

Development of numerous test projects in each business domain Expansion of matched to needs and test results

Future Development Plan

Upgrading and optimization of

operations

Public risks Product & services risks

Monitoring of urban

areas

Prediction of energy demand

Optimal distribution of

resources

Prediction of transaction

prices

Quality control for transportation

vehicles

Manufacturing quality control

Detection of infrastructure

abnormalities

Monitoring of plant

repair predictors

Strengthening of information governance

Human resources matching

Automatic issuance of orders based on demand predictions

ＮＥＷ

ＮＥＷ

ＮＥＷ

ＮＥＷ

Strengthening of information

management, Detection of crime

and fraud

Kaizen, improvement in

value of products & services

Matching of products &

people

Tourism matching

Customer behavior analysis

Customer acquisition

and support, sales promotion

© NEC Corporation2013

NEC Group Vision 2017

To be a leading global company

leveraging the power of innovation

to realize an information society

friendly to humans and the earth

innovation of social value through big data - pldt...

Documents