WeChat Social to Intelligent Connection

杨强

What is WeChat?

A New Lifestyle

• Mobile App for 500M+

• Multimedia

• A Way to Connect: A Platform

Rapid Growth

0

2

4

6

8

10

12

1.0 4.68

Oversea Users By Aug 2013

Active Users per Month

By Sep 2014

> 1 0

Users（Unit：100M）

1 Billion Accounts, 550M Active Users

8M Service Accts

20 Languages, 200 Countries

微信 Growing Up Path

Birth

Speech

Nearby

Shake It!

Scan

Service Platform

Video Chat

Payment Online Shopping

Red Packet

2014-10-1

Video Clips

Moment Ads

1 2

3

2012-3-29

2012-9-17

2013-1-15

Growth Path (Unit: 100M)

~6

2015

2010

2011

2012

2013

2014

微信诞生

-扫一扫

-视频聊天

-支付

-游戏

-表情商店

-红包

-小视频

-卡券

-广告

1亿

2亿

3亿

2015

-语音信息

-附近的人

-摇一摇

-漂流瓶

-朋友圈

-公众平台

2012-3-29

2012-9-17

2013-1-15

2014-10-1

433 days: from 0 to 100 M

6 Months Later: doubled to 200M

3 months later: 300 M

3 Milestones

COLLECTIVE USER EXPERIENCE: SHAKING FOR RED POCKET IN 2015 SPRING FESTIVAL

Red envelope data：Over 1.01 billion times、Peak at 550 000 / min、Shaking 11

billion times, Peak 810 million times / min

• INTERACTION / COMMENTS / VOTING / LOTTERY / SHARE WITH FRIENDS

LINK

BigData @ WeChat

WeChat Big Data

Big

Dat

a

relational data

contacts

subscribed public accounts

group

non-relational

explicit information

behavior

posts

text

images

videos comments user feedback

Relationship Types (%)

陌生人

网友

老师或领导

同事

亲人或亲戚

同学

现实生活中的朋友

24.7

32.1

50.4

70.8

75.7

81.4

90

Friends in Real Life

Strangers

Classmates

Artificial Intelligence @

Image Understanding Lifelong Learning Agent Spammer and Rumor

detection

Location-based social

networks

NLP and Text Recommendation Feature Engineering Provence of information

Speech Understanding Social Search Trust Assessment User Modeling

Crowd Intelligence Sentiment Analysis Event Detection Network Analysis

WeChat User Modeling & Transfer Learning

Case Study: ADs IN WeChat MOMENTS

• PRESENTS IN MOMENTS / TARGETING BASED ON BIG

DATA / INNTERACTIVE COMMUNICATION

• SPREADING BETWEEN FRIENDS

User Modeling for Ads

• Data Sources • Demographic information

• Articles read

• Public accounts subscribed

• Techniques: • DNN

• Multi-task Learning

Source 1

Source 2

Source 3

Source 1

Source 2

Source 3

Topic modelling

Topic modelling

Topic modelling

Co-training

tag 1

tag n

Seed users

Advertise

Image Understanding for Ads

Cross Domain Transfer Learning

• Predicting User Feedback from Social Data

source domain: BMD advertisement with abundant feedbacks

Target domain: SOHO advertising with no previous data

Source domain: BMW advertisement with user feedback

Crowd Intelligence @ WeChat

2010

2011

2012

2013

2014

微信诞生

-扫一扫

-视频聊天

1亿

2亿

3亿

2015

-语音信息

-附近的人

-摇一摇

-漂流瓶

-红包

-游戏

-表情商店

2012-3-29

2012-9-17

2013-1-15

-朋友圈

-公众平台

-小视频

-卡券

-广告

Grow with Users

- Games

- Red Pocket

- Moments Ads

- Shaking TV Program

Crowd Intelligence

Charity by the Millions: Voice Donor

LINK

Crowd Intelligence for the Visually Impaired

Collect Voices Filter by Standard Models

Speech Recognition

• Large Mandarin Corpus: DNN (deep neutral network)

• Language model: • N-gram, DNN • Low-rank matrix • GPU training

• Decoder: • WFST framework • Large, parallel search space

Audio Fingerprinting

• Challenge • noisy environments,

• compactness of fingerprint, and

• service scalability when song database is huge (10M)

• Application: WeChat “Shake” Music, lunched in Jan, 2013

• Big Music Database（10M songs） Fast Recognition (3-5 seconds)

• Daily Page View > 8M, User View > 3M

Audio Fingerprinting：WeChat Live TV recognition

Recognize live TV program from audio

fingerprinting

• Challenge: High concurrent throughput

• SHAKE-TV:

• Can recognize > 500 TV channels

across China

• User View: 1M simultaneously

• Rich Cross-TV Screen User Experience

• Fully integrated with social networks

Image Understanding @ WeChat

Configuration Speed-up

2 GPUs Model

Parallelism

1.71

2 GPUs Data Parallelism 1.85

4 GPUs Model + Data

Parallelism

2.52

4 GPUs Data Parallelism 2.67

GPU0

GPU1

GPU2

GPU3

Mariana CNN on GPUs

Mariana CNN is Tencent’s Deep Convolutional Neural Network based on Single-machine, Multi-core GPU Computation.

• Data parallelism and model parallelism

• Partition models for parallel execution

• Model scalability and performance had major improvement

• Scan for information or services

• Local and Global Image Feature Descriptor

• Highly Efficient Feature Indexing and Matching

• Mobile video and image quality assessment

• Challenges:

• Variable lighting, Non-planar recognition

• WeChat “Scan” on covers, lunched in 2013

• Large Image databases (~10M)

• Open interfaces for developers

Mobile Visual Search

LINK

• OCR on a mobile device

• Camera OCR based language Translation

• Certificate and ID OCR on mobile or cloud

• Face Technology:

• Detection, Alignment, Tracking, Recognition/Verification

• User modeling based on images

• Targeted Adverts

Mobile Image Tech.

LINK

Augmented Reality

3D animation w/ embedded video on designs • Rich interaction with users

Challenges： • Real-time and precise target

detecting/tracking, model rendering

Applications： • lunched in WeChat movie ticket App “微票” (Jan 2015)

LINK

Natural Language Understanding @ WeChat

WeChat: Closed-loop NLP • Closed-loop Feedback in WeChat Services

• Always online: real-time message platform

• Massive user base: 549 million monthly active users

• Payment User Intention

…

payment

WeChat NLP

Word Multi-Embedding

Learn Embedded Word Representation

WeChat NLP

Semantic Matching of Questions and Answers

h

output

R(query,doc1)

WARP loss

query doc1 doc2

R(query,doc2)

h x

x h

x x

h

h x

x h

x x

h

h x

h x

x x

output output

Dependency-Tree RNN model

• semantic match

• semantic answer ranking

Semantic Match

• Dependency-Tree RNN

•Word multi-embedding match

•BM25

•Other Features:

-Sentence type recognition

-Synonym

-Antonym

-Parsing

WeChat NLP

Text/Voice

Answer

Speech Recognition

TTS

Data management

Pattern based

Parsing and LM based

NLU Query Analysis

Query Inference

Semantic Pattern

intent identification

Self-learning system

query rewrite NER/NED Parsing

Sentiment Analysis Graph Search

RDF based Inverted index based

NLG

Log/Session

management

WeChat

knowledge

graph

Dialog

management

User Profile

User behavior

Context

management

Semantic Search

Semantic Match

ReRanking

Inverted index

Text Search

Recommendation

NLP & Question Answering

NLP Dialog

微信 Future

Connect Everything

A New Connection Model

Becomes a New Lifestyle

Will extend the connection to daily life, commerce &

entertainment

Provide Mobile Internet Service to Industries

People

Service Things

Connecting People, Services and Things

People, things and services An ecosystem for connection, a new solution provider

Internet +

Intelligent Business Solutions

Integrate internet and other businesses，Smart

Cities，Improved User experience, Connecting

everything

Eight Principles

1、Bring New Value

2、Remove geographical restrictions

3、Remove middle men

4、Distributed

5、Ecosystem

6、Evolutionary Service Platform

7、Social Centric

8、Users’ Interests Always #1

THANK YOU