mmai final ohmytype

1

Oh! My TypeFind your ideal type with a smart

website

Prof. Winston H. HsuTA.   Yan-Ying ChenTeam 8 洪三權 周融瑋 張衡

2012.01.10 NTU MMAI course term project presentation

2

Outline• Motivation

o Why are we doing this?

• Existing systemo Why existing system sucks

• Observationo What things that people care about

• System introductiono What methods are we using

• Website Demoo How’s the website

• Evaluationo What are people thinking about

• Conclusiono Is it good?o Q&A

You are my cup of tea

3

Why are we doing this?

Most of the time,we are looking for a soul mate.

4

Why are we doing this?

• Most of the time, we are seeking for another part of our life

• Sometimes… we just fell boring XD

State of the art• Blog

o Wretcho Beauty wall o Yam blog

• BBSo ptt [Beauty] [NTUBeauty]o ptt2 [ZZZZZZZZZ9] o dispBBS [Beauty#Beauty]

• Newso 今日我最美o News paper[Star news]

• Movie & TV…

5

6

Observation• People love to look around• But they tend to meet new friends in familiar

place or familiar social circles

7

What we need is …• Look for date• More option• Familiar social circles• Find their own type

8

We present you

Oh! My type

http://ohmytype.csie.org/beta/

9

Oh! My type• It’s a website based on plurk social network • You can browse all the photos and guess your

type• You may also link to someone’s blog or wretch

10

We gatheralmost 50000 plurk account

• 9800/50000 use portraits• we also gather their friends, social relation.• And we try to find their wretch Album• Then, we do a little experiment on their photo

and social relationship for recommendation

11

How do we do it?• Image feature• Social Network Degree (friends)• Recommendation• UI design

12

Image feature [Total histogram]

• 195*195 pixel , RGB 32 HD colors• Each pixel has total 8*3 features

o R8 B8 G8

• Calculate histogram

R0R2

R4R6

G0G2

G4G6

B0B2

B4B6

CE/ 通用格式

CE/ 通用格式

CE/ 通用格式

total Histogram

13

Image feature [Face histogram]

• Using tools to find face• Restore to 50*50 pixel, HSV 32HD colors • Using mask to block that should not be face

• Calculate histogram (16*3 features )

-

14

Image feature [Eigenface]

• Base on 50*50 pixel, 8 bytes color • Calculate Eigenface• Restore first 100 Eigen Vector

• Eigenface =[ 1.00755753e+002, 9.59269257e+001, 9.29569550e+001,8.94374390e+001,… ]

15

Social network degree • How many friends does he/she have• More friends gets more score

16

Recommendation• Feature vector

o EX: FEATURE 5566 = [ [Total_Color_Histogram_24 ],[Face_Color_Histogram_48 ],[eigenface_100 ][Social_dgree_1 ],[Sex_1 ] ]

• Relevance feedbacko FEATURE search = FEATURE new *(0.7) + FEATURE old(sum) *(0.3)

• Calculate DISTANCE o FEATURE search = [ [V_a1] ,[V_a2],[V_a3].[V_a4],[V_a5] ]o FEATURE 5566 = [ [V_b1] ,[V_b2],[V_b3].[V_b4],[V_b5] ]

o DISTANCE search_to_5566 = Distance (Search ,5566 )

17

UI design• Simple photo wall• Single click to select• Drop out for abandon

18

Demo• Once again. We emphasize that

relationship is base on inside, not outside.

19

User study• For user study, we make two version to compare.• 1. Optimal version (16)

o Recommended base on selection

• 2. Random version (6)o Just random

• Age : 18~30• Career: 21 student ,1 Teacher

性別 - 性向統計 (Optimal)男 - 女男 - 男女 - 女女 - 男男 - 其他其他 - 其他

性別 - 性向統計 (Random)男 - 女男 - 男女 - 女女 - 男男 - 其他其他 - 其他

20

User study我明明就選女生啊，為什麼推出來的還是男生居多哩 XDDDD

就前三排而言，我覺得結果沒有很準，我會看得出來被列出來的圖片們的相似之處，但 .. 也許是帥哥 / 美女的 feature ，不僅僅是在五官的相似上，換句話說 .. 五官長得像帥哥 / 美女的人，不一定也是帥哥 / 美女另外，有個疑問是如果把不相關的圖片刪掉，會增進系統的結果嗎？最後，建議關於刪除圖片的功能，建議可以在旁邊說明一下，或者新增一個移除按鈕在圖片上

good job! *4

可以透過照片看出幽默風趣！！？

XD" 所以對某特定圖片點一次 MORE ，該張圖片的 rank 應該要提升對嗎 ? 可是 .. 怎麼好像有點亂跑，雖然通常都還是會在前三排，這是正常的嗎？

另外，建議可以放一點點使用解說在旁邊，譬如說 .. 圖片是可以拉到旁邊丟掉的 XD"

21

User study

我明明就選女生啊，為什麼推出來的還是男生居多哩 XDDDD

就前三排而言，我覺得結果沒有很準，我會看得出來被列出來的圖片們的相似之處，但 .. 也許是帥哥 / 美女的 feature ，不僅僅是在五官的相似上，換句話說 .. 五官長得像帥哥 / 美女的人，不一定也是帥哥 / 美女另外，有個疑問是如果把不相關的圖片刪掉，會增進系統的結果嗎？最後，建議關於刪除圖片的功能，建議可以在旁邊說明一下，或者新增一個移除按鈕在圖片上

good job! *4

XD" 所以對某特定圖片點一次 MORE ，該張圖片的 rank 應該要提升對嗎 ? 可是 .. 怎麼好像有點亂跑，雖然通常都還是會在前三排，這是正常的嗎？

另外，建議可以放一點點使用解說在旁邊，譬如說 .. 圖片是可以拉到旁邊丟掉的 XD"

可以透過照片看出幽默風趣！！？

Random Optimal

22

Conclusion• Girls love to use Photoshop (CS4 )

o And take photo in optimal angle

• Boys love to put on their female idols imageo Some of them also put on CG anime images

• Initial photos are importance to final result

• UI can help users to do relevance feedback

• Face organs is hard to detect

23

Thank you!Q&A time