A Crowd-sourcing Based Mobile Image Translation and
Knowledge Sharing Service
Department of Computer ScienceWaseda University, Tokyo, Japan
1Helsinki Institute for Information Technology
2Eindhoven University of Technology
Yefeng Liu, Vili Lehdonvirta1, Mieke Kleppe2, Todorka Alexandrova, Hiroaki Kimura, Tatsuo Nakajima
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus
• Introduction
• Human Mobile Image Translation
• Preliminary Study
• Discussion
• Future Directions
Outline
2
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus
Introduction
A menu board outside a restaurant, Tokyo
3
“...I can’t wear tie here?? Should I
take off my tie?..”
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus
Real World Problem
• Digital pocket translators or online translation services are useless if you don’t know how to input the characters.
4
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus
(Typical) Mobile Image Translation
Image Text
OCR Optical Character Recognition
MT Machine Translation
Irregular fonts or formats, handwriting, etc.
Poor performance
English Text
5
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus
Image Text
Question of the image
TranslatorCommunityOutsourcing
English Text
• Better quality in text recognition and translation
• Human worker can provide richer interpretations and responses in addition to literal answers.
Our Solution: Human Mobile Image Translation
6
Crowdsourcing
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus
Image Based Translator + Mobile Q&A
• NOT only a translator
• But also a knowledge broker that allows users to share high level information pertinent to the situation at hand, e.g.
• advices
• instructions
• suggestions
7
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus
Basic work-flow overview
Open call
etc.
Requester Translators Requester
Scoring
EnglishKanji
Best answer
8
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus
Preliminary study
• A preliminary study and design research aims to
• verify the feasibility of the design
• identify real user requirements and design issues
9
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus
Preliminary study - MethodCollected around a hundred pictures/questions from potential users
Fifteen characteristic cases were selected from the collected images
Interviewed the requesters what kind of answers they were expecting
Assigned questions to invited translators
Interviewed translators for their feedbacks
Compared the results with the requesters’ expectations
10
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 11
Preliminary Study Cases - Example 1
“...how long do I have to wait?”
Information in the picture is insufficient to answer this question.
However, most of the repliers can still suggest an approximate waiting time according to their life experiences.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 12
Preliminary Study Cases - Example 2
“What are the events between 5th and 8th?”
Poor question text.
Some translators misunderstood the question, thus provided useless answers.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 13
Preliminary Study - Findings (1)
1. Communication between requester and worker.
Better communication Better understanding Better result
2. Question/Answer style
• Short, but clear (e.g clarify to what level of details is wanted);
• Question with choices is better;
• Asking for links (of image/web page/etc) is a good way to lower the difficulty and faster the response time.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 14
Preliminary Study - Findings (2)
3. “Tweet” and “keywords” style answer is preferred
- Many translators use English as 2nd or 3rd language, they oftenface the problem of being unable to explain in long sentence.
a). “Pork, spicy, famous chinese food”
b). “Twice cooked pork (huiguo rou)”
- meaningless if don’t know the name
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 15
Discussion (1)
1. Quality of outcome
Misunderstanding between requester and worker strongly affects quality of outcome.
- Requesters may use unclear or too complicate English.- Workers often are not native English speaker.
suggests a single reply can hardly be trusted.
- Human always make mistakes.
- Malicious replies.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 16
Discussion (2)
An additional proofreading phase.
open call
Requester Translators Requester
Scoring
EnglishKanji
Best answerProofreaders
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 17
Discussion (3)2. Different user types (user requirements)
Client Users
Short-term stay Long-term stay
Need immediate answer
Need immediate answer
Waitable Waitable
A B C D
may have different preference on the accuracy vs. timeliness trade-off
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 18
Future Directions (1)
1. Dynamical task allocation with real time requirement
i. capable for the task
ii. available for the task
In this study case, local context of the requester and background information of the worker is important to determine the capacity.
Not only about if the worker is free, but also involves other factors like expertise, properties of question, etc.
• Task is better be assigned to worker who is:
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 19
Future Directions (2)
2. Motivation and Incentive
Social and Intrinsic incentive: game play
A location-based mobile game is designed
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 20
Current Status
Early Test
Prototype Implementation
Redesign
Usability Test/On field studyRedesign
Preliminary Study
Design
This paper
(some images here..)
Thanks for your attention!
Distributed & Ubiquitous Computing Lab.Depart. of Computer Science, Waseda University
http://www.dcl.info.waseda.ac.jp/
Yefeng Liu, PhD candidate