Identifying American Sign Language Attributes Using ASL Novices on Mechanical Turk

By Kyle Rector

My experience with ASL

English to ASL Dictionaries

Can ASL be put in a database?

Written forms of ASL

ASL to English Dictionaries

Can I use ASL novices?

Grosjean and Lane (1977) non signer can distinguish a pause for phrase boundaries or sentence ends

Gibet, Courty, Duarte (2011) ASL signer observed animations of French signed language to see if they could replicate LSF

Socially Computed Scripts to Support Social Problem Solving Skills

Real-Time Captioning by Groups of Non-Experts

AudioWiz: Nearly Real-time Audio Transcriptions

Better Vocabularies for Assistive Communication Aids: Connecting Terms using Semantic Networks and Untrained

Annotators

How are those last projects similar?

Identifying American Sign Language Attributes Using ASL Novices on Mechanical Turk

By Kyle Rector

What are ASL Attributes?

Identifying American Sign Language Attributes Using ASL Novices on Mechanical Turk

By Kyle Rector

Research Questions

– Is it possible to have ASL novices identify attributes of a sign?

– How accurate are ASL novices when identifying parameters?

– What attributes of a visual language are easier or harder to solve with lay people?

– What types questions are needed to get this data?

How can a novice do this?

• 33 Handshapes• 10 Locations• 9 Orientations• 30 Movements• 6 Relative Positions

Mechanical Turk

• 50 words• 72 questions• 3 people• 2 cents per person• $216 total• Answered overnight!

ASL Expert

http://asl2eng.cs.washington.edu

Same 50 wordsFinished over an average of 4.5 hours

Evaluation - differences

Handshape: #:54 %:3.27Location: #:34 %:6.8Orientation: #:213 %:47.33Movement: #:143 %:20.43Relation: #:44 %:14.67Total: #:488 %:13.56

Understanding the differences

Mturk: yes

ASL Expert: no

ASL Novice errors

• Completely missed it• Crowd undecided• Close resemblance• Confused relative position and movement

ASL Expert errors

• Completely missed it• Close resemblance• ASL Experience

Is this sustainable?

• “I liked that they were different and I'm interested in sign language so I found them interesting. I did these awhile back and just came back looking for more to do and found this! :)”

• “They were interesting and quick. I suppose if I was in contact with any one who used sign language I might pick up a few words.”

Future Work

• Use technology or ASL Experts to compensate– Orientation– Movement

• Only use possible combinations of attributes– How do we determine what these are?

Future Work

• Put this data into ASL to English dictionary• Close resemblance in hand shapes or locations– search metric

http://asl2eng.cs.washington.edu

Conclusion

• Getting data from novices, good or bad is useful for future applications

• Orientation is very hard for novices to comprehend

• ASL Experts entering a term in the dictionary seems easy, but can be costly

Thank you!

• Advisor: Richard Ladner• Research Assistant: Travis Smith• Grad students: Shiri Azenkot, Lydia Chilton,

Supasorn Suwajanakorn