crowdsourcing speech data science and ai
Post on 22-Jan-2018
244 Views
Preview:
TRANSCRIPT
Daniela Braga, PhDCEO
daniela@definedcrowd.com
DefinedCrowd: Crowdsourcing,
Speech Data Science, AI
Crowdsourcing Week, June 15th 2017
definedcrowd confidential 15
The challenges of crowdsourcing NLP data
Crowd quality Data quality
• Language tests• Job specific tests• Real Time Audits• Built-in language/spam
validators
• Referral system• System of tokens• Legal/privacy compliance
(under NDA)
Quality gateways
Controlled crowd
• Checking for suspicious crowd behavior (multiple accounts creation, peaks of activity, specific job spam, IP check against country of living)
Machine Learning
Data quality control
• Validation steps• Inter-annotator
agreements• Precision and Recall
metrics
definedcrowd confidential 16
DefinedCrowd combines the best of professional services with SaaS companies
top related