chalearn hep competitions isabelle guyon, chalearn and lri, universit paris sud, orsay for
DESCRIPTION
Recent challenges Challenges in Machine LearningTRANSCRIPT
![Page 1: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/1.jpg)
ChaLearnHEP competitions
Isabelle Guyon, ChaLearn andLRI, Université Paris Sud, Orsay
http://chalearn.org
for
![Page 2: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/2.jpg)
What is Chalearn?
• California non-profit (US tax exempt).• Organizes challenges since 2005.• Funded by government grants and industry sponsors.• 10 directors; 4 officers; dozens of collaborators.• CiML book series (JMLR reprints) and NIPS workshops.• Now 3 to 4 challenges per year in 4 main tracks:
– Automatic Machine Learning.– Computer vision: Looking At People (LAP) challenges.– Causality and network reconstruction.– High Energy Physics.
http://chalearn.orgChallenges in Machine Learning
![Page 3: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/3.jpg)
http://chalearn.org
Recent challenges
Challenges in Machine Learning
![Page 4: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/4.jpg)
Result submission
http://chalearn.org
Code
Challenges in Machine Learning
![Page 5: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/5.jpg)
Code submission on Codalab
http://chalearn.org
BIGDATA
User
Code
Results
Tasks
Scores
Challenges in Machine Learning
![Page 6: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/6.jpg)
Fully automatic machine learning without ANY human intervention
automl.chalearn.orgDecember 2014 – January 2016
$30,000 in prizes
Testing code submissionAutoML Challenge
>500 participants
![Page 7: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/7.jpg)
Challenge design
Challenges in Machine Learning http://chalearn.org
![Page 8: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/8.jpg)
Challenges in Machine Learning http://chalearn.org
• High impact problem.• Good data.• Enough data.• Clear objective.• Good metric. • Simple rules.• No IP contingencies (except for winners).
1. Define
![Page 9: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/9.jpg)
2. Implement
Challenges in Machine Learning http://chalearn.org
• Three-way data split (train/valid/test):– Immediate feed-back on validation set.– Final results on test set.
• Encourage sharing and novelty (coopetition):– Forum.– Worksheets or scripts.– Monitor downloads / likes / thanks.
• Type of submission:– Result.– Code.
• Beta testing.
![Page 10: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/10.jpg)
3. Attract
Challenges in Machine Learning http://chalearn.org
• Exciting problem description.• Tutorial.• Video.• Starting kit.• Bootcamp/hackathon.• Publication outlets.• Advertising.• Gamification.
![Page 11: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/11.jpg)
4. Reward
Challenges in Machine Learning http://chalearn.org
• Multiple prizes (phases and/or tracks).• Medium size prize pool (~ $5000-$10,000).• Travel awards.• Best paper awards.• Give credit and rewards to all actors.
![Page 12: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/12.jpg)
5. Harvest
Challenges in Machine Learning http://chalearn.org
• Open-source code (winners).• Fact sheets.• Post-challenge analyses.• Workshops (broader scope than just the
challenge theme).• Proceedings.• Crowd-sourced paper.
![Page 13: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/13.jpg)
Build/maintain/expand
Challenges in Machine Learning http://chalearn.org
Create recurring events
![Page 14: ChaLearn HEP competitions Isabelle Guyon, ChaLearn and LRI, Universit Paris Sud, Orsay for](https://reader033.vdocuments.net/reader033/viewer/2022052710/5a4d1b897f8b9ab0599bdeb9/html5/thumbnails/14.jpg)
Thank you!
http://automl.chalearn.org
DirectorsKristin Bennett, Rensselaer Polytechnic Institute, New-York, USAGideon Dror, Academic college of Tel-Aviv Yaffo, IsraelHugo-Jair Escalante Balderas, Nat. Institute of Astrophysics, Optics and Electronics (INAOE), MexicoSergio Escalera, University of Barcelona, SpainChih-Jen Lin, National Taiwan University, TaiwanFlorin Popescu, Fraunhofer Institute, Berlin, GermanyMehreen Saeed, FAST, National University of Computer and Emerging Sciences, Lahore Campus, PakistanDanny Silver, Acadia University, CanadaAlexander Statnikov, New York University, New-York, USAIoannis Tsamardinos, University of Crete, Greece
OfficersPresident: Isabelle Guyon, ClopiNet, California, USAVice-president: Gideon Dror, Academic college of Tel-Aviv Yaffo, IsraelSecretary: Vincent Lemaire, Orange Labs, FranceTreasurer: Kristin Bennett, RPI, USA
Sponsors