autolab workshop
TRANSCRIPT
![Page 1: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/1.jpg)
WORKSHOP7th of November, 2014
![Page 2: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/2.jpg)
Background
● Started in 2010 by Prof. Dave O’Hallaron○ 15-213 followed by 15-112
● Completely student-run
● Timeline:○ < 2010: Single machine○ > 2010: VM support○ > 2011: Autodriver○ > 2013: Rails upgrade and Tango Web API
![Page 3: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/3.jpg)
Some Numbers
● Approximately 2500 students use it every semester● 77 classes since 2010 between CMU and CMU-Qatar● 2 coding challenges● 2014 (as of 3:20PM today): 320,538 -> ~1034 jobs/day
2012 2013 2014
![Page 4: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/4.jpg)
Becoming Open Source
● Higher customizability● Larger adoption rate● Decoupling of two core components● CMU spirit● Add your own features!
○ PDF annotation○ Online quizzes○ Distributed Job Queue
![Page 5: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/5.jpg)
Overview of Front-end
● Ruby on Rails○ REST○ Model-View-Controller (MVC)
■ Model: Internal structure■ View: What user sees■ Controller: Supply objects to view for rendering,
page redirecting, etc.○ Javascript, CSS, etc.
![Page 6: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/6.jpg)
Overview of Front-end
● Important model relations
CUD: CourseUserDatumAUD: AssessmentUserDatum
User CUD Course
AssessmentAUD
Submission
![Page 7: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/7.jpg)
Overview of Front-end
● Resources & Routing○ RESTful CRUD○ In the process of switching to a more traditional
REST interface for models○ Each Resource Gets 7 routes by default
![Page 8: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/8.jpg)
Overview of Front-end
● For example, Users○ GET /users => index○ GET /users/new => new○ POST /users => create○ GET /users/:id => show○ GET /users/:id/edit => edit○ PATCH /users/:id => update○ DELETE /users/:id => destroy
![Page 9: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/9.jpg)
Overview of Tango
● External, standalone, job dispatch service● Stateless● Highly flexible● Web API
![Page 10: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/10.jpg)
Overview of Tango
Tango
creates directory
stores files
poll for output file
optional callback
open
upload
addJob
poll
callback
job queue
![Page 11: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/11.jpg)
Overview of Tango
TangoVM pool
(EC2/CMU)
rhel.img
rhel.img
rhel601.img
rhel210.img
worker thread initializeV
MwaitVM
copyIn
copyOut
destroyVM
runJob
vmms
job queue
![Page 12: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/12.jpg)
Overview of Tango
● Virtual Machine Management System (VMMS)● Extending Tango for OpenStack, Docker, Vagrant, Xen
etc
![Page 13: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/13.jpg)
Autograding
Autolab
Upon submission#validateHandin#saveHandin#autogradeAfterHandin- #createVm
Upon receiving Callback#autogradeDone#saveAutograde- #parseAutoresult
Tango
creates directory
stores files
poll for output file
optional callback
VM pool (EC2/CMU)
rhel.img
rhel.img
rhel601.img
rhel210.img
open
upload
addJob
poll
callback
worker thread initializeVM
waitVM
copyIn
copyOut
destroyVM
runJob
/courses/COURSE_ID/assessments/ASSET_ID/handin
.c
tangoREST
vmms
![Page 14: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/14.jpg)
Docs
● http://docs.autolab.cs.cmu.edu/
● https://github.com/autolab/Tango/wiki
![Page 15: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/15.jpg)
Getting Started...
● What’s the hardest part of joining a project?
o Setting up your environment!
● We’ve set up an environment for you!
● See the Autolab README for full instructions
o Later, you will need your own DB
o ...and possibly your own Tango for Autograding
![Page 16: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/16.jpg)
Getting Started...
1. Fork Autolab repo on github (https://github.com/autolab/Autolab.git)
1. You can’t push to our repo, so you need to fork your own
2. ssh [email protected]
3. git clone https://github.com/<github_id>/Autolab.git <your_andrewID>
4. cd <your_andrewID>
5. cp lib/autoConfig.rb.template lib/autoConfig.rb
6. cp config/database.yml.template config/database.yml
![Page 17: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/17.jpg)
6. In autoConfig.rb change:- RESTFUL_HOST = “greatwhite.ics.cs.cmu.edu”- RESTFUL_PORT = 3000- AUTOCONFIG_COURSE_DIR =
“/home/autolab/<andrew_ID>/courses”- RESTFUL_KEY = “demo”- RESTFUL_COURSELAB = “autolab-demo”
Getting Started...
![Page 18: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/18.jpg)
Getting Started...
7. In config/database.yml change:- database: <andrew_ID>_autolab_development- username: root- password: ‘autolab-demo’
8. Populate Autolab by running:- bundle exec rake db:create db:migrate - rake autolab:populate
9. Run Autolab!- rails s -p <fav_port_num_between_9000_and_9999>- 54.201.118.81:<port>- Log in via “Developer Login”, enter email “[email protected]”.
![Page 19: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/19.jpg)
Contributing to Autolab
1. Fork the repository and create a local clone of your forka. https://github.com/autolab/Autolabb. https://github.com/autolab/Tango
2. Assign yourself to a new or existing issue on the projecta. https://github.com/autolab/Autolab/issuesb. https://github.com/autolab/Tango/issues
![Page 20: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/20.jpg)
Contributing to Autolab
3. Create a new local branch “fix_issue” and commita. git checkout -b "fix_issue”b. git add .c. git commit -m “commit_msg. Fixes #<issue_num>”
4. Push the branch to GitHuba. git push origin “fix_issue”
![Page 21: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/21.jpg)
Contributing to Autolab
4. Submit a pull requesta. navigate to your fork of the projectb. click the large green “compare and review” button
5. We will review and merge useful pull requestsa. documentation and comments on pull requests are
useful in this regard
![Page 22: Autolab Workshop](https://reader035.vdocuments.net/reader035/viewer/2022062220/55cf0f20bb61eb601b8b46b9/html5/thumbnails/22.jpg)
Contributing to Autolab
6. Keep your forked repository synced with the projecta. git remote add upstream
https://github.com/autolab/Tango.gitb. git remote add upstream
https://github.com/autolab/Autolab.gitc. git checkout masterd. git fetch upstream e. git merge upstream/master