18-847f: special topics in computer systems foundations of ...gaurij/18-847f... · big data gold...
TRANSCRIPT
![Page 1: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/1.jpg)
1
18-847F:SpecialTopicsinComputerSystems
FoundationsofCloudandMachineLearning
Infrastructure
![Page 2: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/2.jpg)
2
Lecture1:IntroductionandLogistics
FoundationsofCloudandMachineLearning
Infrastructure
![Page 3: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/3.jpg)
GraduateSeminarClass
3
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject(NoExams!)
![Page 4: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/4.jpg)
LearningObjectives
4
o Knowthestate-of-the-artframeworksincloudandmachinelearningandtheirtheoreticalfoundations
o Readandprovideconstructivecriticismofresearchpapers
o Presenttoanaudience,andanswertheirquestions
o Docreative,collaborateresearch
![Page 5: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/5.jpg)
WhystudyCloudandMLinfrastructure?
5
Whatarethelargestwordsafter‘BigData’?
![Page 6: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/6.jpg)
BigDataGoldRush
6
WhogotrichintheCaliforniagoldrush?
![Page 7: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/7.jpg)
BigDataGoldRush
7
WhogotrichintheCaliforniagoldrush?
IntheBigDatarush,it’stheinfrastructurecompanies
![Page 8: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/8.jpg)
TopicsCovered
8
CloudCompu)ng DistributedStorage
MachineLearning
Modelreplica
PARAMETERSERVERw’=w–αΔw
Modelreplica
Modelreplica
w Δw
a b a+b
![Page 9: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/9.jpg)
TopicsCovered
9
CloudCompu)ngo SchedulinginParallelComputing
o MapReduce,Spark
o StragglerReplication
o TaskReplicationinQueueingSystems
o CodedMapReduce
![Page 10: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/10.jpg)
TopicsCovered
10
DistributedStorageo Codingforlocality/repair
o Systemsimplementationofcodes
o Reducinglatencyincontent
download
a b a+b
![Page 11: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/11.jpg)
TopicsCovered
11
MachineLearning
Modelreplica
PARAMETERSERVERw’=w–αΔw
Modelreplica
Modelreplica
w Δw
o SGDandSupportVectorMachines
o Backpropagation,LeNet,AlexNet,GoogleNet
o DistributedGradientDescent
o Hyper-parametertuning
o GANs,Deepreinforcementlearning
![Page 12: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/12.jpg)
TopicsCovered
12
CloudCompu)ng DistributedStorage
MachineLearning
Modelreplica
PARAMETERSERVERw’=w–αΔw
Modelreplica
Modelreplica
w Δw
a b a+b
![Page 13: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/13.jpg)
Instructor:GauriJoshi
13
SM+PhD2010-2016
B.Tech+M.Tech2005-2010
ResearchStaffMember2016-2017
AssistantProfessorFall2017-
Internships
![Page 14: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/14.jpg)
Haveworkedinalltheseareas
14
CloudCompu)ng DistributedStorage
MachineLearning
Modelreplica
PARAMETERSERVERw’=w–αΔw
Modelreplica
Modelreplica
w Δw
a b a+b
![Page 15: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/15.jpg)
StudentIntroductions
15
o Name?
o Department?
o MastersorPhD?
o Previousrelatedclasses(ifany)?
o Whatyouarelookingtolearnfromthisclass?
![Page 16: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/16.jpg)
ClassHoursandWebsite(s)
16
o When:Mon,Wed4:30-6:00pm
o Where:ScaifeHall222
o ClassWebsite(Readings,Schedule):https://www.andrew.cmu.edu/user/gaurij/18-847F-Fall-2017.html
o CanvasSite(Readings,Assignments,Projects):https://canvas.cmu.edu/
![Page 17: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/17.jpg)
ReadingMaterial
17
PaperswillbepostedontheclasswebsiteoronCanvaso Bookchapters
o Surveypapers
o Theorypapers(Scheduling,Queuing,Coding,Optimization)
o Systemspapers(Cloud,MachineLearning)
Additionalreferencebookslistedinthesyllabus
![Page 18: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/18.jpg)
InstructorandOfficeHours
18
Prof.GauriJoshi,ECEDeptEmail:gaurij[AT]andrew.cmu.eduOfficeLocation:CIC4105OfficeHours:Wed2:00-3:00pmorbyappointment
![Page 19: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/19.jpg)
GraduateSeminarClass
19
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject
![Page 20: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/20.jpg)
Lectures
20
o Nextweek:DeeperOverviewofcoursetopics
o 3-4Guestlecturesduringthesemesterbyauthorsofpapersrelevanttothisclass
![Page 21: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/21.jpg)
GraduateSeminarClass
21
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject
![Page 22: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/22.jpg)
Homeworks(45%)
22
o Classdividedintotwogroups;readthepaperassignedtoyourgroup
o Submitpaperreview(due9:00ambeforeclass)o Tworeviewsperweek(advise:finishthemtogether!)
o Discussionwithyourgroupisokay,butwritereviewsinyourownwords.Listcollaboratorsinthehomeworksubmission
![Page 23: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/23.jpg)
PaperReviewFormat
23
o BriefSummaryofthepaper
o Reflectsyourunderstandingofthepaper
o Nojudgments(positiveornegative)here
o High-leveltechnical/writingcomments
o Significance&correctnessofresults(Don’tbemean!)
o Paperorganization
o Low-leveltechnical/writingcomments
o Smallerclarifications,corrections,typos
o DiscussionQuestionsforClass(atleast2)
o Confusionsaboutthepaper,openresearchdirections
![Page 24: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/24.jpg)
TentativeGradingRubric(Total:10pts)
24
o Clarity,Organization(1pt)
o Understandingofthepaper(5pts)
o High-levelcomments(2pts)
o Low-levelcomments(2pt)
![Page 25: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/25.jpg)
GraduateSeminarClass
25
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject
![Page 26: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/26.jpg)
ClassPresentations(20%)
26
o Onestudentfromeachgroupsignsupforpresentation,atleast1weekinadvance
o Eachstudentwillpresent~2timesinthesemester
o 20minpresentation,followedby25mindiscussiono MotivationandRelatedworko Summaryofmainresultso Yourviewsonthepaper
![Page 27: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/27.jpg)
TentativeGradingRubric(Total:10pts)
27
o Motivation(2pts)
o Clarity(2pts)
o Correctness(2pts)
o Engagingtheaudience(2pts)
o Extraresearch,goingbeyondthepaper(2pts)
![Page 28: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/28.jpg)
GraduateSeminarClass
28
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject
![Page 29: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/29.jpg)
TentativeGradingRubric(Total:5pts)
29
o Discussionquestions(2pt)
o Attendanceandattention(1pt)
o Speakingupinclass(1pt)
o InsightfulQuestions/Comments(1pt)
![Page 30: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/30.jpg)
GraduateSeminarClass
30
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject
![Page 31: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/31.jpg)
ResearchProject(25%)
31
o Groupsof2or3
o Originalresearchonatopicofyourchoiceo Topicsalignedwithyourresearchallowedandencouragedo Ifyoucan’tthinkoftopics,cometalktome!
o PossibleProjectTypes:o Newtheoreticalanalysiso Implementationusingoneoftheframeworksdiscussedo In-depthliteraturesurveyofaparticulartopic
![Page 32: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/32.jpg)
Timeline
32
o 1-pageproposaldueSept27
o Publishablequalityreport(max5pg)inACMformato Initialdraftdue:Nov22o Finalreportdue:Dec8
o Peer-review2otherreportso Last2weeksofclass:Presentations(20mineach)
![Page 33: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/33.jpg)
TentativeGradingRubric(Total:20pts)
33
o Originality(1pts)
o ReviewofRelatedWork(1pts)
o WritingandOrganization(2pts)
o TechnicalResults(5pts)
o Peer-ReviewofOtherreports(1pts)
o Finalpresentation(10pts)
![Page 34: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/34.jpg)
InSummary..
34
o PaperReadingo SubmittingReviewso ClassPresentations(~2inthesemester)o FinalProject
Mightseemlikealotofworkbut..o Youwillgetfastandefficientatreadingpaperso Theprojectwillbeafun,collaborativeexerciseo Noexams!
![Page 35: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/35.jpg)
TODO
35
o Formgroupsforclassreadings
o Sign-upforpresentation
o Formgroupsforclassprojects
o Startreadingthepapers
o Startthinkingaboutprojects
![Page 36: 18-847F: Special Topics in Computer Systems Foundations of ...gaurij/18-847F... · Big Data Gold Rush 7 Who got rich in the California gold rush? ... , Deep reinforcement learning](https://reader035.vdocuments.net/reader035/viewer/2022070823/5f2b412485ecf6061e40e6b1/html5/thumbnails/36.jpg)
NextClass
36
Historyandoverviewofcloudandmachinelearninginfrastructure
Willgiveyouadditionaltimeforthefirstpaperreviewsandpresentations