final project presentationsintrotodeeplearning.com/2019/materials/2019_6s191_projects.pdf · [3]...
TRANSCRIPT
![Page 1: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/1.jpg)
Final Project PresentationsMIT 6.S191
February 1, 2019
![Page 2: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/2.jpg)
![Page 3: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/3.jpg)
6.S191 Project Group 1Varnika SinhaJulia WangEmily Zhang
![Page 4: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/4.jpg)
Problem
Cybersecurity Challenges:
❖ Unprecedented attacks❖ Cyber espionage❖ Data theft
Current security systems are not robust enough to adequately protect user data.
Group 1
![Page 5: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/5.jpg)
Solution & Algorithms
cat:mouse::attacker:defense
❖ Reinforcement Learning❖ Reward system❖ Policy gradient
Group 1
![Page 6: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/6.jpg)
Applications & Impact
❖ Password protection❖ Cloud security❖ Protection against malware
& viruses❖ Voting systems security❖ Internet of things security
Group 1
![Page 7: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/7.jpg)
![Page 8: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/8.jpg)
Identification of Heartbeat Systems in Photometric
Surveys
Baichuan Mo, Erik Tamre, Prajwal Niraula, Yunpo Li
6.S191Feb 1, 2019
![Page 9: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/9.jpg)
Thompson et al. 2012
Heartbeat Systems
![Page 10: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/10.jpg)
Scientific Motivations
Probing the Internal Stellar Structure
As method of finding exoplanet
Tidal Circularization
![Page 11: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/11.jpg)
Distinguishing the HeartBeat SystemsHB System Non - HB
![Page 12: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/12.jpg)
Proposed method
Deep Learning! But why?
We will try to run the models on both time domain and frequency domain
Method one: 1-D Convolutional Neural Network
Motivation: Extract local features in the time domain (e.g., peaks, valleys, etc.)
Method Two: Recurrent Neural Network (LSTM)
Motivation:
1. Time series input data
2. Different duration of each photometric survey : Handle variable length input
![Page 13: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/13.jpg)
Challenges and path forwardChallenges at current stage
More than 100,000 unlabeled observed data, small amount of labeled data
To push forward this work
Manual labeling vs. model labeling
Introduce data other than photometric survey
![Page 14: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/14.jpg)
![Page 15: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/15.jpg)
Advanced Scoliosis Detection with Deep Neural Nets
Group 3Sandra LiuEric MagliarditiNathan Rebello
![Page 16: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/16.jpg)
Scoliosis is an abnormal lateral spinal curvature
● Starts before 15 yrs. old
● 600,000 patient visits/year
● 30,000 children fitted w/brace
● 40,000 undergo spinal fusion
surgery
Brace May 2014-July 2015
https://www.aans.org/Patients/Neurosurgical-Conditions-and-Treatments/Scoliosis
![Page 17: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/17.jpg)
How do we map early signs to advanced scoliosis?
Mild● No predictive methods
● Costs:
○ $5K on bracing
○ $100K/surgery
○ $1K/year on checkups
● If detected, preventative
measures can improve posture
>10°>40°
Severe
https://health.costhelper.com/scoliosis.html
![Page 18: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/18.jpg)
Use Convolutional Neural Nets & Supervised Learning to predict severe scoliosis
Convolution Neural Net
1. Learns features present in patients who had mild
scoliosis but became severe after a time, T
3. Assigns weights to features
4. Outputs classification
Input Data:
X-Ray of patient with early signs of Scoliosis: Curvature
~10-20°
Labels: Future Scoliosis Severity (Mild, Moderate, Severe)
Class 1: Mild Scoliosis after time T
Class 3: Severe Scoliosis after time T
Class 2: Moderate Scoliosis after time T
*Classification Subject to Change
Classification
![Page 19: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/19.jpg)
Goal: CNN Detect Subtle Features that Trigger
Advanced Scoliosis
● Severe:
○ Spinal curving
● Subtle:
○ Uneven Shoulders
○ Ribs at different heights
○ Head not centered above
pelvis
○ Uneven waist
○ Hips raised high
Severe
![Page 20: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/20.jpg)
What are the challenges?
● Validation● Data Acquisition
○ 600,000 patient visits/year
○ Potential privacy issues with hospitals
![Page 21: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/21.jpg)
Further Applications
● Individual-specific physical therapy treatments● Potential using physician/therapist data with the deep neural
net to develop effective therapy treatments● Detecting features in X-ray images that might lead to severe
scoliosis
https://www.webmd.com/back-pain/what-are-the-symptoms-of-scoliosis#1
![Page 22: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/22.jpg)
Thank you!
![Page 23: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/23.jpg)
![Page 24: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/24.jpg)
One case every 3 seconds.131.5 million by 2050.
Costs above a $1 trillion in 2018.
![Page 25: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/25.jpg)
Music can help
...but which music works best?
...
![Page 26: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/26.jpg)
Infer Mood Personalized
music for therapy
Wristbands offer:- Activity (calories)- Temperature- Electrocardiogram
LSTM Feedback
Recommender system
![Page 27: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/27.jpg)
![Page 28: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/28.jpg)
Pred1ct: Predicting Useful CRISPR-Cas9 Outcomes
David Li, Joshua Park, Akshaj Kadaveru6.S191 Group 5
![Page 29: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/29.jpg)
CRISPR/Cas9 allows for targeted gene editing● CRISPR/Cas9 cuts at specific locations given a guide RNA● Cells have own mechanisms to repair cuts in DNA
![Page 30: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/30.jpg)
Existing Approaches● Library of guide RNAs (target different areas)● Treated cells containing target sequences with Cas9● Sequence and compare
![Page 31: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/31.jpg)
Pred1ct Network Architecture- Feedforward neural network- Input: 20 dimensional vector; each dimension can be one of four values
- (A, C, T, G)20
- Output: Percentage of insertions/deletions that are one nucleotide insertions- Value between 0 and 1
- Existing training set of ~42,000 sequences
... ... ......
![Page 32: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/32.jpg)
Consequences● Small frameshifts make up 24% of mutations that manifest in currently
recognized genetic disease.● Accurate prediction of +1 frequencies allows for the design of useful guide
RNAs that would allow correction of these diseases● For example:
○ Cystic Fibrosis○ Crohn's Disease○ Tay-Sachs Disease
![Page 33: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/33.jpg)
![Page 34: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/34.jpg)
Rational Design of Electrocatalysts
Nathan Ricke and Eric Alt6.S191Group 6
![Page 35: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/35.jpg)
Electrocatalysts for Storing and Recovering Energy
![Page 36: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/36.jpg)
Computationally Generate and Test Catalysts
LASSO Regression
![Page 37: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/37.jpg)
G
G
Smiles StringFC2ccc1nc(G)c(G)nc1c2
LSTM Neural Network
![Page 38: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/38.jpg)
![Page 39: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/39.jpg)
GANs for Automotive Exterior Design
Group 7: Yiwen Huang, Greg Allan, Michael Schmid
[2]
[1]
[3]
![Page 40: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/40.jpg)
Automotive Design - State of the Art
BMW 8 series concept sketch [4]
BMW 8 series concept car [5] BMW 8 series final design [6]
Attractive design requires creativityBUT there are no new elements on cars!
6.S191 GANs for Automotive Exterior Design
![Page 41: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/41.jpg)
Idea: GANs in Automotive Design
Generative Network ?Slowly increase or
decrease single latent variables (luxury, sports,
race)
Idea from lecture:Exploit the creativity of Generative Networks
design suggestions[7], [8], [9]
[10]
[11]
6.S191 GANs for Automotive Exterior Design
![Page 42: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/42.jpg)
6.S191 GANs for Automotive Exterior Design
Thank you for your attention!
[12]
![Page 43: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/43.jpg)
![Page 44: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/44.jpg)
Deep Reinforcement Learning for Radiation Therapy Planning
Group 8: Susu Yan (Listener), Michelle Jiang (Credit)MIT 6.S191 presentation
![Page 45: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/45.jpg)
Radiation Treatment Plan: Tumor and Organs-at-risk
Tumor
Esophagus
Spinal cord
![Page 46: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/46.jpg)
Radiation Treatment plan: Dose Distribution
3D distribution of dose in patient shown on CT
![Page 47: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/47.jpg)
Actions
![Page 48: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/48.jpg)
Actions (cont.)
There are thousands of parameters that can be modified to generate a radiation therapy plan.
![Page 49: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/49.jpg)
Reward: Minimizing or Maximizing Dose Values and Meeting Clinical Goals
The goal is to kill all tumor cells and minimize radiation damage to healthy tissues.
![Page 50: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/50.jpg)
Challenges
![Page 51: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/51.jpg)
Challenges ● Requires years of training and
experience ● Time consuming ● Never sure if a better plan
exists● Patient needs to be treated as
soon as possible
![Page 52: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/52.jpg)
Reinforcement Learning
Actions:Change parameters
Treatment Planning System
Reward: Meeting clinical goals
State:Dose in each voxel
CT Scan of Patient with segmentation of contours
Image adapted from https://adeshpande3.github.io/Deep-Learning-Research-Review-Week-2-Reinforcement-Learning
![Page 53: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/53.jpg)
Thank you!Group 8: Susu Yan (Listener), Michelle Jiang (Credit)
![Page 54: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/54.jpg)
![Page 55: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/55.jpg)
Diagnosing Defects in 2D Materials with Deep Learning
Nina & Jovana Andrejevic
Project Group 9
![Page 56: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/56.jpg)
Introduction
Project Group 9
2D materials exhibit tunable electronic and optical properties, exciting for development of next-generation electronic and optoelectronics devices
Quality is critical, but challenging to monitor
Raman spectroscopy provides one signature of material quality
Need a high-throughput technique for rapidly identifying and quantifying defects to satisfy industry-scale growth and processing
![Page 57: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/57.jpg)
Proposal
Project Group 9
- is resistant to artifacts produced by defects- distinguishes different materials in an unsupervised manner
We use an autoencoder to learn a compressed representation of materials’ signatures that
Can unsupervised deep learning automate the screening of “defect fingerprints”?
![Page 58: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/58.jpg)
Data Generation
Project Group 9
shift asymmetry
Defect Fingerprints
Example Spectrum
Generate 3 different materials classes with increasing complexity
![Page 59: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/59.jpg)
Project Group 9
![Page 60: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/60.jpg)
Preliminary Results
Project Group 9
Asy
mm
etri
c
Sh
ifte
d
P
rist
ine
Material 1
cluster by material
Material 2
Material 3
![Page 61: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/61.jpg)
Conclusion
Our preliminary results show:
Project Group 9
- the suitability of autoencoders for recovering salient features of Raman signatures corrupted by defects
- the network’s ability to learn a well-separated representation of different materials’ signatures in an unsupervised manner
Next steps:
- Train on experimental data, possibly supplemented by simulation- Quantify defect concentration
![Page 62: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/62.jpg)
References
[1] “Raman Spectroscopy Quality Control of New 2D Materials.” Spectroscopy Europe/Asia, 12 July 2017, www.spectroscopyeurope.com/news/raman-spectroscopy-quality-control-new-2d-materials.
[2] “Keras: The Python Deep Learning Library.” Keras Documentation, https://keras.io/.
Project Group 9
Thank you!
![Page 63: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/63.jpg)
![Page 64: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/64.jpg)
Using GANs in Filmmaking to replace traditional VFX
Baptiste, Nick, Suraj, Brandon - Group 10
![Page 65: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/65.jpg)
Current CGI Implementation- Generate 3D models
- Texture, lighting, and color
- Animate the CGI
- Avatar
- Music videos
![Page 66: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/66.jpg)
Current Uses of Machine Learning in VFX- CGI overlaid on real images
- Wire removal & green screen detection
- Rotoscoping
- Deepfakes
- Human faces created out of nothing
- Already large dataset
![Page 67: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/67.jpg)
![Page 68: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/68.jpg)
Group 11
![Page 69: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/69.jpg)
![Page 70: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/70.jpg)
![Page 71: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/71.jpg)
![Page 72: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/72.jpg)
![Page 73: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/73.jpg)
![Page 74: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/74.jpg)
![Page 75: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/75.jpg)
![Page 76: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/76.jpg)
![Page 77: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/77.jpg)
![Page 78: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/78.jpg)
![Page 79: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/79.jpg)
![Page 80: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/80.jpg)
![Page 81: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/81.jpg)
![Page 82: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/82.jpg)
![Page 83: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/83.jpg)
![Page 84: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/84.jpg)
![Page 85: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/85.jpg)
![Page 86: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/86.jpg)
![Page 87: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/87.jpg)
![Page 88: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/88.jpg)
![Page 89: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/89.jpg)
![Page 90: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/90.jpg)
![Page 91: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/91.jpg)
![Page 92: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/92.jpg)
![Page 93: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/93.jpg)
![Page 94: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/94.jpg)
![Page 95: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/95.jpg)
...
![Page 96: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/96.jpg)
![Page 97: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/97.jpg)
![Page 98: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/98.jpg)
![Page 99: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/99.jpg)
![Page 100: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/100.jpg)
![Page 101: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/101.jpg)
![Page 102: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/102.jpg)
![Page 103: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/103.jpg)
![Page 104: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/104.jpg)
![Page 105: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/105.jpg)
![Page 106: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/106.jpg)
![Page 107: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/107.jpg)
![Page 108: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/108.jpg)
![Page 109: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/109.jpg)
![Page 110: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/110.jpg)
![Page 111: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/111.jpg)
THANKS FOR LISTENING!Group 11
![Page 112: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/112.jpg)
![Page 113: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/113.jpg)
AI-assisted parentingZhenhua (Ray) Rui
& Kai Jin
Group 12
![Page 114: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/114.jpg)
parenting is a sophisticated job
Kids development
LanguageMotor skills
MentalHabits
...
![Page 115: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/115.jpg)
learn how kids learn and suggest the next move
6 month 12 month 18 month 24 month
“en”, “a”, ... “en”, “a”, “mama”
dance
“car” ...
“en”, “a”, “mama”
dancetempo
“car”,“pickup truck”
choose books...
![Page 116: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/116.jpg)
challenge
Can AI help parents raise better humans?
![Page 117: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/117.jpg)
![Page 118: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/118.jpg)
Using Deep Learning to assist Colorblind people
Victor HortaLuis Covatti
![Page 119: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/119.jpg)
Agenda
What is Color blindness, and why it matters?
What is the problem we are trying to solve?
Proposed solution
Potential applications
![Page 120: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/120.jpg)
What is Color blindness, and why it matters?
8% of men are colorblind [1]
1 in every 200 women is colorblind [1]
What number do you see?
Source: L. T. Sharpe, A. Stockman, H. Jagle, and J. Nathans, “Color vision: From genes to perception chapter Opsin genes cone photo pigments,” in Color Vision and Color Blindness. Cambridge, U.K.: Cambridge Univ., 1999, pp. 3–51.
![Page 121: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/121.jpg)
What is Color blindness, and why it matters?
8% of men are colorblind [1]
1 in every 200 women is colorblind [1]
Source: L. T. Sharpe, A. Stockman, H. Jagle, and J. Nathans, “Color vision: From genes to perception chapter Opsin genes cone photo pigments,” in Color Vision and Color Blindness. Cambridge, U.K.: Cambridge Univ., 1999, pp. 3–51.
![Page 122: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/122.jpg)
How can we generate image filters that increase the world readability by slightly tilting colors, while still maintaining them as true to their original as possible?
The problem
Colorblind people usually can distinguish standalone colors
But things get harder when certain shades are very small and close to each other
Red Green
???? Easy!
Surroundings seem to matter
![Page 123: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/123.jpg)
What has been done so far?
Mechanical solutions Recoloring algorithms
“The results show that the glasses introduce a variation of the perceived color, but neither improve results in the diagnosis tests nor allow the observers with CVD to have a more normal color vision.” [1]
Source: [1] https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-22-28693 [2] Jimmi Lee, Wellington P. dos Santos. An Adaptive Fuzzy-Based System to Simulate, Quantify and Compensate Color Blindness [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color blindness
Drastic change in original colors (unnatural)Do not preserve image details
Enchroma glasses Aim to improve color differentiation
Deep Correct[3]
Original
Adaptive Fuzzy[2]
Corrected
![Page 124: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/124.jpg)
The model/system
Input image
Output image
Loss-function dependent on:
Interpretability by colorblind people Assessed by mathematical transformation and field tests (feedback)
Distortion from original image
max
min
[Illustrative]
CNN
Using colors within the domain of
visualization for colorblind people
![Page 125: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/125.jpg)
Potential applications
Visual Arts(paintings, movies)
Entertainment(games)
Internet(maps readability, web design)
Improve accessibility to digital content
![Page 126: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/126.jpg)
Thank you
![Page 127: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/127.jpg)
![Page 128: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/128.jpg)
All Dolled Up:How Deep Learning Can Teach Children to Love ThemselvesBy: DIna Atia, Faduma Khalif, Yousef Mardini
Group 17
![Page 129: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/129.jpg)
Background● National Black Doll Museum● Topsy Turvy Dolls ● Mattel Barbie: 1968, 1980
![Page 130: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/130.jpg)
All Dolled Up:● Learns what you look like ● Maps your features to
doll feature set
● Doll that looks like you!
![Page 131: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/131.jpg)
Methodologycollecting ground truth data
training a set of Classifiers
applying the most accurate classifier to raw, unannotated data.
making the necessary corrections to focus on the weak points of the classifier
![Page 132: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/132.jpg)
The ClassifierA CNN
![Page 133: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/133.jpg)
Thanks For Listening!!Please Give Us Prizes
![Page 134: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/134.jpg)
![Page 135: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/135.jpg)
Background● National Black Doll Museum● Topsy Turvy Dolls ● Mattel Barbie: 1968, 1980
![Page 136: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/136.jpg)
Customized interior design using generative models
Group 18: Keran Rong ; Mia Hong
![Page 137: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/137.jpg)
Customized interior design using generative models
Interior design Current situation:
• A high demand in market
• High expertise is needed
• Fashion – sensitive
•Customized design is expensive
![Page 138: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/138.jpg)
![Page 139: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/139.jpg)
Customized interior design using generative models
![Page 140: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/140.jpg)
Thank you! Any Questions?
![Page 141: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/141.jpg)
![Page 142: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/142.jpg)
Neural Networks as an early-stage Architecture Design & Sustainability Tool
Group 23Yu Qian Ang (credit)Klo’e Ng (listener)
6.S191
![Page 143: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/143.jpg)
PROBLEMLARGE AMOUNT OF WASTE
THE BUILT ENVIRONMENT:
>3 Billion tonnes of raw materials consumed annually
ENERGY INTENSIVEBuilt environment consumes >30% to 70% of total primary energy use
CARBON EMISSIONSBuildings are key contributor to greenhouse gas emissions around the world
Group 23
![Page 144: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/144.jpg)
EARLY STAGE DESIGN IN BUILT ENVIRONMENT
WHY MACHINE LEARNING / NEURAL NETWORK:
- Human design inherently subjective
- Opportunity for impact downstream (enhance sustainability)
- Insufficient time/manpower to explore many design options
- Human error, blind-spots, and bias
PROPOSED APPLICATIONGroup 23
![Page 145: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/145.jpg)
GENERATIVE ADVERSARIAL NETWORKS (GAN)PROPOSED APPLICATION
A Identify key parameters/features to optimise
B Design and develop GAN model
(Low) carbon footprint
(Low) energy use/wastage
Cost efficiency
(Low) material wastage
Group 23
![Page 146: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/146.jpg)
GENERATIVE ADVERSARIAL NETWORKS (GAN)
GENERATIVE NETWORK
MODEL
MODEL GENERATESCONCEPTUAL DESIGN OPTIONS
PROPOSED APPLICATIONData | Samples
Latent Space
12
3
Data/samples from past projects, or design options developed manually
Feed data into model to train iteratively, aim to minimize loss
Group 23
![Page 147: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/147.jpg)
IMPACT & CHALLENGES
POTENTIAL IMPACT
more/better building design options
Enhanced sustainability Less wastage Less human
error/bias
CHALLENGES
Validation of GAN outputs
(may need to run physics based simulations)
Contextualizing the GAN outputs
(architecture is sometimes highly contextualized)
Mode collapse: generator keeps generating similar designs
(limited diversity of samples)
Group 23
![Page 148: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/148.jpg)
SIMILAR APPROACHES/APPLICATIONS
1 2 3Generating Anime Girls
Jin et al (2017)(Fudan & Carnegie Mellon)
Ma et al (2018)(KU-Leuven & ETH Zurich)
Generating Pose-guided Apparel
CycleGAN: Generating photos from paintings etc
Zhu et al (2017)(UC Berkeley)
Group 23
![Page 149: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/149.jpg)
THANK YOU
![Page 150: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/150.jpg)
![Page 151: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/151.jpg)
![Page 152: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/152.jpg)
DEEP DOODLEDEEP LEARNING METHODS TO GENERATE SKETCHES FROM LABELS
![Page 153: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/153.jpg)
WHAT DEEP LEARNING CONCEPTS ARE WE USING?
![Page 154: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/154.jpg)
Semantic segmentation of one cat image from a dataset used to train image recognition neural nets
Some cat doodles from Google QuickDraw
![Page 155: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/155.jpg)
WHAT DEEP LEARNING CONCEPTS ARE WE USING?
![Page 156: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/156.jpg)
POSSIBLE APPLICATIONS
• Used in cognitive interviews
• Animating children’s books in real time.
• Assistive technology for teaching kids to draw.
![Page 157: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/157.jpg)
![Page 158: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/158.jpg)
Demo
https://colab.research.google.com/drive/1hHMR3Q2-Iugs8fb5GXG3SGmwXkMpo2FC
https://magenta.tensorflow.org/assets/sketch_rnn_demo/index.html
![Page 159: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/159.jpg)
![Page 160: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/160.jpg)
Deep Learning in Major League Baseball
Maximilian Porlein and Jack Phifer, MIT 2022
![Page 161: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/161.jpg)
Inspiration
MLB Beat the Streak: choose up to two Major League players daily. String together a 57-game hit streak to beat Joe DiMaggio’s record of 56 games and you win $5.6 million dollars. If either of your players goes hitless, you start all over.
![Page 162: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/162.jpg)
Why not just pick the top hitters in the MLB?
![Page 163: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/163.jpg)
Why not just pick the top hitters in the MLB?
- Too many factors can influence a player’s ability to make a hit
![Page 164: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/164.jpg)
Why not just pick the top hitters in the MLB?
- Too many factors can influence a player’s ability to make a hit- Pitcher:
Source: FanGraphs PITCHF/x, Feb 2011
![Page 165: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/165.jpg)
Why not just pick the top hitters in the MLB?
- Too many factors can influence a player’s ability to make a hit- Weather:
- Air temperature can change a baseball's trajectory- Air density can play a role in how far a ball travels- High and low temperatures can affect a pitcher's grip- Cloud coverage can affect how players see the ball- Windy conditions
Source: Alan Nathan, University of Illinois Department of Physics
![Page 166: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/166.jpg)
Why not just pick the top hitters in the MLB?
- Too many factors can influence a player’s ability to make a hit- Location:
- Home team advantage
Source: SB Nation, Jan 2011
![Page 167: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/167.jpg)
Our Model - Formulation● 16 Different Variables
○ Data compiled from several different sources including mlb.com and baseball-reference
○ Most categories didn’t directly exist but were compiled in python○ All categories were scaled to a value between 0 and 1
![Page 168: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/168.jpg)
Our Model - Implementation
● Used 1 hidden layer○ Consisted of 28 nodes
● Outputted probability of a hit● All active players are fed into
NN and player with the highest output is selected for that night
![Page 169: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/169.jpg)
Our Model - Conclusion
● Model was trained using data from 2016 and 2017 season○ 2018 was used as the testing data
● Model was largely unsuccessful as it could never put together high streaks○ Highest streak was 9
● Reasons the model fell short○ Baseball is an imperfect game with human variation○ Not enough testing data (data for previous years wasn’t as accessible)○ Not enough training time
![Page 170: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/170.jpg)
Extensions of Our Model
- Allows individual teams to choose rosters before playing specific teams
- Teams can use to determine their most consistent players- Can be used in recruitment for colleges and teams
- See which players are most consistent
![Page 171: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/171.jpg)
Extensions of Our Model
MLB Beat the Streak:
- Even if the model is not completely perfect…- MLB frequently gives “off-day” exceptions to streaks longer
than 10-15 days - Prizes (like merchandise) are still awarded to streaks as
short as 5 days
![Page 173: Final Project Presentationsintrotodeeplearning.com/2019/materials/2019_6S191_Projects.pdf · [3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color](https://reader033.vdocuments.net/reader033/viewer/2022041713/5e495ccf478260013e214d95/html5/thumbnails/173.jpg)
Final Project PresentationsMIT 6.S191
February 1, 2019