on-demand logistics at doordash how machine learning powers · how machine learning powers...
TRANSCRIPT
![Page 1: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/1.jpg)
How Machine Learning PowersOn-Demand Logistics at DoorDashGary Ren, Machine Learning Engineer
March 25, 2020
![Page 2: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/2.jpg)
Last mile, on-demand logistics
![Page 3: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/3.jpg)
Last mile, on-demand logistics
![Page 4: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/4.jpg)
Machine Learning at DoorDash
Logistics Engine
Reinforcement Learning for Logistics
Impact of GPUs
Outline
![Page 5: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/5.jpg)
Machine Learning at DoorDash
![Page 6: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/6.jpg)
Merchants
ConsumersDashers
Marketplace
![Page 7: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/7.jpg)
Merchants
ConsumersDashers
Convenience
Selection
Reach
Revenue
Flexibility
Earnings
Marketplace
![Page 8: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/8.jpg)
Merchants
ConsumersDashers
ML at DoorDash
Recommendations/PersonalizationSearch RankingDemand Distribution
Supply/DemandDynamic PricingDelivery Time
AssignmentsTravel EstimatesHotspots
![Page 9: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/9.jpg)
Merchants
ConsumersDashers
ML at DoorDash
Recommendations/PersonalizationSearch RankingDemand Distribution
Core DispatchBatching AlgorithmsHotspots
![Page 10: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/10.jpg)
Merchants
ConsumersDashers
ML at DoorDash
Recommendations/PersonalizationSearch RankingDemand Distribution
Core DispatchBatching AlgorithmsHotspots
![Page 11: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/11.jpg)
Merchants
ConsumersDashers
ML at DoorDash
Supply/DemandDynamic PricingDelivery Time
![Page 12: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/12.jpg)
Merchants
ConsumersDashers
ML at DoorDash
AssignmentsTravel EstimatesHotspots
![Page 13: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/13.jpg)
Merchants
ConsumersDashers
Food Prep TimeSelection IntelligenceParking Prediction
Lifetime ValueFraudPromotions
Pay CalculationSupply ForecastingIncentives
Recommendations/PersonalizationSearch RankingDemand Distribution
Supply/DemandDynamic PricingDelivery Time
AssignmentsTravel EstimatesHotspots
ML at DoorDash
![Page 14: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/14.jpg)
Logistics EngineThe AI system that powers deliveries on DoorDash
![Page 15: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/15.jpg)
Merchants
ConsumersDashers
Logistics Engine
Recommendations / PersonalizationSearch rankingDemand distribution
Supply/DemandDelivery TimeDynamic Pricing
AssignmentsTravel EstimatesHotspots
![Page 16: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/16.jpg)
Merchants
ConsumersDashers
Logistics Engine
Recommendations / PersonalizationSearch rankingDemand distribution
Supply/DemandDelivery TimeDynamic Pricing
AssignmentsTravel EstimatesHotspots
![Page 17: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/17.jpg)
Goal: Fast and efficient deliveries
On-time delivery to consumer
Increase marketplace efficiency
Logistics Engine
![Page 18: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/18.jpg)
Logistics Engine
Balance Supply/Demand Dasher/Delivery MatchingPlan Routes
![Page 19: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/19.jpg)
Logistics Engine
Balance Supply/Demand Dasher/Delivery MatchingPlan Routes
![Page 20: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/20.jpg)
Undersupply● More deliveries than Dashers
Dasher incentives● Bonus $ per delivery
Upfront Dasher communications● Dashers can plan ahead
Supply Demand Balance
![Page 21: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/21.jpg)
Complexity
Weather, holidaysPromotions, events
VarianceForecasting
Future demandFuture supply
Every regionEvery time of day
Challenges
Region $0 $1 $2 ...
1
2
3
...
![Page 22: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/22.jpg)
Machine Learning (ML)● Predict demand● Predict supply by incentive amount
Operations Research (OR)● Given predictions from ML and
business goals● Determine cost function ● Determine constraints● Solve using mixed integer
programing
ML meets OR
![Page 23: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/23.jpg)
Logistics Engine
Balance Supply/Demand Dasher/Delivery MatchingPlan Routes
![Page 24: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/24.jpg)
Logistics Engine
Balance Supply/Demand Dasher/Delivery MatchingPlan Routes
![Page 25: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/25.jpg)
In plain English● Pick the best Dasher for each delivery
In canonical operations research● Vehicle Routing Problem
DoorDash specific considerations● Real-time fulfillment● Optimize supply for future demand
Optimal Matching
![Page 26: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/26.jpg)
Complexity
Merchant operationsTraffic, weather
VarianceTime constraint
Asap deliveryReal-time demand
CombinatoricsDelivery constraints
Challenges
![Page 27: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/27.jpg)
Machine Learning (ML)● Predict food prep times● Predict travel times● Predict delivery times
Operations Research (OR)● Given predictions from ML, deliveries, Dashers,
and business goals● Determine cost function● Determine constraints● Solve using mixed integer programming
ML meets OR again!
![Page 28: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/28.jpg)
Cost function
Constraints
Mixed integer programming
Optimal matching
Delivery data Location Type Size
Dasher data Location Capacity Constraints Equipments
Total delivery duration
Dasher travel duration
Food prep duration
Dasher assignment
Data Predictions Optimizations Actions
Optimal Matching: Summary
![Page 29: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/29.jpg)
Total delivery duration
Dasher travel duration
Food prep duration
Reinforcement learning?
Delivery data Location Type Size
Dasher data Location Capacity Constraints Equipments
Dasher assignment
Data Predictions Optimizations Actions
Self learning optimization?
![Page 30: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/30.jpg)
Reinforcement Learning for Logistics
![Page 31: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/31.jpg)
● Agent: The entity that takes actions and tries to learn the best policy.
● Environment: The world that the agent interacts with.
Quick Context
![Page 32: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/32.jpg)
Quick Context
![Page 33: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/33.jpg)
● State: The current status of the environment. It represents all the information needed to choose an action.
● Action: The chosen move out of the set of all possible moves at a state.
● Reward: The feedback as a result of the action. Note that rewards can be immediate or delayed.
● Policy: The strategy used to choose an action at each state.
Quick Context
![Page 34: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/34.jpg)
Quick Context
![Page 35: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/35.jpg)
● State: Deliveries and Dashers
● Action: Assignment algorithms
● Reward: Delivery speeds and Dasher efficiency
● Agent: Deep neural network
● Environment: Assignment simulator
For DoorDash Logistics
![Page 36: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/36.jpg)
For DoorDash Logistics
![Page 37: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/37.jpg)
● 6 second improvement in delivery speed
● 1.5 second improvement in Dasher efficiency
Results
![Page 38: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/38.jpg)
Impact of GPUs
![Page 39: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/39.jpg)
● CPU → GPU: 10x improvement
● Single GPU → Multiple GPUs: 3x improvement
Faster training speeds
![Page 40: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/40.jpg)
Merchants
ConsumersDashers
Food Prep TimeSelection IntelligenceParking Prediction
Lifetime ValueFraudPromotions
Pay CalculationSupply ForecastingIncentives
Recommendations/PersonalizationSearch RankingDemand Distribution
Supply/DemandDynamic PricingDelivery Time
AssignmentsTravel EstimatesHotspots
More ModelsMore Experiments
![Page 41: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/41.jpg)
Takeaways
● Machine learning has many use cases at DoorDash
● Machine learning + operations research help efficiently solve supply demand balance and optimal matching problems
● Reinforcement learning fits well and has potential in logistics
![Page 42: On-Demand Logistics at DoorDash How Machine Learning Powers · How Machine Learning Powers On-Demand Logistics at DoorDash Gary Ren, Machine Learning Engineer March 25, 2020](https://reader033.vdocuments.net/reader033/viewer/2022052018/6031168e4138813c7e129b30/html5/thumbnails/42.jpg)
Articles / Talks
● DoorDash Engineering Blog (doordash.engineering)○ Optimal Matching○ Reinforcement Learning○ Personalization○ Experimentation
● Other Talks○ Software Engineering Daily○ O’Reilly AI○ QCon NY