data science in dft tom ewing dept for transport · pdf file · 2017-08-10data...
TRANSCRIPT
Data Science in DfT Tom EwingPrincipal Data ScientistDept for Transport
About me
● Joined Government in 1996
● Became a Statistician in 2008
● Became a Data Scientist in 2015
● First 18 months as a data scientist: ○ Engaging○ Traction○ Prototypes
● Last 6 months as a data scientist:○ Team Building○ Development○ Delivery
Applying Data Science to Policy
● Public Consultations
● Cycling & Walking Investment Strategy (CWIS).
● Five Questions to the Public
● 4000 responses
● Variety of Media
Processing and Analysing the Results...
● Majority manually processed
● Time Consuming + Boring + Not digital
● No tools for analysis
● Contracting responsibility out
Applying Data Science
● Focus on ‘pain points’:○ Collation○ Analysis
● Our solution:○ Opens emails and extracts the content and metadata○ Opens any attachments:
■ MS Word (.doc , .docx)■ PDF (.pdf)■ Image (.jpeg , .png , .tiff , .bmp ….)
○ Extracts the data○ Saves everything into an Excel Spreadsheet○ Builds and visualises an LDA topic model
Data Science to the Rescue! Part 1 (Automation)
● Focus on ‘pain points’:○ Collation○ Analysis
● Our solution:○ Opens emails and extracts the content and metadata○ Opens any attachments:
■ MS Word (.doc , .docx)■ PDF (.pdf)■ Image (.jpeg , .png , .tiff , .bmp ….)
○ Extracts the data○ Saves everything into an Excel Spreadsheet○ Builds and visualises an LDA topic model
Prototype Demo:
https://goo.gl/9jsr7z
Alpha Demo: http://51.140.61.200:3838/janus/janus-
public/shiny-dashboard/
Innovation
● Digital Tools & Methodologies
● IMCreate / Hacktrain
● Internal Hackathon: DfT Hackclub
● DfT Hacks!
The Future...
● Recruiting & Expanding our team
● New Strategy: ‘The Data Science Vision’
● Data Science ‘Hub’
● New Projects (Aviation, Maritime, Rail, Mobile Phone Data etc.)
● DfT Hacks 2.0
Questions?
https://www.facebook.com/tom.ewing1
TomEwing1979
http://www.github.com/Tommo565