f# data: making structured data first class citizens
DESCRIPTION
Slides from a talk at DCP 2014 workshop, about F# Data type providers. For live code samples, see the YouTube links in the slides!TRANSCRIPT
![Page 1: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/1.jpg)
F# DataMaking structured data first-class citizens
Tomas Petricek, University of Cambridge
Project homepage: http://fsharp.github.io/FSharp.DataGet in touch: @tomaspetricek | [email protected]
![Page 2: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/2.jpg)
F# Software Foundation
http://www.fsharp.org
software stackstrainings teaching F# user groups snippets
mac and linux cross-platform tutorials
F# community open-source MonoDevelop
contributions research support consultancy mailing list
![Page 3: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/3.jpg)
F# Data type providers
First-class dataCSV, REST, WorldBank…
R Type provider
Statistics & visualization5000 tested packages
www.fslab.org
![Page 4: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/4.jpg)
Deedle data frame
Data explorationIndexing and aggregation
F# Charting library
Simple & composable Interactive style
www.fslab.org
![Page 5: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/5.jpg)
What are type providers?Integrating WorldBank and R
http://youtu.be/7r2-B-5H_io
![Page 6: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/6.jpg)
The confusion of languages
![Page 7: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/7.jpg)
What are type providers?
![Page 8: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/8.jpg)
What are type providers?
![Page 9: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/9.jpg)
Type provider research questions
Data vs. Schema Laziness
Mapping to types
Schema inferenceSchema
inference
![Page 10: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/10.jpg)
Schema inferenceLoading Titanic data from CSV
http://youtu.be/yjBdZduc0ko
![Page 11: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/11.jpg)
Inferring primitive types
null int
⊤
bool
string decimal
float
𝜏1+𝜏2
![Page 12: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/12.jpg)
Structure inferenceWorking with XML and JSON data
http://youtu.be/_DjX0ybaXZYhttp://youtu.be/SkZBzlREOMo
![Page 13: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/13.jpg)
Inferring structured types
person { name :
string }person { name : string,
age : int }
person { name : string, age : int option }
[ { num : int } ]
[ { str : string } ]
[ { num : int option, str : string option } ]
int{ value :
int }
int + { value : int }
![Page 14: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/14.jpg)
Does it scale?Query movies using Apiary provider
http://youtu.be/-Am2uRUv39c
![Page 15: F# Data: Making structured data first class citizens](https://reader033.vdocuments.net/reader033/viewer/2022061300/54c651074a7959b1098b45ce/html5/thumbnails/15.jpg)
Conclusions
Inference from small-scale samples works!
Schema is (very) often missing
But data is (very) often regular
Check out F# Data and contribute!
Project homepage: http://fsharp.github.io/FSharp.DataGet in touch: @tomaspetricek | [email protected]