get good data out of bad @nacnudus spreadsheets garmonsway … · 2019. 5. 21. · duncan...
TRANSCRIPT
![Page 1: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/1.jpg)
Duncan Garmonsway
@nacnudus
Get good data out of bad spreadsheetsWith tidy tools
![Page 2: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/2.jpg)
https://twitter.com/albertocairo/status/963133927530483712
![Page 3: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/3.jpg)
https://twitter.com/albertocairo/status/963133927530483712
![Page 6: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/6.jpg)
![Page 7: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/7.jpg)
@sh
arla
gelfa
nd h
ttps:
//tw
itter
.com
/sha
rlage
lfand
/sta
tus/
9967
3912
7562
3915
52
@dv
augh
an32
http
s://b
log.
davi
svau
ghan
.com
/pos
t/tid
ying
-exc
el-c
ash-
flow
-spr
eads
heet
s-in
-r/
![Page 8: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/8.jpg)
https://www.brodrigues.co/blog/2018-09-11-human_to_machine/
![Page 9: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/9.jpg)
http
s://n
acnu
dus.
gith
ub.io
/nzm
igra
tion/
@kuri_pablo http://170.238.108.227/binhnd/RpWebEngine.exe/Portal?BASE=CPVHND2013NAC&lang=ESP
![Page 11: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/11.jpg)
https://www.mic.com/articles/160125/pokemon-go-cp-changes-balance-update-hits-every-pokemon-but-onehttps://www.reddit.com/r/TheSilphRoad/comments/5e6jev/silph_research_group_on_cp_balancing/https://docs.google.com/spreadsheets/d/1_WS9FIGSlKVk6XAJTE3TxXIqlBPIQ5Lsx5qifE72vXY/edit?usp=sharing
![Page 12: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/12.jpg)
http
s://i
nfor
mat
ioni
sbea
utifu
l.net
/vis
ualiz
atio
ns/b
est-i
n-sh
ow-w
hats
-the-
top-
data
-do
g/ v
ia @
Jenn
yBry
an
![Page 13: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/13.jpg)
![Page 14: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/14.jpg)
![Page 15: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/15.jpg)
![Page 16: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/16.jpg)
![Page 17: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/17.jpg)
![Page 18: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/18.jpg)
![Page 19: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/19.jpg)
![Page 20: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/20.jpg)
![Page 21: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/21.jpg)
![Page 22: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/22.jpg)
filter(row == 3, col == 1)$character
![Page 23: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/23.jpg)
readr::melt_csv()
googlesheets4::sheets_cells()
tidyxl::xlsx_cells()
![Page 24: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/24.jpg)
![Page 25: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/25.jpg)
![Page 26: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/26.jpg)
![Page 27: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/27.jpg)
![Page 28: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/28.jpg)
![Page 29: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/29.jpg)
![Page 30: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/30.jpg)
![Page 31: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/31.jpg)
![Page 32: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/32.jpg)
![Page 33: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/33.jpg)
![Page 34: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/34.jpg)
![Page 35: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/35.jpg)
![Page 36: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/36.jpg)
![Page 37: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/37.jpg)
![Page 38: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/38.jpg)
![Page 39: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/39.jpg)
![Page 40: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/40.jpg)
![Page 41: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/41.jpg)
1. One-row-per-cell format:
![Page 42: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/42.jpg)
1. One-row-per-cell format:tidyxlreadrgooglesheets4
![Page 43: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/43.jpg)
1. One-row-per-cell format:tidyxlreadrgooglesheets4
2. behead() each header in turn
![Page 44: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/44.jpg)
1. One-row-per-cell format:tidyxlreadrgooglesheets4
2. behead() each header in turnunpivotr
![Page 45: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/45.jpg)
![Page 46: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/46.jpg)
Via
Dav
id R
obin
son
and
Jenn
y B
ryan
http
s://r
pubs
.com
/dgr
two/
tidyi
ng-e
nron
![Page 47: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/47.jpg)
![Page 48: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/48.jpg)
((IF((103-B$89)=103,0,(103-B$89)))+(IF((200-B$95)=200,0,(200-B$95)))+(IF((196-B$98)=196,0,(196-B$98)))+(IF((200-B$101)=200,0,(200-B$101)))+(IF((70-B$104)=70,0,(MIN(40,(70-B$104))))+(IF((78-B$109)=78,0,(MIN(50,(78-B$109)))))+(IF((103-B$114)=103,0,(MIN(66,(103-B$114)))))+(IF((195-B$119-B$124-B$129-B$134-B$139)=195,0,(MIN(70,(195-B$119-B$124-B$129-B$134-B$139)))))+(IF((64-B$144)=64,0,(MIN(50,(64-B$144)))))+(IF((48-B$149)=48,0,(MIN(20,(48-B$149)))))+(IF((44-B$154)=44,0,(MIN(20,(44-B$154)))))+(IF((130-B$159)=130,0,(MIN(20,(130-B$159)))))))
![Page 49: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/49.jpg)
![Page 50: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/50.jpg)
![Page 52: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools](https://reader034.vdocuments.net/reader034/viewer/2022052009/601e962e0e8a320c8134fdd7/html5/thumbnails/52.jpg)
My stuff:https://github.com/nacnudus/tidyxlhttps://github.com/nacnudus/unpivotrhttps://nacnudus.github.io/spreadsheet-munging-strategies/https://rstudio.cloud/project/269487
Related:https://github.com/luisDVA/unheadrhttps://github.com/ianmoran11/tidyABShttps://databaker.sensiblecode.io/
Duncan Garmonsway
@nacnudus
Get good data out of bad spreadsheets