jiscad viz
DESCRIPTION
TRANSCRIPT
![Page 1: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/1.jpg)
Visualising Activity Data
Dept of Communication and Systems,The Open University
Scattered puzzle pieces next to solved fragment by Horia Varlan
Tony Hirst
![Page 2: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/2.jpg)
Today’s link shortener is bit.ly
Read: [ jlKwGq ]as: http://bit.ly/jlKwGq
![Page 3: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/3.jpg)
Visual Analysis
vs.
Presentation Graphics
![Page 4: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/4.jpg)
![Page 5: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/5.jpg)
This is NOT a presentation about:- data discovery- data preparation- data cleansing
![Page 6: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/6.jpg)
BUT…
![Page 7: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/7.jpg)
ScraperWiki[ aGhJtK ]
![Page 8: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/8.jpg)
Search and replace…
…add regular expressions and you have search and replace “on steroids”
![Page 9: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/9.jpg)
Google Refine[ aq1jUE ]
Example: walkthrough (@jenit) [ awGQPT ]Example: merging two tables by column [ pWK3C0 ]
![Page 10: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/10.jpg)
DataWrangler[ gmE3yz ]
![Page 11: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/11.jpg)
Data has shape and structure
![Page 12: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/12.jpg)
Hierarchical Data
![Page 13: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/13.jpg)
Treemaps
Many Eyes[ qY5786 ]
![Page 14: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/14.jpg)
![Page 15: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/15.jpg)
![Page 16: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/16.jpg)
![Page 17: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/17.jpg)
![Page 18: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/18.jpg)
plot srcfile using ($1):(column(focusCar) -$2) with lines title "VET", srcfile using ($1):(column(focusCar) -$3) with lines title "WEB", srcfile using ($1):(column(focusCar) -$4) with lines title "HAM", srcfile using ($1):(column(focusCar) -$5) with lines title "BUT", srcfile using ($1):(column(focusCar) -$6) with lines title "ALO", srcfile using ($1):(column(focusCar) -$7) with lines title "MAS", srcfile using ($1):(column(focusCar) -$8) with lines title "SCH", srcfile using ($1):(column(focusCar) -$9) with lines title "ROS", …
![Page 19: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/19.jpg)
Or heatmaps in R:[ qXmPgs ]
![Page 20: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/20.jpg)
Text processing with Unix tools[ m5tz63 ] [ lOVySX ]
Count number of lines in a file: wc -l L2sample.csv
View first few lines in a file: head L2sample.csv or head -n 4 L2sample.csv
View last few lines in a file: tail L2sample.csv or tail -n 15 L2sample.csv
Sample contiguous rows from start or end of file:head -n 1 L2sample.csv > headers.csvtail -n 20 L2sample.csv > subSample.csvcat headers.csv subSample.csv >
subSampleWithHeaders.csvSample contiguous rows from middle of file:head -n 15 L2sample.csv | tail -n 6 >
middleSample.csvSplit large file into smaller files:split -l 15 L2sample.csv subSamples
Search for lines containing a term:grep mendeley L2sample.csvgrep EBSCO L2sample.csv >
rowsContainingEBSCO.csv
![Page 21: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/21.jpg)
More text processing tricks
Extract columns:cut -f 3 L2sample.csvcut -f 1,2,14,17 L2sample.csv > columnSample.csv
Sort data in a column:cut -f 40 L2sample.csv | sort
Identify distinct entries in a column:cut -f 40 L2sample.csv | sort | uniq
Count how many times each distinct term appears in a column:cut -f 40 L2sample.csv | sort | uniq –c
Sort can also sort by column (-k), reverse order (-r):cut -f 40 L2_2011-04.csv | sort | uniq -c | sort -k 1 -r > uniqueSID.csv
![Page 22: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/22.jpg)
[ dAdIo3 ]
![Page 23: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/23.jpg)
![Page 24: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/24.jpg)
Time series data
![Page 25: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/25.jpg)
aka “seasonal subseries”
[ j3HODr ]
![Page 26: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/26.jpg)
![Page 27: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/27.jpg)
Trends
#time series data in d#first differencefd=np.diff(d)
Autocorrelation
matplotlib[ qSIcrV ]
![Page 28: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/28.jpg)
Graphs and Networks
![Page 29: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/29.jpg)
digraph test {
CSV [shape=box]KML [shape=box]JSON [shape=box]XML [shape=box]RDF [shape=box]HTML [shape=box]GoogleSpreadsheet [shape=Msquare]RDFTripleStore [shape=Msquare]"[SPARQL]" [shape=diamond]"[YQL]" [shape=diamond]"[GoogleVizDataAPI]" [shape=diamond]"<GoogleGadgets>" [shape=doubleoctagon]"<GoogleVizDataCharts>" [shape=doubleoctagon]"<GoogleMaps>" [shape=doubleoctagon]"<GoogleEarth>" [shape=doubleoctagon]"<JQueryCharts_etc>" [shape=doubleoctagon]
"[SPARQL]"->RDF;"[SPARQL]"->XML;"[SPARQL]"->CSV;"[SPARQL]"->JSON;JSON-> "<JQueryCharts_etc>";CSV->"{GoogleRefine}"CSV->ScraperWikiJSON->ScraperWiki"[YQL]"->ScraperWikiScraperWiki->CSVHTML->ScraperWikiHTML->"[YQL]""[SPARQL]"->"[YQL]""{GoogleRefine}"->CSV [style=dashed]CSV->"<Gephi>" [style=dashed]"<Gephi>"->CSV [style=dashed]RDF->"[YQL]”}
Graphviz
![Page 30: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/30.jpg)
Gephi
![Page 31: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/31.jpg)
[ nKoB4b]
![Page 32: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/32.jpg)
[ nKoB4b]
![Page 33: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/33.jpg)
![Page 34: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/34.jpg)
![Page 35: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/35.jpg)
![Page 36: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/36.jpg)
![Page 37: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/37.jpg)
Statistical Graphs
![Page 38: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/38.jpg)
R
![Page 39: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/39.jpg)
![Page 40: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/40.jpg)
Graphics Libraries
![Page 41: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/41.jpg)
Protovis
![Page 42: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/42.jpg)
Processing
![Page 43: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/43.jpg)
![Page 44: Jiscad viz](https://reader033.vdocuments.net/reader033/viewer/2022061217/54b449654a7959612b8b457d/html5/thumbnails/44.jpg)
I hope that’s been
ouseful.info ….?