1 giscience and the big data age yihong yuan department of geography texas state university
TRANSCRIPT
![Page 1: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/1.jpg)
1
GIScience and the Big Data Age
Yihong Yuan
Department of Geography
Texas State University
![Page 2: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/2.jpg)
2
About me
• Yihong YuanAssistant Professor
[email protected]. ELA 366, 512-245-3208
• Research Interests– Spatio-temporal data mining– Human mobility and activity patterns– Big data analytics
![Page 3: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/3.jpg)
3
Geography and Big Data
• GIS– Not only about mapping functions
• Big Geo-data– Information and communication technologies
(ICTs)• Greater mobility flexibility• A wide range of spatio-temporal data sources• Align marketing campaigns to spatial patterns.
![Page 4: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/4.jpg)
4
• “Geography is one of the most natural, logical and intuitive ways to discover, visualize, overlay, compare, slice, sort and apply big data to a problem”
• “GIS used to be about the analysis of relatively static
institutional data, but new data streams mean that today’s GIS problems look very much the same as today’s big data problems: extract meaningful information from a fire hose of inputs”
![Page 5: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/5.jpg)
5
• Traditional geographic knowledge discovery– e.g., high resolution trajectories
• Incomplete Spatio-temporal datasets– Low resolution– Few individual attributes– Uncertainty?
![Page 6: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/6.jpg)
6
Past Research• Georeferenced mobile phone data analytics
– Individual-oriented research– Activity space
» Measurements: Radius, Eccentricity, entropy» Correlation between phone usage and activity space
– Trajectory and sequence patterns» Time series analysis
– Urban-oriented studies• Spatial clusters • Spatial rhythms
• Dynamic clustering• Functional time points
![Page 7: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/7.jpg)
7
UML Model about Geo-referenced mobile phone data
Knowledge Discovery Tasks
Generalize types of information in mobile
phone datasets
Urban-oriented research
Construct an UML model
Analyze individual activity space
Individual-oriented research
Measure trajectory similarity
Correlate activity space with phone usage
Correlate activity space with individual and
supra-individual attributes
Identify urban hotspots and clusters
Dynamic clustering and time series analysis
Extract functional time points
![Page 8: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/8.jpg)
8
• Mobile Phone Connections in 10 cities in northeast China– Time, Duration, and Locations of Mobile
Phone Connections in 9 days– Age and Gender Attributes of the Users– Possibility of simulated data
Example Mobile Phone Dataset
![Page 9: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/9.jpg)
9
Analysis of Activity space
• Three measurements– Radius -> Scale
• eigenvectors of trajectories
– Eccentricity -> Shape• Range [0,1]• Closer to a straight line or a circle
– Entropy->Regularity• How random the visiting patterns are
2 2
1
ˆ 1-( )
ˆ
ee
e
2e1e
21
- logN
i ii
E p p
![Page 10: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/10.jpg)
10
Correlation between individual activity space and phone usage
![Page 11: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/11.jpg)
11
Results
• For People with Higher Mobile Phone Usage: – Larger Activity Space– Trajectories are Closer to a Circle– Movement is More Random, Less
Predictable
![Page 12: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/12.jpg)
12
Activity space vs Trajectory
![Page 13: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/13.jpg)
13
Analysis of trajectory patterns
• Compare trajectories from phone records– Sequences of cell IDs
• Edit distance Method– String matching and auto-correction
![Page 14: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/14.jpg)
14
Analysis of trajectory patterns (Cont.)
• Applications– Identify similar users
• Clustering analysis
– Identify outlier users
![Page 15: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/15.jpg)
15
Knowledge Discovery Tasks
Generalize types of information in mobile
phone datasets
Urban-oriented research
Construct an UML model
Analyze individual activity space
Individual-oriented research
Measure trajectory similarity
Correlate activity space with phone usage
Correlate activity space with individual and
supra-individual attributes
Identify urban hotspots and clusters
Dynamic clustering and time series analysis
Extract functional time points
![Page 16: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/16.jpg)
16
• The changing clustering of urban area
Urban hotspots and clusters
Weekdays Weekends
T2: 2pm-3pm
T1:8am-9am
T3: 7pm-8pm
![Page 17: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/17.jpg)
17
• Mobility patterns of different population groups– Weekday 2pm-3pm
Urban clusters (Cont.)
Age: 12-17 Age: > 60
![Page 18: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/18.jpg)
18
• Provide input for urban infrastructure planning– Are public facilities where people are??
Urban clusters (Cont.)
Age: > 60
A park
![Page 19: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/19.jpg)
19
Dynamic Clustering
• Focus on “rhythms” instead of just “clusters”
• Various mobility patterns in urban area– How to explore? – time series analysis
CBD, Beijing Suburb, Beijing
![Page 20: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/20.jpg)
20
Dynamic Clustering (Cont.)
• Methods– Divide study area
• Voronoi polygon (based on towers)• What to compare: 24-hour series for each
polygon based on mobility count
• Outlier detection e.g., traffic congestion
![Page 21: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/21.jpg)
21
Outlier polygons
• 15 outliers for weekdays and 18 for weekends
Weekday Weekends
![Page 22: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/22.jpg)
22
Mobility patterns in outlier areas• Outlier Polygon 238
– Night clubs and other leisure facilities– International trading center
• Outlier Polygon 125– Several community colleges – Not many night clubs, bars, etc.
Polygon 238 Polygon 125
![Page 23: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/23.jpg)
23
Current and future research
![Page 24: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/24.jpg)
24
Setting up functional time in cities
• Standardization of time – Determination of the beginning/end of a day
• The development of ICT– Real-time activity patterns– More flexibility in time management and
activity scheduling• i.e., fixed parking hour policy may not be
applicable in Central business districts
![Page 25: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/25.jpg)
25
Setting up functional time in cities
![Page 26: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/26.jpg)
26
Cross-country comparison for Social Media websites
• Flickr data, 100 million records and geo-tagged photos
• Similarity and dissimilarity of human mobility in various cities– “A tale of many cities”
![Page 27: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/27.jpg)
27
Current and future research• Mobility patterns in
developing and developed countries– China as a focus
• Weibo and Twitter check-in data– Comparison study for
special time period– Holiday patterns
![Page 28: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/28.jpg)
28
Current and future research
• Mass media and Social Media– GDELT dataset
• Geo-tagged news Events from 1970s
– Public relations and interaction between countries
![Page 29: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/29.jpg)
29
(a) (b)
![Page 30: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/30.jpg)
30
Big data and GIS jobs…• Traditional GIS jobs:• GIS Technician/Analyst/consultant• GIS manager/researcher• ……• Where are the positions?• Public sector… NGA, USGS, State and local Gov, DOT,
planning dept.• Private company…Oil&Gas, Mapping companies, Land
management, Utility…• Non-profit agency… Nature Conservancy, International
Crane Foundation• Consulting firms…Surveying, Remote Sensing…
![Page 31: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/31.jpg)
31
Example: Private Sector Jobs
• Mapping Companies• Software Developers• Utilities• Land Development• Non-Profits• Others
![Page 32: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/32.jpg)
32
Job Skills
• Project Management• Technical Support• Report Writing• Public Speaking• Research/Literature review• Programming
![Page 33: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/33.jpg)
33
Software Skills (cont.)
• GIS software packages• ArcGIS, ENVI, GDAL• Mobile & Web Technology
– Silverlight / Flex /HTML / ASP– Android Dev
• Python / C#...• Database: Access, SQL
Server, PostgresSQL
![Page 34: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/34.jpg)
34
Job Postings• Company Website
– ESRI summer internship program• Relevant Employment Websites
– General sites: Monster.com / Indeed.com– Linkedin.com– Glassdoor.com– GIS Jobs Clearinghouse (gjc.org)– GISjobs.com & Geojobs.org– GeoCommunity – GIS Café – WI State Cartographers Office
• http://www.sco.wisc.edu/jobs/jobs.php
![Page 35: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/35.jpg)
35
Job Postings
• Internal Company Postings• Company Website• Relevant Employment Websites
– GIS Jobs Clearinghouse (gjc.org)– GISjobs.com & Geojobs.org– GeoCommunity – GIS Café – Monster.com
![Page 36: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/36.jpg)
36
Job Postings
• Internal Company Postings• Company Website• Relevant Employment Websites
– GIS Jobs Clearinghouse (gjc.org)– GISjobs.com & Geojobs.org– GeoCommunity – GIS Café – Monster.com
![Page 37: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/37.jpg)
37
Big data jobs…• Spatial data are inherently big data…• For GIS major…
– Data Scientist• This is a more “General” term• Focus on big (geo)data analytics• Highly competitive salary• Graduate degree (MA possible, PhD preferred)• Many opportunities…
• Skill set:• Strong statistical background• Strong and programming: Python, R, etc,
![Page 38: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/38.jpg)
38
Example positions
• Data Scientist @ ESRI– http://www.simplyhired.com/job/data-scientist-agriculture-job/esri/5jjxyxjt4b?cid=n
tvzgigizsvnqhofbuscopqozjkxqugd
• Research Data Scientist– http://www.americasjobexchange.com/job-detail/job-opening-AJE-56966
1132?source=indeed&utm_source=Indeed&utm_medium=cpc&utm_campaign=Indeed
• Other potential groups: Apple geo-group, Twitter geo-group, Facebook data science group
![Page 39: 1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University](https://reader036.vdocuments.net/reader036/viewer/2022063020/56649e365503460f94b25f1c/html5/thumbnails/39.jpg)
39
Thanks!Questions and Comments?