data mining holly b. smith university of north texas

DATA MINING

Holly B. Smith University of North Texas

“Copyright and Terms of Service

Copyright © Texas Education Agency. The materials found on this website are copyrighted © and trademarked ™ as the property of the Texas Education Agency and may not be reproduced without the express written permission of the Texas Education Agency, except under the following conditions:

1) Texas public school districts, charter schools, and Education Service Centers may reproduce and use copies of the Materials and Related Materials for the districts’ and schools’ educational use without obtaining permission from the Texas Education Agency;

2) Residents of the state of Texas may reproduce and use copies of the Materials and Related Materials for individual personal use only without obtaining written permission of the Texas Education Agency;

3) Any portion reproduced must be reproduced in its entirety and remain unedited, unaltered and unchanged in any way;

4) No monetary charge can be made for the reproduced materials or any document containing them; however, a reasonable charge to cover only the cost of reproduction and distribution may be charged. Private entities or persons located in Texas that are not Texas public school districts or Texas charter schools or any entity, whether public or private, educational or non-educational, located outside the state of Texas MUST obtain written approval from the Texas Education Agency and will be required to enter into a license agreement that may involve the payment of a licensing fee or a royalty fee.

Call TEA Copyrights with any questions you have.

Copyright © Texas Education Agency, 2012. All rights reserved. 2


1.8

TEN

2020


“Truth is a gem that is found at a great depth…”

Lord Byron


BY

• Definition• Methods Used

OF

• Who Mines/Uses Data• Storage Issues

FOR

• TEKS• Classroom Activities


Patterns


A 6 step process1.Defining2.Preparing3.Exploring4.Building5.Exploring and validating6.Deploying and Updating


Questions that Data Mining answers:

What goods should be promoted to this customer?

What is the probability that a certain customer will respond to a planned promotion?

Can one predict the most profitable securities to buy/sell during the next trading session?

Will this customer default on a loan or pay back on schedule?

Can you think of others?


1990 Data Mining was introduced

Family tree has three branches:classical statisticsartificial intelligencemachine learning


Techniques in Data MiningAssociation Rule:

• to discover interesting associations between attributes contained in a database. • market basket analysis. • tells if item X is a part of the event, then what is the percentage of item Y is also part of the event. • Who uses the Market Basket analysis?

Clustering:• to find appropriate groupings of elements for a set of data. • a kind of undirected knowledge discovery or unsupervised learning; that is, there is no target field, and the relationship among the data is identified by bottom-up approach. • Who uses Clustering?


Techniques in Data MiningDecision Trees:

• predicts the value of a target• based on several variables• simple to interpret and understand, performs well with a large amount of data in a short time


Techniques in Data MiningNeural Network:

• often represented as a layered set of interconnected processors, or neurodes• uses a complex mathematical process to resemble the brain• Artificial Neuron Networks can

examine entire warehouses of databases• large companies can spot trends (i.e., silly bands)• science and engineering community, geographical systems, and gaming (machine playing chess)

14

Data Mining Example #1

• Local Aware Mobile Social Networks

• Loopt https://www.loopt.com/• Whrrl (recently acquired by Groupon) • foursquare https://foursquare.com/• buddycloud http://buddycloud.com/,

and • brightkight http://brightkite.com/

(currently being revised)

https://www.loopt.com/

https://foursquare.com/

http://buddycloud.com/

http://brightkite.com/


“mental pain and distress, far greater than could be inflicted by mere bodily harm.”

Louis Brandeis


Counterterrorism


“the Industrial Revolution of data”

Joe HellersteinComputer ScienceUniversity of California in Berkeley


STORAGE

SOFTWARE METADATA

Extracts data from warehouse

Manage new database

Retrieve and analyze the data

Information about the data in the warehouse

Used to run searches

Used to identify key pieces of data


Storage Issues


There is an upside

22

Data Mining Example #1

• Reality Mining

• http://www.google.org/flutrends

• http://www.google.org/flutrends/video/GoogleFluTrends_USFluActivity.mov

http://www.google.org/flutrends

http://www.google.org/flutrends

http://www.google.org/flutrends/video/GoogleFluTrends_USFluActivity.mov

http://www.google.org/flutrends/video/GoogleFluTrends_USFluActivity.mov


Define fields

Enter database structure

Analyze company’s data requirements

Access information in the database system

Create a meaningful data set

Import and export databases


Data EntryData AnalysisData Collection


www.wpi.edu

http://www.wpi.edu/


I Want YOUto Mine Data


Resources

Slide Source3 http://en.thinkexist.com/search/searchquotation.asp?search=gems4 http://thoughts.forbes.com/thoughts/truth-lord-byron-truth-is-a6 http://www.unc.edu/~xluan/258/datamining.html 7 http://terrorism.about.com/od/counterterrorism/a/DataMining.htm8 Journal of Engineering Science and Technology Vol. 6, No. 2 (2011) - By Francisca

Nonyelum Ogwueleka9 http://www.unc.edu/~xluan/258/datamining.html10 http://www.time.com/time/printout/0,8816,2058205,00.html.10 http://www.unc.edu/~xluan/258/datamining.html11 www.itworld.com/050805datamining12 http://www.unc.edu/~xluan/258/datamining.html 13 http://www.unc.edu/~xluan/258/datamining.html 14 http://www.time.com/time/printout/0,8816,2058505,00.html15 http://www.theatlantic.com/health/print/2012/05/using-data-mining-to-predict-epidemics16 http://terrorism.about.com/od/counterterrorism/a/DataMining.htm19 http://seqcc.icarnegie.com/content/SSD/SSD7/1.5.2/normal/pg-trends/pg- datawarehouse/pg-datawarehouse.html20 http

://articles.cnn.com/2008-02-20/living/cb.top.jobs.top.industries_1_job-growth-median-annual-salary-physician-assistants?_s=PM:LIVING

http://en.thinkexist.com/search/searchquotation.asp?search=gems



http://thoughts.forbes.com/thoughts/truth-lord-byron-truth-is-a

http://thoughts.forbes.com/thoughts/truth-lord-byron-truth-is-a

http://www.unc.edu/~xluan/258/datamining.html



http://www.itworld.com/050805datamining

http://seqcc.icarnegie.com/content/SSD/SSD7/1.5.2/normal/pg-trends/pg-



http://articles.cnn.com/2008-02-20/living/cb.top.jobs.top.industries_1_job-growth-median-annual-salary-physician-assistants?_s=PM:LIVING





Resources

Slide Source

23 http://3.bp.blogspot.com/-0mZJJX8zPnc/Tbcusl4kDzI/AAAAAAAAAIY/vf7apxeUxEA/s1600/Bizarre-Architecture-Nord-LB-Building-Hanover-Germany.jpg

23 http://fashionshow007.files.wordpress.com/2010/04/design-for-fashion.jpg24 http://www.databaseguides.com/wp-content/uploads/2009/01/database-backup.jpg25 http://sp.life123.com/bm.pix/database1.s600x600.jpg26 http://www.ehow.com/info_7896179_middle-school-math-data-projects.html#ixzz1vu1kC

5VT26 http://chinwag.com/files/images/photos/junk_mail.jpg28 http://donmamporro.files.wordpress.com/2010/04/poster1-3-i-want-you.jpg29 http://www.zealeap.com/wp-content/uploads/2010/11/Thank-you-sign1.jpg

http://3.bp.blogspot.com/-0mZJJX8zPnc/Tbcusl4kDzI/AAAAAAAAAIY/vf7apxeUxEA/s1600/Bizarre-Architecture-Nord-LB-Building-Hanover-Germany.jpg

http://3.bp.blogspot.com/-0mZJJX8zPnc/Tbcusl4kDzI/AAAAAAAAAIY/vf7apxeUxEA/s1600/Bizarre-Architecture-Nord-LB-Building-Hanover-Germany.jpg

http://fashionshow007.files.wordpress.com/2010/04/design-for-fashion.jpg

http://www.databaseguides.com/wp-content/uploads/2009/01/database-backup.jpg

http://sp.life123.com/bm.pix/database1.s600x600.jpg

http://www.ehow.com/info_7896179_middle-school-math-data-projects.html

http://www.ehow.com/info_7896179_middle-school-math-data-projects.html

http://chinwag.com/files/images/photos/junk_mail.jpg

http://donmamporro.files.wordpress.com/2010/04/poster1-3-i-want-you.jpg

http://www.zealeap.com/wp-content/uploads/2010/11/Thank-you-sign1.jpg

data mining holly b. smith university of north texas

Documents

state of texas

texas charter schools

texas public school

education service centers

reproduced materials

related materials

data mining holly

data mining answers