Business Intelligence

Lynn Langit/MSDN Developer Evangelist Southern California

http://blogs.msdn.com/SoCalDevGal

Predictive AnalyticsPredictive Analytics

PresentatioPresentationn

ExplorationExploration DiscoveryDiscovery

PassivePassive

InteractiInteractiveve

ProactivProactivee

Role of SoftwareRole of Software

Business Business InsightInsight

Canned reportingCanned reporting

Ad-hoc reportingAd-hoc reporting

OLAPOLAP

Data miningData mining

What and Why BI?

SQL Server 2008 BI & Tools OLTP – SQL Server Engine

SSMS / Profiler and other mgmt tools Reporting – SSRS

No need for IIS, BIDS / Report Manager to design

Integrates with SharePoint ETL – SSIS

Part of SQL Server, BIDS to design OLAP – SSAS

Multidimensional Cubes, BIDS / SSMS Data Mining – SSAS

Algorithm-based models – BIDS / Excel / SSMS

Why BI? Faster reports

OLAP can be 1,000% faster Flexible

click to query using pivot tables, add calculated members, create custom views

Proactive ‘discover’ patterns in data, ‘predict’ future

Reduce load on OLTP source systems

Scalable no manual index tuning, data de-normalization

SQL Server 2008 Languages OLTP – SQL Server Engine

T-SQL, .NET (CLR), XML Reporting – SSRS

RDL + queries ETL – SSIS

XMLA metadata + queries, .NET extendable

OLAP – SSAS MDX, XMLA

Data Mining – SSAS DMX, XMLA, PMML

Cubes vs. Data Mining

Where do I start? Understand OLAP modeling

Star schema + grain statements Review AdventureWorks DW sample

From www.CodePlex.com Realistically access source data quality

Plan for ETL, learn SSIS Leverage Excel

Light-weight data mining designer and client

OLAP cube pivot table client

Demo 1 – SSAS Cubes

Data Mining Add-ins for Office 2007

Table Analysis Tools for Excel 2007Table Analysis Tools for Excel 2007

Data Mining Template for Visio 2007Data Mining Template for Visio 2007

Data Mining Client for Excel 2007Data Mining Client for Excel 2007

Information Information WorkerWorker

BI AnalystBI Analyst

Data Mining Data Mining SpecialistSpecialist

Demo – Data Mining

DM - From Scenarios to Tasks

From Tasks to Techniques

Understand & Prepare specifics

Modeling Specifics

New to SQL Server 2008 Microsoft Time Series algorithm improved

ARIMA plus ARTxp method, and a blending algorithm = better results

New prediction mode allows adding new data to time series models

Holdout Support added Easily partition data into training and test sets that are stored in mining structure & available to

query after processing

Ability to build mining models based on filtered subsets added Results in less structures, i.e. can just filter existing

Drillthrough functionality extended makes all mining structure columns available, not just columns included in the model

allows you to build more compact models

Cross-validation added allows users to quickly validate their modeling approach by automatically building temporary

models and evaluating accuracy measures across K folds. The feature is available through a new cross-validation tab under Accuracy Charts in BIDS, in addition to being accessible programmatically via a stored procedure call.

Summary

Data Mining in SQL Server 2008 is mature, powerful and accessible

Can use Excel 2007 Familiar client for BI – OLAP cubes AND Data Mining

models Model Creators / Users Excel Data or Server Data

SSAS and Excel both support the full DM Cycle Data Understanding Data Preparation Modeling Validation Deployment

DM Webcasts

Fri, 02 Nov 2007MSDN Webcast: Build Smart Web Applications with SQL Server Data Mining (Level 200)Thu, 08 Nov 2007MSDN Webcast: Building Adaptive Applications with SQL Server Data Mining (Level 300)Mon, 19 Nov 2007MSDN Webcast: Extending and Customizing SQL Server Data Mining (Level 300)Fri, 30 Nov 2007MSDN Webcast: Creating Visualizations for SQL Server Data Mining (Level 300)Thu, 01 Nov 2007TechNet Webcast: Deliver Actionable Insight Throughout Your Organization with Data Mining (Part 1 of 3): Your First Project with SQL Server Data Mining (Level 200)Thu, 15 Nov 2007TechNet Webcast: Deliver Actionable Insight Throughout Your Organization with Data Mining (Part 2 of 3): Understand SQL Server Data Mining Add-ins for the 2007 Office System (Level 200)Thu, 29 Nov 2007TechNet Webcast: Deliver Actionable Insight Throughout Your Organization with Data Mining (Part 3 of 3): Use Predictive Intelligence to Create Smarter KPIs (Level 200)

BI Resources from Lynn Langit

1. “Foundations of SQL Server 2005 Business Intelligence”(published by APress in April 2007)

2.2. http://blogs.msdn.com/SoCalDevGal

3.3. “Building Business Intelligence “Building Business Intelligence Solutions with SQL Server 2008” Solutions with SQL Server 2008” (MSPress Fall 2008)(MSPress Fall 2008)

DM Resources

Technical Communities, Webcasts, Blogs, Chats & User Groupshttp://www.microsoft.com/communities/default.mspx

Microsoft Developer Network (MSDN) & TechNet http://microsoft.com/msdn http://microsoft.com/technet

Trial Software and Virtual Labshttp://www.microsoft.com/technet/downloads/trials/default.mspx

Microsoft Learning and Certificationhttp://www.microsoft.com/learning/default.mspx

SQL Server Data Mininghttp://www.sqlserverdatamining.comhttp://www.microsoft.com/bi/bicapabilities/data-mining.aspx

END – END - END End of first set

Business Intelligence

Lynn Langit/MSDN Developer Evangelist Southern California

http://blogs.msdn.com/SoCalDevGal

Session Prerequisites – Session Two Working SQL Server 2005 Developer Understanding of OLAP concepts Working SQL Server Analysis Server

2005 Developer Interest in or basic knowledge of Data

Mining concepts

Session Two Objectives and Agenda Understand what’s new SQL Server 2008

Business Intelligence SSAS OLAP cubes SSAS Data Mining Structures

Demo – Simplified Cube / Dim Wizards

Demo – New Aggregation Designer

Data Mining Are you using it now?

Data Mining – Logical Model

Mining ModelMining Model

Mining ModelMining Model

Training DataTraining Data

DB dataDB dataClient dataClient dataApplication dataApplication data

Data MiningData MiningEngineEngine

To To PredictPredict

Predicted DataPredicted Data

Mining ModelMining ModelDB dataDB dataClient dataClient dataApplication dataApplication data““Just one rowJust one row””

Data MiningData MiningEngineEngine

algorithmalgorithm

Evaluation Specifics

Analysis ServicesAnalysis ServicesServerServer

Mining ModelMining Model

Data Mining AlgorithmData Mining Algorithm DataDataSourceSource

Data Mining - Physical Model

Your ApplicationYour Application

OLE DB/ ADOMD/ XMLAOLE DB/ ADOMD/ XMLA

DeploDeployy

BI Dev BI Dev StudioStudio (Visual (Visual Studio)Studio)

App DataApp Data

Data Mining Interfaces – APIs

Analysis Server (msmdsrv.exe)

OLAP Data Mining

Server ADOMD.NET

.Net Stored Procedures Microsoft Algorithms Third Party Algorithms

XMLAXMLAOver TCP/IPOver TCP/IP

OLEDB for OLAP/DM ADO/DSO

XMLAXMLAOver HTTPOver HTTP

Any Platform, Any Device

C++ App VB App .Net App

AMO

Any App

ADOMD.NET

WANWAN

DM Interfaces

Configuration Model Creation/Management

Database Administrators Session Mining Models

Model Application Permissions on models Permissions on data sources

Deployment Browse

Copy to Excel Drillthrough

Query Default Advanced

Excel Services Manage models and structures

Export/Import Rename

Connection Database Trace

Excel Functions*

DMPREDICTTABLEROW ( Connection, ModelName, PredictionResult, TableRowRange[, string CommaSeparatedColumnNames])

DMPREDICT ( Connection, Model, PredictionResult,

Value1, Name1, [...,Value32, Name32])

DMCONTENTQUERY (Connection, Model, PredictionResult[, WhereClause])

Data Mining Extensions (DMX)

CREATE MINING MODELCREATE MINING MODEL CreditRiskCreditRisk

(CustID(CustID LONG KEY, LONG KEY,

Gender TEXT DISCRETE,Gender TEXT DISCRETE,

Income Income LONG LONG CONTINUOUS,CONTINUOUS,

Profession TEXT DISCRETE,Profession TEXT DISCRETE,

RiskRisk TEXT DISCRETE PREDICT) TEXT DISCRETE PREDICT)

USINGUSING Microsoft_Decision_Trees Microsoft_Decision_Trees

INSERT INTOINSERT INTO CreditRisk CreditRisk

(CustId, Gender, Income, (CustId, Gender, Income, Profession, Risk)Profession, Risk)

Select Select

CustomerID, Gender, Income, CustomerID, Gender, Income, Profession,RiskProfession,Risk

From CustomersFrom Customers

SelectSelect NewCustomers.CustomerID, NewCustomers.CustomerID, CreditRisk.Risk, CreditRisk.Risk, PredictProbability(CreditRisk.Risk)PredictProbability(CreditRisk.Risk)

FROMFROM CreditRisk CreditRisk PREDICTION JOINPREDICTION JOIN NewCustomersNewCustomers

ONON CreditRisk.Gender=NewCustomer.GenderCreditRisk.Gender=NewCustomer.Gender

ANDAND CreditRisk.Income=NewCustomer.Income CreditRisk.Income=NewCustomer.Income

AND AND CreditRisk.Profession=NewCustomer.ProfessionCreditRisk.Profession=NewCustomer.Profession

DMX Column Expressions Predictable Columns Source Data Columns Functions - Predict

“Workhorse”Discrete scalar valuesContinuous scalar valuesAssociative nested tablesSequence nested tablesTime SeriesOverloaded to

PredictAssociationPredictSequencePredictTimeSeries

PredictProbability PredictSupport PredictHistogram Cluster ClusterProbability GetNodeId IsInNode

Arithmetic operators Stored Procedure Subselect

Select from nested tables

Data Mining Interfaces – XMLA ++

Analysis Server (msmdsrv.exe)

OLAP Data Mining

Server ADOMD.NET

.Net Stored Procedures Microsoft Algorithms Third Party Algorithms

XMLAXMLAOver TCP/IPOver TCP/IP

OLEDB for OLAP/DM ADO/DSO

XMLAXMLAOver HTTPOver HTTP

Any Platform, Any Device

C++ App VB App .Net App

AMO

Any App

ADOMD.NET

WANWAN

DM Interfaces

New to SQL Server 2008 Microsoft Time Series algorithm improved

ARIMA plus ARTxp method, and a blending algorithm = better results

New prediction mode allows adding new data to time series models

Holdout Support added Easily partition data into training and test sets that are stored in mining structure & available to

query after processing

Ability to build mining models based on filtered subsets added Results in less structures, i.e. can just filter existing

Drillthrough functionality extended makes all mining structure columns available, not just columns included in the model

allows you to build more compact models

Cross-validation added allows users to quickly validate their modeling approach by automatically building temporary

models and evaluating accuracy measures across K folds. The feature is available through a new cross-validation tab under Accuracy Charts in BIDS, in addition to being accessible programmatically via a stored procedure call.

Summary

Data Mining in SQL Server 2008 is mature, powerful and accessible

Can use Excel 2007 Familiar client for BI – OLAP cubes AND Data Mining

models Model Creators / Users Excel Data or Server Data

SSAS and Excel both support the full DM Cycle Data Understanding Data Preparation Modeling Validation Deployment

DM Webcasts

Fri, 02 Nov 2007MSDN Webcast: Build Smart Web Applications with SQL Server Data Mining (Level 200)Thu, 08 Nov 2007MSDN Webcast: Building Adaptive Applications with SQL Server Data Mining (Level 300)Mon, 19 Nov 2007MSDN Webcast: Extending and Customizing SQL Server Data Mining (Level 300)Fri, 30 Nov 2007MSDN Webcast: Creating Visualizations for SQL Server Data Mining (Level 300)Thu, 01 Nov 2007TechNet Webcast: Deliver Actionable Insight Throughout Your Organization with Data Mining (Part 1 of 3): Your First Project with SQL Server Data Mining (Level 200)Thu, 15 Nov 2007TechNet Webcast: Deliver Actionable Insight Throughout Your Organization with Data Mining (Part 2 of 3): Understand SQL Server Data Mining Add-ins for the 2007 Office System (Level 200)Thu, 29 Nov 2007TechNet Webcast: Deliver Actionable Insight Throughout Your Organization with Data Mining (Part 3 of 3): Use Predictive Intelligence to Create Smarter KPIs (Level 200)

BI Resources from Lynn Langit

1. “Foundations of SQL Server 2005 Business Intelligence”(published by APress in April 2007)

2.2. http://blogs.msdn.com/SoCalDevGalhttp://blogs.msdn.com/SoCalDevGal

3.3. “Building Business Intelligence “Building Business Intelligence Solutions with SQL Server 2008” Solutions with SQL Server 2008” (MSPress Fall 2008)(MSPress Fall 2008)

DM Resources

Technical Communities, Webcasts, Blogs, Chats & User Groupshttp://www.microsoft.com/communities/default.mspx

Microsoft Developer Network (MSDN) & TechNet http://microsoft.com/msdn http://microsoft.com/technet

Trial Software and Virtual Labshttp://www.microsoft.com/technet/downloads/trials/default.mspx

Microsoft Learning and Certificationhttp://www.microsoft.com/learning/default.mspx

SQL Server Data Mininghttp://www.sqlserverdatamining.comhttp://www.microsoft.com/bi/bicapabilities/data-mining.aspxhttp://www.microsoft.com/bi/bicapabilities/data-mining.aspx