the analyst's perspective: advanced bi with powerpivot...

11

The Analyst's Perspective: Advanced BI with PowerPivot DAX, SharePoint Dashboards, and SQL Data Mining

Rafal LukawieckiStrategic Consultant, Project Botticelli Ltdrafal@projectbotticelli.com

22

Objectives

Introduce more advanced BI analytics from Microsoft

Discuss using SharePoint 2010 as a BI Dashboard environment

The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. Thematerial presented is not certain and may vary based on several factors. Microsoft makes no warranties, express, implied or statutory, as to theinformation in this presentation.

Portions © 2010 Project Botticelli Ltd & entire material © 2010 Microsoft Corp. Some slides contain quotations from copyrighted materials byother authors, as individually attributed or as already covered by Microsoft Copyright ownerships. All rights reserved. Microsoft, Windows,Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. Theinformation herein is for informational purposes only and represents the current view of Project Botticelli Ltd as of the date of thispresentation. Because Project Botticelli & Microsoft must respond to changing market conditions, it should not be interpreted to be acommitment on the part of Microsoft, and Microsoft and Project Botticelli cannot guarantee the accuracy of any information provided after thedate of this presentation. Project Botticelli makes no warranties, express, implied or statutory, as to the information in this presentation. E&OE.

This seminar is based on a number of sources including a few dozen of Microsoft-owned presentations, used with permission. Thank you to Chris Dial, Tara Seppa, Aydin Gencler, Ivan Kosyakov, Bryan Bredehoeft, Marin Bezic, and Donald Farmer with his entire team for all the support.

3

PowerPivot on SharePoint 2010Manageability

44

PowerPivot for SharePoint 2010Managed Self-Service Business Intelligence

Collaborative, shared gallery of PowerPivots

IT Pro managementLifecycle & Workflow

Server Resource Management

55

Share Insights Common view of organizational performance

6

1. PowerPivot for SharePoint: Uploading Documents to Server

2. Galleries

77

Managing the BI Environment

User driven application administration and monitoring

Manage and facilitate access to secure organizational data

8

1. PowerPivot Management Dashboard2. Anticipating a self-created BI that can

become an organisational concern

99

PowerPivot Infrastructure Overview

SharePoint Farm

WFE

App Servers

Content dBs

NLB

Excel, RB, PerfPoint

Power User

Data Sources

Excel Services

PowerPivot Mid-Tier

AS Engine

Browser

Standard User

PowerPivot Add-In

1010

PowerPivot Infrastructure: Excel

SharePoint Farm

WFE

App Servers

Content dBs

NLB

Excel Services

Gemini Mid-Tier

Gemini Engine

Browser

Standard User

Excel, RB, PerfPoint

Power User

Data Sources

• Use of IMBI Engine: In-Memory Column-

Based store

• Once data is imported, all calculations

are performed on client

• Excel now has it’s own local SSAS

engine

• Added Excel power functions for Gemini

called DAX (Data Analysis eXpressions)

• Use of new compression algorithm to

significantly compress the data ~ 10:1

• Added slicer functionality: not just for UI

but for smoother SharePoint integration

PowerPivot Add-In

1111

Excel, RB, PerfPoint

Power User

Data Sources

Browser

Standard User

SharePoint Farm

WFE

App Servers

Content dBs

NLB

Excel Services

PowerPivot Mid-Tier

AS Engine

PowerPivot SharePoint Integration: ECS Viewing

Excel Web Access

1212

Excel, RB, PerfPoint

Power User

Data Sources

Browser

Standard User

SharePoint Farm

WFE

App Servers

Content dBs

NLB

Excel Services

PowerPivot Mid-Tier

AS Engine

PowerPivot SharePoint Integration: Server Action

Excel Web Access

13

PowerPivot DAX

1414

Data Analysis Expressions (DAX)

Simple Excel-style formulas

Define new fields in the PivotTable field list

Enable Excel users to perform powerful data analysis using the skills they already have

Has elements of MDX but does not replace MDX

1515

Data Analysis Expressions (DAX)

No notion of addressing individual cells or ranges

DAX functions refer to columns in the data

Sample DAX expression Means:= [First Name] &“ ”& [Last Name] String concatenation just like Excel

=SUM(Sales[Amount])SUM function takes a column name

instead of a range of cells

=RELATED (Product[Cost])new RELATED function follows

relationship between tables

1616

DAX Aggregation Functions

DAX implements aggregation functions from Excel including SUM, AVERAGE, MIN, MAX, COUNT, but instead of taking multiple arguments (a list of ranges,) they take a reference to a column

DAX also adds some new aggregation functions which aggregate any expression over the rows of a table

SUMX (Table, Expression)

AVERAGEX (Table, Expression)

COUNTAX (Table, Expression)

MINX (Table, Expression)

MAXX (Table, Expression)

1717

More than 80 Excel Functions in DAXDate and Time Information Math and Trig Statistical TextDATE ISBLANK ABS AVERAGE CONCATENATEDATEVALUE ISERROR CEILING, ISO.CEILING AVERAGEA EXACTDAY ISLOGICAL EXP COUNT FINDEDATE ISNONTEXT FACT COUNTA FIXEDEOMONTH ISNUMBER FLOOR COUNTBLANK LEFTHOUR ISTEXT INT MAX LENMINUTE LN MAXA LOWER

MONTH Logical LOG MIN MIDNOW AND LOG10 MINA REPLACESECOND IF MOD REPTTIME IFERROR MROUND RIGHTTIMEVALUE NOT PI SEARCHTODAY OR POWER SUBSTITUTEWEEKDAY FALSE QUOTIENT TRIMWEEKNUM TRUE RAND UPPERYEAR RANDBETWEEN VALUE

YEARFRAC ROUND

ROUNDDOWNROUNDUPSIGNSQRTSUMSUMSQTRUNC

1818

Example: Functions over a Time Period

TotalMTD (Expression, Date_Column [, SetFilter])

TotalQTD (Expression, Date_Column [, SetFilter])

TotalYTD (Expression, Date_Column [, SetFilter] [,YE_Date])

OpeningBalanceMonth (Expression, Date_Column [,SetFilter])

OpeningBalanceQuarter (Expression, Date_Column [,SetFilter])

OpeningBalanceYear (Expression, Date_Column [,SetFilter] [,YE_Date])

ClosingBalanceMonth (Expression, Date_Column [,SetFilter])

ClosingBalanceQuarter (Expression, Date_Column [,SetFilter])

ClosingBalanceYear (Expression, Date_Column [,SetFilter] [,YE_Date])

19

1. Simplicity of DAX to Relate and AnalyseData

20

SharePoint 2010 BI Dashboards:PerformancePoint Services

2121

PPS in SharePoint 2010

PerformancePoint Services in SharePoint 2010 improve over PerformancePoint Server 2007:

SharePoint does all security, management, backup, respository of dashboard

Decomposition Tree

KPI Details

Scorecard drilldown, dynamic hierarchies, calculated KPIs

Dynamic, up-to-date filters for time intelligence

SharePoint Dashboard Designer is smoother

Better accessibility

Analytic charts with value filtering and server-based conditional formatting

2222

Monitoring with PPS

Business users can build performance dashboards easily

2323

Analytics with PPS

Integration of KPIs and analytics

Multidimensional slice and dice, drill-across, drill-to-detail, root-cause analysis, prediction and centralized business logic definitions

No coding

2424

Reporting and Consolidation in PPS

Combine operational and financial data into one report

No need to reconsolidate manually

Dynamic and standard reports

Consistent live reports published from Excel to Reporting Services and SharePoint

2525

Dashboard Designer

Details

pane

Workspace

Browser

Workspace

2626

Developing a Dashboard

Choose a dashboard layout

Assign elements to a dashboard zone

Add filters

Preview the dashboard

Deploy to SharePoint

27

1. Building a Dashboard, Scorecard, and a KPI Using SharePoint Server PerformancePoint Services

28

Visualising BI with Microsoft Visio and SharePoint 2010

29

Two Trends that Lead to…The Messy Diagram

30

Data VisualizationFault Analysis Tree

Status Indicators

Color By Value

Text Callouts

Data Bars

31

Data VisualizationManufacturing

Specialized Shapes

3232

Strategy Maps VisualizePPS Scorecard data in context

33

Data Mining with SQL Server

3434

What does Data Mining Do?

Explores Your Data

Finds Patterns

Performs Predictions

3636

Analysis ServicesServer

Mining Model

Data Mining Algorithm DataSource

Server Mining Architecture

Excel/Visio/SSRS/Your App

OLE DB/ADOMD/XMLA

Deploy

BIDSExcelVisioSSMS

AppData

3838

Data Mining TechniquesAlgorithm Description

Decision Trees Finds the odds of an outcome based on values in a training set

Association Rules Identifies relationships between cases

Clustering Classifies cases into distinctive groups based on any attribute sets

Naïve Bayes Clearly shows the differences in a particular variable for various data elements

Sequence Clustering

Groups or clusters data based on a sequence of previous events

Time Series Analyzes and forecasts time-based data combining the powerof ARTXP (developed by Microsoft Research) for short-term predictionswith ARIMA (in SQL 2008) for long-term accuracy.

Neural Nets Seeks to uncover non-intuitive relationships in data

Linear Regression Determines the relationship between columns in order to predict an outcome

Logistic Regression

Determines the relationship between columns in order to evaluate the probability that a column will contain a specific state

40

1. Association Rules for Market Basket Analysis

2. Automatic recommendation engine using DMX queries

4141

Summary

Advanced self-service analysis requires a rich expression language: DAX

Team, and organisational BI dashboards and scorecards are easy to build using SharePoint 2010

Data Mining enables advanced pattern (correlation) discovery in your data

4242

© 2010 Microsoft Corporation & Project Botticelli Ltd. All rights reserved.

The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material presented isnot certain and may vary based on several factors. Microsoft makes no warranties, express, implied or statutory, as to the information in this presentation.

Portions © 2010 Project Botticelli Ltd & entire material © 2010 Microsoft Corp. Some slides contain quotations from copyrighted materials by other authors, asindividually attributed or as already covered by Microsoft Copyright ownerships. All rights reserved. Microsoft, Windows, Windows Vista and other product names areor may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents thecurrent view of Project Botticelli Ltd as of the date of this presentation. Because Project Botticelli & Microsoft must respond to changing market conditions, it shouldnot be interpreted to be a commitment on the part of Microsoft, and Microsoft and Project Botticelli cannot guarantee the accuracy of any information provided afterthe date of this presentation. Project Botticelli makes no warranties, express, implied or statutory, as to the information in this presentation. E&OE.