sap net weaver 2004s enterprise data warehousing

SAP NetWeaver 7.0:Enterprise Data WarehousingOverview

Product Management SAP NetWeaver BI

November 2007

© SAP 2007 /

1. Overview2. Data Modeling

2.1. Data Warehousing Workbench2.2. DataSource2.3. Transformation2.4. DataStore Objects2.5. Modeling Data Marts

3. Data Flow Design3.1. Data Transfer Process3.2. Process Chains

4. Administration & Monitoring4.1. Administration Cockpit4.2. Information Lifecycle Management

5. Maintaining Data Security6. Performance Management

5.1. SAP NetWeaver BI Accelerator5.2. Other Performance Techniques

Agenda

© SAP 2007 /

Enterprise Data Warehousing

e.g.,at business unit level provide a:

local/subsidary viewregional viewglobal view

Provide each organizational unit or better each role with the needed reliable,consolidated, integrated, up-to-date, and historical information

at headquarter levelacross business unitsprovide

regional viewglobal view

© SAP 2007 /

The Challenge of Enterprise Data Warehousing

With an centralised Enterprise Data Warehouse:People will find the right informationRelated information is connected

Collaboration and information exchange between people does work

EDW

© SAP 2007 /

Architecture SAP NetWeaver BI

SAPOperational Data

Non-SAPOperational Data

SAP NetWeaverBI Data3rd-Party BI Data

Information Broadcasting

Analyzer

BI Consumer ServicesBI Consumer Services

Visual ComposerEmbedded BI

BI Kit

Services &B

APIS

Composite

ReportDesigner

Web ApplicationDesigner

WebAnalyzer

Business Explorer Suite (BEx)

SAP NetWeaver Portal

UIs can be embedded

Enterprise Search WorklistsKnowledge Management Collaboration

Data Sources

BI Layer

SAP NetWeaverBI Accelerator

Appliance

Meta D

ataR

epository

Master D

ata

Open HubService

OperationalData Store

Data Marts

Data Warehouse

Virtual-Provider

Query Designer

Planning Modeler

Analytic Engine

PSA

Downstream System

ODBO/XMLA

Web Services

Near-LineStorage

EnterpriseReportBI App MS ExcelAd Hoc

PlanningLayout

© SAP 2007 /

Analytic Engine

BI Architecture:Enterprise BI Data Management

Dat

a Fl

ow C

ontr

ol /

Proc

ess

Cha

ins

Enterprise Query, Reporting & Analysis

Caching

Source Systems

Mon

itorin

g / A

dmin

istr

atio

n

Calculation

Aggregation

Planning Services

Enterprise Data Warehouse

OperationalData Store(volatile) Data Warehouse Layer

(historical)

(Architected)Data Marts Open

HubService

DataSource / PSA

Analysis Process DesignBI A

ccel

erat

or

Security

Met

a D

ata

Rep

osito

ry /

Doc

umen

ts

Info

Obj

ects

/ M

aste

r Dat

a

Nea

r-Li

ne S

tora

ge

© SAP 2007 /

Enterprise Data Warehousing - Processes

Data Modeling for EDWBusiness (Process) Experts can define the basis for the enterprise reporting. They define data containers(InfoProviders) and data consolidation rules. Multiple-layer EDW-architectures are supported.

Data Flow DesignThe data flow (in particular DTP and InfoPackage) is defined at this level.

Maintaining Data SecurityThis process enables organizations to model the company‘s security rules into the software in a highlyflexible matter.

Administration and MonitoringThe BI administrator is offered a central admin and monitoring tool (NetWeaver Administrator), whichfacilitates monitoring of complex landscapes. In addition, lifecycle management tasks can be initiated fromhere.

Performance ManagementProvides all means to accelerate query performance, in particular the SAP NetWeaver BI Accelerator.

Data Modelingfor EDW

Data FlowDesign

Maintaining DataSecurity

Administrationand Monitoring

PerformanceManagement

© SAP 2007 /







Agenda

© SAP 2007 /

Overview

Data Warehousing Workbench with SAP NetWeaver 7.0Modeling and Administration view

© SAP 2007 /

Data Warehousing Workbench

Usability FeaturesFavoritesPersonalizationAdvanced SearchComplete data flow at a glance

© SAP 2007 /







Agenda

© SAP 2007 /

Conceptual Layers of Data Warehousing

(Persistent) Staging Area

OperationalData Store

DataWarehouse

(Architected)Data Marts

Any Source

Information Access

DataSources

© SAP 2007 /

Data Acquisition Layer – Data Sources

SourcesSupport of virtually all sources

ETL Tool

RelationalSource

Multi-Dimensional

SourceFile XMLSAP

Source

DataSource

UDConnect

DBConnect BAPIWeb

ServiceFile

Interface

LegacyApplications

BI ServiceAPI

e.g. IBM DB2,Teradata

e.g. Hyperion e.g. ORACLEFinancials

e.g. SAP CRM e.g. SAPNetWeaver PI(via proxyframework)

© SAP 2007 /

New BI DataSource concept withSAP NetWeaver 7.0

Highlightsunique look and feel for all of the DataSource TypesPSA is attached to DataSource

InfoPackage writes to PSAData Transfer Process writes from PSA to data targets

direct/remote access is optionalpreview feature is standardautomated conversions (e.g. date format detection)

© SAP 2007 /

Source System Tree

Source sytems categories:

SAP vs. non SAPFile vs. databaseRelational vs.Multidimensional DBABAP vs. JavaXML vs. Text/BinaryPull vs. PushRealtime vs. Batch

© SAP 2007 /

DataSource Example – One fits all approach

General InformationDescriptionsReconciliation flag (notfunctional)Opening Balance(inventory)Error handling (duprecs)

© SAP 2007 /

Data Flow Concept in SAP NetWeaver 7.0

SAP NetWeaver BI

Source

InfoProvider

Source System 1

DataSource / PSA

Transformation

Process Chain

InfoPackage

Data TransferProcess

© SAP 2007 /

Data Flow Concept in SAP NetWeaver 7.0Simplified

SAP NetWeaver BI

Source

InfoProvider

Source System 1

DataSource / PSA

Transformation

Process Chain(optional)

Data TransferProcess

XRestrictions:

Not optimized formass data transfer

No packaging of data

Full Mode Only

© SAP 2007 /







Agenda

© SAP 2007 /

Data Flow in SAP NetWeaver 7.0 BI

Any Source

SAP NetWeaver Business Intelligence

DataSource / PSA

Non-SAP SAPSAP NetWeaver BI

SAP NetWeaver PI

InfoProvider

Transformation

InfoPackage

Downstream Systems

Open Hub DestinationInfoProvider

DataTransferProcess

DTP DTP

InfoPackage

Transformation Transformation

© SAP 2007 /

Transformation

Source

Package 1

Transformation

Start Routine

ExpertRoutine

End Routine

Transformation Rule 1Transformation Rule n

Package 2Package m

Target

SemanticGroups

* optional

*

*

*

*

Universal transformation from sourceto target objects

Transformation types:Move, aggregate, constant, master data lookup, …Business rules, e.g. unit + currency translationFormula builder with rich predefined functionslibraryABAP routines incl. regular expressions

SAP NetWeaver 7.0 EnhancementsIntuitive UIUnit conversionUnified transfer + update rules into all-in-onecapabilityIntegration of Open Hub Service

© SAP 2007 /

Transformation – Definition

Access from the Data Warehousing WorkbenchNew transformation

Unification of transfer and update rulesInfoSource not mandatory anymore

Former concept of update rulesSmall square next to the transformation iconAccess from context menu via ‘additional functions’

Links sources and targetNew source: InfoSetOther sources: DataSource, InfoCube, DataStore object, InfoObject, InfoSourceTargets: InfoCube, DataStore object, InfoObject, InfoSource, Open Hub Destination

Transformation

Update rule

© SAP 2007 /

Transformation – Graphical UI

Sourcefields

Targetfields

Rules pergroupNote: Key figures, characteristics and date

fields are shown on the same level(transformation group)

© SAP 2007 /

Transformation Rules

Transformation rule detailsInformation on

Rule typeCurrency/Unit ConversionSource fieldsTarget Fields

© SAP 2007 /

Enhanced Data Flow in SAP NetWeaver 7.0 BI


DataSource / PSA

InfoProvider

DataTransferProcess

InfoProvider

TransformationTransformation

InfoSource (optional)

Transformation (optional)

InfoSource (optional)

Transformation (optional)

© SAP 2007 /

Transformations – InfoSource – 1 –

InfoSourceTransformation directly links from a source InfoProvider (or DataSource) to a targetInfoProviderAn InfoSource is usually not neededNew InfoSource architecture is used (flat InfoObject-based structure)Scenarios for (flexible) InfoSource

A flexible InfoSource is necessary in order to use currency or unit conversionfrom the source DataSource Define InfoSource as an intermediate structureYou can use a flexible InfoSource as a uniform source for several targets; theInfoSource can the be target from different sources (see next slide)

Note: for ‘direct’ InfoSources (for master data updates), there is no differencebetween ‘old’ and ‘new’ InfoSource, i.e. you can define a transformation as well astransfer rules

Pre-requisite: InfoObject is defined as InfoProvider

© SAP 2007 /

Transformations – InfoSource – 2 –

InfoSourceScenario: InfoSource as a uniform source for several targets and as target fromdifferent sources


DataSource

InfoProvider

DataSource 1 DataSource 2 DataSource n…

InfoProvider 1 InfoProvider 2 InfoProvider m…

Transformation…

InfoSource

Transformation…

© SAP 2007 /

Transformation Groups – 1 –

Transformation GroupsSummarize key figures with the same characteristics assignments

All key figures of one transformation are updated based on the same characteristicvaluesIf other characteristic updates are necessary for particular key figures, a newtransformation is created

© SAP 2007 /

Transformation Groups – 2 –

Transformation GroupsUse / Example

Scenario: overview on bonus-relevant sales of all employeesAn employee generates a certain sales volume, which is the basis for his/herbonusThe manager of the employee will be assigned 10% of the employee’s bonus asmanager’s bonus relevant

two transformation groups are generated (e.g. ‘employee’ and ‘manager’)

Giles1000Johnson

ManagerSales VolumeEmployee

Source

Transformation Group 1Employee Employee

Sales Volume Bonus-relevant Sales

100Giles

1000Johnson

Bonus-relevant SalesEmployee

Target Transformation Group 1Manager Employee

Sales Volume*0,1 Bonus-relevant Sales

© SAP 2007 /







Agenda

© SAP 2007 /

DataStore Object Types

DataStore Object types – overview

via APIs,Staging intosubsequenttargetspossibleX

for externalapplicationsand analysisprocesses(APD)XNo

DataStore Object fordirect update

via staging(DTP)X

Staging layeresp. for largesets of datawith(generally)unique keyX

On requestlevelX

Write-optimizedDataStore Object

via staging(DTP)XXX

Deltadeterminationfrom afterimageson record levelXX

Standard DataStoreObject

Activ-ationQueue

ChangeLog

ActiveDataOthers

Fast Access(no activation)

Delta / ChangeData Capture

ODSLayer

EDWlayer

Integrationinto data flow

StructurePrimary Usage

DataStore object type

© SAP 2007 /

Standard DataStore Object – 1 –

DetailsOption ‘Generation of SID Values’

Improves query performanceQueries are also possible if SIDvalues are not generated

Option ‘Unique data records’Only available if ‘Generation ofSID Values’ is setActivation process is optimized(only inserts, no sorting, nobefore image)Note: error if key already existsFor (non-reporting) scenarios, write-optimized DataStores are recommended instead ofstandard DataStore objects with unique flag

Performance ImprovementRollback

Instead of rolling back in serial and in one transactionrollback now is in parallel and for each data package there is a single task

© SAP 2007 /

Standard DataStore Object – 2 –

StructureActivation queue

Used to store data to be updated in DataStore Object which has not been activatedAfter activation the data can be deletedTechnical key: Request SID, Package ID, Record number

Active Data TableSame structure asthe DataStore Objectdefinition

Change LogChange history for delta mechanism from the DataStore Object into other InfoProviderKey fields:– Request GUID, Package ID, Record number

© SAP 2007 /

Write-optimized DataStore Object – 1 –

ScenariosFast EDW inbound layer (no activation needed)For large sets of data records on detailed level (e.g. document level)

“wide” structure is possible (16 key fields, 749 data fields)“Load of new records”:

Every record has a new key No update, only insertsE.g. for POS data

“Load & Drop”:Full Upload into DataStore ObjectUpdate subsequent InfoProviderDrop DataStore Object dataContinue with full uploads

© SAP 2007 /

Write-optimized DataStore Object – 2 –

DetailsDefinition

Only active data table (key: request ID, Packet No., Record No.)– No change log and no activation queue– Technical key is uniquePartitioned on request IDNo SID generation– Nevertheless: Reporting is possible (but not optimized for performance)Fully integrated in data flow: usable as data source and data target– Export into InfoProviders via request deltaCan be included into MultiProvider or InfoSet

Uniqueness of DataCheckbox “Do not check Uniqueness of data”Performance improvement during data load– Does not create/maintain unique index on

semantic key

© SAP 2007 /

DataStore Object For Direct Update – 1 –

DetailsDefinition

Only active data tableCan be used as data target within APDcannot be used for transformation (upload) scenarios

no loading process within BIbut export into next InfoProvider is possible

Reporting is possibleScenarios

Used for direct input of (external) transactional data– E.g. BI table for user interactionAn API is available with a set of function modules(some are RFC enabled)Fed by APD processes

© SAP 2007 /







Agenda

© SAP 2007 /

InfoCube

InfoCubeStar Schema optimized for multi-dimensional reporting

Fact TableFact Table

DimensionDimension

DimensionDimensionDimensionDimension

DimensionDimension

Dimension ID

MasterData

MasterData

MasterData

MasterData

MasterData

MasterData

MasterData

MasterData

Surrogate Key

Support ofdegenerateddimensions

© SAP 2007 /

InfoCube

Example:

InfoCube in Meta DataRepository

© SAP 2007 /

MultiProvider

MultiProviderLogical definition without data persistencyAbstraction level for queriesCan integrate the following InfoProviders via union operator

InfoCubeInfoObjectDataStore ObjectVirtualProviderInfoSetAggregation Level

© SAP 2007 /

InfoSet

InfoSetLogical definition without data persistencyCan integrate InfoCubes, DataStore Objects and InfoObjectsjoin and outer join operator

InfoCube DataStore Object InfoObject /Master Data

InfoSetw/o data

persistency

© SAP 2007 /







Agenda

© SAP 2007 /

Data Transfer Process: Complex Example

DataStore Object 1

DTP

DataStore Object 2

DataStore Object 3

InfoSource

DTPDTP

DTPDTP

DTP

IP IP

TR

TR TR

TR

TRTR

DataSource (PSA)DataSource (PSA) DataSource (PSA)DataSource (PSA)

Source System 1 Source System 2

TRProcess Chain

Process Chain

SAP Netweaver BI

© SAP 2007 /

Benefits of New Data Transfer Process

Data Transfer Process (DTP) - Data Distribution within SAP NetWeaver BILoading data from one layer to others except InfoSourcesSeparation of delta mechanism for different data targetsEnhanced filtering in dataflowImproved transparency of staging processes across datawarehouse layers(PSA, DWH layer, ODS layer, Architected Data Marts)Improved performance: optimized parallelizationEnhanced error handling for DataStore object (error stack)Enables real-time data acquisition

© SAP 2007 /

Filter in Data Transfer Process

With filter it is possible toload a set of data to the

data target instead of thecomplete volume of data.Different data selectionscan be made via different

data transfer processes forthe same or for different

data targets.

Extractionmode: Delta or

Full

© SAP 2007 /

Data Transfer Process

Loading directlyinto Data Target

without PSA

PrerequisiteDataSource is enabled for direct access (table ROOSOURCE)Or SAP Basis Plug-In Release 2005.1 SP8, or SAP Plug-In Release 2004.1 4.6C SupportPackage 13Or if you want to have the correction before the above support packages are released, pleaseapply note 923783.

© SAP 2007 /

Process chaincan automatethe loadingprocess

Error Handling Overview

DTP SchedulerDTP Scheduler

Error StackError Stack

DataSource (PSA)DataSource (PSA)IP DTP

There is no error handlingavailable for an InfoPackage.In case of invalid records,data needs to be reloadedfrom the source system.

Invalid records can be corrected in the errorstack and updated into the data target

Source System

Error DTP

© SAP 2007 /

Error Handling Features

Error HandlingPossibility to choose in the scheduler to...

abort process when errors occurprocess the correct records but do not allow reporting on themprocess the correct records and allow reporting on them

Number of wrong records which lead to a wrong requestInvalid records can be written into an error stackKeys should be defined for error stack to enable the error handling of DataStore objectTemporary data storage can be switched on/off for each substep of the loading processInvalid records can be updated into data targets after their correction.

© SAP 2007 /

Error Handling

Error HandlingOnce errors occur, the wholeData Package is terminated.The request is not releasedfor reporting.

Valid records are updated.After manual release of therequest, data is valid forreporting.

Valid records are updated andavailable for reporting

© SAP 2007 /

Error Stack – 1 –

Error StackStores erroneous records

Automatic checks: Existence of master data, conversion exit (restricted, e.g.Alpha)Customer-defined checks in transformation routines (see appendix for moreinformation)

Keeps the right sequence of records for consistent DataStore handlingKey of error stack defines which data should be detained from the update after theerroneous data recordAfter correction, Error-DTP updates data from error stack to data targetNote: Once the request in the source object is deleted, the related data records inerror stack are automatically deleted

© SAP 2007 /

Error Stack – 2 –

Error StackKey of Error Stack = Semantic Groups

Subset of the key of the target objectMax. 16 fieldsDefining which data should be detained from the update after the erroneousdata record (for DataStore Object)Semantic groups bundle records with the same semantic group key into thesame request see transformation chapter for more details (for DataStoreObject and InfoProvider)

© SAP 2007 /

Temporary Data Storage

Temporary Data StorageHelp for tracing the erroneous records and transformationsData records from different steps within the data transfer process can be storedtemporarilyStores complete set of data (erroneous as well as valid records)Scenario:

If the debugging mode is switched onTrace the erroneous recordsTrace Transformation

© SAP 2007 /

Temporary Data Storage

Settings for Temporary Data StorageLevel of detail

Tracing the erroneous recordsTracing transformation by packageTracing transformation by record

Deletion of temporary storageWith request status ‚green‘If request is deletedAfter X days

Switch on/off thetemporary datastorage for dataloading steps

© SAP 2007 /

Data Transfer Process Monitor – 1 –

DTP MonitorIntegrated in InfoProvider management screenIntegrated in DTP maintenanceAdditional information: duration of each stepTemporary storage access – if activatedError Stack is displayed in DTP Monitor

Data display intemporary storage

Error Stack

© SAP 2007 /

DTP and Open Hub

Open Hub DestinationasDTP DataTarget

© SAP 2007 /

Open Hub Destinations

© SAP 2007 /

DTP Initialization without Data Transfer

Data TransferProcess Initializationwithout data transfer

© SAP 2007 /

DTP Monitor – Header

Monitor Data transferprocess ‚header‘

© SAP 2007 /

New with SPS08:Monitor Datatransfer process‚Detail‘

DTP Monitor – Detail

© SAP 2007 /







Agenda

© SAP 2007 /

Introduction: Typical Data Load Cycle

Data LoadMonitor

Data TargetMaintenance

Start

Load into PSA

Load into DataStore

ActivateData in

DataStoreObject

Load into InfoCube

Roll up to BIAIndex

…

Drop Indices

© SAP 2007 /

Process Chain Example

© SAP 2007 /

Three Different Views in the Transaction

Planning view: Build and change process chainsGrey: unplanned processesGreen: planned prozessesYellow: planned but unknown processesRed: multiple planned processes

Check view: Check for errors in designGreen: Error-free processesYellow: Process with warningsRed: Process with errors

Log view: Monitoring of process chainsGrey: Not yet runGreen: Finished without errorYellow: runningRed: broken or failed

© SAP 2007 /

Process Chains:Failed processes can send email

Planning viewcontext menu

Write a messageand fill in recipient

and type. Infosaved within

process variant.

© SAP 2007 /







Agenda

© SAP 2007 /

BI Administration Cockpit - Motivation

Easy administrationfor complex

Enterprise Data Warehousesusing the BI

Administration Cockpit

Easy administrationfor complex

Enterprise Data Warehousesusing the BI

Administration Cockpit

© SAP 2007 /

BI Administration Cockpit - Scope

Support the BI administrator inStatus trackingPerformance optimizationStrategic administration

…in the areas ofEnterprise Data WarehousingEnterprise Query, Reporting and AnalysisBusiness Planning and Analytical Services

…by providing a central point of entry withcockpits

Real-time monitorsRuntime StatisticsCross system monitoring

…including context-specificDrill-down to detailsProcessing optionsExceptions (optional)

…using proven technologyBI QueriesBI Web ApplicationsSAP NetWeaver Portal

…to make administrationeasier and faster

…and thus to lower the TCO

© SAP 2007 /

BI Administration Cockpit - Overview

Central access to most important BImonitoring information

Monitoring ofmultiple BI systems

in one view

Flexible filtering ofrelevant information

Context menu foraccess to more

detailed informationor BI Transaction

Exception definition forintuitive display of criticalmonitoring data (optional)

Graphical display

© SAP 2007 /

BI Administration Cockpit –Architecture

BISuite

BIPlatform

DataWarehousing

Queries

BI Web Applications

SAP NetWeaver BI

InfoProviders / MultiProviders

Query RuntimeStatistics

Data LoadStatistics

Data LoadStatus

DataSources

iViews iViews


PortalPages iViews

SAP NetWeaver 7.0 BItechnology(software componentSAP_BW)

Technical Content for SAPNetWeaver BI (softwarecomponent BI_CONT,release 7.0.2), TheTechnical Content isentirely based on SAP BW3.x functionality notrequiring BI_JAVA.

Business Package „BIAdministration 1.0“ fromthe Portal ContentPortfolio. BI AdministrationCockpit can run in acentral or in a local portal.

© SAP 2007 /

BI Administration Cockpit –Main building blocks

BISuite

BIPlatform

DataWarehousing

Queries

BI Web Applications

SAP NetWeaver BI

InfoProviders / MultiProviders

Query RuntimeStatistics

Data LoadStatistics

Data LoadStatus

DataSources

iViews iViews


PortalPages iViews

BI StatisticsDetailed Runtime Statistics Data collection for various BI

Objects in Data Warehousing, Enterprise Reporting and Planning

Technical Content (InfoProviders and DataSources)Central Data Basis for BI Administration Cockpit and BI system

load transaction ST03Persistent Data Storage and Remote Access to BI Statistics

Information

Technical Content (Web Application and Queries)Flexible analysis of statistics data and sophisticated

presentation of information (graphs, charts, tables)

BI Administration Cockpit (Business Package)Single point of entry and integration with other (non BI related)

portal content (example: Universal Work List)

Built-In

Mandato

ry

Recommen

ded

© SAP 2007 /

New BI Statistics and Technical Content

Main enhancementsNew Technical Content for new and enhanced BI Statistics

New Query Runtime StatisticsProcess Chain and DTP StatisticsBI Object Request and Process Status

Technical Content for direct access and analysis on persistent dataPer default, queries from the Technical Content filter on reading frompersistent InfoProviders onlyReading from Virtual Providers can be enabled on query level by customers

Technical Content on detailed and aggregated levelFor Query Runtime Statistics

New maintenance for statistics data collectionEnabling statistics and selection of detail level for statistics

© SAP 2007 /

Analysis of BI Statistics data in SAPNetWeaver 7.0

Statistics tables (RSSDSTAT)

Direct analysisof tables RSDDSTAT*

ST03 – BW System Load

New and enhancedTechnical Content

Query Monitor (RSRT)

BI Administration Cockpit

Expert mode “profiling”in the (new) BEx Web

New: Persistentdata storage and

direct access

New: System Loadanalysis for BI basedon Technical Content

New: Ad hocanalysis of

statistics data

© SAP 2007 /







Agenda

© SAP 2007 /

Analytic Engine

BI Architecture:Platform & Data Warehouse

Dat

a Fl

ow C

ontr

ol /

Proc

ess

Cha

ins

Enterprise Query, Reporting & Analysis

Caching

Source Systems

Mon

itorin

g / A

dmin

istr

atio

n

Calculation

Aggregation

Planning Services

Enterprise Data Warehouse

OperationalData Store(volatile) Data Warehouse Layer

(historical)

(Architected)Data Marts Open

HubService

DataSource / PSA

Analysis Process DesignBI A

ccel

erat

or

Security

Met

a D

ata

Rep

osito

ry /

Doc

umen

ts

Info

Obj

ects

/ M

aste

r Dat

a

Nea

r-Li

ne S

tora

ge

© SAP 2007 /

Data-Aging Strategies – Initial Steps

Frequently read /changed data

Very rarely read data

Rarely read data

Classic ArchiveNear-Line StorageOnline Database

Categorizing Information According to Importance:

© SAP 2007 /

Persistent Data Warehouse Layers –Strategic Aspects

Data Warehouse

OperationalData Store(volatile)

Data Warehouse Layer(historical)

ArchitectedData Marts

NLS

NLS

Eng

ine

InfoProviderInfoCubesDataStore-Objects

• Multidimensional Model• High Performance Capabilities• High Volume Capabilities• Optimized TCO

BIA

Eng

ine

BI

© SAP 2007 /

Offline Archive

RDBMS

Modeling Aspects –Perfect InfoCube Design Example

InfoCube

NLSBIA

Staging

Indexing Archiving

BI

© SAP 2007 /

Offline Archive

NLS

RDBMS

InfoCube

BIA Engine

BI

Accelerated Online Nearline Offline

high frequently frequently non frequently rarely

Information Lifecycle Management Aspects

© SAP 2007 /

RDBMS

NLS EngineBIA Engine

BI

Offline Archive

Business Explorer Suite (BEx)Transparent Access No Access

Adjoint InfoProvider

InfoProvider

NearlineProvider

Reporting Aspects

© SAP 2007 /

Sources

NLS EngineBIA Engine

BI

Data Mart

EDW

PSA

DTP

Indexing

DAP

DTP

• timeslices + dimensions• ADK, ADK/NLS, NLS• new process type in

ProcessChains• flexible for structural changes• Archive and delete in one LUW• write protection for removed

areas in Data Store objects

• Reload via DTP available

Dataflow Aspects

© SAP 2007 /

The Near-Line Storage Solution forSAP NetWeaver BI

Near-Line StorageSeparation of frequently used data and rarely used data via Admin CockpitcapabilitiesNLS support for InfoCubes and DataStore objects

Transparent access to „non-archived“ and „archived“ data for queriesOpen interface for certified partnersDevelopment partners

PBS Software – CBW®FileTek – StorHouse®OuterBay - LiveArchive®SAND-Technologies - Searchable Archive®

© SAP 2007 /

SAP NetWeaver 7.0 BI: NLS-Based Archiving

InfoProviderInfoCubesData Store Objects

Data Archiving Process

Defining a flat view of the InfoProviderwithout navigational attributes and SIDs

Scheduling via Process Chain

Archive TypeOffline, ADK only (like BW 3.x)Near-Line onlyOffline and Near-Line (NLSindexing Offline Archive)

Selection SchemaTime-Slice Archivingrelative archiving periods, delta oriented,DSO and compressed InfoCube,range protection for incoming data

Pure Request-basedfor uncompressed InfoCubes

Flexible Selectionsno support for periodic processingArchiveOnline DB

Query PropertiesNear-line storage to be read as well

© SAP 2007 /

Authorizations Levels

Authorizations can be definedOn InfoCube levelOn characteristic levelOn characteristic value levelOn key figure levelOn hierarchy node level

Authorization

Authorization

Autho-rization

On key figure levelOn characteristic value level

On characteristic level

© SAP 2007 /

Introduction to Analysis Authorizations

Authorization Check okQuery results will be shown ifquery selection is a propersubset of the authorization

Authorization Check not okQuery results will not be shown at all (‘not authorized’)– even if parts of the authorizations are met

Authorizations

QuerySelection

Authorizations

QuerySelection

© SAP 2007 /

Authorization Relevant Characteristics

Before restrictingauthorizations oncharacteristics, you haveto mark them asauthorization-relevant.

InfoObject maintenance / transaction RSD1

© SAP 2007 /

Authorizing Characteristic Values – 1 –

Scenario: A group of usersis authorized only tospecific salesorganizations (e.g. Berlinand Birmingham)

Central maintenance for(analysis) authorizations /transaction RSECADMIN

© SAP 2007 /

Authorizing Characteristic Values – 2 –

A group of users isauthorized only to specificsales organizations (e.g.Berlin and Birmingham)

Possible ValuesEQ: single valueBT: range of valuesCP: contains (simple) patterns ending with ‘*’ or ‘+’(e.g. XY*)

(Berlin)(Birmingham)

© SAP 2007 /

Authorizing Navigational Attributes – 1 –

If you want to grantauthorizations onnavigational attributes,mark them in the attributetab strip as authorizationrelevant.

© SAP 2007 /

Authorizing Hierarchies – 1 –

On the same level like thevalue authorization, youcan also grantauthorizations onhierarchy levels.

Assume you’ll have asales organization asdepicted.

© SAP 2007 /

Authorizing Hierarchies – 2 –

Now you grant access forthe complete Americasand France.

You can also usevariables for flexiblyand dynamicallydetermininghierarchy nodes.

© SAP 2007 /

Special Authorizations

Special authorizations* (asterisk): denotes a set of arbitrary characters+ (plus): denotes exactly one character (e.g. 01.++.2005 until 10.++.2005 : allowsaccess only the first 10 days of each month in 2005 - only available for timevalidity (0TCAVALID)): (colon): allows only aggregated access to data (e.g. allows information on allsales areas only on aggregated level – not on particular countries)

Key figure authorizations

For key figure authorizations, you can include 0TCAKYFNM ascharacteristic into the authorization. Note: hierarchy authorizations are notallowed on this characteristic.

Note: Once you define 0TCAKYFNM authorization-relevant, key figuresare checked for every InfoProvider.

© SAP 2007 /

Selection and Authorization

Check of AuthorizationsSelection of query will be checked against the union of the authorizationsExample:

One authorization grants access to cost center 1000 for year 2004, a secondone grants access to the same cost center for year 2005Access to a query selection with cost center 1000 and years 2004 and 2005 willbe granted

Note: In the former concept of authorization objects, the query selection had tobe in the intersection of the two authorization object if the authorization shouldbe checked (i.e. the mentioned query was not authorized)

Year

200

5

Year

CostCenter

Year

200

4

CC 1000

© SAP 2007 /

Comparing Authorization Concept

Comparison Analysis Authorizations<= SAP NetWeaver 2004 vs. SAP NetWeaver 7.0

Most important differences

Analysis AuthorizationAuthorization Objects

<=SAP NetWeaver 2004 SAP NetWeaver 7.0

Technical Foundation

ChangeableNot ChangeableAfterwardsMaintenance

Number of InfoObjectsnot limited10 objectsNumber of objects

IndividuallyOnly on global basisNavigational Attributes

Equivalent to valueauthorizations

Via GUID and0TCTAUTHHHierarchy Authorizations

Union (‚as expected‘)Intersection of businessobjects

Composition ofauthorizations

Only InfoObject settingPer InfoObject ANDInfoCubeAuthorization Relevance

© SAP 2007 /

Migration

Migration SupportABAP program RSEC_MIGRATION (use transaction SA38)No complete, automatic migration, but support

About 80% automatic migration expectedThe more complex the existing authorization concept, the more manualmigration work might be necessaryCustomer-exit variables for 0TCTAUTHH cannot be migrated; the respectivehierarchy nodes must be assigned manuallyIntensive tests are highly recommended

Singular event, not for schedulingDuring migration to the new authorization concept, the existing concept won‘t bechanged

© SAP 2007 /

Customer Pain Points

Increasing datavolume

Increasing numberof information

workers

AdditionalAdministration

effort

Information at thespeed of thought

Quick and easyscalability

Reducecost of operation

significantly

© SAP 2007 /

SAP NetWeaver BI AcceleratorValue Proposition

SAP NetWeaver BI Accelerator

Very fast queryresponse time

Stable queryresponse time

High scalability Low maintenance

Performanceimprovements

by factor 10 – 100

Independent ofDB optimizer,aggregates, ...

No aggregatemaintenance, minimized

roll-up/change run

Implemented for latestblade server hardware

platforms

Increasedquality of

information/ExtendedBI reach

Significantreduction of

operationcosts

© SAP 2007 /

SAP NetWeaver BI Accelerator

SAP NetWeaver BI Accelerator for high performance BIA new transparent approach to boost BI query performance

Performance speedup factor between 10 and 100Without changing the BI user experience (transparent to users)Pre-requisite: BI in SAP NetWeaver 7.0

DBMS

Database

SAP NetWeaverBusiness

IntelligenceBI Accelerator

Queries Queries

X

© SAP 2007 /

SAP NetWeaver BI Accelerator Scenarios

Ready for high data volumesQueries that routinely involve access to many millions of records and may involve up tobillions of recordsExamples: retail, utilities, telephone companies

Challenging response time SLAsExample: service level agreements for call center operators demand short response timesfor good closure rates

Unpredictable types of queriesFar more different data sets and aggregations than traditional optimization and cachingstrategies can handleExcellent response times for any drill-down, slice & dice, …Examples: on-demand reporting for different user groups,ad hoc analyses

Minimizing costs of operationMaintenance of aggregates can be significantly reducedReduced roll-up and change run times

© SAP 2007 /

Aggregates

AggregatesPre-aggregated (sub-)InfoCubesAlternative to SAP NetWeaver BI Accelerator

Database /Selection

Analytic Engine

Aggregate

Month RevenueJuly 30August 30

Month RevenueJuly 30August 30

Month Material RevenueJuly Hammer 10July Nail 20August Hammer 10August Nail 20

© SAP 2007 /

Query Cache

Query CacheStores query results in cross-transactional application bufferRe-use of similar query results – also for other usersCan be actively used for performance improvement pre-load the cache viainformation broadcasting

InfoCube(if BIA is used, InfoCube data on database is not read)

Aggregates orSAP NetWeaver BIA

Query Cache

© SAP 2007 /

Other Performance Options

Modeling optionsMultiProvider (semantic) partitioningLine-item dimensions

Database featuresIndexingDatabase Statistics

© SAP 2007 /

Compression

CompressionMove data from F to E fact tableCompression usually reduces the number of records by combining records withthe same key that has been loaded in separate requestsWhen dealing with non-cumulative key figures, it is highly recommended toregularly compress (also when using SAP NetWeaver BI Accelerator)Double fact table

“F” Table– Request Information– Typically small– Optimised for Loading

and Deleting“E” Table– Optimised for Queries– Typically large– User-defined DB Partitioning

(depending on the DBMS)– But: no information on requests

Fact Table

REQUEST No. Time Material Sales

F - Table

REQUEST No. Time Material Sales

E - Table

Compression

Upload

As InfoPackages are added, Ffact table partitions are created

© SAP 2007 /

Copyright 2007 SAP AGAll rights reserved

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changedwithout prior notice.Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge and other SAP products and services mentioned herein as well as theirrespective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned andassociated logos displayed are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This documentcontains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy,and/or development. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, orother items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties ofmerchantability, fitness for a particular purpose, or non-infringement.

SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitationshall not apply in cases of intent or gross negligence.The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in thesematerials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages

Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche Genehmigung durchSAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden.Einige von der SAP AG und deren Vertriebspartnern vertriebene Softwareprodukte können Softwarekomponenten umfassen, die Eigentum anderer Softwarehersteller sind.SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge und andere in diesem Dokument erwähnte SAP-Produkte und Servicessowie die dazugehörigen Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und in mehreren anderen Ländern weltweit. Alle anderen in diesem Dokument erwähntenNamen von Produkten und Services sowie die damit verbundenen Firmenlogos sind Marken der jeweiligen Unternehmen. Die Angaben im Text sind unverbindlich und dienen lediglich zuInformationszwecken. Produkte können länderspezifische Unterschiede aufweisen.

Die in diesem Dokument enthaltenen Informationen sind Eigentum von SAP. Dieses Dokument ist eine Vorabversion und unterliegt nicht Ihrer Lizenzvereinbarung oder einer anderenVereinbarung mit SAP. Dieses Dokument enthält nur vorgesehene Strategien, Entwicklungen und Funktionen des SAP®-Produkts und ist für SAP nicht bindend, einen bestimmtenGeschäftsweg, eine Produktstrategie bzw. -entwicklung einzuschlagen. SAP übernimmt keine Verantwortung für Fehler oder Auslassungen in diesen Materialien. SAP garantiert nicht dieRichtigkeit oder Vollständigkeit der Informationen, Texte, Grafiken, Links oder anderer in diesen Materialien enthaltenen Elemente. Diese Publikation wird ohne jegliche Gewähr, wederausdrücklich noch stillschweigend, bereitgestellt. Dies gilt u. a., aber nicht ausschließlich, hinsichtlich der Gewährleistung der Marktgängigkeit und der Eignung für einen bestimmten Zwecksowie für die Gewährleistung der Nichtverletzung geltenden Rechts.

SAP übernimmt keine Haftung für Schäden jeglicher Art, einschließlich und ohne Einschränkung für direkte, spezielle, indirekte oder Folgeschäden im Zusammenhang mit der Verwendungdieser Unterlagen. Diese Einschränkung gilt nicht bei Vorsatz oder grober Fahrlässigkeit.Die gesetzliche Haftung bei Personenschäden oder die Produkthaftung bleibt unberührt. Die Informationen, auf die Sie möglicherweise über die in diesem Material enthaltenen Hotlinkszugreifen, unterliegen nicht dem Einfluss von SAP, und SAP unterstützt nicht die Nutzung von Internetseiten Dritter durch Sie und gibt keinerlei Gewährleistungen oder Zusagen überInternetseiten Dritter ab.

Alle Rechte vorbehalten.

sap net weaver 2004s enterprise data warehousing

Internet

data containersinfoproviders

data modeling2

data security6

data flow designthe

enterprise data warehousinge

data securitythis process

data flow design3

data consolidation rules