SQL Server Integration Services Dataflow

Data FlowThe Data Flow Task

Encapsulates the data flow engineExists in the context of an overall control flowPerforms traditional ETL in addition to other extended scenariosIs fast and scalable

Data Flow ComponentsExtract data from SourcesLoad data into DestinationsModify data with Transformations

Service PathsConnect data flow componentsCreate the pipeline

Data Flow TaskOne of the most valuable control flow tasks is the Data Flow Task.

Encapsulates the data flow engine

Load

Extract

Transform

Understanding a ETL Processing

Data Flow ElementsSQL Server Integration Services provides three different types of data flow components:

Data flow source - Sources extract data from data stores such as tables and views in relational databases, files, and Analysis Services databases. Data flow transformations - Transformations modify, summarize, and clean data.Data flow destination - Destinations load data into data stores or create in-memory datasets.

Integration Services PathsA Path connects two components in a data flow by connecting the output of one data flow component to the input of another component. A path has a source and a destination.

Defining Data Flow SourcesIn SSIS, a source is the data flow component that extracts data from different external data sources and makes it available to the other components in the data flow. Sources have one regular output, and many sources in addition also have one error output.All the output columns are available as input columns to the next data flow component in the data flow.

Sources extract data from: Relational tables and views Files Analysis Services databases

Understanding Data Flow Sources

OLEDB Oracle Connection

Data SourceSource Adapter

Data Flow DestinationsDestinations are the data flow components that load the data from a data flow into different types of data sources or create an in-memory dataset. Destinations have one input and one error output.

Destinations load data to: Relational tables and views Files Analysis Services databases and objects DataReaders and Recordsets

Enterprise Edition only

Understanding Data Flow Destinations

ADO.NET Connection TargetDestination

Adapter

Defining Data Flow TransformationsSSIS Transformations are the components in the data flow of a package that give you the ability to modify and manipulate data in the data flow. A transformation performs an operation either on one row of data at a time or on several rows of data at once. For example aggregate, merge, distribute, and modify data and also can perform lookup operations and generate sample datasets.

Understanding Data Flow Transformations

Mapping Columns and Dataflow Pipeline

DimProduct

ProductKey

Color

Name

Cost

DimProduct

ProductKey

Color

Name

Cost

DimProduct

ProductKey

Color

Name

Cost

Source

Transformation

Destination

Transformations

We can logically group them by functionality: Row Transformations Rowset Transformations Split and Join Transformations Auditing Transformations Business Intelligence Transformations Custom Transformations

Row Transformations The most common and easily configured transformations perform operations on rows without needing other rows from the source. These transformations, which logically work at the row level, often perform very well.

Update column values or create new columns

Transform each row in the pipeline input

Rowset Transformations Create new rowsets that can include

Aggregated values

Sorted values

Sample rowsets

Pivoted or unpivoted rowsets

Split and Join Transformations Distribute rows to different outputs

Create copies of the transformation inputs

Join multiple inputs into one output

Perform lookup operations

Auditing Transformations Integration Services includes the following transformations to add audit information and count rows.

Business Intelligence Transformations

cleaning data

updating of a slowly changing dimension

looks up values

mining text

running data mining prediction queries

The final grouping of transformations lets you perform advanced operations on rows in the data flow pipeline.

ADO.NET Connection

Dataflow Summary

Sources

Transformations

Destinations

OLEDB Oracle Connection

EXCELConnection