nate locklin ◦ database analyst, ppg industries ◦ [email protected]@locklin.org steve tirone...
TRANSCRIPT
Three Best Practices with SSIS
Nate Locklin◦ Database Analyst, PPG Industries◦ [email protected]
Steve Tirone◦ Data Warehouse Analyst, Amerinet◦ [email protected]
About Us
Survey:◦ What is SSIS?◦ Use it occasionally◦ Use it every day◦ I wrote the book on advanced SSIS
Given that we didn’t know, we wanted to present something generally useful, hence◦ “Three best practices to keep in mind when
developing and deploying SSIS packages”
SSIS Experience?
Open BIDS (Business Intelligence Development Studio)
Available in VS2005, VS2008, but not VS2010 (yet)
Incidentally, the Import/Export wizard in Management Studio runs SSIS under the hood
First, a Baseline Intro
Several tabs at the top◦ Control Flow: your top level workflow workspace◦ Data Flow: details of data transfer operations◦ Event Handlers: where you can handle raised
events ◦ Package Explorer: like a site map of your package
Start by dragging Tasks to your Control tab Most commonly used tasks are probably:
◦ Data Flow◦ Execute SQL
Constructing your Package
Let’s import the DJIA stock prices◦ downloaded from Yahoo Finance into an Excel
spreadsheet
Quick Example
Demo: Quick Package
Two often used log providers◦ Text Files◦ SQL Server (sysssislog in 2008, sysdtslog90 in
2005) – created when first used Many events possible to log
◦ OnError and OnTaskFail events most common◦ Can reduce detail to Computer, Source, and
Message◦ Here you will find all the error messages:
http://msdn.microsoft.com/en-us/library/ms345164.aspx
Best Practice #1: Logging
Demo: Logging
Properties of objects can be changed at run-time
No need to change the package itself to promote from test to production◦ Connection string is all that’s needed to change
Several formats:◦ XML configuration file most commonly used
Best Practice #2: Configurations
Demo: Configurations
Checkpoints let a package start where it left off previously, such as when an error occurs in the middle of your package◦ Lets you fix the error, then restart the package,
without it re-running the parts already done
Best Practice #3: Checkpoints
Set the following properties of the Package◦ CheckpointFileName = <some name>◦ CheckpointUsage = IfExists◦ SaveCheckpoints = True
Set the following properties on all the tasks:◦ FailPackageOnFailure = True
Setting up for Checkpoints
Demo: Checkpoints
Most importantly: ◦ “A package can be restarted only at the control
flow level. You cannot restart a package in the middle of a data flow.”
Also:◦ “When a package is restarted the Foreach Loop
containers and its child containers are run again. If a child container in the loop runs successfully, it is not recorded in the checkpoint file, instead it is rerun.”
Checkpoint Caveats
RetainSameConnection◦ Using lots of connections? Use this, please!◦ Can be found as a property of the Connection Mgr
Packet size◦ Consider increasing (32,767) for larger data
Row Per Batch Size◦ Sometimes speed is enhanced when setting the
“Rows per batch” to something other than blank, like 1000, 5000 or 10,000 (particularly when run through BIDS over the network)
Other Design Considerations
Property of the Connection Manager
Retain Same Connection
On All page of the Connection Manager
Packet Size
Package protection level
Default is Encrypt with User Key, but you want Encrypt with Password (all or sensitive)
Deployment Considerations
Know your environment (64 bit vs 32 bit)◦ Might get
[Connection manager "Excel Connection Manager 1"] Error: SSIS Error Code DTS_E_OLEDB_EXCEL_NOT_SUPPORTED: The Excel Connection Manager is not supported in the 64-bit version of SSIS, as no OLE DB provider is available.
◦ Or Class not registered
The 64 bit issue may be addressed by telling Integration Services to use 32 bit runtime
Deployment Considerations
In the SQL Agent job step:
64/32 bit Options
Property of the PROJECT:
64/32 bit Options (BIDS)
Great flexibility in error handling if you “program” the OnError handler
Error Handling
Implement Logging Use Configuration Files for flexibility Checkpoints can be useful Know your environment Keep small design considerations in mind
Summary
Nate Locklin◦ Database Analyst, PPG Industries◦ [email protected]
Steve Tirone◦ Data Warehouse Analyst, Amerinet◦ [email protected]
Slides are at http://tinyurl.com/SSISBestPractices
Questions?
Future BI Topics?◦ Integration Services (SSIS)◦ Reporting Services (SSRS)◦ Analysis Services (SSAS)
SQL Saturday?◦ BI Edition?◦ Volunteers to help coordinate?
Email Gina or Steve
Another Survey! Yeah!