We’re gonna talk about...

B A MOO !

• How is BAM working? Details....• Challenges• AIMS BAM setup....in theory• Practial implementation, bugs and recommendations• Live experiences• Conclusion

Agenda

• Data from ports & orchestrations• Realtime data (close to)• Not many prerequisites• Performance data• Low impact - > tracking / pipelines is not an option• Automated deployment • Autmated maintenance / updates• Supports BTS2006R2 and newer

Challenges

• Integrated BizTalk functionality• Easy to enable, few prerequisites• Selectable tracking• Low overhead compared to tracking (unless you use it

the same way as tracking)• Global tracking can be disabled• No big changes in different BizTalk versions• Tracking profiles can be deployed «in flight», adapts to

changes

We ended up with BAM

How is BAM working?

MsgBox

BAMPrimary(activities)

AsynchTDDS

SynchEventstreams

Tracking

Tracking

• Scan BizTalk MgmtDb for artifacts & dependencies• Scan DTADb for orchestration XML to find shapes /

orch ports• Dynamically deploy new activities / tracking based on

changes

How to automate the monitoring setup with BAM?

That was the theory......sounds simple right?

• One activity for all orchestrations (ports & call / start shapes)

• One activity for all receiveports• One activity for all sendports

• Three trackingprofiles; receiveports / sendports / orchestrations

• Used Microsoft.BizTalk.Bam.TrackingCompiler.Track-ingCompiler / bm.exe

ISSUES• Limit in number of fields in an activity definition• We had to create one unique field per shape per

orchestration

AIMS BAM – take 1

• One activity per each common orchestration shape• One activity for all receiveports• One activity for all sendports

• One tracking profile receiveports, one for sendports, separate tracking profiles for each orchestration common shape activity

• Used Microsoft.BizTalk.Bam.TrackingCompiler.Track-ingCompiler / bm.exe

ISSUES• Win32Exceptions on the TrackingCompiler (limit on open

handles exceeded) due to large amount of ports in orchestrations

• GUI handles in the TrackingCompiler, probably for TPE• Timeouts

AIMS BAM – take 2

• One activity per each common orchestration shape• One activity for all receiveports• One activity for all sendports

• One tracking profile receiveports, one for sendports, separate tracking profiles for each orchestration common shape activity

• Used Microsoft.BizTalk.Bam.TrackingCompiler.Track-ingCompiler / bm.exe

ISSUES• Win32Exceptions on the TrackingCompiler (limit on open

handles exceeded) due to large amount of ports in orchestrations

• GUI handles in the TrackingCompiler, probably for TPE• Timeouts

AIMS BAM – take 2

• One activity per each common orchestration shape• One activity for all receiveports• One activity for all sendports

• One tracking profile receiveports, one for sendports, separate tracking profiles for each orchestration common shape activity

• Bttdeploy.exe/ bm.exe

ISSUES• Win32Exceptions on the bttdeploy.exe due to large

amount of ports in orchestrations• GUI handles in the TrackingCompiler, probably for TPE.

TPE dependencies in bttdeploy.exe (no logic, should been the other way around)

• Timeouts

AIMS BAM – take 3

• One activity per each common orchestration shape• One activity for all receiveports• One activity for all sendports

• One tracking profile receiveports, one for sendports, separate tracking profiles for each orchestration common shape activity

• Bttdeploy.exe/ bm.exe but this time more & smaller tracking profiles

ISSUES• Win32Exceptions on the TrackingCompiler due to large

amount of ports in orchestrations• GUI handles in the TrackingCompiler, probably for TPE• Timeouts

AIMS BAM – take 4

• One activity for each orchestration shape• Multiple activities for all receiveports (100 per activity)• Multiple activities for all sendports (100 per activity)

• Multiple tracking profiles for receiveports, multiple for sendports, separate tracking profiles for each orchestration shape activity

• Bttdeploy.exe/ bm.exe

ISSUES• Time consuming, thousands of trackingprofiles / activities• No changes to the BizTalk possible while deploying• Orphaned trackingprofiles

AIMS BAM – take 5

• Good up to certain configs (6000+ components) / high throughput

• TDDS not «cleaning» MsgBox quickly enough, leading to throttling

• Condition worsening if tracking hosts & BAM are not properly scaled

• Messaging has priority over TDDS if running TDDS on shared host (which still happens!)

• Tested on multi MsgBox setups, scaled out BAM (dedicated server), single box setup, multi server setup, different clustering of BizTalk servers and SQL servers

Performance

• Test & dev environments are subject to rapid changes

• New TP deployments / removals necessary for each change

• Takes time......and in the middle of this someone runs a deploy / undeploy

• In many cases lead to orphaned tracking profiles

Test / Dev environments

• BTS 2009 / 2010 with BAM and passthrough pipelines -> stopped processing of messages

• No existing fixes for BTS2009• BTS2010 CU 2-5 solved issue, CU 6-7 breaks support

• DB locking during long deploys, caused uncomplete setup

• XML encoding issues in tracking profiles (unknown chars)

• Orchestrations without XML in DTADb

Other bugs & issuesXML declaration is removed at a receive location that uses BAM tracking and the PassThruReceive pipeline in BizTalk Server 2009 / 2010

• Use in stable environments with low frequency on changes and make sure you update your BAM tracking accordingly

• Make sure you scale TDDS hosts, use dedicated tracking hosts

• Use synch BAM if you want to bypass the MsgBox. However, synch «eats» cycles of the processing and can only be done from code (orchestrations, pipelines, externals etc)

• Use asynch if you care about performance. Only option when using bttdeploy.exe. MsgBox is used for caching so monitor its size carefully

• Ensure that your BAM SSIS packages are running

• Know how to clean out orphaned tracking

• Know when to use tracking instead of BAM. BAM is not necessary or good for everything

Recommendations

AIMS conclusion