Distributed Transaction &

Long-running transactions

Rossen ZhivkovFreelance SharePoint Consultant

January 19th, 2008Sofia, Bulgaria

Krasimir ParushevConsultant, ITCE ltd.

What if…

What if you have to update a SQL Server database and Oracle database in one transaction?What if you have to call two web services in one transaction?What if you have to implement transaction that lasts for days?

What is…

What is distributed transactionWhat is 2-phase commit protocolWhat is transaction coordinatorWhat is resource manager

What is long-running transactionWS-Coordination, WS-AtomicTransaction, WS-Business ActivityWhat is BPEL4WS

Roles in Transaction System

ApplicationBegins the transactionMakes changes to resourcesCan Commit or Abort the transaction

Transaction ManagerTransaction demarcation, participants, and statePrepare result is made durableCan Abort the transaction

Resource ManagersControls changes to resources under the scope of a transactionUses a (durable) log for recoveryCan Abort the transactionNeed to participate in 2PC (Atomicity)Provide Isolation of its resourcesProvide Durability of its resources

Local Transactions

DBMSDBMS ApplicationApplication

ActivityActivity

ActivityActivity

ActivityActivity

ADOADOConnConn

Client 1Client 1

Client 2Client 2

Client NClient N

Transaction Transaction ManagerManager

LockLockManagerManager

DataData

VB VB ObjectObject

ADOADOConnConn

VB VB ObjectObject

ADOADOConnConn

VB VB ObjectObject

Local Transactions

All updates within a single DBMSOr other “resource manager” (RM)

DBMS provides transaction demarcationBegin (start), end (commit), abort (rollback)

DBMS provides transaction managerDBMS provide lock manager

Computer 3Computer 3

RM3 - ORACLERM3 - ORACLE

Participating DTCParticipating DTC

Computer 2Computer 2

RM2 -SQL ServerRM2 -SQL Server

Participating DTCParticipating DTC

Computer 1Computer 1

The Coordinating DTC The Coordinating DTC

RM1 - MSMQRM1 - MSMQ

A Distributed Transaction

DataData

DataData

Your Tx COM+ Application Your Tx COM+ Application

QueueQueue

RM ProxyRM Proxy

RM ProxyRM ProxyRM ProxyRM Proxy

Distributed Transactions

Transactions may span multiple RMsDatabases, Queues, etc.

Distributed Transaction ManagerProvides common transaction demarcationCoordinates commit/abort across RMs

2 Phase Commit (2PC) ProtocolRMs are “prepared” to either commit or abortRMs are notified of final outcome

RM recovery

Resource ManagersNeed to participate in 2PC (Atomicity)Provide Isolation of its resourcesProvide Durability of its resources

Two-Phase Commit

Transaction ManagerDispenses new transactions to clientsTracks the resource managers participating in the transactionCoordinates the transaction outcome

Resource ManagerA subsystem that allows updates to transaction protected data

SQLSQLServerServer

TransactionManager

SQLSQLServerServer

FileFileSystemSystem

ResourceManagers

Resource Manager

Two-Phase Commit

TransactionManager

ResourceManager

Prepare

Commit

Prepare and Force write “Prepared” record to RM log

Prepared

Force write “Commit” record to TM log

Lazy write “Committed” record to RM log and Release locks

CommittedLazy write “Committed” record to TM log

Two Phase Commit – Failure

TransactionManager

ResourceManager

Prepare

Commit

Prepare & Write “Prepared” record to RM log

Prepared

Write “Commit” record to TM log

Write “Committed” record to RM log & Release locks

Committed

Write “Committed” record to TM log

Abo

rt o

n F

ailu

reC

omm

it

Abo

rt o

nF

ailu

re

Indo

ubt

on F

ailu

reC

omm

it

Will

Abo

rtW

illC

omm

it

LIVE DEMO

MS-DTCDistributed Transaction Coordinator

Windows NT ServiceTwo Phase Commit (2PC) and Recovery

InfrastructureCOM Interfaces (a.k.a., DTC proxy)

Used for App TM, TM RM communication

Local TM, remote TM, TM failover

Supported Resource Managers (RM)SQL Server, MSMQ, BizTalk, COM+ CRM, …Oracle, DB2, …

SQL Server And Dist Tx

Client initiated distributed transactionExisting DTC transaction is passed to SQL Server using data access API

ODBC SQL_ATTR_ENLIST_IN_DTC connection attribute

OLE-DB ITransactionJoin::JoinTransaction

ADO Only implicit via MTS/COM+

System.Data.SqlClientv1.1 SqlConnection.EnlistDistributedTransaction

System.TransactionsLater…

DB-Library – dbenlisttrans

COM+, System.EnterpriseServices, WCF

SQL Server And Dist. Tx…

Server initiatedBEGIN DISTRIBUTED TRANSACTIONLinked Server callExtended Stored ProceduresX-TransactionsPartitioned Tables

NT NT

SQLSQLServerServer

SQLSQLServerServer

MS DTC

ApplicationApplication

MS DTC

Using distributed transactions in .NET System.Transactions

System.Transactions = API for programming local & distributed transactions

Enlisting resources in distributed transactionsAmbient TransactionsTransaction Promotion

Transaction Flow

Transaction Flow

scope 4

{Suppress}scope 3{RequiresNew}

Transaction A

Transaction B

scope 1

{Required}

Code block withno ambienttransaction----- -- - - -- -- ---- –- – - - ----- -- –-- –- - –-- ------- -- - - –- –- –

scope 2

{Required}

Transactions from .NET application

Custom Resource Managers

Why to write own resource managersIncorporate custom resources in distributed transactionsImplement logic for two-phase commit steps (Prepare, Commit, Rollback)

Volatile resource managersTransactional variablesTransactional collectionsTransactional files….

Sample

Transactional NTFS & Registry

Kernel Transaction MemoryKernel objects in transaction (distributed / local)Allows atomic operations for:Many operations on single fileOperations for multiple filesOperations for many computersRegistry modifications

Sample

Windows Communication Foundation

What is WCF?Web services + WSE + WS-specifications + MSMQ + EnterpriseServices(COM+) + RemotingReliable messagingSecurable servicesProtocols support

New abstraction levelData AbstractionCommunication Abstraction

WCF and Transactions

WCF services & other resource managers can be executed within the scope of distributed transactionTransactable Services

In the Internet – using WS-AtomicTransactionIn the Intranet - DTC, OleTransaction protocol, System.Transaction

MSDTC and WS-AtomicTransactionsTransaction Flow

Incorporate web services in distributed transactions using WCF

Long-running transactions

The business scenarioWe ”cannot” lock the service sessions (and the underlying database) for that long time

Are ACID properties maintained?Is it OK to hold locks for days

Compensation actionsLogic to “undo” the effects from previously committed action

LIVE DEMO : Long-running transactions

WS-* specifications

WS-CoordinationDistribution of the transaction context

WS-AtomicTransactionPure ACID for short transactions with 2PC or similarLocking resources

WS-Business ActivityLong living transactions

To coordinate large work implemented by several small WS-AtomicTransaction services

CompensationsSOAP “compensate” message

Business Process Execution Language For Web Services - BPEL4WS

What is BPEL4WS:Orchestrating web services to implement business processComplex transactions modeled as … WORKFLOW

Provides sequences, branches, iterations, etcBound to local scope

Use WS-BusinessActivity to incorporate distributed scopes

Comparison: Distributed vs. Long-running

Transaction treesIn distributed transactions:

Parent NEVER commits before the child commits (2PC)Parent / Child hold locks till end of transaction

In long-running transactions:Parent can commit independently of the child transactionIf Child then rollbacks, parent executes compensation logicParent / Child release locks immediately after commit

Workflow Foundation

Business Processes orchestrationSupport for Distributed transactions via TransactionScopeSupport for Long-running transactions via Compensations

Persistence and TrackingIntegration with WCF & Web Services

Published as WCF/web serviceUtilizes WCF/Web service

Implement long-running transactions in Windows Workflow Foundation

Summary

Distributed Transactions2-Phase Commit ProtocolSQL Server 2005 & Distributed transactions.NET & Distributed TransactionsCustom Resource Managers

Distributed Transactions in Web ServicesWindows Communications Foundation & WS-AtomicTransaction

Long-running transactionsWorkflow Foundation BPEL4WS & WS-BusinessActivity

Next steps

Thank You for you time

Rossen Zhivkovrzhivkov@gmail.com

Krasimir Parushevkrasimir@itce.com