![Page 1: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/1.jpg)
Session: 17901 MQ for z/OS – The Insider Story
Paul Kettley
PLM for Messaging on z
![Page 2: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/2.jpg)
Agenda
• Transactions
• Queue Managers
• Resource Managers
• What does a transaction look like?
• Scenario Walkthrough
• Summary
![Page 3: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/3.jpg)
Agenda
• Transactions
• Queue Managers
• Resource Managers
• What does a transaction look like?
• Scenario Walkthrough
• Summary
![Page 4: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/4.jpg)
Transactions
ACID
STATE 1
CHANGE
Transaction Contract
Message Transaction STATE 2
![Page 5: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/5.jpg)
Agenda
• Transactions
• Queue Managers
• Resource Managers
• What does a transaction look like?
• Scenario Walkthrough
• Summary
![Page 6: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/6.jpg)
The Queue Manager - Delivering Messages
Queue
Database Application
EXEC
SQL QUERY
EXEC
SQL INSERT
MQPUT
MQGET WebSphere MQ
IMS
CICS
DB2
z
/
O
S
MQI
![Page 7: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/7.jpg)
...
A1
C3
B4
Pigeon Holes Pageset
1 12
2
3
4
5 6
7
8
9
10
11
Conveyor Belt Log
C3 A1 B4
C3 A1 B4
Core Technology - Logging and Buffering
?
![Page 8: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/8.jpg)
Accessing the MQI - Stubs and Adapters
CICS
Region Batch/TSO Channel
Initiator
IMS
Regions
MPP/BMP/FP
Appl
MQI Stub
RMI TRUE
Process
MQI Stub
Adapter Adapter
Program calls
Queue Manager
Appl
MQI Stub
Appl
MQI Stub
ESAF exits
(QRPL)
Control
![Page 9: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/9.jpg)
Agenda
• Transactions
• Queue Managers
• Resource Managers
• What does a transaction look like?
• Scenario Walkthrough
• Summary
![Page 10: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/10.jpg)
Building Blocks - Resource Managers
Buffer
Manager
Data
Manager
Log
Manager
Recovery
Manager
Connection
Manager
Message
Manager
CF
Manager
UR
Backout
MQI
Commit
Rlse
Redo
Undo
Ckpt
Ckpt
MQOPEN
UR
Lock
Manager
![Page 11: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/11.jpg)
Handling Applications - Connection Manager
MQCONN
MQOPEN
MQPUT
...
Control Blocks
Control
Blocks Logs
Data
Other RMs
THREAD
Application
Connection
manager
EOT
EOM SYNCPOINT
COORDINATOR PHASE 2
PHASE 1
![Page 12: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/12.jpg)
Controlling the MQI and MQSC - Message Manager
MQI
MQOPEN
Message Manager
Command, Runtime, Group servers
MQGET
MQPUT
MQPUT1
MQINQ
MQSET
Application
MQOPEN
MQCLOSE
Plus
Validation
Consistency
Triggering
Get-Wait
MQSC
CONSOLE
QSG
Objects
SYSTEM.CLUSTER
BASE
…Locking …Security
![Page 13: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/13.jpg)
Controlling Messages and Objects - Data Manager
Log Messages
Pagesets 1-99
Space group
Scavenger
SubQueue
Msg pointers
SubQueue
Msg pointers
Queue attributes
SubQueue
Msg pointers
Pageset 0
Objects
Space group
REDO
UNDO
Space group
1 12
2
3
4
5 6
7
8
9
10
11
![Page 14: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/14.jpg)
Bringing High Performance - Buffer Manager
checkpoint
write pageset
pageset
read pageset
DEFINE BUFFPOOL(bpid)
BUFFERS(nnnn)
buffer
pages
dirty
dirty
dirty
DEFINE PSID(psid)
BUFFPOOL(bpid)
NO FORCE
FORCE DIRTY PAGES > 3 CKPTS
CKPT,
LRU, STEAL,
WRITE AHEAD
OLDEST
![Page 15: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/15.jpg)
Local Message Queue Storage
Head
Start Scan
Tail
M1 (del) M2(del)
100MB
up to 100MB
very long message
...
Text up to 4MB MD First part of text Next part of text ...
M4
long message
... 4MB + of text Next part of text Text up to 8MB
M3
short messages 4K
4MB
Text up to 4MB MD First part of text Next part of text ...
Subqueue
![Page 16: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/16.jpg)
RESTART/RECOVERY log
pagesets
Handling Transactions - Recovery Manager
1 12
2
3
4
5 6
7
8
9
10
11
TRANSACTION STATES
1 12
2
3
4
5 6
7
8
9
10
11 1 12
2
3
4
5 6
7
8
9
10
11 1 12
2
3
4
5 6
7
8
9
10
11 1 12
2
3
4
5 6
7
8
9
10
11
COMMITTED
CKPT
BEGIN
IN
FLIGHT
IN
DOUBT
IN
COMMIT
1 PHASE
COMMIT BEGIN
IN
FLIGHT
IN
COMMIT COMMITTED
BEGIN
IN
FLIGHT
IN
BACKOUT
BACKED OUT BACKOUT
log
2 PHASE
COMMIT
LOG
LOAD
![Page 17: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/17.jpg)
Providing Logging Interfaces - Log Manager
• Log read and write functions
• Log Shunting
• Multiple active log data sets and archive log datasets
• Archive inventory management
• Duplexed for reliability
• “Bootstrap” file • End of log location
• Archive inventory
• Various Utilities
![Page 18: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/18.jpg)
Agenda
• Transactions
• Queue Managers
• Resource Managers
• What does a transaction look like?
• Scenario Walkthrough
• Summary
![Page 19: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/19.jpg)
Examining Transactions - A Log Print
00000000D569 URID(00000000D569) RM(RECOVERY) TYPE( START UR ) **** 00640024 00200001 03000000 0000D569 00000000 D545
0000 00240000 0000D000 00000000 00000700 00000000 00000000 00000000 0000D6C4
0020 D6E6C4C1 4040B5B4 8FA08793 02864040 40404040 4040C2C1 E3C3C840 4040D6C4
0040 D6E6C4C1 40400000 00000000 0000
00000000D5CD URID(00000000D569) RM(DATA) LRID(00000000.00000E01) TYPE( UNDO REDO )
SUBTYPE( DECREMENT BY )
**** 002A0064 0600000F C9000000 0000D569 00000000 D569
0000 00000000 00000E01 00040326 00000001 00000001
00000000D5F7 URID(00000000D569) RM(DATA) LRID(00000001.00000201) TYPE( UNDO REDO )
SUBTYPE( DELETE )
**** 0026002A 06000008 C9000000 0000D569 00000000 D5CD
0000 00000001 00000201 00000000 00000E01
00000000D61D URID(00000000D569) RM(RECOVERY) TYPE( START COMMIT1 )
**** 007C0026 00200002 03000000 0000D569 00000000 D5F7
0000 00240000 0000D000 00000000 00000700 00000000 00000000 00000000 00004040
0020 40404040 40400000 00000000 00000000 00000000 00000000 00000000 00000000
0040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
0060 00000000 0000
00000000D699 URID(00000000D569) RM(RECOVERY) TYPE( PHASE 1 TO 2 )
**** 0024007C 0020000C 03000000 0000D569 00000000 D61D
0000 00240000 0000D000 00000000 0000
00000000D6BD URID(00000000D569) RM(RECOVERY) TYPE( END COMMIT2 )
**** 00240024 00200010 03000000 0000D569 00000000 D699
0000 00240000 0000D000 00000000 0000
![Page 20: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/20.jpg)
Concurrency and Isolation - Lock Manager
1 12
2
3
4
5 6
7
8
9
10
11 1 12
2
3
4
5 6
7
8
9
10
11 1 12
2
3
4
5 6
7
8
9
10
11 1 12
2
3
4
5 6
7
8
9
10
11
request commit allocation
M1 M2
Y
Z
X
Y Z X
MQOO_INPUT_EXCLUSIVE
MQOO_INPUT_SHARED
MQOPEN
vs
DELETE QLOCAL
API Request
Transaction
Lifetime
Shared
Exclusive Exclusive
![Page 21: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/21.jpg)
Managing Shared Messages - Coupling Facility Manager
CF manager
QM3 MQ
DB2
CF manager
QM1
CF manager
QM2
List
Structure
DEFINE QL(queue)
CFSTRUCT(structure)
QSGDISP(SHARED)
IXL*
![Page 22: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/22.jpg)
committed puts
List Structure
uncommitted puts queue details
uncommitted gets
expired messages
SQ2 LH
SQM1 LH
SQM1 LH
MQCMIT
MQGET
SQ1 LH
(MQGET)
MQPUT MQGET
MQCMIT
KEYS
DATA
WRITE
READ
MOVE
DELETE
MONITOR
MsgId/CorrelId
State Priority Time Qmgr
Shared Message Queue Storage Using CF List Structures
![Page 23: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/23.jpg)
Agenda
• Transactions
• Queue Managers
• Resource Managers
• What does a transaction look like?
• Scenario Walkthrough
• Summary
![Page 24: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/24.jpg)
Scenario - MQPut to a Triggered Queue
Application Message
Manager
Data Manager Buffer
Manager
Recovery
Manager
Log Manager Lock Manager
MQOPEN
ACQUIRE
LOCK
LOCATE QUEUE
IN HASH TABLE
SECURITY
BASE NAME
ACQUIRE
HANDLE
MQPUT
USE HANDLE
LOCATE PAGE
TO HOLD MSG
BUFFER
PAGE
START UR LOG RECORDS
LOG RECORDS
CHECK
TRIGGER
RULES
MQCMIT
FORCE LOG
RELEASE
LOCKS
![Page 25: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/25.jpg)
Scenario - MQGet from a Queue
Application Message
Manager
Data Manager Buffer
Manager
Recovery
Manager
Log Manager Lock Manager
MQOPEN
ACQUIRE
LOCK
LOCATE QUEUE
IN HASH TABLE
SECURITY
BASE NAME
ACQUIRE
HANDLE
MQGET
USE HANDLE
FIND MSG
(INDEX / NEXT)
BUFFER PAGE
START UR LOG RECORDS
LOG RECORDS
MQCMIT
FORCE LOG
RELEASE
LOCKS
![Page 26: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/26.jpg)
Summary
• Delivers transactional messaging
• Enables robust business applications
• Complex, but well organised
• Adapters, Address spaces, Resource Managers
• Designed for throughput, availability and scalability
• Logging, Buffering, Locking, Communications
![Page 27: Session: 17901 MQ for z/OS The Insider Story...2 PHASE COMMIT LOG LOAD Providing Logging Interfaces - Log Manager Log read and write functions Log Shunting Multiple active log data](https://reader035.vdocuments.net/reader035/viewer/2022071107/5fe1ed01a48cc3790b473c69/html5/thumbnails/27.jpg)
Session: 17901 MQ for z/OS – The Insider Story
Paul Kettley
PLM for Messaging on z