magnet: database delta delivery tool
TRANSCRIPT
Magnet: solution for database delta
deliveries
Martins Kemme
The Challenge
You can’t deliver all database objects every time, because
• it may take too much time to recreate all database objects
• there are objects (like tables) that can’t be deleted and reinstalled (valuable data are there!)
Delta Delivery
“Delta delivery” - deliver only the items, that were modified recently
TimePreviously delivered changes
Changes that are new and need to be delivered now
Delta Delivery Challenges
• Take only modified items plus related items
• Decide what to do if this item has been modified several times?
• Find out detailed info about each file• what it is? • how it must be installed?
• Know the installation sequence
• Make a note that now these changes have also been delivered
Magnetfor Subversion repositories
What is Magnet?
Java tool, that operates on SVN repositories and can
• retrieve from Subversion repository only these files that are referenced in selected revisions
• create installation file for files referenced in selected revisions
• collect Subversion log messages from selected revisions into one text file (i.e. create documentation for the delivery)
• add specific properties to revisions in Subversion repository marking that these revisions are collected
The Algorithm
1. Determine file type either by location (directory) or by file name
2. Determine file installation method from file type
3. Determine installation method parameters
4. Create installation script that contains calls of installation methods for each file
Magnet: the result
• The following slides will present some examples of installation scripts generated automatically by the Magnet
• These scripts can be run automatically by continuous integration engine or by person manually
Unix Script Example#!/bin/bash
### Load all installation functions. ./installation_functions.sh
### Check if required Login Ids are entered else prompt to get themCHKSCHEME "APPS_ID" "APPS_ID"
echo [INSTALLER] $SYSDATE Starting installation...
echo [INSTALLER] Installing revision 5930INSTALLBINARY XXCMP "../applnode-concurrent/appl_top/xxcmp/11.5.0/bin/xxcmp_ar_loader.ctl"
echo [INSTALLER] Installing revision 5935RUNSQLPLUS $APPS_ID "@../dbinstall/apps/data_scripts/CMP-5048_DATAFIX.sql"
echo [INSTALLER] $SYSDATE Installation finished.
SQL*Plus Script ExampleSET verify ONSET timing ONSPOOL &spool_fileSET define off prompt [INSTALLER] Starting installation... prompt [INSTALLER] Installing TABLEconn &&ZV_HORIZON@branches/horizon/DB_scripts/ZV_HORIZON/table/gl_invoices.sql prompt [INSTALLER] Installing PACKAGE_BODYconn &&ZRZ_OWNER@trunk/source_code/DB_scripts/ZRZ_OWNER/package/p_legal_persons-b.sql prompt [INSTALLER] Installing GRANTconn &&ZV_ADMIN@trunk/source_code/DB_scripts/ZV_ADMIN/grant/771_setup_new_form.sql
Windows BAT file@echo offecho [INSTALLER] Starting installation... echo [INSTALLER] Installing TABLEcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/table/dmf_parts.sqlcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/table/labels.sql
echo [INSTALLER] Installing PACKAGE_SPECIFICATIONcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/package/p_zc_prices-h.sqlcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/package/p_legal_persons-h.sql
echo [INSTALLER] Installing PACKAGE_BODYcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/package/p_zc_prices-b.sqlcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/package/p_legal_persons-b.sql
echo [INSTALLER] Installation finished.GOTO :EOF :RUNSQL
...etc...
Magnet
Setup file(either .sh, or .bat or .sql)
Folder of collected files
List of deleted filesConsolidated log messages
Updated repository
Martins.Kemme (at) gmail.com
Thank you!