informix user group france - 30/11/2010 - fonctionalités ids 11.7

User Group Informix France

Informix 11.70: fonctionalités

Olivier Bourdin

[email protected]

Mardi Mardi 3030 Novembre Novembre 20102010


Overview

� Introduction

� Administration

� Usability/Extensibility

� Performance/Warehousing

� Grid Availability

� Security

� RAS


Administration

� Table Defragmenter (OLTR)

� Storage Provisioning

� Storage Space Schema

� Embedability Toolkit

� Deployment Utility and Assistant

� Informix 11.70 Install Changes


Table Defragmenter - OLTR

� Perform similar task as PC Disk defragmenter

� Reorganize the table into fewer and larger contiguous extent(s)

� Can be performed online, so no downtime, no unavailability of the table

� Originally called Table Reorganization (OLTR)


Storage Provisioning

� Highly configurable, automatic expansion of dbspaces, temporary dbspaces, sbspaces, temporary sbspaces and blobspaces

� Chunks are extended and/or created as needed

� IDS can also expand spaces before they are full

� Feature is fully incorporated into OAT.


Storage space Schema

� An enhancement of dbschema utility

� Generate schemas for dbspace, chunks, physical logs and logical logs

� Generated necessary commands for recreate existing dbspaces, chunks, physical log and logical logs

– Commands can be SQL Admin API

– Alternatively onspaces and onparams (non-sql) commands



� Usage: dbschema -

USAGE:

dbschema { [-q] [-t tabname] [-s user] [-p user] [-r role] [-f procname]

[-hd tabname] -d dbname [-w passwd] [-seq sequence] [-l [num]]

[-u [ia] udtname [all]] [-it [Type]] [-ss [-si]] [filename]

[-sl length] [-nw] } |

{ [-q] -c [-ns] [filename] }

-q Suppress the db version from header

-t table name or "all" for all tables

-s synonyms created by user name

or "all" for all users

…

…

-c generate storage space and logical log definitions

-ns generate commands in non-sql format



� Admin API output:

-- Dbspace 1 -- Chunk 1

-- EXECUTE FUNCTION TASK

('create dbspace', 'rootdbs', '/usr2/as_support/sanjitc/1170/rootdbs',

'200000', '0', '2', '500', '100');


EXECUTE FUNCTION TASK

('create dbspace', 'datadbs1', '/usr2/as_support/sanjitc/1170/

datadbs1_1170', '1000', '8', '2', '100', '100');



('create dbspace', 'datadbs3', '/usr2/as_support/sanjitc/1170/

datadbs3_1170', '1000', '0', '4', '200', '200');


('add mirror', 'datadbs3', '/usr2/as_support/sanjitc/1170/datadbs3_1170',

'0', '/usr2/as_support/sanjitc/1170/mdatadbs3_1170', '12');


('start mirroring', 'datadbs3');



� Non-SQL output:

# Dbspace 1 -- Chunk 1

# onspaces -c -d rootdbs -k 2 -p /usr2/rootdbs -o 0 -s 200000 -en 500 -ef 100

# Dbspace 2 -- Chunk 2onspaces -c -d datadbs1 -k 2 -p /usr2/datadbs1_1170 -o 8 -s 1000 -en 100 -ef 100

# Dbspace 3 -- Chunk 3onspaces -c -d datadbs2 -k 4 -p /usr2/datadbs2_1170 -o 0 -s 1000 -en 200 -ef 200

# Dbspace 4 -- Chunk 4onspaces -c -d datadbs3 -k 4 -p /usr2/datadbs3_1170 -o 0 -s 1000 -en 200 -ef 200 -m /usr2/as_support/sanjitc/1170/mdatadbs3_1170 12


Embeddability Toolkit

� Informix Embeddability Toolkit empowers the customer with the tools and the information they need for easy automation of reduced footprint Informix deployments.

� It includes: • Informix Deployment Assistant (DA)

• Informix Deployment Utility (DU)

• DU's Configuration File

• Additional Scripts for Embedding Informix

• A Tutorial on an end to end Deployment Solution


Deployment Assistant (DA)

� Enables users to take snapshots of Informix instances and / or data spaces for deployment on other target computers.

� Allows for reduction of the packaged instance to the user's minimum desired configuration

� Provides both GUI and Command-Line Interfaces developed using Java/Eclipse SWT

� Snapshots produced by DA are ready for use by the Deployment Utility (DU) for deployment on the target computers



� Specify the location in which the snapshot of the Informix instance and/or data will be saved.

� The archive formats available are:

• BZip2 (*.tar.bz2)

• GZip (*.tar.gz)

• Tar (*.tar)

• Zip (*.zip)



� A summary of the snapshot to be created is presented

� Selecting Next will begin creation of the snapshot with the selected features and/or data


Deployment Utility (DU)

� Standalone tool to rapidly deploy packaged Informix instances in an embedded environment

� Configuration file - $INFORMIXDIR/etc/ifxdeploy.conf

� Command line tool with customization options during deployment

ifxdeploy

[-config <configfile>] [-clone] [-extractcmd <command>]

[-verbose] [-l <logfile>] [-p <password>] [-silent] [-sqliport <port>]

[-drdaport <port>] [-servernum <num>] [-rootpath <rootdbs>]

[-file <tgz file>] [-relocate <path>] [-force] [-keepdis]

[-start [<secs>] [-autorecommend] [-y]


Usability/Extensibility

� Automatic DBA Procedures

� Automatic Table Locator

� Core Programability Enhancements

� Unique Event Alarms

� SPL Debugger

� Auto Registration


Autonomic DBA Procedures

� A collection of new database administration tasks templates is now added to the sysadmin database. They enable the DBA executing core and useful administration task automatically by the database server using the scheduler functionality instead of implement and running them manually.

� There are the new tasks

– Detect and recreate bad indexes

– InPlace alter removal

– Automatic CPU allocation at database server start

– Detect and remove idle user sessions

– Detect fragments which are candidates for a compression, repack, shrink or defragmentation (Data compaction)


Automatic Table location

� New table space location and fragment management introduction using the sysadmin task() function interface

� Space management – predefined one or more dbspaces attached to the database for locating newly

created tables

EXECUTE FUNCTION SYSADMIN:TASK(“autolocate database”, “stores_demo” , “datadbs”)

� Fragmentation management – automatically create a fragmented table in the dbspaces assigned to the current

database

– add new fragments to the table for out of disk space or server limitation like out of fragments conditions

– deferred extent allocation until first rows are inserted

� Table content management – round robin will not blindly add the row to the next fragment, empty or smaller

fragments are preferred for choosing a target table fragment for the insert


Core Programability Enhance.

� Consistent return codes for server initialization – Very helpful in Informix embedded environments

– Take appropriate action based on return code

#!/bin/sh

# Execute the oninit program

oninit

#Get the return code from oninit execution

RC=$?

# Validate the retun code and take necessary action

case $RC in

0) echo "RC=0: The database server was initialized successfully." ;;

1) echo "RC=1: Server initialization has failed." ;;

187) echo "RC=187: Check the entries in sqlhosts file." ;;

221) echo "RC=221: DUMPDIR missing. Creating DUMPDIR."

mkdir $INFORMIXDIR/tmp

chmod 770 $INFORMIXDIR/tmp ;;

*) echo "Return Code=$RC !" ;;

esac


Unique Event Alarms

� IDS uses the event alarm mechanism to notify the DBA about any major problems in the database server.

� Change in alarmprogram.sh/.bat EVENT_SEVERITY=$1

EVENT_CLASS=$2

EVENT_MSG="$3"

EVENT_ADD_TEXT="$4"

EVENT_FILE="$5"

EVENT_UNIQID="$6“

� IDS 11.70 has 79 Event Class Ids Class ID: 22 Class message: Long transaction detected

Event ID: 22002 message: Continuing Long Transaction (for COMMIT): tx:

Class ID:22 Class message: Long transaction detected

Event ID:22003 message: Aborting Long Transaction: tx:


Auto Registration

� Terminology change Built-in DataBlade Modules are now known as “Database Extensions”

� The Following Database Extensions will be Automatically Registered upon there first use after instance initialization:

• Basic Text Search

• Hierarchical Data Type (Aka Node Type)

• Binary Data Type

• Informix Web Feature Service for Geospatial Data

• Large Object Management

• MQ Messaging

� In addition the following DataBlade Modules will be Automatically Installed and Automatically Registered

• Spatial DataBlade

• Timeseries DataBlade

� On the fly create sbspaces, start VP


Performance/Warehousing

� Forest of Trees (FOT)

� MULTI-INDEX Scan

� STAR Index

� Fragment Level Statistics

� Fragment Level List/Interval

� C-UDR Preload


Forest Of Trees (FOT)

� This is a variation of a normal B-tree index and might provide an increase in performance depending on certain factors and usage. The creation of a FoT index is simple as it is a small addition to the create index syntax.

� Two problems were seen with a B-tree index ... • Root Node contention can occur when many session are reading the

same index at the same time.

• The depth of large B-tree index increases the number of levels created, which results in more buffer reads required.

� The FoT feature will address these problems by ... • Splitting a B-tree index into smaller subtrees.

• Each of these will have a separate Root Node, so queues will be shorter and quicker because we have spread the mutex contention cross many Root Nodes.

• Each smaller subtree will have less levels to navigate.



create [ unique ] index index_name on table ( column1 [, column2 ] ) [ in dbspace ]

hash on ( column1 ) with N buckets;

Hash on key value to pick a bucket / rootnode

Bucket 1 Bucket 2 Bucket 3

2,189,... 1,198,... 4,201,...

2,... 189,... 1,... 198,... 4,... 201,...

Rootnodes

Leaves

Key Value Bucket

1 2

2 1

3 2

... ...

47 3

221 1



� When to consider using a FoT index ...

– onstat -g spi output showing spin locks with large numbers of waits and loops:

Num Waits Num Loops Avg Loop/Wait Name

1427723 60715268 42.53 fast mutex, 7:bf[299] 0x3400006 0x108b7908000 security_s_symb_s_co_id

879324 40682456 46.27 fast mutex, 7:bf[24] 0x3400003 0x108b74bc000 pk_fk_last_trade

477038 19177763 40.20 fast mutex, 3:bf[391] 0x320000a 0x10177fe2000 daily_market_idx

436564 17239046 39.49 fast mutex, 7:bf[26] 0x3400004 0x108b74c4000 ecurity_s_symb_s_num_out

330244 14563521 44.10 fast mutex, 7:bf[30] 0x3400008 0x108b74d4000 security_s_symb_s_ex_id_s

176874 5183331 29.31 fast mutex, 7:bf[38] 0x3400008 0x108b74f4000 security_s_symb_s_ex_id_

27050 3636325 134.43 mutex lock, name = log

16555 1301066 78.59 fast mutex, lockhash[3104]

18101 885933 48.94 fast mutex, AIOSHCB lock

17274 879611 50.92 fast mutex, 3:bf[319] 0x1800002 0x10177f52000 trade_history_idx

– Reduces the B-Tree depth on larger tables ... • Reducing a 4-level B-tree index to a 3-level B-Tree might reduce the buffer reads

by 25 percent.


STAR/Snowflake Join Support

� Star Join is a new query processing method (ported over from XPS)

� Improves query performance for star and snow flake schema queries

� Utilize multi-index scan and skip scan

� Uses pushdown technology


MULTI-INDEX Scan

� Queries with multiple predicates on columns of a table will use the composite index only if the query has equality predicates on the leading columns of the index.

• Composite index (c1,c2) will be used for “c1 = 10 and c2 between 1 and 5”

• Cannot be used for “c1 between 1 and 10 and c2 = 5”; needs another index (c2,c1)

• For varying combination of the predicates, various combinations of composite indexes with the same set of columns will have to be created

� Multi-Index scan can now use multiple single column indexes

� Improves performance of queries that have predicates on multiple columns

� Reduces the total number of indexes as the various composite indices are not needed; reduces disk usage as a result


Fragment Level Statistics

� Allows storing of statistics at the fragment level and aggregating table level statistics from its constituent fragment's statistics.

� New catalog table sysfragdist stores statistics for each fragment for each table-column.

� Fragment level statistics are encrypted and stored in a smartblob column “encdist” in sysfragdist catalog.

� Controlled by new table property STATLEVEL [CREATE | ALTER] TABLE ... STATLEVEL [TABLE | FRAGMENT | AUTO]


Fragmentation List / Interval

� 2 new fragmentation strategies – LIST and INTERVAL

� LIST – Useful when fragment key has finite set of values. e.g: departments in an organization, colors, states of the country

� INTERVAL – Fragmentation based on an interval (days, months, hour, integer value) – DBA does not have to create all the possible fragments; defines the interval

value and initial fragments

– IDS will automatically create new interval fragments (if not present) when rows are inserted with fragment key values in that interval

� Supports these ALTER FRAGMENT options: – ADD/DROP fragment or dbspace name, ATTACH/DETACH fragment, INIT,

MODIFY partition name/expression/dbspace, ONLINE(altered table not exclusively locked)



� Fragments data based on a list of discrete values

– e.g. states in the country, departments in an organization, colors


C-UDR Preload

� Preloading a C-UDR shared library allows IDS threads to migrate from one cpuvp to another during the execution of the C-UDR

� With is feature, Lift the thread binding restriction. – Eliminate the thread binding and unbinding maintenance code

� Allowing thread migration during C-UDR execution can increase performance.

� Onconfig parameter PRELOAD_DLL_FILE

– PRELOAD_DLL_FILE $INFORMIXDIR/extend/bts2.0/bts.bld

– PRELOAD_DLL_FILE /app/myudr.so


GRID / ER / MACH11

� DDL on Updatable secondaries

� Rolling Upgrade support

� Replicate data using ER without PK

� ER: Logical log lag action configuration

� MACH11: Transaction Survival

� CDR Check repair with Timestamps

� Connection Manager for ER

� Informix Flexible Grid


DDL on Updatable Secondaries

� This feature allows sessions that are connected to any type of

updatable secondary to perform DDL SQL statements.

� This feature is enabled as soon as onconfig parameter, UPDATABLE_SECONDARY, is set to a value > 0.

� List of DDL commands which still can not be run: Update statistics, Create database (with no logging) Create raw table, Create temp table (with logging) Create external table, Create/Drop xadatasource, Create/Drop xadatasource type, Creating any type of virtual table


Rolling Upgrade Support

� Provide a server side infrastructure to support rolling upgrade of HDR or RSS system

� Two main usages of this feature • Easily instantiate an ER system

• Convert existing HDR or RSS system into an ER system for rolling upgrade support

� Ability to convert HDR or RSS pair to ER and setup replicate for every table in the system automatically

• Uses ERKEY on tables with no primary key

� One of the nodes continue to be online and allow queries including DML while the other undergo product upgrade

“Product upgrade without downtime”


ER without PK

� This is a feature that enable replication of tables without primary key.

� The idea behind this feature is to support rolling upgrade and grid replication.

� Two new external interfaces added to SQL and CDR utility to enable automatic creation of primary key.

• create table .... with erkey

• alter table ... add erkey

� Three erkey columns, two integer type and one smallint will be added to table.

� The table will have an unique key constraint built on the 3 erkeys.


ER: Logical log lag action

� When using Enterprise Replication if the current log position comes close to overwriting a log that has not yet been processed then the db server may take one of two actions:

• Add additional logical logs.

• Enter DDRBLOCK mode.

� Whilst in DDRBLOCK mode user update operations are blocked.

� The Logical Log Lag Action configuration parameter CDR_LOG_LAG_ACTION provides additional options:

• 1 Logical log staging

• 2 Dynamic log addition

• 3 Ignore

• 4 Shutdown (ER)

• 5 DDRBLOCK


Transaction Survival

� Transaction survival allows transactions started on updatable secondaries to survive failover to a new primary, in the event of the original primary's failure.

� Turned on with 1 new onconfig parameter: FAILOVER_TX_TIMEOUT

Primary

New Primary

HDR Secondary

RSSClients

Clients


Connection Manager for ER

� Connection Manager on ER provides a form of Network Virtualization for application clients connecting to a named “Grid” of Enterprise Replication nodes within an ER Domain

� Network Virtualization for the Grid • Provide the ability to distribute connections from client applications via

Service Level Agreement definitions to designated Grid nodes

• Distribution based on LATENCY, FAILURE and WORKLOAD

– Note for LATENCY and FAILURE SLAs, Quality Of Data monitoring must be defined and started

� Enables customers to automate allocation of client connections to appropriate nodes within a Grid

• A database administrator may want to implement Connection Manager for ER in order to ensure appropriate distribution of connections


CDR check repair with timestamps

� Cdr check currently uses a master node to sync other nodes to

• Mismatched rows on target are updated to values contained on source

• Missing rows on target are inserted to match source

• Extra rows on target are deleted unless using the --extratargetrows option

� Cdr check can now synchronize data based on timestamps from any server

• Timestamp – all servers are updated with row containing the highest timestamp

• Deletewins – if row exists in delete table it is deleted, otherwise the highest timestamp wins


Grid Replication

� Grid is an extension to Enterprise Replication that allows ER to replicate the execution of statements

� DML/DDL statements, procedures, and functions can be replicated to, then run on grid target servers

� Replicates can be automatically created when new tables are created

� Eliminates the need for a primary key for replicates

� Adds the means to support Connection Manager on top of ER


Grid Replication

� Grid greatly simplifies administration of database objects for many servers

� Let's say we have 1000 servers in a grid. If we want to create a new table on all 1000 servers we only have to run the create table command one time

� We can also set up replication automatically when creating tables, thus reducing the effort and knowledge required to use ER

� Admin api routines can also be replicated across servers in a grid to reduce the effort of performing server administration tasks


RAS

� Oninit –i block

� Ifxcollect

� Dbschema no name


Security

� Row Level Auditing - SRLA

� Database Mapped Users

� Trusted Context


Row Level Auditing (SLRA)

� This feature allow to specify which tables are audited at row level

� The purpose is to avoid performance drop during auditing whenever all tables and rows are audited

� This feature is turned on by setting ADTROWS in adtcfg file, or dynamically with onaudit -R <value>

• Backward compatibility

� New SQL command are provided to define which tables are audited at row level

� An audit trail file is generated and filtered with onshowaudit utility


Database Mapped User

� Authenticate external users without an OS account

Database users no longer need

• An account Look-up in the local OS

• Enable password properties for external authenticators

� The DBSA “maps” an external user to:

• An existing OS user or

• A database defined UIG/GID pair

� Uses an extension to the GRANT/REVOKE SQL statement

� Requires PAM or SSO authentication

� Traditional password based authentication is still available


Trusted Context

� This feature allow to reuse a connection for a different user without the need to establish a new connection

� The purpose is to avoid connection overhead when there is a need to connect on behalf of new users, which is the case in application server

� This also provide higher security level and better auditing granularity, as each user connect and switch on a trusted connection with their own credentials and privilege

� New SQL statement are provided and/or enhanced to create trusted context, establish trusted connection and provide connection switch capabilities

� All users properties are reported for the switched connection user when established


MerciMerci

Olivier Bourdin

[email protected]

Mardi Mardi 3030 Novembre Novembre 20102010

informix user group france - 30/11/2010 - fonctionalités ids 11.7

Documents