Abstract:

Columnar in-memory analytics is a hot topic that everyone is talking about these

days. DB2 10.5 delivered new technology known as BLU Acceleration. In this

session Chris will give an overview of BLU Acceleration but will spend most of the

hour discussing the technical comparisons between BLU Acceleration and other in

memory alternatives on the market like Oracle in-memory option, SQL Server

columns store, Teradata, SAP HANA and more. This will be an all technical

comparison so that you can walk away understand how BLU Acceleration fits in the

in-memory columnar landscape.

1

Objectives:

Understand the basics of BLU acceleration

Technical comparison to Oracle

Technical comparison to Microsoft

Technical comparison to Teradata

Technical comparison to SAP HANA

2

At this point in the conference (seven session in) you have probably already heard a

lot about BLU Acceleration so I will just focus on the key points that differentiate it

from competitors.

This slide describes at a high level what DB2 with BLU Acceleration is. What is the key business

value of implementing BLU Acceleration?

This is a new technology that has been developed by IBM and integrated directly into the DB2

engine. BLU Acceleration is a new storage engine along with integrated runtime (directly into the

core DB2 engine) to support the storage and analysis of column organized tables. The BLU

Acceleration processing is parallel to the regular, row-based table processing found in the DB2

engine. This is not a bolt-on technology nor is it a separate analytic engine that sits outside of

DB2. Much like when IBM added XML data as a first class object within the database along with

all the storage and processing enhancements that came with XML, now IBM has added column

organized tables directly into the storage and processing engine of DB2.

Simply put, this is a column-organized table store in DB2. Along with this store are many benefits

including significantly improved performance, massive storage savings and ease of

implementation and ease of management.

This feature allows us to deliver on these performance and storage innovations while also

optimizing the use of main-memory, improving I/O efficiency and exploiting high performance

CPU instructions and hardware characteristics that enhance the value derived from your

database investments.

5

This slide describes the at a high level the technology that underpins BLU Acceleration and warrants some discussion.

Dynamic In-Memory: This refers to the fact that BLU is optimized for in-memory analytics but it goes much further than that. We know that in a big data world there is no way you will always be able to fit all the data you have in memory at any given time. In fact most clients can’t even fit the amount of active data in memory without it becoming cost prohibitive. BLU Acceleration was designed with this in mind and is optimized for in-memory analytics without having to have all the active data fit in memory (unlike Oracle’s and SAP’s implementations). But we have taken this even further. With BLU Acceleration we are aware of the hardware capabilities we are running on and optimize the data on disk with a knowledge of the cache sizes and register sizes of the machine the database is on. We optimize access to columnar data to exploit the L2, L3 caches as well as optimize the data in the CPU registers to keep the CPU doing productive work at all times (a CPU that has to go back to main memory to get data is not productive) – today even main memory is too slow so we optimize the in chip analytics to take advantage cache lines and CPU cache prefetching.

Actionable Compression: With BLU acceleration the compression is extremely good (about 10x compression on average for tables). But making the data smaller is only one benefit. The bigger benefit is that the way the compression has been implemented allows DB2 to act on the data while it is still compressed. DB2 can perform predicate evaluations (equality and range predicates) as well as joins and aggregations while the data remains compressed. Some other vendors can do simple equality predicates on compressed data but it requires the patented order preserving compression that is in DB2 BLU Acceleration to be able to perform the broader set of operations that IBM delivers. This provides huge CPU savings as well as benefits for cache lines and main memory utilization and is one of the key advantages to BLU acceleration.

Parallel Vector Processing: Today’s CPU have special instructions that were initially designed for image processing and gaming applications. Knows as Single Instruction Multiple Data (SIMD) instructions, these operations allow an application to pack a CPU register with multiple data values and then perform an operation (like compare for example) on all the data values in the register with a single instruction. This can multiple the performance of the CPU by several factors (imaging packing the CPU with 4, 8 or more column values and then comparing them all at once with one CPU instruction compared to having to load value, compare, load value, compare, etc. etc. DB2 BLU also makes use of all the CPUs on a system even for a single query and has unique algorithms for parallelizing workloads to a very high degree all automatically.

Data Skipping: This is a technique whereby DB2 stores metadata about the values in a column for set of rows and then using that metadata during query processing to skip over data we know will not qualify for a given query. BLU keeps track of the minimum and maximum value for a set of columns in the table for every 1024 “rows”. By doing so if I have a query that is looking for example for rows that have DATE_COL > ’01/15/2014’ then DB2 can use the minimum and maximum information to not even both to look at data blocks that have a max DATE_COL of ’12/13/2013’ because we would know that there are no rows in that block that qualify. This helps to avoid I/Os and makes better use of main memory. Several products have this capability but what makes BLU unique is that we use data skipping even if the data is already in main memory. That is, if another query accessed some data and that data is already in the bufferpool but doesn’t qualify for my query, DB2 will skip over that data and not even bother to pull that data through the CPU cache and will not waste CPU cycles to find out that it doesn’t qualify. Again this makes BLU more CPU and memory efficient.

5

6

This slide shows the performance improvements some customers have

experienced with BLU Acceleration. The goal for BLU Acceleration was to

make 10x-25x performance improvements over row based tables the norm. The

good news is that many customers are exceeding our expectations and

achieving even higher levels of performance. Your mileage will vary of course

but we are seeing some queries that have in fact improved by over 1000x with

10-25x being the average performance improvement for a given workload.

First let’s compare to Oracle Exadata which actually has no in-memory columnar

capabilities to speak of but it’s the only thing publicly available from Oracle (at the

time of this conference).

The only columnar available in Oracle is with Exadata (not available on non Exadata platforms)

and is called Hybrid Columnar Compression. This is not a true columnar store but rather they

take sets of data pages and then rearrange the data on those pages by column rather than by

row and then compress those data pages. The entire set of pages makes up what is called a

compression unit and is the smallest unit of I/O for Oracle Exadata. When a row is needed

Oracle must read in the entire compression unit and cannot perform column level I/O.

Note: In Oracle, every compression unit has all columns – so you are not able to just read a

single column.

Let’s first look at a set of data pages (called blocks in Oracle) within a table.

Normal row storage is used for regular tables.

Oracle Exadata Hybrid Columnar Compression simply reorders the columns within

these set of data pages to place all the C1s close together on a page or set of pages,

C2s move together, and so on.

What you end up with is the same set of data pages where all the column values are

grouped together on a page or set of data pages. However, the full row is made up

into what is called a compression unit. In Exadata, a compression unit is the

smallest unit of I/O. This means that you cannot just read in column C1 into

memory for analysis…you still have to read all the columns for all the rows you

need to examine.

And when you update column in a row the row is removed from the compression

unit and DECOMPRESSED and stored at the end of the table. With BLU

acceleration the row is moved to the end of the table but remains in columnar

format and remains fully compressed.

The end result is that the table ends up with lots of holes in the compression unit

and the newly updated data is uncompressed thus making the table significantly

larger as update activity modifies the table over time.

As mentioned the compression unit is the smallest amount of I/O that you can do in

Exadata with HCC so when you want to just read C3 and C4 from above, you must

put all the data into the cache for processing, even the columns that you have no

interest in. Clearly not a columnar object.

As you can see the buffer cache or flash cache becomes polluted with all the

columns for all the rows that need to be processed. This wastes both memory and

CPU resources to go to the columns of interest.

This slide shows the storage index feature. Note that the storage index is not persistent or

permanent. It is dynamically created, and stored in exadata memory. It is created

automatically the first time a query arrives with a column predicate. The first query

submitted will see no performance gain, but will trigger the creation of the index.

Subsequent queries that filter on the same column are supposed to see the benefit. Oracle

only keeps track of 8 columns per storage cell object (i.e. table).

The problem with this technique is that the storage index has to be recreated every time the

storage cell is shut down for any reason. Also the columns of interest for one query may not

be those of interest for another so data skipping capabilities are severely hampered and in

fact non deterministic (a query may run fast one time and slow another time).

17

DB2 with BLU Acceleration has a vast array of advantages over Oracle Exadata. In fact all of the features

we spoke about on slide 5 of this presentation are competitive differentiators vs. Exadata as described

above.

18

DB2 with BLU Acceleration has a vast array of advantages over Oracle Exadata. In fact all of the features

we spoke about on slide 5 of this presentation are competitive differentiators vs. Exadata as described

above.

19

A typical data warehouse or data mart requires a lot of manual work to optimize and tune.

Picking partitioning strategy and compression options and creating and loading the data is

just the start. Then you have to add in auxiliary structures (indexes, aggregates, etc) to

optimize performance along with the memory and I/O tuning and query rewriting that often

is required for complex analytic systems. These added structures and tuning tasks are an

iterative process – someone comes up with a new query that isn’t optimized for the current

set of indexes and you have to go and tune yet again and perhaps add even more indexes.

It is not uncommon to see data marts where there is more storage allocated to indexes than

to data.

This all goes away with BLU Acceleration because there are no auxiliary data structures.

The only indexes are for primary key or unique column constraint enforcement. You just

create the tables, load the data and run queries.

In the fall of 2013 Oracle announced a new “In Memory Option” at OpenWorld

which is supposed to be generally available some time in 2014. After reviewing

their presentation material from that conference and publicly available sources

describing the technology on the internet, it’s clear that BLU Acceleration has a

number of significant advantages.

What is this new in memory option? What Oracle is announcing is that they intent to allow database

administrators to create a new type of “index” on top of their existing row organized tables. These new

“indexes” will be a full representation of the table but instead of being row organized they will be column

organized and they will live in memory only (not persisted on disk).

What this means is that when the database starts up, Oracle will scan through all the data in the row

organized tables the DBA wants to be “in-memory” and they will break the tables up into columnar format

and load those columns into memory for faster analytic queries. You can think of this as a columnar in-

memory cache of the row based data. They will keep the data in the columnar format in sync with the row

format as transactions modify the row format of the tables.

From a technical perspective, you can think of this like a DB2 Materialized Query Table (MQT) that is

only in memory in columnar format.

The DBA must specify which tables (or which partitions of which tables) they would like to be duplicated in-memory as columns stores. The DBA must also specify the amount of memory that they want to allocate to these in-memory duplicate copies of the data (separate from their traditional buffer pools or buffer caches). In most cases, analytic queries only access a small portion of the columns that make up the tables in the database so the DBA also has the option to include or exclude specific columns from the in-memory duplicate cache. This will become yet another huge tuning and management burden on the Oracle DBA to first determine what columns of what partitions of what tables are hot and would benefit from in-memory columnar caching. Then the DBA must calculate how big these objects will be in memory and allocate enough memory to hold them. Then the DBA must monitor this area to see if they are running out of space (as users add more data to the tables) and tune/adjust the space or add more objects as needed. Certainly not the “Load and Go” simplicity that DB2 with BLU Acceleration provides.

These in-memory columnar duplicates are loaded into memory either at first access or at system startup time (as defined again by the overworked Oracle DBA). This means that the system will perform huge numbers of I/Os to read in all of the data and then consume vast amounts of CPU cycles to break the rows into columns, compress the data and store it in memory. This data is not persisted on disk so every time you have to recycle the database or when the server or database crashes this process needs to run again to rebuild these in-memory columnar duplicates again. The last thing you need to happen when your database is being restarted is to have to consume extra CPU and extra I/O to rebuild these objects while you are trying to restart the database to make it available to your end users.

Note that the row tables still go into the regular buffer pool (Oracle calls them buffer cache) in memory so in effect Oracle isduplicating the memory for these objects by storing them once in row format and a second time in columnar format in memory. And in the case of the columnar in-memory duplicates, they only exist in memory so they must all fit in memory. They have not concept of bring parts of the data into memory and then overwriting those with other data that may be needed by other queries.

Finally Oracle did mention that they will have compression for these in-memory objects but they also mentioned that the compression won’t be very good (OK they didn’t exactly say that, they said that it would be similar to the OLTP compression which Oracle already has which only gives you 2x-4x compression…which isn’t very good). Especially when compared to BLU Acceleration which as many customers have reported delivers 10x compression or more.

They say it’s simple to use but look at the next slide and judge for yourself if it is really as simple as they

say.

25

As you can see there is a lot more work required to manage Oracle In-Memory columnar

objects then there is to manage DB2 BLU.

This page summarizes the differences between Oracle’s future in-memory database option and the currently available DB2 10.5 with BLU Acceleration.

The first is that BLU has been available since June 2013 which is likely at least a year before Oracle delivers anything to the market. That’s a years worth of business benefits (storage savings, lowered hardware costs, better and faster business results) that clients can take advantage of. What is the opportunity costs that you lose by waiting?

In Oracle there is no persistence for the column store (it is in memory only) and so when the server fails or when you apply a patch, you have to completely rebuild the column store. Can your business afford to wait that long to be able to get access to the data in an acceptable amount of time.

Simplicity – in the case of Oracle this is yet another option that the DBA much choose (amongst the other options for auxiliary data structures like indexes, materialized views, etc). And the DBA has to not only select what to put in memory (tables, partitions, columns) they must also monitor that space to ensure they don’t run out of memory for columnar objects. If they run out of space the columnar tables will become out of date as they can no longer fit in memory as changes are made to the row store (and they don’t exist on disk).

The column store objects in Oracle only exist in memory so they must fit into available memory (all the data even cold data must fit there).

And what is worse, is that their compression capabilities are only 2x-4x compression according to announcements made at OpenWorld. This makes DB2 BLU 5x better at compressing data. And we only keep the one copy of data compressed in memory. In the case of Oracle the columnar data is cached in memory and the row version of the table still exists on disk and will be loaded into the buffer pool when needed. Effectively Oracle can and will have two copies of the data in memory at the same time reducing their memory density by 2x (that is if they get 2x compression but store the data twice in memory then they have zero savings).

We will have to see what they do on the compression front in terms of being able to act on the data compressed but BLU is already well ahead being able to act on compressed data for a broad range of operations (equality, range predicates, joins, aggregations, and more).

BLU is included in the advanced editions of DB2 with all the other capabilities IBM offers. Oracle will likely sell this as yet another add-on only available for Enterprise Edition customers and you will also likely need Advanced compression and Partitioning options to make these work in production (otherwise you will have to load full tables in memory rather than partitions of tables).

Now let’s look at SQL Server 2012, 2014 and PDW

A Non-Clustered Columnstore Index is a non-clustered index first introduced in SQL Server 2012 which is organized on data pages by column rather than by row. It can be created on base tables that are clustered indexes or heaps, and can co-exist with other non-clustered indexes. While columnstore indexes can be compressed more than a traditional row-based SQL Server tables with data compression, they are still additional objects requiring storage over and above that consumed by the base tables on top of which they are built.

Data in an columnstore index is processed, or transferred from disk to memory in batches called “column segments”. Each column segment contains a subset of values from a single column (typically 1M rows in size). Each segment is stored within a single LOB, is the unit within which the compression algorithm operates (compression occurs for patterns within each segment, not across segments) and the unit of movement of data between disk and memory. The segments making up a set of rows are called a row group.

The Clustered Columnstore addresses the two major advantages of the Non-Clustered Columnstore. It is available in SQL Server 2014 (out 1H 2014) and currently available in the SQL Server Parallel Data Warehouse-based appliances.

28

DB2 BLU Acceleration will continue to have advantages over the SQL Server 2014 iteration of SQL

Server’s Columnstore Index in flexibility, manageability and performance.

DB2 BLU uses a patented compression algorithm which compresses patterns across the entire table. The

unit of compression for CCIs is a column segment, which is ideally 1M rows in size but can also be

smaller depending on insert patterns. Table-wide compression translates into a superior compression ratio

for BLU.

Unlike BLU which automatically handles maintenance tasks such as space reclaim without any

interruption to the workload, maintenance of CCIs involves significant DBA involvement as well as

downtime.

The implementation of BLU has several aspects which translate into superior performance over CCIs. Min

and max values for BLU are maintained at a very granular level while the column segment is the unit of

row elimination for CCIs, with a targeted size of 1M rows per segment. Inserts of less than a certain

number of rows are first inserted in an intermediate row-based store before being inserted into the actual

Columnstore. If there are row store objects which have not yet been merged into the Columnstore query

performance will be negatively affected.

While constraints can be defined on BLU tables this is not the case for Clustered Columnstores.

Finally, CCIs are only included in SQL Server Enterprise Edition whereas BLU can be acquired with DB2

29

Advanced Enterprise Workgroup Edition.

29

Now let’s compare this to Teradata compression and columnar objects

Teradata is similar to Oracle Exadata where they have a “hybrid” approach where groups of rows

are reordered and stored in column format. Much like Exadata when a row is updated, it is

removed from the compression/columnar grouping and stored at the end of the table

uncompressed. This empty space left in the table will need to be cleaned up at some point in the

future. Teradata uses a number of compression techniques but the DBA must then decide which

one to use for optimal performance.

One big issue is that you cannot use Teradata load utilities to get data into these tables. The only

supported method to get data into the table is INSERT from SELECT.

Tables that are using the column store option are append only and do not support primary index

which means there is no hash based partitioning of these tables (they are round robined across

the nodes in the cluster). This will result in poor data distribution across the cluster

It would seem that Teradata does not have much faith in their columnar technology. The SQL Data

Definition document from Teradata they state that you have to call Teradata support to enable columnar

capabilities?

Why? Is it unstable? Does it suffer from performance problems? What are they not telling clients that

would cause them to restrict the use of this functionality?

Now let’s look at a real columnar vendor – SAP HANA.

The first real columnar vendor to look at here is SAP HANA. Note that although IBM competes

with SAP in the database space now, our technical partnership with SAP remains strong with the

application team at SAP who continue to advance the integration between SAP and DB2 (with

SAP developers continuing to work in the IBM Toronto Lab and IBM developers continuing to

work in SAP facilities in Germany) for our mutual clients success.

We do however, have difference of opinion when it comes to how to deliver the best in-memory

columnar technology. We feel that row organization is still superior for OLTP workloads and that

pureScale delivers higher availability and better scalability for this kind of work (and we have

performance proof points to back that statement up).

For analytics DB2 with BLU Acceleration has some clear advantages we will discuss on the next

few pages.

The closest competitor to having a columnar solution that is like BLU Acceleration is SAP HANA.

With HANA the tables can be defined as row or column store oriented and the data is stored in

memory and persisted on disk. With HANA, newly inserted rows are not put directly into the table

but instead are stored in row format in a side table which is then periodically merged into the

column store. SAP HANA has several forms of compression for the columnar data.

Dictionary encoding is much like DB2 table compression where a dictionary of values is created

and then the data in the column is a pointer/symbol for the dictionary value. The issue here is that

in the case of HANA all of the column values must be in the dictionary so it is most useful when a

column has a limited number of possible values. Run length encoding is used when there are

several column values in sequence that are all the same value. In this case you simply store the

fact that the next N values are all the same (rather than storing that value multiple times). Cluster

encoding is when a column contains a sequence of values that occurs more than once in the

table. In this case the sequence is replaced by a symbol.

Column based structures are optimized for read operations. Unfortunately, compression induces the

problem that changes on a compressed structure are prohibitively slow, because – depending on the

employed compression algorithm – much more data than actually manipulated must be de- and

recompressed.

Every relation stored in a column-oriented layout is associated with an additional write-optimized insert-

only structure called differential buffer (or Delta Store). All inserts, updates, and

delete operations are performed on the differential buffer. The read-optimized main part of the data, called

the main store, is not touched by any data modifying operation. The differential buffer in conjunction with

the compressed main store represents the current state of the data.

To address the disadvantages of data compression, write performance is traded for query performance and

memory consumption by using the differential buffer as an intermediate storage for multiple

modifications. To derive a consistent view, read operations must access the main store as well as the

differential buffer, while data modifying operations manipulate the differential buffer only.

36

From a cost perspective, DB2 with BLU Acceleration on Power is only a fraction of the cost of SAP HANA on Intel. The

above example shows a comparison for 10TB of raw user data. The recommendation for 10TB of raw user data for DB2 10.5

with BLU acceleration is to use 32 cores, 1TB of memory and the database size will be around 2TB after compression (note that

for SAP BW some of the tables in DB2 10.5 would remain as row organized while others would be set as column organized.

For this we configure a 32-core p770 with 1TB DRAM and 12TB of SSD storage (the database only requires 2TB of storage but

for additional space for redundancy, local backups, logs, etc we have costed out 12TB). The cost for this hardware is $878,884.

For software DB2 10.5 AESE can be purchased using Terabyte Pricing which costs $94,000 per terabyte of compressed data in

the database. Therefore for 2TB of compressed data the software cost including first year of support would be only $188k. This

brings the total IBM solution to only $1,066,884 USD list price.

For SAP the costs are much greater. For 10TB of data SAP recommends 4TB - 5TB of Memory (10TB / 4 * 2 is what appears

in SAP Note 1637145 however there are other sizing guidelines from field specialists that suggest 5x compression instead of 4x

which is why we have a range of 4TB – 5TB of memory). Also they recommend that you have 5x the amount memory for

database storage which in this case would be 20TB of storage for 4TB of memory. For cores SAP recommends 16GB of

memory per core for 8-core CPUs and 12.8GB of memory per core for 10-core CPUs (per SAP NOTE 1637145). In our

example above we use 10-core processors so for the 4TB of memory you would need (4096GB/12.8GB = 320 cores). This

means for 10TB of raw data SAP HANA would require a cluster of 8 40-core servers. The pricing shown above is for 9 (8 + 1

spare) IBM x3590s each with 40cores, 512GB of memory and 2.5TB of storage. The total cost for the SAP HANA on Intel is

just almost $9 Million dollars when you include 1st year of support!

SAP hasn’t optimized the I/O path and therefore you need to load in the columns that need to be accessed and thus they need

more memory – IBM has done a lot of work on dynamic in-memory and uses are paging algorithms to help here. If this was

DB2 on an Intel environment,

Don’t forget to include the added administration costs for running 9 servers in a cluster vs. one p770 server (performance

tuning, software patching, storage management, etc).

Now let’s look across the board at a number of other vendors and how they stack up

with BLU Acceleration

Columnar support for AIX: One key benefit of DB2 10.5 with BLU Acceleration is that it runs on both AIX and on Linux on x86. All other columnar vendors run only on

Linux including SAP HANA. If a client is a strong supporter of AIX, then DB2 10.5 with BLU Acceleration has a key competitive advantage.

All of the major columnar vendors support the ability to bring in only a small portion of the database into memory, process that data and then bring more data into

memory if needed EXCEPT SAP HANA. HANA is very immature in this space and in fact the first releases of HANA required that all of the data for all tables fit into

memory. As of HANA 1.0 SP5 they allow some data to be paged out to make room for more data to come into memory but the entire working set of columns must still

fit in to memory. That means that if 100 queries are running accessing 1000 columns of table data then all the data for all 1000 columns must fit in memory on HANA

at once. Otherwise queries will start to fail. Some clients have noted that HANA crashes if it runs into these memory problems.

Flexible deployment with either column or row storage: Some columnar vendors like Vertica and ParAccel support only columnar storage while others like Oracle and

SQL Server don’t really support columnar table storage fully. This flexibility in DB2 10.5 allows clients to adopt the right technology for the right business needs.

Encryption support is available with DB2 BLU tables using IBM Guardium Encryption Expert. Vendors like Vertica and Vectorwise have no ability to encrypt columns

while other vendors like Teradata rely on 3rd parties for encryption support.

Another key advantage of having DB2 BLU columnar tables fully integrated into the core database engine is that we pick up the security and separation of duties

native to DB2 LUW. Specifically the ability to restrict DBAs from accessing data they should not have access to and having a separate role for Security Administrators

(SECADM). Oracle provides some support with the addition of the separately priced Data Vault option and SQL Server allows separation of duties but none of the

columnar vendors are advanced enough in security to provide this type of SOX or HIPPA requirement.

Simplicity. For BLU you just load the data and go. With Oracle you still must determine which compression algorithms to use and which indexes you should or should

not create (creating indexes on Exadata can cause queries to run slower do to immature optimizer choices). SAP is simple in this regard but the Teradata, SQL

server, Greenplum and Vertica columnar solutions are very complex with many restrictions and many choices a DBA must make in order to get data into the tables

and set up optimally. Similarly ParAccel requires a separate Analyze phase and reorganization where as with BLU the Analyze and Compress phases are built right

into the load utility.

No need to create indexes for performance. Most of the columnar vendors require some form of indexes in addition to the columnar tables in order to deliver improved

performance. BLU only supports a primary key for uniqueness enforcement and does not require indexes do to the advanced performance of SIMD, compression we

can act directly on, multi core parallelism and more.

With DB2 10.5 BLU column organized tables are automatically compressed and DB2 select the optimal compression algorithms for each column without any DBA

intervention. In fact there is no way for the DBA to even specify any different behavior. Some of the columnar vendors require the DBA to select the compression

algorithms for each column. While some provide advisors for this, it is still extra work on the DBA to run these tools and select the optimal compression types for each

column.

An issue with highly parallelized analytic workloads is that when many, many users access the system at the same time, it is possible for the server to quickly be

overwhelmed with work to do. DB2 10.5 deploys automatic workload management to queue requests if the server is too busy. This happens automatically and delivers

optimal throughput through the system without requiring the DBA to manage any of it. Most of the other vendors do not offer this capability and those that do often

require the DBA to manually set up the system to govern resources.

Finally, graphical tools are available from most of the traditional DBMS vendors who have been in the business for a long time and realize administrators need tooling

to simplify their lives even for systems that run fast out of the box. However, the newer columnar vendors have yet to invest in tooling support and so database

administrators are left to solve problem on their own without the aid of intelligent monitoring and diagnostic tooling from the vendor.

Maintain columnar organization in memory. DB2 BLU tables remain compressed and in columnar format in memory. This is also the case with SAP HANA, Teradata,

Vertica, Vecrotwise and ParAccel. With SQL server it is only the index that is columnar. With Oracle the data is not truly columnar. As with the on disk structures, the

data is hybrid columnar. Also Greenplum stores the data in a buffer but as soon as they access the data it is uncompressed and coverted back to rows. Greenplum

specifically calls this “early materialization”.

Data can stay compressed when being worked on: With DB2 with BLU acceleration, values are not decompressed until as late as possible. This means that for values

(or rows) that do not qualify for the result set, data may never be decompressed since predicates and joins take place on the compressed values. DB2 10.5 with BLU

has the richest set of operations that can be performed on compressed data compared to any other vendor. In Oracle the data must be decompressed to be

evaluated and with SQL Server the table data is always will be decompressed in order to be selected back since it is only the index that is compressed. SAP HANA,

Teradata and other columnar vendors can do simple equality predicates on compressed data but not much else.

Insert/update maintains columnar: With DB2, inserts and updates to BLU tables keep the data in columnar format. In Oracle the values are uncompressed and stored

in row format if there is an insert or update on a compressed columnar table. With SAP HANA they insert new rows into a differential table and then merge that data

back into the columnar format at regular intervals. Teradata and Greenplum support is the same as DB2 and with SQL Server the table is read only if you create a

columnar index on top of a table so inserts and updates are not supported. Other vendors work like SAP with a differential table which is then populated into columns

store.

Data Skipping: DB2 10.5 with BLU Acceleration keeps a map of all the data in columnar format so it knows the minimum and maximum column values of every 1024

rows. This allows DB2 to skip the processing of data for which we know there is no qualifying records in that range of rows. The only other vendors that can do this

are Vertica and Vectorwise. Oracle Exadata does have some limited capabilities here but they keep the data for only 8 columns and also they build the info on the fly

which means that every time the database is recycled they lose the information and have to rebuild it when queries start to run.

SIMD exploitation: Only DB2 and Vectorwise take advantage of the latest processor capabilities to run a single instruction on multiple data inputs (SIMD) for a wide

array of operations (aggregates, predicates, joins, etc). That is, we can pack the registers with many pieces of data and perform a single instruction on all of that data

simultaneously. Thus increasing performance significantly. DB2 can also simulate SIMD processing in software to get an even further advantage on top of the

hardware assisted SIMD capabilities. No other vendors have the range we have. Some like SAP HANA can only do simple things with SIMD like decompression or

simple counts.

Informational constraints: Only DB2 10.5 BLU and Oracle allow for the definition of informational constraints. This allows the optimizer to build access plans knowing

that a column contains unique values without having to pay the performance overhead of actually requiring the database engine to enforce the uniqueness. Since

uniqueness is often enforced in the ETL layer it can be redundant for the database to enforce it but it is still useful for the optimizer to have the knowledge that the

data is unique.

Uniqueness enforcement: Even though BLU queries do not use indexes we still have the ability to enforce uniqueness on data in a blue table. Other vendors can do

this by creating secondary indexes with the exception of Teradata who do not support primary key indexes on their columnar tables and SQL Server and ParAccel

which do not allow unique constraints on columnar data.

Don’t forget to fill out your evaluations on this session and others you attend to help

us all improve.

43