introducing postgres plus advanced server 9.4

24
© 2015 EnterpriseDB Corporation. All rights reserved. 1 Introducing Postgres Plus Advanced Server 9.4 Presenters; Regional Sales Manager Asia & Pacific Erik Baardse, [email protected] Ashnik Database consultant; Sameer Kumar, [email protected]

Upload: enterprisedb

Post on 14-Jul-2015

260 views

Category:

Technology


3 download

TRANSCRIPT

Page 1: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 1

Introducing Postgres Plus Advanced Server 9.4 Presenters;

Regional Sales Manager Asia & Pacific Erik Baardse, [email protected]

Ashnik Database consultant; Sameer Kumar, [email protected]

Page 2: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 2

•  EnterpriseDB and Postgres Plus Overview •  Postgres Plus Advanced Server 9.4

−  Resource Management −  Partitioning −  Enhanced SQL (Cube/Rollup/Grouping Sets, Connect_By_Root) −  UTL_HTTP −  SQL/Protect & MTK Enhancements

Agenda

Page 3: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 3

POSTGRES innovation

ENTERPRISE reliability

24/7 support

Services & training

Enterprise-class features, tools &

compatibility

Indemnification

Product road-map

Control

Thousands of developers

Fast development

cycles

Low cost

No vendor lock-in

Advanced features

Enabling commercial adoption of Postgres

Page 4: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 4

EDB is a Gartner Magic Quadrant Leader

Page 5: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 5

The Database PostgreSQL / Postgres Plus Advanced Server

Tools

xDB Replication Server

Failover Manager

Management and Monitoring

Back and Recovery Tool Postgres Enterprise

Management (PEM)

Stack Builder & Update Monitor

Cloud Enablement Cloud Database for

Amazon Web Services Open Stack Support*

Core

OR PostgreSQL

Postgres Plus Advanced Server

Extension Components (PostGIS, pgPool, pgBouncer,

SQL/Protect, PL/Perl Python TCL, FDW)

Connectors (.Net, JDBC, ODBC,OCL)

Utilities (EDBLoader, EDBPlus, ECPGPlus)

Migration Tool Kit

* Roadmap

EnterpriseDB Product Overview

APIs to enable the Platform as a Service*

Page 6: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 6

from PostgreSQL core from EDB Development

•  64 bit LOBs up to 4TB in size

•  Custom background workers

•  Writable Foreign Data Wrappers

v9.1

EDB contributions to PostgreSQL core

• No restore In-place version upgrades

v9.2

v9.3

v9.0

• Materialized Views

•  Deferrable unique constraints and Exclusion constraints

•  Streaming replication

•  Windows 64 bit Support

•  Hot standby

•  Synchronous replication

•  Serializable Snapshot Isolation

•  In-memory (unlogged) tables

•  Writeable Common Table Expressions (WITH)

•  Cascaded streaming replication

•  JSON support, Range Types

•  VARRAY support •  SQL Profiler

•  Index Advisor •  Parallel Bulk Data

Load

•  Row Level Security •  Declarative Partitioning syntax

•  Table() function support for nested tables

•  INSERT APPEND hint

•  xDB Multi-master replication

•  Expanded Object Type support

•  Partition Read Improvements over 75x

•  Support for 1000s of Partitions

•  Partition write improvements over 400x

• MySQL Foreign Data Wrappers for SQL/MED

Postgres Plus Advanced Server Key Feature Development

• Index-only scans (covering indexes)

• Linear read scalability to 64 cores

v9.4 • pg_prewarm • ALTER SYSTEM • Concurrently updatable Materialized Views

• Mongo FDW & MySQL FDW

•  Logical Decoding for Scalability

•  JSONB Data Type

•  JSONB

Indexing •  Expanded

JSON functions •  Delayed

Application of Replication

•  3x Faster GIN indexes

•  Support for Linux Huge Pages

•  CPU & I/O Resource Management

•  SQL Aggregation with CUBE, ROLLUP and GROUPING SETS

•  Comprehensive UTL_HTTP Package

•  Hash Partitioned Tables

•  Connect_By_Root Operator for hierarchical

queries •  SQL/Protect

Logging to DB Table

•  EDB*Loader Improved Error handling

Page 7: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 7

CPU & I/O Resource Management

Hash Partitioned Tables

SQL Aggregation with CUBE, ROLLUP and GROUPING SETS

Comprehensive UTL_HTTP Package

Connect_By_Root Operator

ICU Collation

EDB*Loader Improvements

SQL/Protect Logging to DB Table

Migration Toolkit Enhancements

Postgres Plus Advanced Server Postgres Community

Feature Highlights for Postgres Plus Advanced Server (PPAS) 9.4

Many new features including: Logical Change Set Extraction JSONB Data Type Time Delayed Standby ALTER SYSTEM pg_prewarm() Materialized View Refresh Concurrently Ordered Set Aggregates and more… http://www.depesz.com/ is a good reference site https://commitfest.postgresql.org/ is the global community site for patches review

Page 8: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 8

Postgres Plus Advanced

Server

Resource Manager

(CPU & I/O)

Reporting

Transactions

80%

20%

Run Mixed Workloads More Efficiently with PPAS 9.4 Resource Management

•  DBA assigns CPU & I/O to job groups

•  Allocates and prioritizes consumption of resources

•  Low priority jobs don’t hurt high priority jobs

Page 9: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 9

•  Create Resource Groups and assign the CPU Rate Limit

•  Use these resource groups during a psql session

Run Mixed Workloads More Efficiently with PPAS 9.4 Resource Management

- Statements executed will be limited in CPU or writing to shared_buffers per resource group. - Individual limits are computed based on recent CPU / shared_buffer usage. - Processes sleep when necessary & avoid sleep when holding critical locks. - The limits are adjusted regularly based on current usage. - If multiple processes in the same group are being executed, aggregate usage will be limited

CREATE RESOURCE GROUP resgrp_a; CREATE RESOURCE GROUP resgrp_b; ALTER RESOURCE GROUP resgrp_a SET cpu_rate_limit = .25; ALTER RESOURCE GROUP resgrp_a SET dirty_rate_limit = 12288;

SET edb_resource_group TO res_grp_a;

Page 10: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 10

Chapter 13 of EDB Oracle Compatibility Guide for details on usage

One logically large table is broken into smaller physical pieces. •  Worthwhile when a table would otherwise be very large. •  Exact point to see benefits will depend on the application.

When should I Partition? •  Good rule of thumb is that the size of the table should exceed the physical memory

of the database server. •  When most accessed rows are in a single partition or a small number of partitions. •  When there is a lot of concurrent inserts or updates (Hash partitioning may benefit) •  When a query or update accesses a large percentage of a single partition. •  For Bulk Loads / Unloads (ALTER TABLE faster than bulk load, avoids VACUUM

overhead from DELETE). •  When actively archiving seldom-used data to less expensive storage.

Support Larger Tables with Partitioning

Page 11: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 11

Use PPAS Syntax To Minimize Partitioning Errors and Complexity

CREATE TABLE sales ( dept_no number, part_no varchar2, country varchar2(20), date date, amount number ) PARTITION BY RANGE(date) ( PARTITION q1_2014 VALUES LESS THAN('2014-Apr-01'), PARTITION q2_2014 VALUES LESS THAN('2014-Jul-01'), PARTITION q3_2014 VALUES LESS THAN('2014-Oct-01'), PARTITION q4_2014 VALUES LESS THAN('2015-Jan-01') ); CREATE INDEX sales_date on sales(date);

Simple Declarative PARTITION BY and SUBPARTITION BY

Single Index command

•  More SQL syntax provide sophisticated data management techniques −  ADD / DROP to augment the partitions −  SPLIT to divide the data −  EXCHANGE to swap in a new partition −  TRUNCATE to remove all data but leave

structure in tact −  MOVE to shift data to a different tablespace

•  PPAS Improved performance with Fast Pruning and Constraint Exclusion support

•  System Catalog Views for Partitions −  ALL_PART_TABLES −  ALL_TAB_PARTITIONS,

ALL_TAB_SUBPARTITIONS −  ALL_PART_KEY_COLUMNS,

ALL_SUBPART_KEY_COLUMNS

Page 12: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 12

List, Range or Hash Partition Rules •  Provide the constraints to define where data is stored •  For PPAS, also used to support Fast Partition Pruning •  Consider how data stored will be queried, include often-queried columns

in partitioning rules.

•  List – Single partitioning key column; based on exact value •  Range – One of more partitioning key columns; based on values between

two extremes •  Hash (New 9.4) – Data divided amongst equal sized partitions based on

hash value. * Internal tests have shown hash partitioning can improve performance when many hundred concurrent connections insert/update to the same table*

PPAS 9.4 Supports Various Partitioning Rules

Page 13: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 13

Advanced Server’s Query Planner uses two optimization techniques to compute an efficient plan: •  Constraint exclusion (constraint_exclusion = partition or on)

−  Provided by PSQL −  SELECT w/ WHERE: Query Planner must examine the CHECK constraints defined

for each partition before deciding which partition to send query fragments to. •  Fast pruning (edb_partition_pruning = on)

−  Occurs earlier in query planner process. Understands the relationship between partitions in an Oracle style partitioned table.

−  SELECT w/ WHERE: Query Planner can reason that only a certain partition holds the values without examining the constraints defined for each partition.

−  Can be used for LIST or single value RANGE Partitions (not usable on subpartitioned tables or multi-value range partitioned tables)

−  Works with >, >=, =, <=, <, AND, BETWEEN operators in the WHERE Clause

PPAS Improves Partitioning Performance

Page 14: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 14

•  Aggregation Functions (CUBE / ROLLUP / GROUPING SETS) −  From the edb-sample.sql, see dept, emp and jobhist tables to aggregate

employees based on their locations, departments and jobs. −  Chapter 2.2.6 of EDB Database Compatibility Guide for Oracle for more

details on GROUP BY ROLLUP / CUBE / GROUPING SETS and the subtotals, groupings and result sets generated.

−  Your own data for analysis might be involve sales or support related information. You could report on revenues or tickets grouped by geographic regions, assigned sales or support engineer, product utilized, etc.

•  Connect_By_Root −  Support additional hierarchal query needs. With our edb-sample data, we

can report on various levels of manager employee hierarchies. −  Chapter 2.2.5.6 of EDB Oracle Compatibility Guide for more details on

sample usage −  Your own data for reporting might look at hierarchies of products or modules

Enhanced SQL for Aggregation and Connect_By_Root Syntax

Page 15: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 15

•  GROUP BY ROLLUP produces subtotals for each hierarchal group based on a left to right listing of items in the expression

SELECT loc, dname, job, COUNT(*) AS "employees" FROM emp e, dept d

WHERE e.deptno = d.deptno GROUP BY ROLLUP (loc, dname, job) ORDER BY 1, 2, 3;

Multidimensional Analysis - Rollup

Page 16: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 16

•  GROUP BY CUBE produces groupings and subtotals for every permutation of items in the expression

SELECT loc, dname, job, COUNT(*)

AS "employees" FROM emp e, dept d

WHERE e.deptno = d.deptno GROUP BY CUBE (loc, dname, job) ORDER BY 1, 2, 3;

Multidimensional Analysis - Cube

……………

Page 17: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 17

•  GROUP BY GROUPING SETS produces one result set that is a concatenation of multiple result sets based on different groupings (i.e. UNION ALL)

SELECT loc, dname, job, COUNT(*) AS "employees" FROM emp e, dept d

WHERE e.deptno = d.deptno GROUP BY GROUPING SETS (loc,

dname, job) ORDER BY 1, 2, 3;

Multidimensional Analysis – Grouping Sets

Page 18: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 18

•  UTL_HTTP – Built in package that provides a way to use the HTTP(S) protocol to retrieve information found at a URL. Commonly requested functions now provided: −  REQUEST, REQUEST_PIECES −  GET/SET_FOLLOW_REDIRECT −  GET/SET_RESPONSE_ERROR_CHECK −  GET/SET_HEADER, GET_HEADER_BY_NAME, GET_HEADER_COUNT −  READ_RAW, READ_TEXT −  BEGIN/END_REQUEST, GET_RESPONSE

•  Ch 8.17 of EDB Enterprise Edition / Ch 7.17 of EDB Oracle Compatibility Guide for more details on usage

Develop Applications To Retrieve Data With UTL_HTTP Package

Page 19: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 19

Preventing attacks is normally the responsibility of the application developer. But with SQL/Protect, DBAs can now provide another layer of protection to prevent corruption or co-opting of the database.

Injection Attack Report

SQL/ Protect

Attacker

PREVENTION TECHNIQUES

Unauthorized Relations

Utility Commands (e.g. DDL)

SQL Tautology

Unbounded DML

EDB SQL/PROTECT

DBA Managed with Centralized SQL Injection Protection

Threat Deleted

New in 9.4 - View edb_sql_protect_queries contains Injection Attack Report information (Ch 4.1.1.2.3 of EDB Enterprise Edition Guide for more information)

Page 20: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 20

A More Robust Migration Toolkit

New in 9.4 •  A detailed log with well defined error codes to

allow DBAs to better understand which capabilities of their database applications are migrate-able to PPAS.

•  See Ch 9 of Migration Toolkit documentation for a list of the error codes provided

•  Migration of schemas, tables, constraints, indexes, data, views and more

•  Online (immediate) or offline (DDL scripts) migration

•  Customize migrations with bulk inserts, row filters, change datatypes inline, subsets of schema objects

•  Parallel data movement techniques, bypass logging for faster data loads or use native connectivity to source database.

Page 21: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 21

1.  Customers running mixed workloads.

2.  Application Developers who integrate with external Web servers.

3.  Customers with large tables where they often search for exact matches or have many concurrent inserts/updates.

4.  Users who think they need a NoSQL database.

5.  Customers with reporting or data warehousing databases.

6.  DBAs who use need to bulk load data.

7.  DBA’s concerned with Security and SQL Injection Attacks.

Recap: Postgres Plus Advanced Server (PPAS) 9.4 Use Cases

Page 22: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 22

•  EnterpriseDB Exclusive Master Partner for South East ASIA Region

•  Certified EDB Training Partner in South East Asia

•  Brings Power of Enterprise Open Source Solutions to customers in the region through Partner Ecosystem

•  Provides services and onsite support for Postgres Plus deployments in the ASEAN region. Ashnik Team of Certified Postgres Plus Professional can help customers during Postgres Plus Installations, configurations, HA clustering setups, Security Hardening and Backup and Archiving.

About Ashnik

Page 23: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 23

How can I learn more?

•  Download Postgres Plus Advanced Server:

•  http://www.enterprisedb.com/download-advanced-server

•  General information feature table:

•  http://www.enterprisedb.com/postgres-plus-advanced-server

•  Set up technical call with a Postgres Expert •  [email protected] | [email protected]

Page 24: Introducing Postgres Plus Advanced Server 9.4

© 2015 EnterpriseDB Corporation. All rights reserved. 24