transforming it infrastructure

Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved

Transforming IT Infrastructure

Tim GormanTechnical Consultant

Oracle OpenWorld 2014


Virtualized storage is fast becoming the new norm.


Virtualized storage is fast becoming the new norm.

Nobody can justify provisioning non-production environments the way they

did up to now.

Private and confidential© 2014 Delphix. All Rights Reserved© 2014 Delphix. All Rights ReservedPrivate and confidential 4

Accelerating the tempo of application development

Fortune 1 Retailer #1 Social Network #1 Financial Services #1 Network Equipment #1 Cable

#1 Wholesale #1 Food Service Co. #1 Office Supplies #1 Apparel & Footwear #1 Chip Manufacturing

#1 Pharma #1 Auction Marketplace #1 Total Healthcare #1 Aerospace #1 Computer Access

#1 CPG #1 ETL Software #1 Availability Service #1 Mutual Life Ins. #1 Satellite TV

#1 State Gov #1 Cruise Line #1 Retirement Fund #1 IT Services #1 Game Software

Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 5

• The problem• Attempting to find a solution• A solution• Delphix• Wrap-up / Q&A

Agenda


• PROD is what we care most about…

The problem

PROD


• PROD is what we care most about…– But behind every PROD…

• There is QATEST…

The problem

PROD QATEST



• There is QATEST, DEV…

The problem

PROD QATEST DEV



• There is QATEST, DEV…• …and other project-specific environments to support

The problem

PROD QATEST DEV


• Also, as PROD grows…

The problem

PROD PROD


DEVQATEST

• Also, as PROD grows…– Then so do the supporting non-PROD environments…

The problem

PROD


The problem

CEO

COO

CFO CMO CSO CIO

• And so inevitably the situation shifts from this…


The problem

CEO

COO

CFO CMO CSO CIO

StorageAdministrator

• To this…

* Inspired by Connor McDonald (http://connormcdonald.wordpress.com/)


• Some try to mitigate the problem by copying subsets of PROD…

Attempting to solve the problem…

PROD QATEST DEV UAT



• …but it is difficult and time-consuming and expensive to create referentially-correct subsets of databases.


PROD QATEST DEV UAT



• …but it is difficult and time-consuming and expensive to create referentially-correct subsets of databases. Most give up…


PROD QATEST DEV UAT


• What inevitably happens is that cloning is performed less frequently…

• …meaning that schemas and data are older, dirtier due to previous testing, and too small to accurately represent reality


PROD QATEST DEV UAT


PROD

• Creating snapshots with mirror splits…– Mirror the volume one extra copy


PRODPROD


PROD

• Creating snapshots with mirror splits…– Mirror the volume one extra copy– Then cease writes to that extra copy at a specific time and

make it separately addressable as a separate volume


PRODPROD


PROD


make it separately addressable as a separate volume– To create another snapshot, “re-silver” the volume into the

mirror set


PRODPROD


PROD



mirror set– And it’s ready to split again


PRODPROD


PROD




– While we’ve solved the problem of quickly creating new copies


PRODPROD


PROD




– While we’ve solved the problem of quickly creating new copies• We haven’t solved the problem of using a full image


PRODPROD


PROD




– While we’ve solved the problem of quickly creating new copies• We haven’t solved the problem of using a full image• And re-silvering is not a fast operation, so we need time to “reload”


PRODPROD


Attempting to solve the problem

CEO

COO

CFO CMO CSO CIO


We’re still at square one…



CEO

COO

CFO CMO CSO CIO



…and it gets even worse



CEO

COO

CFO CMO CSO CIO




Because now we have to predictthe size of everything withperfect accuracy



CEO

COO

CFO CMO CSO CIO





and then pre-allocatefor copies as well as volumes


So, in a perfect world what we’d like to see is something like this…


PROD

DEV for Tom

DEV for Dick

DEV for Sally

DEV for Harry

DEV for Becky



PROD

DEV

Tom Dick Sally

Harry

Becky

…in the real world we see this…


• It’s not easy for developers or testers to share environments simultaneously



• It’s not easy for developers or testers to share environments simultaneously– Changes made by each user conflict with one another




• So it becomes necessary to serialize access to the limited number of environments




• So it becomes necessary to serialize access to the limited number of environments– Each user queues at the resource, so their project takes more

time elapsed





time elapsed– Each user destructively changes the resource






• And now we need a way to reset or clean up the environment between each use






• And now we need a way to reset or clean up the environment between each use– Reset or clean up takes time, which causes further delays

between uses



• The biggest problem with cloning environments and databases is all the duplication

A solution…


• The biggest problem with cloning environments and databases is all the duplication– Most (if not all) of the blocks comprising the database or file-

system exactly the same

A solution…




• What if we store only one copy of each version of a block?

A solution…




• What if we store only one copy of each version of a block?– Copy-on-write (COW)

• Never modify a block, but insert a new version instead

A solution…






• Then let the storage figure out what we’re supposed to be seeing?

A solution…






• Then let the storage figure out what we’re supposed to be seeing?– The storage maintains the relationships and versions on

retrieval

A solution…







retrieval

• And then make it portable so that ANY platform (Unix/Linux, Windows) can use it?

A solution…







retrieval

• And then make it portable so that ANY platform (Unix/Linux, Windows) can use it?– NFS (network file system)

A solution…


• Cloning the old way…

A solution…

PROD DEV


• Cloning the smart way…

A solution…

PROD DEV


• Cloning the smart way…– As changes are made to the clone, only those changed blocks

are stored…

A solution…

PROD DEV



are stored…

• Takes less storage, faster to copy• The cost of the clone has gone way down in terms of

storage and time

A solution…

PROD DEV



are stored…


storage and time

A solution…

PROD

CapEx

DEV



are stored…


storage and time

A solution…

PROD

CapEx OpEx

DEV

Delphix Engine

• Software appliance installed as a virtual machineStorage LUNs are presented through the hypervisorVirtualized data is presented to target servers using NFS

Delphix Engine


• Target server(s) are installed with Oracle databaseVirtual databases are just Oracle databases on NFSDatabase files reside on the Delphix Engine

Delphix Engine


• Target server(s) are installed with Oracle databaseVirtual databases are just Oracle databases on NFSDatabase files reside on the Delphix Engine

• Source database(s) are copied into the Delphix Engine• Using standard RMAN functionality

• Not just a point-in-time image, but a continuous TimeFlow

© 2014 Delphix. All Rights ReservedPrivate and confidential 56

The Delphix Ecosystem: Source Environment

Source Environme

nt

Target Environments

SSH, JDBC

NFS

Any Storag

e

FC

The Source Environment is the host or cluster on which the instance(s) for the source database runs. Typically this is the production environment.

SSH, DSP

JDBC


The Delphix Ecosystem: Source Database

Target Environments

SSH, JDBC

NFS

Any Storag

e

FC

The Source Database is the collection of Oracle files that make up an Oracle Database. Using RMAN APIs, a full level 0 backup is initially taken directly into Delphix. Level 1 incremental backups and archive/redo data will be used to keep the Delphix System up to date. RMAN operations are run via the Delphix SnapSync service. Log shipments are performed via the LogSync service.

SSH, DSP

JDBC

Source Environme

nt


The Delphix Ecosystem: Delphix Server

Target Environments

SSH, JDBC

NFS

Any Storag

e

FC

The Delphix Server resides in a VMWare infrastructure and can use any storage as a backend. It hosts the Delphix management GUI and can be accessed via a web browser, the Delphix CLI, or RESTful API.

SSH, DSP

JDBC

Source Environme

nt


The Delphix Ecosystem: dSource

Target Environments

SSH, JDBC

NFS

Any Storag

e

FC

The dSource is the Delphix Server’s compressed representation of a Source Database. Incremental change data is ingested into the dSource, building a window of change data over time called the Timeflow. This Timeflow data serves as the basis for all Virtual Database storage within Delphix.

SSH, DSP

JDBC

Source Environme

nt


The Delphix Ecosystem: Target Environment

Target Environments

SSH, JDBC

NFS

Any Storag

e

FC

A Target Environment is a host or cluster that runs the Oracle instance(s) for Virtual Databases. This is typically the same database servers where development, QA, reporting, or other target databases run today.

SSH, DSP

JDBC

Source Environme

nt


The Delphix Ecosystem: Virtual Database

Target Environments

SSH, JDBC

NFS

Any Storag

e

FC

A Virtual Database, or VDB, is a fully functional read/write database that runs on a Target Environment. Storage for the VDB is completely managed by the Delphix Server and provided to the Target Environment via NFS. All changes to the VDB are stored back on the Delphix Server.

SSH, DSP

JDBC

Source Environme

nt

Delphix Engine features and use cases

• Platforms DatabasesVMware, Amazon EC2 Oracle, SQLServer, PostgreSQL, Sysbase

• Application DataUnix/Linux and Windows file-systems

• Data MaskingIntegrated compliance and security

• Business intelligence and data warehousingAid ETL with production copies, or offload BI/DW workload

• Upgrades, migration, and modernizationUpgrade OS/database, migrate to Linux, migrate to new data center or cloud

• ReplicationHigh-availability and fault-tolerance for the Delphix Engine

Private and confidential© 2014 Delphix. All Rights Reserved© 2014 Delphix. All Rights Reserved

THANK YOUANY QUESTIONS?

Private and confidential

Em: [email protected]: @TimothyJGorman

Web: http://Delphix.com/

Make yourself at home at booth #821!

mailto:[email protected]

mailto:[email protected]

http://Delphix.com/

transforming it infrastructure

Technology

problem prod

prod growsprod prod

mirror setprod

solution delphix wrapup

extra copyprodprodprod

supportprod qatest dev

devprod qatest dev

specific time