big iron to big data - user · big iron to big data: a fast-growing market segment ... •zpsaver...

Big Iron to Big Data: Do more with the data you’ve got

Kevin Markwick

Regional Sales Director

Leading the next technology [r]evolution:

Big Iron to Big Data

Big Iron to Big Data: a fast-growing market segment composed of solutions that optimize traditional data systems and deliver mission-critical data from these systems to next-generation analytic environments.

Syncsort Confidential and Proprietary - do not copy or distribute 2

Trusted Industry Leadership


500+Experienced & Talented

Data Professionals

>6,000Customers

196850 Years of Market Leadership

& Award-Winning Customer Support

84of Fortune 100 are Customers

3xRevenue GrowthIn Last 12 Months

The global leader in Big Iron to Big Data

Macro Trends Require Big Iron to Big Data Strategies

Digital business is driving significant growth in workloads managed by legacy data systems

▪ IBM i and Mainframes run the core transactional applications of most enterprises

▪ Growing touchpoints on mobile & online increasing transaction volumes and workload unpredictability

▪ Critical focus on data infrastructure optimization & availability initiatives to meet demand and manage costs

Innovation is enabling analysis of larger, more diverse data

▪ Enterprises making major investments in new Big Data repositories for greater insights

▪ Mobile & online data sources must beanalyzed quickly & efficiently

▪ Next-gen analytic environments must contain valuable app & machine data from legacy systems that is liberated, integrated & trusted

4


Syncsort Confidential and Proprietary - do not copy or distribute

Big Iron to Big Data Market Size and Forecast


$20.4 billionlarge, stable

Data Infrastructure Optimization &

Availability software market

$8 billion~20% annual

growth

Big Datasoftware market

$1.2 billion~25% annual growth

Big Iron to Big Data software market


Big IronBig Data

source: Estimates based on 2016 Wikibon study

5

And These Strategies Must Drive Value Across Multiple Domains

▪ Reduce computing costs on legacy data systems including mainframes & IBM i Power Systems

▪ Improve availability, reliability and integrity

▪ Meet growing security and compliance requirements

Data Liberation, Integration & Integrity

▪ Unlock mainframe and IBM i data for machine learning and advanced analytics

▪ Access, transform, integrate & deliver data to analytic environments

▪ Ensure data quality, lineage, security ▪ Enable data consumption on premise

and in the cloud

6

Data Infrastructure Optimization & Availability



Customer Use Cases & Strategic Partnerships


Data Infrastructure Optimization

• Mainframe Optimization

• Cross-Platform Capacity Management

• EDW Optimization

• Application Modernization

Data Availability

• High Availability & Disaster Recovery

• Mission-Critical Migration

• Cross-Platform Data Sharing

• IBM i Data Security & Audit

Big Iron to Big DataA fast-growing market segment composed of solutions that optimize traditional data systems and

deliver mission-critical data from these systems to next-generation analytic environments.

7



Data Integration

• Mainframe Access & Integration for Machine Data

• Mainframe Access & Integration for App Data

• High-performance ETL

Data Quality

• Data Governance

• Customer 360

• Big Data Quality & Integration

• Data Enrichment & Validation



8




• Mainframe Optimization

• Application Modernization

• EDW Optimization

• Cross-Platform Capacity Management

Data Availability

• High Availability & Disaster Recovery

• Mission-Critical Migration

• Cross-Platform Data Sharing

• IBM i Data Security & Audit

• Mainframe Access & Integration for Machine Data

• Mainframe Access & Integration for App Data

• High-performance ETL

Data Integration

Data Quality

• Data Governance

• Customer 360

• Big Data Quality & Integration

• Data Enrichment & Validation



9

Differentiated Product Portfolio & Technical Expertise



Data Availability

Data Integration

Data Quality

Market-leadingdata quality capability

Best-in-class resource utilization and performance, on premise

or in the cloud

#1 in high availability forIBM i and AIX Power Systems

Industry-leading mainframe data access and highest

performing ETL

• Trillium Software System

• Trillium Quality for Big Data

• Trillium Precise

• Trillium Cloud

• Trillium Global Locator

• DL/2

• Zen Suite

• MFX® for z/OS

• ZPSaver Suite

• EZ-DB2

• EZ-IDMS• DMX & DMX-h

• DMX AppMod

• athene®

• athene SaaS®

• MIMIX Availability & DR

• MIMIX Move

• MIMIX Share

• iTera Availability

• Enforcive IBM i Security

• Ironstream®

• Ironstream® Transaction Tracing

• DMX & DMX-h

• DMX Change Data Capture



10

USE CASE CHALLENGES DESCRIPTION & BUSINESS VALUE CUSTOMERS

Mainframe Optimization

• Contain hardware and software costs

• Improve performance• Meet/exceed SLAs to

support the business

• Best-in-class solutions for optimizing mainframe performance and resource utilization

• Dramatically reduce the costs by reducing billable CPU time, containing overall software costs and delaying hardware upgrades

• Meet SLAs/batch windows to better serve the business

Cross-Platform Capacity Management

• Capacity, performance incidents impact business

• Difficult to get single view across enterprise

• Lack of time/resources

• Single solution to proactively manage capacity across your enterprise IT services and infrastructure

• Avoid service disruptions• Predict and minimize IT infrastructure costs• Free up staff, maximize accuracy

EDW Optimization

• EDW costs are escalating, performance suffering due to ELT/ETL processing

• Moving data and processing to data lake is complex, takes a long time

• Solutions to shift data and ELT workloads to the data lake quickly and efficiently

• Lower EDW costs and improve performance for Business Intelligence • Increase productivity and get faster time to value by automating,

optimizing and future-proofing development without requiring special skills

Application Modernization

• Minimize risk • Match performance of

application on new system• Lack of skills/resources• Time to value

• Enables migration of applications off the mainframe, with minimum or no modifications, to run on other platforms with the same or faster performance

• Seamless integration with leading re-hosting solutions• No manual coding or mainframe expertise required

Delivering Business Value: Data Infrastructure Optimization



High Availability & Disaster Recovery

• Minimize risk and costs of downtime

• Meet SLA, compliance requirements

• Resource constraints

• Proven IBM i solution virtually eliminates downtime and data loss for businesses of all sizes, from SMB to enterprise

• Reduce risk and costs of planned administrative downtime, component failures, software defects, operator error, malicious users, and local/regional disaster

Mission-Critical Migration

• Complexity, downtime and risk of migrations

• SLAs, compliance

• Near-zero downtime workload and system migration• Mitigate risk• Ease strain on internal IT resources

Cross-Platform Data Sharing

• Data spread across systems leads to stale, incomplete information

• Traditional approaches of data movement are inefficient, costly

• Real-time data sharing between all leading databases, operating systems, platforms (physical, virtual, cloud)

• Improve decision-making, business productivity with accurate, up-to-date information

• Easy to use – automated, no programming required• Cost effective

IBM i Data Security & Audit

• IBM i has no comprehensive security management

• Risk to your business• Compliance, governance• Lack of IBM i visibility/skills

• Most comprehensive, easy-to-use security & compliance solution for IBM i

• Get complete oversight/control of IBM i security without special skills• Protect critical data from ever-changing threats – external and internal• Satisfy auditors and meet the strictest compliance regulations

Delivering Business Value: Data Availability



Mainframe Access & Integration for Machine Data

• Mainframe data left out of enterprise-wide views

• Data is diverse, complex• Skills, expertise shortages

• Collect, transform and forward mainframe operational and security data into analytics platforms like Splunk Enterprise

• Include mainframe with your other systems for 360-degree view of your enterprise

• Easy to use - no mainframe expertise required

Mainframe Access & Integration for Application Data

• Critical enterprise data trapped in Mainframe silo –missing from data lake

• Mainframe is expensive• Working with Mainframe

and Big Data requires hard-to-find skills

• Unparalleled technology and expertise for loading Mainframe data into the data lake and integrating it with MapReduce or Spark

• Deliver powerful new insights by combining mainframe data with other data sources

• Save on Mainframe storage and processing costs• Achieve faster time to value• Use skills you already have - no mainframe expertise required

High Performance ETL

• Manual coding is resource-intensive, slow, inflexible and difficult to maintain

• Data Integration software requires tuning for performance/efficiency

• ETL jobs must be rewritten for new execution frameworks

• Fast, efficient, simple no-code ETL solution that runs on any platform – including MapReduce and Spark – on premise or in the cloud

• Get business insights faster with up to 10x lower elapsed processing times, linear scalability

• Shift staff to higher-value initiatives• Future-proof your environment• Save money by optimizing your system resources

Delivering Business Value: Data Integration



Data Governance

• Regulatory compliance, mitigate risk

• Data volume/complexity• Lack of visibility,

accessibility, trust in the data

• Data profiling integrated with leading data governance and data lineage solutions to identify/assess data across your organization and determine compliance with business rules – on premise or in the cloud

• Identify data issues fast so you can fix them and stay compliant• Have confidence in your data• Empower users with collaborative, web-based tools

Customer 360

• Grow customer base, share of wallet, loyalty

• Increasing volume, complexity and sources of customer data

• Data profiling, quality and enrichment solutions deliver a common, unified, trusted view of your customer data, across all systems and sources – on premise or in the cloud

• Maximize the value, efficiency and integrity of every customer interaction, across the entire customer lifecycle

Big Data Quality & Integration

• Maximize the value/ROI of the data lake

• Ensure the data lake is trusted source of data

• Highly-scalable, enterprise-level data quality that supports Big Data frameworks like MapReduce and Spark to process the largest and most complex data volumes in the data lake – on premise or in the cloud

• Trusted the insights from your most critical data assets• Achieve fast time to value with flexible deployment and ease of use

Data Enrichment & Validation

• Need complete, accurate data for effective customer engagement

• Customer data constantly changes

• Real-time data enrichment, validation and verification of third-party data sources, including email, landline/mobile phone, postal address and internet connectivity

• Engage with your customers and prospects more effectively• Mitigate risk by validating customer data

Delivering Business Value: Data Quality

Leading U.K.

Bank


DATA AVAILABILITY….AND MORE

Security Journal (QAUDJRN)

QHIST Log

File Journals

Message Queue

NEW: Operational Data

Exit Point Traffic*

Encryption*

•View DataView Data*

SQL Audit*

IBM iData Stream

Cross-Platform

Audit

Syslog /SIEM

IFS FilesSPLU

NK

Output TypesData Export

Market Landscape and Key Concepts: IBM i Data

*Requires use of our security products

Having The Big Picture allows organizations analyze these data sets to proactively and reactively

mitigate different risks. Typical reasons to perform data analysis include; compliance with security

policies and/or regulations, oversight of IT operations, and system troubleshooting.

Key Differentiators: Robust Filtering, Batch or Real-Time, Performance Optimized

Market Landscape and Key Concepts: Splunk

High-Level Summary of Splunk • First machine data analysis software

• Market leader

• The “Google for log files” enterprise tool

• Can be expensive –particularly with growing data volumes

• Original target for our Ironstream® products

• More mature partnership

Market Landscape and Key Concepts: SIEM

• SIM: Log management, analytics, compliance reporting

• SEM: Real-time monitoring & incident management

• Advanced Threat Detection

• Basic security monitoring

• Investigation and incident response

Security Information & Event Management

Market Landscape and Key Concepts: SIEM & Big Data

Data Type Use Case Main Players Data Size

API for Other Data

Sources

OS logs can be

correlated against

application activity

Splunk, Elastic

Stack, Sumo Logic

Even Bigger Data

Operations Data Security data &

operations data

converge

Splunk ITSI, Elastic

Stack, Sumo Logic,

Logpoint, QRadar,

LogRhythm

Big Data

SIEM Data Security threat

detection, monitoring

& response

QRadar, Splunk ES,

LogRhythm,

ArcSight, AlienVault

Little Data

SPLUNK

Market Landscape and Key Concepts:Stream Your Big Iron Data into The Industry-Leading Platform For Machine Data

Machine Data: Any Location, Type, Volume

Online

ServicesWeb

Services

ServersSecurity

GPS

Location

StorageDesktops

Networks

Packaged

Applications

Custom

AppsMessaging

TelecomsOnline

Shopping

Cart

Web

Clickstreams

Databases

Energy

Meters

Call Detail

Records

Smartphones

and Devices

RFID

On-

Premises

Private

Cloud

Public

Cloud

Platform Support (Apps / API / SDKs)

Enterprise Scalability

Universal Indexing

Answer Any Question

DeveloperPlatform

Report &analyze

Custom dashboards

Monitor & alert

Ad hoc search

Mainframe

We make Critical IBM i and Mainframe Machine Data USEABLE in Splunk with Ironstream®

IBM i

What Can You Do with Security Analytics Using IBM i Data?

Detect Data Movements – Inbound/Outbound FTP

Dataset access operations– Determine potential security threats based on

unauthorized access attempts– Ensure only authorized users are accessing critical

datasetsPrivileged/non-privileged User Activity Monitoring

– Unusual behavior pattern – off hours connections– High number of invalid logon attempts

Attack Detection– Intrusion, Scans, Floods

Authentication Anomalies– Entered the building at 08:30 but logged on from

another country at 09:00Maintain Compliance

– SOX, HIPAA, PCI, GDPR, Etc.

Key Use Cases – Problems we Solve

Enterprise Security and Compliance/SIEM

– Detect and prevent security threats

– Ensure compliance

– Ensure audits pass

IT Operations Analytics/ITOA/Service Intelligence

– Bigger picture of what's happening in the environment

– Make better decisions to take control of the IT infrastructure

– Ensure SLAs Met

What Can You Do with Security Analytics Using IBM i Data?

Detect Data Movements – Inbound/Outbound FTP

Dataset access operations– Determine potential security threats based on

unauthorized access attempts– Ensure only authorized users are accessing critical

datasetsPrivileged/non-privileged User Activity Monitoring

– Unusual behavior pattern – off hours connections– High number of invalid logon attempts

Attack Detection– Intrusion, Scans, Floods

Authentication Anomalies– Entered the building at 08:30 but logged on from

another country at 09:00Maintain Compliance

– SOX, HIPAA, PCI, GDPR, Etc.

Hadoop | The New Centerpiece of the Modern Data Architecture

What do customers want to use their Hadoop clusters for?

1. ETL

2. Analytics*

3. Data Blending

4. Active Archive

5. EDW / Mainframe Optimization

What do customers get out of it? – Business Benefits

Increase Productivity

Reduce Costs

Next-Gen Analytics

Increase Revenue and Growth

Archive Data

Increase Agility

Get More for EDW/ Mainframe Investment

Retain Data for Compliance

Free Mainframe Resources

Implementation Challenges

1. Data Quality: Assessing and improving quality of data as it enters and/or in the data lake.

2. Skills/Staff: Need to learn a new set of skills, Hadoop programmers are difficult to find and/or expensive.

3. Data Governance: Including data lake in governance initiatives and meeting regulatory compliance.

4. Rapid Change: Frameworks and tools evolve fast, and it’s difficult to keep up with the latest tech.

5. Fresh Data (CDC): Difficult to keep data lake up-to-date with changes made on other platforms.

6. Mainframe: Difficult to move mainframe data in and out of Hadoop/Spark.

7. Data Movement: Difficult to move data in and out of Hadoop/Spark.

0

10

20

30

40

50

% of People Who Consider this a Top Challenge (Rated 1 or 2)

Big Data Challenges

Data Quality Skills Governance Rapid Change

CDC Mainframe Data Movement Cost

Key Concepts: Security and Compliance in IBM i Environments

The IBM i is not inherently a secure system, it is however, extremely securable

Being in compliance doesn’t automatically mean the system is secure

Segregation of duties

– Auditors tend not to approve of tools developed in-house or modules that are part of an ERP

Key Concepts: Security and Compliance in IBM i Environments

New/open-source protocols create access headaches

– Legacy, proprietary protocols and experienced administrators now cohabitate with new/open-source protocols and young techies. All of the access points need to be controlled.

The IBM i is now more exposed to the worldwide hacker community (they finally discovered that it’s a very valuable target)

– First session dedicated to IBM i was presented at Def Con in 2015

Most customers have a security program in placeand claim to be confident in it

Customers are much more involved in, and knowledgeable about, security than in the recent past, which should send a clear message to security software vendors: “We aren’t neophytes so don’t waste our time explaining things we already know, get directly to the point about how you can solve my problems”

From Vision’s 2017 Security Survey - IBM Power

My organization’s IT systems are less secure

My organization’s IT systems are about as secure as they were last

year

My organization’s IT systems are more secure

Don’t know

0% 10% 20% 30% 40% 50% 60%

How secure are your organization’s IT systems today compared to a year

ago?

Yes

No

Don't know

0% 10% 20% 30% 40% 50% 60% 70% 80%

Do you have a formal security program in place for IT systems?

Very confident

Somewhat confident

Neither confident norunconfident

Somewhat unconfident

Very unconfident

0% 10% 20% 30% 40% 50%

How confident are you in the effectiveness of your

organization’s security program?

Compliance is a primary driver of IT security

• It is rare to find a company today that doesn’t need to comply with a regulation or external audit requirement

• Countries in secondary markets have their own regulations (Angola, Lebanon, Colombia, etc…) or are in the process to bringing them online (China)

Compliance needs

Industry best practices

Results of security audit

Risk-based framework

Cost-value analysis

Experience from past attacks

Board recommendations

Don’t know

Other (please specify)

0% 10% 20% 30% 40% 50% 60% 70% 80%

What resources and information were used to define your organization’s security program?

From Vision’s 2017 Security Survey - IBM Power

From Vision’s Risk Management Survey – IBM i, Dec. 2016 – respondents are Vision customers

What IBM i security software is already implemented?

45% of respondents have not implemented security software and 12% don’t know if they have (yellow highlight)

40% have implemented a Help Systems security product (in red)

4.5% have implemented Enforciveand 1.7% Cilasoft

Numbers are likely biased toward North America so would likely be different in other geographies

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Help Systems Powertech tools

Help Systems Bytware tools

Townsend Software

Enforcive Enterprise Security

Help Systems SkyView tools

NetIQ Secure Configuration…

Help Systems CCSS tools

Linoma Crypto Complete

Help Systems Safestone tools

Cilasoft Suite of products

IBM Power SC

Raz-Lee iSecurity suite of products

Cosyn Software

Secure Sphere Imperva

I don't know

Other (please specify)

We have not deployed security…

Security Solutions

Finding Opportunities with the Security Risk Assessment Tool

What it Is:

– Free, self-service assessment tool that enables IBM i administrators and security officers to identify security vulnerabilities by conducting real-time, fully controlled penetration attempts on their IBM i environment.

– Customer receives an Executive Summary of the results, along with specific recommendations for remediation.

– Customer registers for access on our website, downloads the tool, installs the software. Assessment runs as a native job on the target computer.

It´s all about Data replication

Todays market

Storage Replication

• Easy to use

• Pushed by IBM/partners

Software Replication

• Near ´Zero´downtime

• Continous Data Protection

• Optimum in RPO and RTO

Mimix for FlashCopy

• Hybrid solution

• Different Data with indivdual

RPO/RTO requirements

The Game-changer

The Idea• Opens the market of storage replication that was mostly inaccessible

• Win net new logos

• Provides a new opportunity to return to customers we lost to SAN replication

The Channel• Opens a whole new channel of partners focused on SAN storage

• Vision direct customers with storage replication in partnership with sophisticated partner

IBM FlashCopy Challenges

Switched offIPL phaseApplication downQUIESCEFORCE WRITE DATAJust FlashCopy

CertainCompromise

Uncertain

File 3

File1

File2

File 3

File 1File 2

tFlash-suspendtr2

t r3

t r1

Rollback Flash-none suspend

RCV0001

RCV0001 RCV0002

RCV0001

t1

RCV0001 RCV0002

t2 t3 t4

Tue 06:00am

Tue12:00pm

Tue 06:00pm

FlashCopy Consistency

FlashCopy Journal Chain FlashCopy Timeline

FlashCopy Commit & Rollback

Flash1

Flash2

Flash3

LIVE

big iron to big data - user · big iron to big data: a fast-growing market segment ... •zpsaver...

Documents