big iron to big data: how your mainframe data completes the puzzle

Big Iron to Big Data: How Your Mainframe Data Completes the Puzzle

Session Abstract and Speakers

Big Data and the “data lakes” for analytics and machine learning that power it are only useful if they

contain the most valuable data. For enterprises, critical operational data originates and resides on the mainframe, making it the all-important key to your business getting the benefits and competitive advantages revealed by Big Data-powered analytics. This webinar outlines how the relationship between mainframe (“Big Iron”) and Big Data analytics can finally now work to solve the last “data” problem and unlock the full promise of Big Data-powered analytics and decision-making. Mainframersand Big Data professionals have never been as important to the business or needed each other more than today so join us to find out the “why” and the “how.”

2Syncsort Confidential and Proprietary - do not copy or distribute

David Hodgson, GM,

Mainframe

Steven Menges, Director,

Product Management

Justin Eastman,

Senior Engineer

Big Data is No Longer a “Future”

Syncsort Confidential and Proprietary - do not copy or distribute 3

Big Data Poll


Q1.Which Big Data analytics platforms does your company use today?

o Hadoop

o Splunk

o Other Data Warehouse

o Don’t Know

(Check all that apply)

Mainframe (aka “Big Iron”) in 2016


Mainframe “Log” Data


Security

Operational

Application Monitoring

Mainframe Application

RACF

Intrusion Detection

Related Mainframe Logs/Data Types

SMF Type 80

SyslogD

Operator logs for DB2, CICS, IMS, etc

Syslog

DB2 Accounting Records

CICS Accounting Records

WebSphere

Job / Step Accounting Records

SMF Type 101

SMF Type 110

Log4j

SMF Type 30


Big Iron and Big Data in 2016: Important, Yet Often Separate

Big Iron to Big Data Poll


Q2. Is Mainframe “log” data going into your big data platform/repository?

o Yes, it is being streamed into it today

o Yes, it goes into it via periodic batch/other input method

o No, but that data has been requested/is desired

o No

o Don’t Know


• Analytics on new and unique data sets is critical to maintaining and increasing competitiveness

• Data lakes for analytics and machine learning are only useful if they contain the most valuable data

• For most larger enterprises, critical operational data originates and resides on a mainframe

• Correlating mainframe data with emerging sources like web and telemetry data is a key to success

• Working with mainframe data is hard and requires the right technology, expertise and focus

Big Iron to Big Data: Why?

Big Iron to Big Data is Mandatory for Gaining Competitive Advantage


• Analytics on new and unique data sets is critical to maintaining and increasing competitiveness

• Data lakes for analytics and machine learning are only useful if they contain the most valuable data

• For most larger enterprises, critical operational data originates and resides on a mainframe

• Correlating mainframe data with emerging sources like web and telemetry data is a key to success

• Working with mainframe data is hard and requires the right technology, expertise and focus

You need this data access to make you successful

Big Iron to Big Data is a Critical Intersection


• Stable market • Mainframes process 71% of Fortune 500 companies’ core

business transactions and store 80% of enterprise data • New data sources including web, mobile and IoT applications

are increasing mainframe transaction volumes• Innovations in mainframe systems are attracting new

workloads – z13 is able to process 2.5 billion transactions a day or 100 Cyber Mondays each day of the year

Big Iron

source: IBM 1

“Industrial Internet Insights for 2015,” GE and Accenture“Practical Challenges Mount as Big Data Moves to Mainstream,” Gartner

2

• Fast-growing market • 76% of enterprises already investing in Big Data or plan to

within a year• Desire to leverage new data sources created by web, mobile,

social and IoT driving increasing rate of deployments• 84% of companies believe Big Data is shifting the competitive

landscape • 89% say failure to adopt Big Data is a risk to market share2

1

2

Big Data

Big Iron to Big Data

Big Iron

MVPs: Always Important, Big Iron and Big Data Functions, Staff Now Critical


“BMC Annual Mainframe Research Results 2015”1

Big Iron to Big Data Big Data

USE CASE: THE NEED/PROBLEM

Security threats on the mainframe due to lack of visibility.

Highly sensitive PHI (Protected Health Information) escaping as data was moved from the production to test environment despite having fences and an automated scrubbing process.

Security information and event management (SIEM) was required.


USE CASE ALTERNATIVES: IN-HOUSE, OTHER

Do nothing and wait for an audit, or even worse, a security exposure

Attempt to perform post-exposure forensics

Manually extract and process logs, SMF records, etc. and produce audit reports to demonstrate compliance

Do solution vendor search and utilize Gartner Magic Quadrant, etc. for SIEM


USE CASE: SOLUTION AND RESULTS

SIEM Solution (Gartner SIEM Leader Splunk)

BIBD Solution to access z/OS log data in Splunkfor real-time alerts (Splunk’s chosen mainframe partner Ironstream)

Combined solution for mainframe logs provides fast access to:

Unusual data movements, amount of movements, and protocols being used

How much of the data movement is compliant, non-compliant, or unknown

Sources of inbound traffic relating to any anomalies


Organizational confidence in ability to audit data access compliance!

Big Iron

Big Iron to Big Data Use Case: Enterprise Security - Recap


• Unlock mainframe data for machine learning and advanced analytics

• Deliver data from any legacy source to analytic environments at the speed of business

• Ensure data lineage, security and efficiency

• Easy, high-volume data ingestion to Big Data repository

• Integrate and transform any data source data for advanced analytics and machine learning

Growing security and compliance requirements for enterprises

1


Ironstream

Big Iron, Big Data and Big Iron to Big Data: Additional Use Cases?

17

Syncsort Confidential and Proprietary - do not copy or distribute

17

Syncsort Confidential and Proprietary - do not copy or distribute

Security & Compliance (SIEM)

• Access Control

• Data Movement

• Real-time Intrusion Detection

• Others?

IT Operations (ITOA)

• Systems Performance and

Tuning

• Capacity Planning

• Others?

Other Monitoring & Analytics?

Big Iron

Enterprises Must Drive Business Value, Reduce Risk Across Multiple Domains


• Unlock mainframe data for machine learning and advanced analytics

• Deliver data from any legacy source to analytic environments at the speed of business

• Ensure data lineage, security and efficiency

• Easy, high-volume data ingestion to any Big Data repository

• Integrate and transform any data source data for advanced analytics and machine learning

• Enable data consumption on premise or in the cloud

• Reduce mainframe computing costs by optimizing peak processing

• Improve availability, reliability and integrity

• Meet growing security and compliance requirements

1

“BMC Annual Mainframe Research Results 2015”1


Solutions for New and “Old” Requirements


High-performance sort for z Systems™

Best Sort for z Systems

Offload Copy & SMS Compression and Sort work to zIIP processors

Savings with zIIP

Unlock value of IMS/VSAM data by transparently migrating to DB2

IMS and VSAM DB2 Migration

Network Managementz/OS® network management & security components

Big Data integration with market-leading support for integration and access of mainframe and legacy data sources

Data Access for Big Data

Collect, transform and stream mainframe app and system log data in near real time to Splunk Enterprise

Log Data Access for Big Data

High-performance Big Data integration software – Linux/Unix/Windows; Hadoop & Spark; on premise and in the cloud

Big Data Integration

The most advanced sort features for Unix, Linux, and Windows platforms

Best Sort for Distributed Platforms

Faster application modernization with less hardware

AppMod

Populate enterprise data lake at the push of a button

Data Funnel

Big Iron Big Iron to Big Data Big Data

BIBD Value at Scale for the Most Demanding Environments - Examples


• Standardizing on MFX and ZPSaver globally for more than 23,000 MSU mainframe environment

• Millions of dollars of savings in annual mainframe costs

• Run DMX Sort on more than 10,000 CPUs to sort billing data for more than 120 million customers

• Ironstream® delivers mainframe security logs in real time to Splunk® Enterprise platform to protect customer information

• Enables enterprise-wide view correlating events across platform that might not raise suspicion alone

• Use DMX-h and Data Funnel to populate Hadoop with mainframe data including 100s of tables and TBs of data from variety of sources

• Leverage unique capability of DMX-h to support data lineage for governance and compliance

• Use DMX-h to bring more data into their analytics platforms for better business insights and to enable clients to make better decisions

• Improved processes from months to hours, opening up valuable cycles for lead developers

• Using DL/2 to migrate 500 IMS databases to a relational model for easier app maintenance and flexibility

• Reduce risk associated with support of legacy applications as skills shortage accelerates

Big Iron Big Iron to Big Data Big Data

Global Bank

Automotive

Insurance

Financial Services Information Services

Telecommunications

Questions and More Information

Additional Questions for David and Justin?

For More Information:

syncsort.com/ironstream

blog.syncsort.com/

Try Ironstream for Free:

syncsort.com/ironstreamstarteredition

Comments/Other:

Steven Menges: [email protected]


http://www.syncsort.com/ironstream

http://blog.syncsort.com/

http://syncsort.com/ironstreamstarteredition

mailto:[email protected]

big iron to big data: how your mainframe data completes the puzzle

Technology