experiences in mainframe-to-splunk big data access

Experiences in Mainframe-to-

Splunk® Big Data Access: Learn What Your Peers are Doing

October 2016

Housekeeping

Webcast Audio:

– Today’s webcast audio is streamed through your computer speakers.

– If you need technical assistance with the web interface or audio, please reach out to us using the chat window.

Questions Welcome:

– Submit your questions at any time during the presentation using the chat window.

– We will answer them during our Q&A session following the presentations.

Recording and Slides:

– This webcast is being recorded. You will receive an email following the webcast with a link to download both the recording and the slides.

2

Session Abstract and Speakers

The requirement to add mainframe data to the stream of machine-to-machine or “log” data for

operational and security/compliance purposes is real. This webinar details 4 organizations who faced these requirements and tells their individual stories as to what requirement/mandate they faced, what options they considered, and how they ultimately addressed it. There will be a live Q&A for participants to ask follow-up questions as to their stories and how they’re doing today.

3Syncsort Confidential and Proprietary - do not copy or distribute

David Friedman,

Senior Systems Engineer

Steven Menges, Director,

Product Management

Justin Eastman,

Senior Engineer

Big Data is No Longer a “Future”

Syncsort Confidential and Proprietary - do not copy or distribute 4

DB2SYSOUT

Live/Stored

SPOOL Data

Alerts

Network

Components

Ironstream API

Application Data

Assembler

C

COBOL

REXX

USSLog4jFile

Load

SYSLOGSYSLOGD

logs

security

SMF

50+

types

RMF

Up to 50,000

values

Mainframes Still Host the Most Critical Applications at Big Orgs


71%Fortune 500

2.5 Billion Bus. Transactions / day / per MF

23of Top 25 US Retailers

of World’s Top Insurers10Top World

Banks92

Source: IBM Organizations Overall2000+

Organization #1: Justin Eastman


Justin Eastman,

Senior Engineer

Reminder:

Type in your questions at any time

during the presentation using the chat

window.

We will answer them during our Q&A

session following the presentations or

afterward.

USE CASE: THE NEED/PROBLEM

Incidents occurring in the organization would result in the need to turn on additional CPUs to recover from system being overloaded.

There was a need to get visibility into the health of multiple sub-systems across different systems in order to monitor to the load to proactively react to these situations.

No single tool or monitor gave visibility into all the different subsystems and across the entire organization.


USE CASE ALTERNATIVES: IN-HOUSE, OTHER

Continue with the human flare gun approach currently used in which multiple groups get involved elongating the mean time to resolution.

Use existing monitors that require multiple sessions and SMEs to access and perform triage.

Expand capacity to ensure that systems are not overloaded.

Continue to rely on the customer to indicate when the services provided become less responsive.

Look for a new solution to address their issues.


USE CASE: SOLUTION AND RESULTS

BIBD Solution to access z/OS log data in Splunk® for real-time monitoring of critical subsystem performance

Creating single view into the health of all the systems and their corresponding subsystems

Creating drill down dashboards that provide the KPIs to identify where the source of the issue resides to that the immediate source

Resulted in a significant reduction of MTTR and a improve ability to detect problems before the customer does.


Big Data Poll


Q1.Which Big Data analytics platforms does your company use today?

o Hadoop

o Splunk

o Other Data Warehouse

o Don’t Know

(Check all that apply)

Organization #2: David Friedman


David Friedman,




Customer had an audit and compliance mandate with approaching deadline.

Using another product to manually retrieve information on a daily basis.

Unable to monitor user log-on attempts, password changes, and access violations on their mainframe environment.

Unable to obtain information in real-time.


Home-grown solution option explored; determined would not meet implementation deadline (and may not have satisfied requirement).

POC bake-off (Syncsort Ironstream performed very well in a POC against competitive product).

Validated the ability to replace the manual processes they were using with Ironstream.



Monitoring security activity on their mainframeapplications to meet audit and compliance requirements outlined in regulation, including:

– log-on attempts

– password changes

– user access violations

– other security events

Get the information in real-time (and eliminated manual processes previously accomplished using zSecure)

Filtering enables selection of only the SMF records needed to produce desired results


SecurityCompliance

Organization #3: Justin Eastman


Justin Eastman,

Senior Engineer


Security threats on the mainframe due to lack of visibility.

Highly sensitive PHI (Protected Health Information) escaping as data was moved from the production to test environment despite having fences and an automated scrubbing process.

Security information and event management (SIEM) solution required.



Do nothing and wait for an audit, or even worse, a security exposure.

Attempt to perform post-exposure forensics.

Manually extract and process logs, SMF records, etc. and produce audit reports to demonstrate compliance.

Do solution vendor search and utilize Gartner Magic Quadrant, etc. for enterprise-class SIEM.



SIEM Solution (Gartner SIEM Leader Splunk®)

BIBD Solution to access z/OS log data in Splunk® for real-time alerts (Splunk’s chosen mainframe partner

Ironstream)

Combined solution for mainframe logs provides fast access to:

Unusual data movements, amount of movements, and protocols being used

How much of the data movement is compliant, non-compliant, or unknown

Sources of inbound traffic relating to any anomalies


Organizational confidence in ability to audit data access compliance!

Big Iron to Big Data Poll


Q2. Is Mainframe “log” data going into your big data platform/repository?

o Yes, it is being streamed into it today

o Yes, it goes into it via periodic batch/other input method

o No, but that data has been requested/is desired

o No

o Don’t Know

Organization #4: David Friedman


David Friedman,


Reminder:

Type in your questions at any time

during the presentation using the chat

window.

We will answer them during our Q&A

session following the presentations or

afterward.



Disbursed transaction information systems.

Current tools provide partial solutions.

Require comprehensive analytics across operation.

Enterprise IT Operational Analytics (ITOA) dashboard desired.


Organization selected Splunk® Enterprise as their ITOA solution for distributed computing environment.

Considered Syncsort to access mainframe logs and get comparable data from mainframe systems.

Conducted thorough POC of Ironstream in conjunction with Splunk®

After POC, they were able to quickly start deploying it as a comprehensive monitoring solution.



Complete picture of overall system health.

Meaningful correlation of information from disparate sources for faster triage and shorter MTTR.

Company now able to monitor entire IT infrastructure to detect potential issues before they become critical.


Reduce MTTR

Big Iron, Big Data and Big Iron to Big Data: Additional Use Cases?

24

Syncsort Confidential and Proprietary - do not copy or distribute

24

Syncsort Confidential and Proprietary - do not copy or distribute

Security & Compliance (SIEM)

• Access Control

• Data Movement

• Real-time Intrusion Detection

• Others?

IT Operations (ITOA)

• Systems Performance and

Tuning

• Capacity Planning

• Others?

IT Service Intelligence?

Other Monitoring & Analytics?

Big Iron

MVPs: Always Important, Big Iron and Big Data Functions, Staff Now Critical


“BMC Annual Mainframe Research Results 2015”1

Big Iron to Big Data Big Data

Syncsort Solutions for New and “Old” Requirements


High-performance sort for z/OS®Best Sort for z Systems

Offload Copy & SMS Compression and Sort work to zIIP processors

Savings with zIIP

Database Optimization Suites for IBM DB2® and CA IDMS™

Network Managementz/OS® network management & security components

Big Data integration with market-leading support for integration and access of mainframe and legacy data sources

Data Access for Big Data

Collect, transform and stream mainframe app and system log data in near real time to Splunk Enterprise

Log Data Access for Big Data

High-performance Big Data integration software – Linux/Unix/Windows; Hadoop & Spark; on premise and in the cloud

Big Data Integration

The most advanced sort features for Unix, Linux, and Windows platforms

Best Sort for Distributed Platforms

Faster application modernization with less hardware

AppMod

Big Iron Big Iron to Big Data Big Data

Data FunnelPopulate enterprise data lake at the push of a button

Transparently migrate IMS to DB2IMS and VSAM DB2 Migration

Powerful new tools for your databases

Questions and More Information

Additional Questions for David and Justin?

For More Information:

syncsort.com/ironstream

blog.syncsort.com/

Try Ironstream for Free:

syncsort.com/ironstreamstarteredition

Comments/Other:

Steven Menges: [email protected]


http://www.syncsort.com/ironstream

http://blog.syncsort.com/

http://syncsort.com/ironstreamstarteredition

mailto:[email protected]

experiences in mainframe-to-splunk big data access

Technology