simple data automation using base sas in a ... group...automation best practices: • always test...

SIMPLE DATA AUTOMATION USING BASE SAS IN A WINDOWS ENVIRONMENT

Jennet Gambino

Bond Brand Loyalty

Jennet.Gambino@bondbrandloyalty.com

“You’re either the one that creates

the automation or you’re getting

automated.”

— Tom Preston-Werner

Why do we automate data tasks?

• Increased productivity

• Reduced direct human

labor cost and expenses

• Improved quality and

consistency of output

Client customer transactional files and survey

response data files originating from multiple

application systems will need to be

extracted, transformed and load (ETL) at

specific time daily.

CHALLENGES:

• Multiple files are coming from different systems at

different time (daily or hourly)

• Data processing business rules are massive

• Need email alerts to notify someone if any of files

are missing or if there are any issues with the content

SAS Process

ReportsMFT Server

Reports/ Error Log

What a manual work flow looks like

Check that each file is

posted on the ftp

hourly

Other

Systems/

Applications

At specific time of the

day submit each SAS

code needed to

transform and generate

data files

Manual

Data Upload

SAS Process

Reports / Emails

(Automated)MFT

Server

Reports / Error Log

Simple Workload Automation with SAS Processing

Scheduled Data Feed

(Automated)Other

Systems

Applications

Scheduled SAS Batch

Processing (Automated)

Email File

Issues

Scheduled

Data Feed

(Automated)

SETUP REQUIREMENTS:

• Managed File Transfer (MFT) tasks

• SAS code (performing all the business rule/ data

processing)

• Windows Task Scheduler (server)

• Windows batch file script calling the necessary SAS

code and program (ex. Content of Step1_runSAS.bat)

"C:\PROGRAM FILES\SASHOME\SASFOUNDATION\9.X\SAS.EXE“ -SYSIN

T:\DATA_CLEANER.SAS -CONFIG

"C:\PROGRAM FILES\SASHOME\SASFOUNDATION\9.X\SASV9.CFG"

THE WINDOWS TASK SCHEDULER

1. At the windows explorer, search “Task Scheduler”,

2. Choose Action in the menu, create basic task, fill all the necessary information,

name and description

Create a Basic Task

THE WINDOWS TASK SCHEDULER

3. Next 2 screens, choose the frequency (and the time if applicable)

Create a Basic Task

THE WINDOWS TASK SCHEDULER

3. Then, next 2 screens, will be the Action section settings. Enter the location of the

windows batch script.

Create a Basic Task

THE WINDOWS TASK SCHEDULER

3. Final screen, review the summary of the tasks , then complete the settings by

clicking on the Finish button.

Create a Basic Task

OUTPUT OF THE SCHEDULED TASK

• Formatted data files needed by other systems

• .LST file and output file containing all output information

• Emailed report(s) only if applicable

Create a Basic Task

AUTOMATION BEST PRACTICES:

• Always test all your SAS codes manually before

scheduling the automated tasks.

• Create the overall data flow chart, data processing and

schedule information as part of your documentation.

• Obtain all the necessary approvals from your clients.

Think about data security and privacy.

• Before going “LIVE” in the server, schedule a performance

test with your IT department.

Me after a successful automation deployment.