simplifying company processes with perl stephane barde arthys / ezperl oreilly open source...

22
Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl O’Reilly Open Source Conference 2000 Stephane Barde 7/19/2000

Upload: alexandra-cole

Post on 27-Mar-2015

217 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

O’Reilly Open Source Conference 2000

Stephane Barde

7/19/2000

Page 2: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

I - Value Added Information Channels

-> Problem related to information flow management-> Creating the value added channel

II - Perl: a key enabler of process re-engineering

-> Resources use maximized with Perl-> Key Advantages

III - Flat Files as a Central Data Repository:

-> Dimensioning.-> Retrieving Data.

Page 3: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

I - Value Added Information Channels

What this is all about:Company standard processes generating information, that are not part of the core business, but generate important loss of productivity.

Improve Efficiency Globally by Giving Value to the Information

Examples:• Travel Request and Booking Information• Bug Tracking System• Application Access Request• New Employee Check Lists: how to get a new employee productive in a minimum amount of time.

Page 4: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

A Typical Information Stack: the Approval Chain

User Need

Approval

Detailed Information

References

User

Manager

Tech/Functional Support

Buyer, Finance, Receiver...

Different People Providing Different Pieces of Information

Travel Request

Departure/Return Date, Hotel, Car.

Approval up to a certain amount. Overnight stay, in lieu off approval...

Detailed Travel Info: Schedule, addresses...

PO #, Invoice #, Folder #, booking #…

Page 5: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

Financial Consolidation and Analysis

Travel Requester

Travel Booker

Travel Approver

A Small Company, one Department

Travel Requester

Travel Booker

Travel Approver

Department

Travel Requester

Travel Booker

Travel Approver

Department

MS Excel

!?

A Small Company: Single Vertical DimensionManual process -> time consuming, either the booker or the financial analyst spend time to consolidate data.

Loss of Productivity on Administrative Tasks

Multi-dimensional model: Processes are different within the organization. Consolidation end up being a nightmare as Data don’t mean the same and are not handled the same way across the organization.

A Typical Information Stack:

Page 6: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

Issues with information Flow Management:

How Efficient are You at Managing and Using Information?

Do we (we = all users involved in the process) all talk the same language through the entire organization? Do we all use the same information pieces to complete a given action?

Am I (I = anyone in the organization) able to report on given metrics globally (what is spent and how it is spend, what is the mean time to resolve a problem)? Can I easily aggregate the information I work on?

Am I able to quickly retrieve the information I need to analyze, compare, and make the right decision at the right time?

Page 7: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

Issues with Information Flow Management:

Take advantage of the Information You Generate

Consistency: No Systematic Processing.

Consolidation: Someone Someday will have to put everything together to report on specific metrics.

Accessibility: No searching capabilities, data not stored on a centralized repository.

Capitalization: Any data repository is a potential wealth: Developing Analysis Capacity will enable to Gain Productivity, Sharpen your Strategy, Refine your Goals.

Page 8: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

Travel Request

Approval No

Approved

Travel Info

Travel DB

Travel Request Process Map

Raw Information

Travel Info Travel Agency

Travel Info

XML Invoice

Travel Booker

A/P

XML Invoice

Page 9: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

End User

UserTest

Bug DB

BugFiltering Tech.

Support

Resolution

Incident closed

OK

Not Solved

Bug Tracking Process Map

Help for Resolution

Page 10: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

Perl

Information Flow Management

Consistency ConsolidationAccessibility &Capitalization

“Process Sticky” Development

Web Based Interfaceto Data

Efficient Data Access

II - Perl as a Key Enabler for Process Re-Engineering

Perl sets a favorable context to Manage Data Flow

Page 11: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

ITManagement

Need for a Web Based Solution

What Management says to the IT guy:“We need to (re-)engineer this process. You’ve got no resources, no money and no time. Just do it. When do you thing it will be ready?”

Time / Resource / $ ?

II - Perl as a Key Enabler for Process Re-Engineering

Will Do… with Perl

You’re the IT guy!What Do You Say?

Page 12: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

Perl

Other Programming language

Programmer Learning Curve

Time

Productivity

Quick developer learning curve makes project staffing easier.

Perl Key Advantages

• IT resources primarily focused on core business applications development and support.

• No time to waste in experimenting: need for efficient, documented, pre-existing solutions.

• Perl reminds of many other programming language.• Quick Learning Curve.• Lot of documented free resources available (CPAN).• Perl is free and highly portable.• Perl is low on Hardware Resources

Short Implementation Timeline with Few Resources

Page 13: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

Dealing with Data

Perl makes Flat File a Good Solution for Web Based Processes

• Perl provides connectors to most of the database types.

• Perl has an efficient set of tools and libraries to develop solutions close to the users process: file processing, triggered email alerts, XML data parsing, synchronization with BackOffice…

• Flat files are a quick way to deploy centralized web based processes.

• They offer an efficient way to store information at no cost.

• Flat files present the advantage of being compatible with most software. For example MS Excel allows further analysis of raw data provided by flat file databases.

Page 14: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

III - Flat Files as a Central Data Repository:

Keep File’s Size Small

• Flat file processing works fine up to 50k lines (and more) depending on the hardware and server workload.

• Well suited for low transactional volume: up to 200 transactions/day.

• Travel Request Example:10.000 employees, 500 frequent flyers (2 trips/month).2000 employees travel twice a year.=> 14.000 rows per year (1k/line => 14 Megs)=> flat files yearly split

To increase search efficiency:• archiving by splitting databases (temporal split / by data value).• Have index files containing keywords and reference to static generated file.

Page 15: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

data.csv

Retrieving information

Looking for ‘ABC’

……………………………...

Search Results

One Data Entry

XML output dynamically generated

XMLtemplate

2 full scans to retrieve the information needed

XSL

XML

Page 16: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

……………………………...

travel.csv

………………………

travel_2000.csv

File split: as the database size decreases, the search is more efficient.

Reducing Response Times: File Split

Example: yearly split of the master data file

Natural Way to Decrease the Size of the File Parsed

Page 17: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

data.csv

Looking for ‘ABC’

……………………………...

Search Results

Static XMLwith Ref #

Ref #

XML

……………………………...

data.csv

XML

1 database entry is associated with 1 static XML file.

XSL

Reducing Response Times: Static Display

Processing Allocated to Data Search

Page 18: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

ref;user_first_name;user_last_name;request_date;from;to;date;preferred_airline;preferred_time;airline;time;………..543;John;Doe;3/5/2000;London;Paris;Air France;8:00 PM;Air France;8:20 PM;……….542;Sam;Tondac;3/6/2000;Los Angeles;Paris;KLM;6:00 PM;Delta Airlines;5:54 PM;……….541;Jennifer;Alister;3/6/2000;Houston;Chicago;American Airlines;5:30 PM;American Airlines;3:33 PM;………….….….

Travel.csv

<travel_request><user><user_first_name>John <user_first_name><user_first_name>Doe <user_first_name></user> <requested_date>3/5/2000<flight><from>London</from><to>Paris</to><preferred_airline>Air France</preferred_airline><preferred_time>8:00 PM</preferred_time>….….

Travel_543.xml

Page 19: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

data_key.csv

Looking for ‘ABC’

……………………………...

Search Results

Static XMLwith Ref #

Ref #

XML

data.csv

……………………………...

+ : Size of the file parsed can decrease by a factor 5 to 10. Can be correlated with temporal split.- : Edit efficiency decreases.

Reducing Response Times: Reference Key & Static Display

XSL

Volume of Data Parsed Significantly Decreases

Page 20: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

ref;user_first_name;user_last_name;request_date;from;to;date;preferred_airline;preferred_time;airline;time;………..543;John;Doe;3/5/2000;London;Paris;Air France;8:00 PM;Air France;8:20 PM;……….542;Sam;Tondac;3/6/2000;Los Angeles;Paris;KLM;6:00 PM;Delta Airlines;5:54 PM;……….541;Jennifer;Alister;3/6/2000;Houston;Chicago;American Airlines;5:30 PM;American Airlines;3:33 PM;………….….….

Travel.csv

<travel_request><user><user_first_name>John <user_first_name><user_first_name>Doe <user_first_name></user> <requested_date>3/5/2000<flight><from>London</from><to>Paris</to><preferred_airline>Air France</preferred_airline><preferred_time>8:00 PM</preferred_time>….….

Travel_543.xml

ref;user_first_name;user_last_name; from;to;airline543;John;Doe;London;Paris;Air France542;Sam;Tondac;Los Angeles;Paris;Delta Airlines541;Jennifer;Alister;Houston;Chicago;American Airlines….….

Travel_key.csv

Page 21: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

INSERT UPDATE

• Lot of Data• High volume of searches• Few Updates• > 100 inserts/day

• Small Volume of Data• Lot of Updates• < 40 inserts/day

File Split

Static Display

Reference Keys & Static Display

Reference Keys & Static Display + File Split

Static Display+ File Split

Search Efficiency

Ex: Bug Tracking System

Ex: Travel Request

Ex: Phone Directory

Page 22: Simplifying Company Processes with Perl Stephane Barde Arthys / Ezperl OReilly Open Source Conference 2000 Stephane Barde 7/19/2000

Simplifying Company Processes with Perl

Stephane Barde Arthys / Ezperl

Perl makes possible the restructuration of the Information Channels with a minimal investment.

For more information:[email protected]