simplifying company processes with perl stephane barde arthys / ezperl oreilly open source...
TRANSCRIPT
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
O’Reilly Open Source Conference 2000
Stephane Barde
7/19/2000
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
I - Value Added Information Channels
-> Problem related to information flow management-> Creating the value added channel
II - Perl: a key enabler of process re-engineering
-> Resources use maximized with Perl-> Key Advantages
III - Flat Files as a Central Data Repository:
-> Dimensioning.-> Retrieving Data.
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
I - Value Added Information Channels
What this is all about:Company standard processes generating information, that are not part of the core business, but generate important loss of productivity.
Improve Efficiency Globally by Giving Value to the Information
Examples:• Travel Request and Booking Information• Bug Tracking System• Application Access Request• New Employee Check Lists: how to get a new employee productive in a minimum amount of time.
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
A Typical Information Stack: the Approval Chain
User Need
Approval
Detailed Information
References
User
Manager
Tech/Functional Support
Buyer, Finance, Receiver...
Different People Providing Different Pieces of Information
Travel Request
Departure/Return Date, Hotel, Car.
Approval up to a certain amount. Overnight stay, in lieu off approval...
Detailed Travel Info: Schedule, addresses...
PO #, Invoice #, Folder #, booking #…
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
Financial Consolidation and Analysis
Travel Requester
Travel Booker
Travel Approver
A Small Company, one Department
Travel Requester
Travel Booker
Travel Approver
Department
Travel Requester
Travel Booker
Travel Approver
Department
MS Excel
!?
A Small Company: Single Vertical DimensionManual process -> time consuming, either the booker or the financial analyst spend time to consolidate data.
Loss of Productivity on Administrative Tasks
Multi-dimensional model: Processes are different within the organization. Consolidation end up being a nightmare as Data don’t mean the same and are not handled the same way across the organization.
A Typical Information Stack:
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
Issues with information Flow Management:
How Efficient are You at Managing and Using Information?
Do we (we = all users involved in the process) all talk the same language through the entire organization? Do we all use the same information pieces to complete a given action?
Am I (I = anyone in the organization) able to report on given metrics globally (what is spent and how it is spend, what is the mean time to resolve a problem)? Can I easily aggregate the information I work on?
Am I able to quickly retrieve the information I need to analyze, compare, and make the right decision at the right time?
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
Issues with Information Flow Management:
Take advantage of the Information You Generate
Consistency: No Systematic Processing.
Consolidation: Someone Someday will have to put everything together to report on specific metrics.
Accessibility: No searching capabilities, data not stored on a centralized repository.
Capitalization: Any data repository is a potential wealth: Developing Analysis Capacity will enable to Gain Productivity, Sharpen your Strategy, Refine your Goals.
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
Travel Request
Approval No
Approved
Travel Info
Travel DB
Travel Request Process Map
Raw Information
Travel Info Travel Agency
Travel Info
XML Invoice
Travel Booker
A/P
XML Invoice
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
End User
UserTest
Bug DB
BugFiltering Tech.
Support
Resolution
Incident closed
OK
Not Solved
Bug Tracking Process Map
Help for Resolution
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
Perl
Information Flow Management
Consistency ConsolidationAccessibility &Capitalization
“Process Sticky” Development
Web Based Interfaceto Data
Efficient Data Access
II - Perl as a Key Enabler for Process Re-Engineering
Perl sets a favorable context to Manage Data Flow
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
ITManagement
Need for a Web Based Solution
What Management says to the IT guy:“We need to (re-)engineer this process. You’ve got no resources, no money and no time. Just do it. When do you thing it will be ready?”
Time / Resource / $ ?
II - Perl as a Key Enabler for Process Re-Engineering
Will Do… with Perl
You’re the IT guy!What Do You Say?
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
Perl
Other Programming language
Programmer Learning Curve
Time
Productivity
Quick developer learning curve makes project staffing easier.
Perl Key Advantages
• IT resources primarily focused on core business applications development and support.
• No time to waste in experimenting: need for efficient, documented, pre-existing solutions.
• Perl reminds of many other programming language.• Quick Learning Curve.• Lot of documented free resources available (CPAN).• Perl is free and highly portable.• Perl is low on Hardware Resources
Short Implementation Timeline with Few Resources
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
Dealing with Data
Perl makes Flat File a Good Solution for Web Based Processes
• Perl provides connectors to most of the database types.
• Perl has an efficient set of tools and libraries to develop solutions close to the users process: file processing, triggered email alerts, XML data parsing, synchronization with BackOffice…
• Flat files are a quick way to deploy centralized web based processes.
• They offer an efficient way to store information at no cost.
• Flat files present the advantage of being compatible with most software. For example MS Excel allows further analysis of raw data provided by flat file databases.
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
III - Flat Files as a Central Data Repository:
Keep File’s Size Small
• Flat file processing works fine up to 50k lines (and more) depending on the hardware and server workload.
• Well suited for low transactional volume: up to 200 transactions/day.
• Travel Request Example:10.000 employees, 500 frequent flyers (2 trips/month).2000 employees travel twice a year.=> 14.000 rows per year (1k/line => 14 Megs)=> flat files yearly split
To increase search efficiency:• archiving by splitting databases (temporal split / by data value).• Have index files containing keywords and reference to static generated file.
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
data.csv
Retrieving information
Looking for ‘ABC’
……………………………...
Search Results
One Data Entry
XML output dynamically generated
XMLtemplate
2 full scans to retrieve the information needed
XSL
XML
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
……………………………...
travel.csv
………………………
travel_2000.csv
File split: as the database size decreases, the search is more efficient.
Reducing Response Times: File Split
Example: yearly split of the master data file
Natural Way to Decrease the Size of the File Parsed
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
data.csv
Looking for ‘ABC’
……………………………...
Search Results
Static XMLwith Ref #
Ref #
XML
……………………………...
data.csv
XML
1 database entry is associated with 1 static XML file.
XSL
Reducing Response Times: Static Display
Processing Allocated to Data Search
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
ref;user_first_name;user_last_name;request_date;from;to;date;preferred_airline;preferred_time;airline;time;………..543;John;Doe;3/5/2000;London;Paris;Air France;8:00 PM;Air France;8:20 PM;……….542;Sam;Tondac;3/6/2000;Los Angeles;Paris;KLM;6:00 PM;Delta Airlines;5:54 PM;……….541;Jennifer;Alister;3/6/2000;Houston;Chicago;American Airlines;5:30 PM;American Airlines;3:33 PM;………….….….
Travel.csv
<travel_request><user><user_first_name>John <user_first_name><user_first_name>Doe <user_first_name></user> <requested_date>3/5/2000<flight><from>London</from><to>Paris</to><preferred_airline>Air France</preferred_airline><preferred_time>8:00 PM</preferred_time>….….
Travel_543.xml
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
data_key.csv
Looking for ‘ABC’
……………………………...
Search Results
Static XMLwith Ref #
Ref #
XML
data.csv
……………………………...
+ : Size of the file parsed can decrease by a factor 5 to 10. Can be correlated with temporal split.- : Edit efficiency decreases.
Reducing Response Times: Reference Key & Static Display
XSL
Volume of Data Parsed Significantly Decreases
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
ref;user_first_name;user_last_name;request_date;from;to;date;preferred_airline;preferred_time;airline;time;………..543;John;Doe;3/5/2000;London;Paris;Air France;8:00 PM;Air France;8:20 PM;……….542;Sam;Tondac;3/6/2000;Los Angeles;Paris;KLM;6:00 PM;Delta Airlines;5:54 PM;……….541;Jennifer;Alister;3/6/2000;Houston;Chicago;American Airlines;5:30 PM;American Airlines;3:33 PM;………….….….
Travel.csv
<travel_request><user><user_first_name>John <user_first_name><user_first_name>Doe <user_first_name></user> <requested_date>3/5/2000<flight><from>London</from><to>Paris</to><preferred_airline>Air France</preferred_airline><preferred_time>8:00 PM</preferred_time>….….
Travel_543.xml
ref;user_first_name;user_last_name; from;to;airline543;John;Doe;London;Paris;Air France542;Sam;Tondac;Los Angeles;Paris;Delta Airlines541;Jennifer;Alister;Houston;Chicago;American Airlines….….
Travel_key.csv
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
INSERT UPDATE
• Lot of Data• High volume of searches• Few Updates• > 100 inserts/day
• Small Volume of Data• Lot of Updates• < 40 inserts/day
File Split
Static Display
Reference Keys & Static Display
Reference Keys & Static Display + File Split
Static Display+ File Split
Search Efficiency
Ex: Bug Tracking System
Ex: Travel Request
Ex: Phone Directory
Simplifying Company Processes with Perl
Stephane Barde Arthys / Ezperl
Perl makes possible the restructuration of the Information Channels with a minimal investment.
For more information:[email protected]