Leader in Data Quality and Data Integration

www.dataflux.com 877–846–FLUX

International +44 (0) 1753 272 020

A DataFlux White PaperPrepared by: David Loshin

Data Best Practices for Spend Analysis

1

Corporate sourcing and procurement organizations should always look for opportunities

to introduce efficiencies, reduce costs, as well as negotiate desirable terms with vendors

and suppliers. These opportunities are revealed in a number of different ways, such as

demand aggregation, improved supplier performance assessment, assurance of

regulatory compliance, determination of rebates and refunds, and identification of non-

compliant spend. All of these business benefits can accrue as a result of a process for

reviewing and analyzing spend data.

However, few companies have the ability to gain a comprehensive perspective of the

products and services purchased and their associated providers. This confounds the

ability to identify opportunities for improvement, and wasteful and duplicate spending

can continue unabated. The difficulty in gaining this enterprisewide perspective is

complicated by a number of factors, such as:

There are often multiple systems used during the procurement process.

With data spread across different data silos, it is difficult to consolidate

spend data to provide summarizations across providers, products or

commodity types.

Different vendors and providers use variant product and service identifiers

and descriptions. The inconsistent naming and identification introduces

challenges when analyzing purchases by product or product category.

Often, transactional data associated with purchasing is missing important

characteristics that are used to influence and inform both the strategic and

the operational decision-making processes for procurement.

Even when improvements such as negotiated product prices have been identified,

ensuring that the negotiated savings can be achieved in practice require additional

visibility into purchasing, supplier fulfillment and delivery data. Yet, according to an

Aberdeen Group study, the top challenges for spend analysis include “poor data

quality,” “too many data sources,” and “lack of standardized processes.”1 In essence,

the biggest challenges to procurement improvements have to do with information, and

the benefits of spend analysis can only be achieved when the spend analysis tools have

access to the right data.

1 “Spend Analysis: Working Too Hard for the Money,” August 2007, Aberdeen Group.

2

In this paper, we look at the business drivers and organizational objectives of a spend

analysis program, and then consider establishing performance indicators and associated

metrics for managing the efficiencies and realizing cost savings. The paper then reviews

spend analysis techniques along with the data management procedures necessary to

enable the process. Last, we consider some of the most important challenges and

associated techniques for driving a successful spend analysis program.

Business Drivers and Organizational Objectives

Strategic sourcing incorporates best practices for evaluating the purchasing patterns

and activities within the organization with the intent of identifying opportunities for

improving the procurement process. These practices focus on examining the products

and services that are bought, how those products are classified, who the suppliers are,

how much is being spent on different types of items, and the terms under which these

items are priced, purchased, and delivered.

This process addresses specific business drivers associated with managing the way the

organization spends money and seeking ways to reduce costs, improve operational

efficiency, and better engage with the providers. Some techniques used for providing

value using spend analysis include2:

Demand aggregation – This is a process of consolidating purchasing

requirements from across the organization into groups of similar items,

thereby opening the possibility for volume discounts, reduced delivery

costs, better purchasing terms and more control over specifications.

Demand aggregation is also good for the supplier, who is able to capture a

greater amount of organizational spend, reduce the cost of doing business,

and streamline manufacturing and delivery efficiency.

Supplier assessment – This process evaluates supplier performance in

terms of objective criteria such as credit scores and market value as well

as scoring suppliers in terms of responsiveness, observance of terms,

delivery times, pricing, product defects and warranties.

Regulatory Compliance – This supports compliance monitoring, especially

with respect to controlled trade, OFAC compliance, doing business with

suppliers from embargoed political regions, and tariffs.

Identify non-compliant (or “maverick”) spend – This process seeks to

identify where individuals in the organization are purchasing products or

services outside of the official procedures. Non-compliant purchases, such

as picking up ink cartridges at the local office supply store instead of

2 For a comprehensive discussion, see the Spend Analysis and Opportunity Assessment wiki,

http://www.esourcingwiki.com/index.php/Spend_Analysis_and_Opportunity_Assessment

3

through the proper organizational channels, can result in paying a

(potentially significant) premium for the purchased item.

Fraud detection – This is intended to identify patterns of fraudulent spend,

such as multiple invoices or payments to ghost companies.

Contracting strategies – These tactics help evaluate potential vendors, the

types of items being purchased and their classifications. They also help in

determining an appropriate procurement approach as well as positioning

the purchaser for negotiation of favorable pricing and service.

Commodity analysis – This process looks at aggregated spending decisions

to determine if the organization is buying commodity products in large

quantities without even realizing it.

Defining Metrics

As with any program intended to improve performance, it is necessary to examine the

impacted processes from a business-driven perspective. Yet, according to a research

report, a significant amount of negotiated savings can still remain unrealized.3 For

example, even when the procurement organization has employed spend analysis

techniques to negotiate better rates with suppliers, there is still a need to communicate

those rates with everyone across the company. This helps make all individuals aware

that their purchases should conform to a well-defined process in order to realize those

negotiated savings.

Therefore, it is critical to define the measures that will be used to indicate when the

expected benefits are achieved. Let’s consider some key business drivers and map them

to defined objectives as a way to understand the key characteristics that are indicative

of better spend decisions, and examine metrics that will be used for those decision-

making processes. For example, if the driver is cost reduction or cost savings, an

objective metric might state “reduce materials cost by 10% within the next 18 months.”

Other performance indicators that reflect improvement may include examples such as:

Negotiated price reductions – As a result of demand aggregation, selected

vendors can be approached to provide lowered prices in return for

recognition as “best supplier” with longer-term contracts. This can be

measured directly as the difference in negotiated price per product.

3 “Spend Compliance Management: Implementing and Sustaining Supply Savings,” December 2004,

Aberdeen Group

4

Communication of negotiated terms – To make all staff members aware of

negotiated rates as well as defined procurement process, there must be a

communication infrastructure in place for corporate awareness. Therefore,

staff awareness goals can be set and measured.

Cost reduction – If the costs associated with procurement should be

reduced through the identification of potential operational efficiencies,

then specific goals for cost reduction can be set as program objectives. One

example may measure time reduction for procurement of unique products.

Internal spend compliance – Once the negotiated rates and terms are

communicated internally, it is important to make sure that staff members

make their purchases through the approved processes and that they are

purchasing the appropriate products through the selected vendors at the

negotiated rates.

Supplier compliance – Once approved rates are negotiated, supplier

performance can be monitored to ensure compliance with targeted pricing

and delivery goals.

Demand reduction – Through improving product reuse and quality

improvements applied to the purchased products, product lifetimes can be

extended, thereby reducing the overall demand, and this can be measured

as the reduction in requests for purchasing selected products.

What Does Spend Analysis Entail?

Spend analysis encompasses the process of aggregating spend data together into a

single framework in order to understand who in the organization is buying, what they are

buying, from which suppliers, where the purchases are performed, and the different

characteristics regarding the terms of the purchases (price, delivery, payment options,

etc.). Even before you can analyze the many transactions, you face a more insidious set

of challenges:

Identifying the systems containing spend data

Collecting the spend data from the identified sources

Transforming that data into a usable format

Consolidating the data into a single repository

A first step is to ask a more basic question: Where does the data come from – and how

can we make it more useful? There are many systems and subsystems, such as accounts

payable, invoicing systems, purchase order management, supplier master data, expense

report data, agreements and contracts, policies, purchasing card data, and others. There

are many different underlying data sources, representations, and definitions, all of which

must be identified, extracted, normalized, cleansed, transformed, classified and

consolidated into a common data store. And since the most significant challenges

5

involve the collection, organization, and presentation of data for analysis, one might say

that spend analysis is ultimately a data-driven process.

The Data-Driven Spend Analysis Process

The spend analysis process is activated by the identification, normalization and

consolidation of data, and consists of a number of distinct phases:

1. Data collection – The first step is to identify the sources of data containing

spend information. This may be no easy task. An organization may have

multiple accounting systems, and there may be multiple processes and

financial databases containing data associated with purchasing. For

example, there may be documentation of product purchases through a

purchase requisition and purchase order application, while vendor data is

managed in a different set of systems. A survey of applications determines

which underlying data sets contain data relating to procurement and spend.

Once the data sources are identified, their structures and contents are

evaluated, and the relevant records and attributes are extracted.

2. Data normalization – Once the data sets have been extracted, data profiling

can identify the similarities and differences to determine data mappings as

well as gaps in completeness. A common model for representing spend data

follows, and the appropriate transformations and normalizations are

applied to convert the many data sets into a standard format.

3. Data cleansing and enhancement – There is bound to be variance between

the contents of the different source data sets – multiple versions of the

same names for vendors, products and additional codes. Not only that,

mergers and acquisitions may have modified corporate structures,

introducing new organizational dependencies in the master vendor

database. At this stage, data cleansing techniques are applied to parse and

standardize vendor and product names and descriptions. Similarly,

duplicate data can be identified and, in certain cases, eliminated. In

addition, third-party data sources can be used for establishing the correct

vendor and supplier corporate hierarchies. Other attributes associated with

each vendor, such as “small business,” “minority-owned,” or “women-

owned,” can be added also, which is important for analyzing certain aspects

of regulatory compliance.

4. Commodity mapping – Product data is particularly challenging in terms of

cleansing and consolidation. Product data has wide variability and

unpredictability when viewed across different business contexts, and is not

necessarily suited to formatted pattern matching. There are differing

standards for classification, presentation, and description. Product

descriptions often contain various abbreviations and shorthand, yet these

fields carry descriptive attribution embedded within free-formed text. Once

6

product data has been normalized, additional enhancements are applied to

assign commodity codes to each product. Again, these enhancements can

be facilitated using third-party data, such as the United Nations Standard

Products and Services Code (UNSPSC).

5. Classification and categorization – The actual analysis looks at transactions

and spending patterns along a number of dimensions, either by corporate

group, geographic region, supplier, and product type, to name a few. This

analysis depends on the proper organization of the data, and at this stage,

spend transactions that documented in the different data sources and have

been consolidated are classified and organized along different dimensions

of categorization. For example, “small paper clips” and “large paper clips”

are both classified as “paper clips,” which is then categorized within the

“paper fastener” category, and so on.

Figure 1: Activating spend analysis via data management techniques. 

At this point the data is ready to be presented for the analysis and decision-making

process. The different purchases can be aggregated to identify the most frequently used

suppliers, which ones provide the best pricing, and how many accounts are active within

the organization. Variance in commodity product pricing can be identified, while

different purchasing characteristics can help in suggesting creative approaches for

sourcing, ranging from reverse auctions for commodity pricing to the larger effort

associated of a request for proposal (RFP) for more complex acquisitions. But long

before any of this can take place, the data management challenges need to be

addressed.

7

Meeting the Technical Challenges

If it is clear that spend analysis will lead to improved operational processes, then, as

with any analytical application, there is a dependence on well-defined data management

practices. And if the top challenges involve the proliferation of data sources and the

quality of the data, then incorporating these types of technologies with data

management best practices will better enable a successful spend analysis program:

Data integration/ETL – It would be reasonable to assume that spend data

would be collected from the numerous data sources and centralized in a

common data warehouse. Therefore, the spend analysis solution should

include data integration tools and utilities to simplify the extraction of

source data, transformation into a common format for analysis, and loading

into the data warehouse.

Data quality – Spend analysis hinges on standardized and cleansed vendor

and product data; traditional data quality management tools can help with

parsing, standardization, and normalization of data, as well as exact and

approximate matching algorithms that help in duplicate identification and

elimination. In addition, inspection, monitoring, and reporting of compliance

with data quality rules will better ensure high quality spend data in

preparation for analysis.

Master data management (MDM) – Any consideration of unique

identification of either products or suppliers suggests managing a master

directory or repository of this data to support the analysis process. Tools

supporting analytical MDM may already combine data integration and data

cleansing with defined data models representing products and parties.

Data enhancement services – Third-party data vendors provide value-added

enhancement services, either by supplying data or providing services for

appending additional data attributes and characteristics.

Process standardization – Of course, the absence of well-defined processes

for managing the data management practices diminishes the value of any

acquired technology.

No matter how advanced the technology is for analyzing spend data, the identified

opportunities for improvement will only be accomplished when the management

supports the organizational change management. Defined performance objectives,

specific criteria for success, accompanied by metrics that monitor how well the

organization adapts to those improvements will determine whether the expected

savings can actually be realized.

8

Conclusion

Corporate sourcing and procurement organizations seeking process improvement can

implement a spend analysis program to identify opportunities to reduce costs while

improving counterparty relationships and establish desirable prices and business terms

with suppliers. Spend analysis not only is used to find these opportunities, but the

additional visibility into purchasing, supplier fulfillment, and delivery data also helps

ensure that the negotiated savings can be achieved in practice. Many opportunities lurk

hidden in spend data: the ability to aggregate purchases, identify preferred suppliers,

negotiate better rates and prices, improve terms of engagement, support regulatory

compliance, identify fraud, and even drive different procurement strategies.

But spend analysis is particularly dependent on high-quality data collected from across

the organization. There are challenges to implementing an effective spend analysis

program, such as the proliferation of systems containing spend and procurement data,

the variance in product names, product descriptions, and vendor names, and the need to

enhance the data with additional characteristics often missing in the source.

Spend analysis is essentially a data-driven process, from the identification of the

appropriate data sources, data extraction, transformation, cleansing, and normalization,

along with the potential need for master product and master vendor directories. At the

same time, implementing best practices in data management and in performance

management will most effectively support the analysis and improvement cycle. To

benefit from spend analysis, a good approach is standardizing and managing the data

management and analysis processes, so that individuals can exploit actionable

knowledge while continuously measuring the success of the program.