master!in!european!studies!2013/2014! thesis!advisor ... ·...
TRANSCRIPT
Impacts and Implications: The Future of Sustainability Reporting and Assessment
Master in European Studies 2013/2014
Thesis Advisor: Professor Andrea Renda
Krystal Crumpler
November 15, 2014
2
TABLE OF CONTENTS INTRODUCTION 4 CHAPTER 1: THE EMERGENCE OF TRANSANTIONAL REGULATORY SCHEMES 8
GAPS IN THE REGULATORY PROCESS 10
A CROSS-‐POLINATION OF AGENDAS 12
TYPOLOGIES AND PROLIFERATION OF TPR 13
PLAYERS AND THE PLURALITY OF INTERESTS 14
THE SUBJECT OF REGULATION AND QUESTION OF LEGITIMACY 16
HOW PRIVATE REGULATION OPERATES 17
THE USE AND ABUSE OF THE SUPPLY CHAIN 18
TPR IN SEARCH OF A DEFINITION 20
HOW GOVERNANCE CAN EFFECT EFFECTIVENSS? 22
WHAT TO MAKE OF EFFECTIVNESS 25
THE (MIS)USE OF INDICATORS 27
RE-‐PIECING THE PUZZLE: COMPLIANCE, EFFECTIVNESS, AND THE LEARNING CURVE
28
THE FIVE Ws OF TPR 31
HOLES IN THE SYSTEM 33
CHAPTER 2: SUSTAINABILITY IN SEARCH OF A DEFINITION 35
RECONSIDERING SUSTAINABILITY VIS-‐À-‐VIS SAFA 36
SUSTAINABLE DEVELOPMENT: WHAT LIES AT STAKE? 37
WHO USES SAFA? 39
DEFINING BOUNDARIES: THE SCOPE OF SAFA 40
SAFA STEP-‐BY-‐STEP 42
STEP #1: MAPPING MATERIAL MATTERS 43
PROCEDURAL ISSUE: HOW TO MAP SMALL-‐SCALE PRODUCERS? 44
STEP #2: CONTEXTUALIZING DATA AND OUR READING OF IT 46
STEP #3: SELECTING APPROPRIATE TOOLS AND INDICATORS 47
3
SUSTAINABILITY PERFORMANCE 49
STEP #4: REPORTING PERFORMANCE IN A “SNAPSHOT” 52
CHAPTER #3: TESTING SAFA VIS-‐À-‐VIS THE PILOT PHASE 54
SAFA “LIVE” FROM THEORY TO PRACTICE 55
SAFA “HOT-‐SPOT” #1: INEFFECTIVE COMPLIANCE CHECKS 56
SAFA “HOT-‐SPOT” #2: INDICATOR CHALLENGES 59
SETTING BOUNDARIES: SPATIAL AND TEMPORAL SPHERES OF INFLUENCE 59
BETWEEN FLEXIBILITY AND RIGIDITY: INDICATORS IN SEARCH OF A THRESHOLD
62
BLURRED THRESHOLDS: WHAT TO DO WITH THE SPACES-‐IN-‐BETWEEN 63
INDICATOR TYPES 65
SAFA “HOT-‐SPOT” #3: (IN)APPROPERIATE METRICS 66
SAFA “HOT-‐SPOT” #4: DATA COLLECTION 69
SAFA “HOT-‐SPOT” #5: IMPLICATIONS FOR SMALL-‐HOLDERS 72
SAFA “HOT-‐SPOT” #6: FUTURE UTILITY? 74
CONCLUSION 77
BIBLIOGRAPHY 79
4
INTRODUCTION In a world of transgressed borders and blurred boundaries, the notion of
globalization evidences the permeation between spheres of the public and the
private and the divide between the national and the international, leading to a sense
of transnationalism—a space and place of transgression(s) between self and other;
the individual and the collective; the national and the international. In a globalized
world comprising cross-‐border flows of products, services, people, and ideas, the
space of the global market becomes an ever-‐more complicated one insofar as its
boundaries are perpetually expanding and its regulation exponentially complicated.
In such a way, governance transcends domestic borders so-‐to-‐speak, escaping
national jurisdiction and calling upon transnational regulation as a means of “making
sense” of these transgressions. In response to the functional and organizational
failures of conventional regulatory bodies (i.e. the weakness of the state, the
fragmentation of international law, or the lack of enforcement mechanisms), the
emergence of transnational private regulation (TPR) represents a new body of rules,
practices, and processes, agreed upon by private actors, firms, NGOs, and experts,
which exercise regulatory power autonomously or by virtue of delegation from
international law or national legislation.
In Chapter One, we shall explore both the emergence and evolution of transnational
private regulation, as an intersection between the public/private and
national/international divide, presenting an opportunity for cooperation among
actors rather than competition between them in the context of transnational
regulatory regimes. In such a way, the nature of global value chains – and the use of
supply chains as instruments for trans-‐national cooperation – represents a radical
new space for transnational regimes to revise the regulatory process. As we shall
discuss throughout the paper, the use of the supply chain as both the site of and
instrument for transnational regulation represents an innovative approach in
effecting effective regulation across multiple forms of governance and among
regulatory relationships. Thus, the use of the supply chain marks a comprehensive
5
approach of transnational private regulation, speaking to its regulatory potential in
the realms of food security, environmental safety, and human rights protection, to
name a few.
Consequently, common frameworks have been established to promote cooperation
among global actors, such as the GlobalGap for food certification schemes, the
Global Reporting Initiative (GRI) for non-‐financial corporate social responsibility
reporting, or the ISEAL Impacts Code for good practices in setting social and
environmental sustainability standards. However, in consideration of the lack of a
hegemonic, international regulator, one may question the legitimacy and
effectiveness of TPR in such absence; that is, the absence of a transnational
arbitrator enforcing and monitoring the implementation and compliance of given
regulatory schemes. In the analysis, then, of the interdependent links between
regulatory relationships and forms of governance, the argument presented herein
calls upon the need for objective evaluation mechanisms, sound ex ante standard-‐
setting and ex post indicator-‐selection, and comprehensive impact assessments in
order to evaluate the effectiveness of TPR. That is, we shall inquire upon the effects
of governance and, successively, how governance can effect effectiveness.
As the proliferation of transnational regulatory schemes continually expands, the
number of environmental, social, and labor codes continually multiply; Koritza
reported in 2013 on the emergence of “several million certified farm and tens of
billions of annual sales of coffee, tea, cocoa, sugar, fruit and other commodities that
meet Fairtrade, Rainforest Alliance or UTZ Certified standards.” 1 Over 20,000
different products bear the blue Marine Stewardship Council (MSC) seal, and 180
million hectares of forest meet the Forest Stewardship Council (FSC) seal. As 106
countries have established national sustainable development strategies, 120
voluntary sustainability standards, eco-‐labels, codes of conduct, and auditor
protocols have emerged, along with 300 different industry codes for social and
1 Koritza, L. 2013, Certification showing it’s worth the investment, Sustainable Brands. (available at http://www.sustainablebrands.com).
6
environmental practices have been developed.2 Despite, however, the increased
number of standards and certification schemes of sustainable performance, the
notion of “sustainability,” in and of itself, becomes ever more evanescent, slipping
between multifaceted forms of its constitution. Hence, the present paper calls upon
the need for an integrated approach in the assessment of regulatory effectiveness.
Considering the intricacy (or, perhaps, impossibility) of compounding multiple
frameworks into a single, common language, lens, and set of rules, we shall analyze
the efforts set for the by the FAO, which seeks to simultaneously simplify
sustainability assessment while defining a new, encompassing framework for it,
establishing a comprehensive, step-‐by-‐step tool for assessing 4 crucial dimensions of
sustainability – good governance, environmental integrity, economic resilience, and
social well-‐being – along food and agricultural value chains.3 In Chapter 2, we shall
present an argument for the merit of the SAFA tool in the field of sustainability
assessment and reporting, as well as its implications for the future. With that in
mind, we shall stress how one must continually call into question what lies at stake
in measuring that which we cannot define; as the definition of “sustainability”
forever escapes the definer, a similar dynamic consequently takes place in the
measurement of it. That is, despite the number of mechanisms, tools, and metrics
for sustainable development perpetually have increased over time, sustainability per
se continuously diminishes.
Finally, in Chapter 3, we shall analyze the SAFA pilot studies performed in 2012 prior
to the finalization of the current SAFA Guidelines of 2013 by 19 small, medium, and
large enterprises in 23 countries and across a variety of agricultural and food sectors,
ranging from tobacco production in Tanzania to organic fruit farmers in Peru. In the
analysis of the SAFA pilots, the paper attempts to critically assess the proposed
2 Cafaggi, F. and A. Renda. 2014. Measuring the effectiveness of private regulatory organizations, forthcoming, p. 8. 3 FAO. 2013. Sustainability Assessment Food and Agricultural Systems Guidelines. Food and Agriculture Organization of the United Nations, Rome (available at http://www.fao.org/fileadmin/templates/nr/sustainability_pathways/docs/SAFA_Guidelines_Final_122013.pdf
7
themes, sub-‐themes, and indicators for sustainability, evaluate the entities’
compliance with the guidelines for self-‐assessment, identify the challenges
encountered within the pilot phase and the major shortcomings of the tool, and
analyze how (and to what degree) the organization incorporated the suggested
measures of improvement within the current version of SAFA available today.
Considering, then, both the merits and limitations of the tool – a tool designed for
effective sustainability assessment – the following discussion seeks to assess the
effectiveness of the tool itself, presenting a qualitative analysis of the “gaps” found
within its regulatory scheme and questioning if and how the new version/vision of
the SAFA framework resolves such shortcomings, serving as a case study by which
the context of transnationalism enables us to interrogate the future of sustainability
assessment—and its particular stake in social progress.
8
CHAPTER 1: THE EMERGENCE OF TRANSANTIONAL REGULATORY SCHEMES
As our global economy becomes ever so more complex, the diversity of our
regulatory schemes, too, must adopt to the conflict, confluence, and
interdependence presented by the dynamic between multiple regulatory processes
and forms of governance. The consequential shift of the “regulatory space” from
public to private spheres and its transition in scope from a national to transnational
one—as well as the interplay between the two extremes—marks a radical revision of
the conventional regulatory space. Instead, this very interplay between the
public/private and domestic/international divide constitutes the emergence of a
new form of regulation, which aims to address the possibilities-‐in-‐between such
regulatory space; that is, transnational private regulation (TPR). In response to
functional and organizational failures of conventional regulatory bodies (i.e. the
weakness of the state, the fragmentation of international law, or the lack of
enforcement mechanisms), transnational private regulation represents a new body
of rules, practices, and processes, agreed upon by private actors, firms, NGOs, and
experts, which exercise regulatory power autonomously or by virtue of delegation
from international law or national legislation.4 In such a way, TPR redistributes
regulatory power across the domestic and global spheres, as well as between public
and private regulators, revealing the role of “actors” and their impacts in the
transnational arena.
Considering the exponentially expanding combination of variables that contribute to
the emergence of private regulation, as Cafaggi and Renda argue, “the regulatory
schemes are not created equal,” in which differentiating factors for any given set of
schemes include, “their origin, the intended beneficiaries, the extent to which these
schemes compete with other similar schemes, and the regulatory functions they
perform” (8). That said, this report aims to underline the importance, thus, of what
lies at stake in such variability, calling upon the need for objective evaluation
mechanisms, sound ex ante standard-‐setting and ex post indicator-‐selection, and
comprehensive impact assessment in order to evaluate effectiveness. Although such
4 Cafaggi, F. and A. Renda, p. 7.
9
metrics could be applied to any form of management, we shall analyze the
interdependent links between governance and regulatory performance by virtue of
the 5 characteristics set forth by Fabrizio Cafaggi and Andrea Renda’s joint report,
Measuring the effectiveness of private regulatory organizations, stipulating the
necessity of (i) a “wider and more sophisticated evaluation” method, (ii) the analysis
of the regulatory process in its entirety, (iii) the distinction between single and
multiple regulatory schemes, (iv) the determination of the governance, standard-‐
setting, monitoring, and enforcement indicators for each regulatory scheme, and (v)
the analysis of effectiveness regarding the cooperation between and among private
and public regulators.5
The very trans-‐national nature of TPR continuously undermines scholars’ ability to
identify and, thus, define it, as private regulation perpetually slips into new
regulatory space(s). Unlike most regulatory systems, transnational private regulation
does not adhere to any common set of principles, as—despite its universal scope—it
lacks any universally agreed-‐upon definition in jurisdictional and academic literature.
Insofar as not a single type of private regulatory scheme exists, the term is rendered
negotiable (e.g. private governance, self-‐regulation, co-‐regulation). However, in
order to contextualize the term within the scope of this paper, Buthe’s broad
definition echoes the encompassing approach found within the present paper, in
which private regulation “entails private actors playing a major role – at one or more
stages beyond implementation or compliance – in what might be called the
‘regulatory process’.”6 In such a way, assessing the effectiveness of transnational
private regulation requires the analysis of the entire regulatory process—and the
sustainability of that process.
5 idem, p.10. 6 idem, p. 17.
10
GAPS IN THE REGULATORY PROCESS
The evolution of TPR can primarily be linked to the weakness of the state as a global
regulator.7 Such failures sparked the emergence of international institutions in the
first part of the 20th century followed by the emergence of transnational private
regulators in the latter. The limitations of the state can be located in the lack of
cross-‐border effectiveness and the inability to both ensure compliance and enact
enforcement. In order to achieve an effective cross-‐border implementation
apparatus, cooperation among states is necessary, resulting however in high
transaction costs. As a result, diverging implementation coverage and capacity
decentralizes the regulatory process, fragmenting its effectiveness. Such
ineffectiveness – and the resulting gaps in the regulatory process – elicits the
creation of private schemes. As Cafaggi argues, these transformations brought about
by the new private regulatory regimes have also modified the unit of analysis,
shifting from a regulatory state to “regulatory capitalism.”8
The issue of globalization underscores the various factors contributing to the shift
towards private regulatory schemes, such as the emergence of global markets and
trade insofar as goods and services cross state boundaries and, hence, cannot be
monitored by single national legislation. The resulting “externalities” of global trade
and governance transcend domestic borders so-‐to-‐speak, escaping national
jurisdiction and calling upon transnational regulation. As Cafaggi and Renda point
out, international regulatory co-‐operation becomes considerably important in
regards to “global public goods” (e.g. deforestation, emission reduction, food safety,
protection of biodiversity) in order to avoid the tendency of “race to the bottom”
market trends, which implies low market prices due to the absence of quality
control, ultimately undermining the aim of regulation itself. Hence, common
frameworks have been established to promote cooperation among global actors
7 Cafaggi, F. 2011a. New Foundations of Transnational Private Regulation,” Vol. 38, “Journal of Law and Society,” EUI WP, p. 3. 8 idem.
11
(e.g. UN Global Compact, FAO SAFA Guidelines, and the ISEAL Alliance) in the form of
transnational private regulation or “meta-‐regulation.”9
Lastly, the surge of informational asymmetries as new technologies require expert
regulatory bodies, the velocity of market transformations in knowledge-‐intensive
and high-‐tech fields calls upon private parties for fast-‐paced and able-‐bodied
regulation, and the role of global value chains all transcend conventional regulatory
spaces, calling upon TPR to regulate in the space beyond the nation. The nature of
global value chains – and the use of supply chains as instruments for trans-‐national
cooperation – represents a radical new space for transnational regimes to revise the
regulatory process. As we shall discuss throughout the paper, the use of the supply
chain as both the site of and instrument for transnational regulation represents an
innovate approach to effecting effective regulation across multiple forms of
governance and among regulatory relationships. Thus, the use of the supply chain
marks a comprehensive approach to transnational private regulation, speaking to its
regulatory potential in areas of food security, environmental safety, and human
rights protection. In such a way, the increasing role of private actors in both
domestic and transnational levels asks us to “reconsider”10 the functions and
boundaries between private and public spheres, subverting the notion of the
private/public divide as mere alternatives and, instead, suggesting their mutual
inter-‐dependence. Such factors indicate that transnational private regimes house the
resources and competences, which can be aligned with the traditional forms of
international organizations and innovative forms of transnational grids in order to
facilitate cooperation among them rather than competition between them. As
Cafaggi and Renda argue, “trans-‐national private regulation is not an alternative but
rather a complement to international public regulation” (14).
Consolidating issues of fragmentation due to the multiplicity of private regimes or
conflicting domestic public legislation, TPR attempts to harmonize these differences.
Although transnational regimes seek to overcome barriers presented by cross-‐
9 Cafaggi, F. and A. Renda, p. 12. 10 Cafaggi, p. 3.
12
border elements and the public/private divide (e.g. food safety and environmental
protection), Cafaggi notes the recent proliferation of TPR at a global level, suggesting
how the multiplication of private regimes (and the competition between them) may
result in even further fragmentation—global fragmentation.11 Hence, such
proliferation further complicates the harmonization efforts of transnational regimes
towards common objectives, ultimately questioning the “effectiveness” of, for
example, standardization in the regulation of public goods and, thus, invoking us to
call into question the implication(s) of fragmentation.
A CROSS-‐POLINATION OF AGENDAS
As private actors often express diverse views and interests, such results in the
emergence of different organizational typologies of regulatory schemes in function
of the relationship between the regulated and the beneficiaries of a given scheme.
The creation of either single stakeholder organizations (e.g. NGOs, industries) or
multi-‐stakeholder organizations (e.g. multiple actors or members) depends upon the
aims of the regulatory body and the concentration of its power.12 Consequently, as
Cafaggi and Renda point out, a regulated firm has a strong interest/involvement in
the regulatory framework as a means of locating its “voice” within the regulatory
structure (such as industry-‐driven interest groups). Similarly, NGOs attempt to
achieve their objectives (such as the protection from gender inequality) by locating
itself within the regulatory process as well. Situating themselves either “within” or
“outside,” actors seek to actualize their goals by virtue of legal or non-‐legal
instruments.13 This, in turns, impacts effectiveness, marking actor involvement as a
key factor of analysis in the evaluation of regime effectiveness.
However, the conflicting interests and differing policies between industries and
NGOs becomes the very point of their conjunction, in which the two conventionally
contrary bodies tend to reconcile their agendas – by virtue of the underlying TPR
conditions – in forms of cooperation rather than competition. This enacts a sort of
11 Cafaggi, p. 4. 12 Cafaggi, F. and A. Renda, p. 15. 13 idem.
13
“cross-‐over effect” between typically rivaling bodies (industry vs. public interest),
which allows for firms to become more involved (and, therefore, accountable) in
private environmental and social regulation while NGOs can cooperate with
industries in order to leverage regulatory impact in areas generally outside of their
reach. By subverting points of conflict into levers of coordination, transnational
private regulation functions as a “cross-‐pollinating” mechanism between divergent
actor agendas and across different forms of governance.
TYPOLOGIES AND PROLIFERATION OF TPR
The agenda of a private regulatory scheme is directly linked to the nature of its
creation; that is, the factors that led to its creation. Generally, three different types
of TPR can be observed. Firstly, the so-‐called “spontaneous private regulatory
schemes”14 may result in the absence of public or private regulatory enforcement,
the fragmentation of public regulation, or in response to other forms of private
regulation. Insofar as multiple private schemes can co-‐exist, the “spontaneous”
nature of one may act in competition or cooperation with another, depending on its
agenda. In such case, the private scheme is a self-‐regulatory one—directly
unaccountable to public regulators and bound only to national or international legal
rules. Secondly, a “co-‐regulatory scheme”15 constitutes the informal engagement
among public regulators in partnership with private actors in order to regulate
specific functions (i.e. rule-‐setting, implementation, monitoring, enforcement, or all
processes). Lastly, a formalized mandate of governance and regulatory requirements
represents that of “formal delegation,” commonly found in the field of technical
standardization. In such a way, the typology of a given private scheme reflects the
issue at hand (e.g. the failure of public regulation in climate change, the fragmented
regulatory schemes for food safety in the private sphere, and high transaction costs
leading to market failure).16
14 idem, p. 47. 15 idem. 16 idem, p. 49.
14
PLAYERS AND THE PLURALITY OF INTERESTS
The variety of organizational frameworks among private schemes is further
complicated by the relationship among actors within those schemes. The
relationship, then, among the regulators, regulated, and beneficiaries determines
the resulting form of governance. As Cafaggi and Renda describe, a variety of
combinations can arise, pointing out three hypotheses. Firstly, the “regulated” can
be located inside the scheme whereas the “beneficiaries” are located outside. As we
shall discuss later, such arrangement of actors mirrors the framework of corporate
social responsibility schemes, allowing for commitment by members of the scheme
and, therefore, benefiting society through strengthened sustainability.17 Reflecting
this scheme, certification standards for food safety and public good regulation in the
protection of human rights both exemplify the same actor relationship. Secondly,
the “regulated” may be outside the scheme while the “beneficiaries” can be found
inside, such as an NGO setting environmental certification schemes (e.g. ISEAL
Alliance). Thirdly, both the “regulated” and “beneficiaries” can act as participants
within the scheme, such as those multi-‐stake holders whom include both suppliers
and retailers within the regulatory scheme (e.g. Global GAP).18
Cafaggi further addresses in his working paper, New Foundations of Transnational
Private Regulation, the complexity and heterogeneity of the private sphere:
Some are mainly driven by industries; some are promoted by NGOs; others by joint endeavor of industry agreements. While at first sight they are all governed by private actors, they pursue different objectives and incorporate multiple degrees of public interest, depending on the composition of their respective governance bodies and effects they have on the general public. (8)
Consequently, the varying (and conflicting) interests underlying TPR schemes speaks
to the notion of a “plurality of interests,” which leads to diverging regulatory
strategies vis-‐à-‐vis the regulatory relationships and choice of governance models.
Thus, Cafaggi calls for a “disentangling” of such plurality as a means of identifying
the underlying dynamic among actors, governance models, and enforcement 17 idem, p. 51. 18 idem, p. 52.
15
mechanisms in order to question accountability—and, consequently, sustainability.
In such a way, the very composition of the regulatory relationship plays a role in the
effectiveness of the private scheme. Cafaggi argues that the role of the
“beneficiaries” within the regulatory process should be examined in terms of
compliance and enforcement as an index of effectiveness. That is, a beneficiary-‐
inclusive arrangement “redefines the nature of responsiveness and the means
through which effectiveness of the regulation should be measured” (9). Specifically,
effectiveness serves not only to measure the compliance of the regulator, but more
importantly, to assess the final impact of the regulatory process on the
beneficiaries.19
In such a way, the multi-‐stakeholder model exemplifies the interplay between
contrary interests and how they are represented in the transnational private
regulatory framework, particularly in regards to the role of the beneficiary, allowing
for a cross-‐sectoral analysis. The “contractual” nature of the multi-‐stakeholder
schemes functions through a system of regulatory contracts (multilateral, network,
or master agreements) placed along the supply chain as a means of regulating the
entire regulatory relationship and process (i.e. from the retailer to the supplier).20
The network created among retailers, suppliers, beneficiaries, and regulators
represents an inter-‐connected formulation of regulatory management, in which all
players contribute – and are therefore held accountable – to the legitimacy of the
scheme. The importance, then, of analyzing who is the beneficiary and the where the
beneficiary is located vis-‐à-‐vis the regulatory framework (whether “inside” or
“outside” the regulatory space) points to the complexity of private regulation,
ultimately calling into question notions of legitimacy, as interests can be (mis)aligned
along the supply chain. Such questions of legitimacy within contractual multi-‐stake
holders are exemplified in both the fields of corporate social responsibility and food
safety—the (il)legitimacy along the “supply chain” can be questioned by means of
dissecting the “chain” into its regulatory “parts.”
19 Cafagggi, p. 9. 20 idem, p. 13.
16
THE SUBJECT OF REGULATION AND QUESTION OF LEGITIMACY
As the combination of actors varies in private regulation, so does that of the subject
of regulation in consideration of its sector-‐specificity. Variables of composition,
scope, geographical influence, and structures of governance condition the specific
regulatory function and objectives of a TPR scheme, ranging from financial markets,
data protection, e-‐commerce, food safety, environmental protection, and technical
standards.21 Although the scope of regulatory regimes is defined by difference,
Cafaggi and Renda argue, “there are paths of policy integration related to regimes
that used to be separate and pursue conflicting objectives” (52). That is, the
permeation between the public/private and domestic/international divide by trans-‐
national private regulatory regimes creates a new interface for compromise between
conventionally contrasting fields of interests (e.g. environmental protection and
social standards, or e-‐commerce and data privacy). The “integrated” approach
towards regulation becomes one of cooperation, juxtaposing the regulatory
processes of environmental risk management and social standardization within a
single regime in order to promote harmonization in all steps of the regulatory
process—and not purely in the name of a theoretical concept of “regulation.” This
integration allows for an effective means of achieving sustainability. The TPR
strategy—as a systematic one—attempts to foster compliance and lower litigation
costs through tactics of coordination. The studies performed by the HIIL project on
TPR in 2014 state that “the decision concerning which objectives should be
internalized and how conflicts among objectives should be solved is a key dimension
of effectiveness.”22 Thus, the inclusion of conflict-‐resolution tools, management
mechanism, and compliance incentives all within a “single regime” among cross-‐
border, cross-‐sector, and cross-‐governance actors engenders TPR as a forum of
“transparent” dialogue: a space to address trade-‐offs and coordinate intersecting
objectives.
The subject of regulation may also be “institutionalized” by horizontal and vertical
complementarities, leading to even more forms of coordination and governance.
21 Cafaggi, F. and A. Renda, p. 52. 22 idem, p 53.
17
Horizontal complementarities represent those public and private regimes, which co-‐
exist at a transnational level whereas vertical complementarities are constituted
when transnational private regimes are complemented by public regulation at a
domestic level (or vice-‐versa).23 Observing the relationships—spontaneously or
formally generated—between the private and public spheres, the role of the
complementarity reveals the relationship between legitimacy and effectiveness at
stake in TPR. Cafaggi argues how the “institutional complementarity approach”
attests to the effectiveness of private regulation as highly dependent on the
legitimacy of public institutions. However, the join working paper set forth by
Cafaggi and Renda, further complicates the vertical/horizontal complex, suggesting
how the cross-‐referencing between the two often lead to an “umbrella” set of
standards; that is, packaging one set of standards into another.24 The meta-‐
regulation, here, acts as a way to “bridge the gap” between the holes in
public/private regulatory schemes, allowing for specific regulatory goals or standards
to be achieved (e.g. ISEAL Alliance). As a result, the multiplicity of regulatory models
reveals not only the potential proliferation of but also the problematic relationship
among notions of legitimacy, effectiveness, and the interaction between the two.
The “institutional complementarity approach” and the role of “meta-‐regulation” play
a significant role in evaluating the accountability of TPR.
HOW PRIVATE REGULATION OPERATES
In order to assess the accountability—and, thus, effectiveness (and vice-‐versa)—of a
regime, we must first observe how private regulators operate. Cafaggi and Renda
propose five distinguishing characteristics of the private regulatory scheme in order
to assess its overall scope: (i) if agenda setting, (ii) whether the scheme regulates
behavior (such as, codes of conduct) or sets standards on products and services, (iii)
whether the scheme assists or manages the implementation of public regulation, (iv)
embeds enforcement mechanisms (i.e. alternative or complementary to public
schemes), and (v) all of the above).25
23 Cafaggi, p. 16. 24 Cafaggi, F. and A. Renda, p. 57. 25 idem, p. 54.
18
The use of standards codifies private regulation, yet voluntarily. That is, the
standardization of specific technical or contractual quantities and qualities is for the
most part voluntary in private regulatory schemes, granting the “regulated” the
choice to adopt or not adopt such standards. However, the notion of “legally
voluntary” is a slippery one insofar as such standards become mandatory if external
factors (such as market or social standards) oblige the “regulated” entity to adopt
them.26 Further, the incorporation of private standards into legislative or
administrative acts are binding in nature. Considering the “assumption of
voluntariness,” Cafaggi and Renda note that such assumption:
Should be qualified by considering many legal and non-‐legal factors that reduce or eliminate the space of choice for regulated entities. Evaluation of effectiveness changes depending on whether regulated entities can choose or are forced to join on the basis of factual circumstances or legal obligations imposed by public regulation. (55)
Consequently, “voluntary” standards are often, de facto, mandatory, challenging the
space for choice by the regulated entity. While observing the effectiveness of private
regulation, one must consider the relationship among the various regimes within the
scheme, in which such interaction may be facilitated by cooperation or plagued by
competition. The markings of cooperation can be found in either the form of
organization (e.g. association, foundation, for profit, not for profit) or contractual
agreements. As Cafaggi and Renda point out, this set-‐up allows for the regulation of
member behavior, but can also have impacts on third parties” (55).
THE USE AND ABUSE OF THE SUPPLY CHAIN
Thus, the role of third parties vis-‐à-‐vis impact – as impacted or impacting – appears
in the transnational networks of TPR, in which agreements or contracts act as the
predominant tools. In such a way, traditional codes of conduct or guidelines are
coupled with commercial contracts in order to ensure compliance along the supply
chain and across state boundaries, imposing standards on third parties as well.
26 idem, p. 55.
19
Cafaggi describes the use of the supply chain as a “vehicle to implement regulation”
(Cafaggi, 2012b, 2013b). As a result, the consumer may act to define the regulatory
regime itself by incorporating demands at the retailer stage, which are translated
into contractual rules, spill-‐over onto the supply chain, and ultimately mandate
compliance of those very rules along the chain. The role of the “third party,” as
exemplified herein, marks a participatory actor of the regulatory regime by indirectly
enforcing standards onto the regime itself. The consumer demand for human rights
protection, eco-‐labeling, certification, or environmental protection to name a few
(e.g. 10% energy consumption targets for the top 100 clothing factories cited by the
OECD in 2010) speak to the power of third party spill-‐over, in which new instruments
become binding by virtue of legal and non-‐legal enforcement mechanisms. 27
The use of a contract becomes instrumental in the enforcement of international soft
law, acting as a vehicle for the “hardening” of soft law and promoting harmonized
standards at an international level (Cafaggi, 30). This phenomenon can be observed
in the field of food safety, as public soft law becomes binding, or corporate social
responsibility, as the contracting of the supply chain implements international soft
law by virtue of delegating regulation to private bodies via trans-‐national private
regulation regimes, obliging compliance along the supply chain and within the
contractual relationship among suppliers and sub-‐contractors.28 The liability of
compliance with standards set forth by international soft law standards (such as the
10 principles cited in the UN Global Compact) can be enforced by corporate social
responsibility provided for by transnational private regimes, implementing its
“hardened” regulation and imposing the obligation of compliance (e.g. human rights
standards) through enforcement mechanisms.
Another area exemplifying the use of contract law in the supply chain is that of food
safety, in which contractual agreements “harden” international soft law. The
voluntary guidelines set forth by the FAO Codex Alimentarius illustrate the
incorporation of product and process standards within the contractual relationship
27 idem, p.55. 28 Cafaggi, p. 31.
20
(or by reference), allowing the retailer to stipulate employment, product safety, and
environmental clauses in the contract with a given supplier. This shift in the “terms
of exchange”29 integrates concerns, which traditionally were not included in the
contractual layout of the supplier/retailer agreement. Such transformation in the
“terms of exchange” revolutionizes the very function of the contract from a mere
trade apparatus to a regulatory instrument of conduct management, integrating
social, environmental, and equality concerns within the clausal relationship.
However, as evidenced in the area of environmental protection and corporate social
responsibility, potential loopholes in the effectiveness of contractual clauses in TPR
may arise in regards to the enforceability of contractual standards as conditioned by
where the regulatory beneficiary is positioned, rendering the effectiveness of the
contract as dependent on this organizational leverage. Secondly, as Cafaggi argues,
the “right” to monitor does not translate into the “duty” to monitor, exposing the
limitations of contract law. Although a supplier may promise to adhere to worker,
environmental, and consumer obligations, the third party’s incentive to monitor
such adherence is often conditioned by the party with “deeper pockets.”30 In such a
way, the theoretical reach of TPR is challenged by the practical limitations of
enforcement and, consequently, the implications on the effectiveness of the TPR
scheme.
TPR IN SEARCH OF A DEFINITION
Insofar as transnational private regulation does not adhere to a commonly agreed-‐
upon definition nor do its various manifestations – its form, regulatory relationship,
and governance model – conform to a pre-‐determined formula, the assessment of
its effectiveness represents, thus, a challenging feat. The current working paper of
Cafaggi and Renda set forth a “conceptual framework” for identifying the four key
dimensions of TPR in order to, then, assess its overall effectiveness. By establishing a
priori fundamental principles of and for effective regulation, we can then evaluate
the mechanics and operations of TPR in light of this conceptual framework, allowing
29 idem. 30 idem, p. 32.
21
for an informed evaluation of a system’s sustainability. Firstly, the “quality” of
private regulation is often measured according to conventional metrics, such as
“certainty, predictability, lack of ambiguity, efficacy, etc.,”31 taking into consideration
that high quality norms should not interfere with existing or co-‐existing regulatory
regimes. The potential conflict between measurements of “quality,” according to the
criteria of alternative regimes, should be accounted for in the evaluation of
regulatory quality insofar as not all metrics are created (or should be created)
equally, but may vary according to the indicators determined.
A second dimension of private regulation can be characterized by the notion of
“legitimacy,” in regards to the voluntary or non-‐voluntary nature of standardization,
the level of responsiveness, the degree of inclusiveness, and the representative
quality of the scheme in relation to the regulated bodies.32 The resulting class of
legitimacy – voluntary or mandatory – rests in a matter of consent in the former case
and formal delegation in the latter. Hence, “voluntary” legitimacy brings with it a
higher expectation of compliance and effectiveness in market-‐based communities
via private regulatory organizations.
Thirdly, the role of “enforcement” in the form of mechanisms of a legal or non-‐legal
nature: arbitration, mediation, administrative execution, domestic and international
implementation, the use of reputation, “blame and shame” methods, etc.33 Due to
the transposition of enforcement mechanism between public and private regulatory
bodies, the “regulatory space” of private norms must be analyzed within these
overlapping spheres, exerting enforcement through both judicial and non-‐judicial
instruments. The increasing use of non-‐judicial enforcement strategies through
transnational networks can often be found in the area of food safety or non-‐
discrimination standardization for example, in which the complementarity between
administrative public agency and private enforcement mechanisms act by
cooperation, promoting compliance and penalize violation.
31 Cafaggi, F. and A. Renda, p. 60. 32 idem. 33 idem.
22
Lastly, and perhaps most significantly, the dimension of regulatory “effectiveness”
defines the effectiveness of TPR in general. Cafaggi and Renda define effectiveness
as “the extent to which private regulation achieves its objectives and the correlation
between means and ends” (61). In such a way, the effectiveness of transnational
private regulation lies in sum of its individual parts; that is, standard setting,
monitoring, and enforcement. Effectiveness considers both ex ante the relation
between “means” and “ends,” and ex post the degree in which its objectives
were/were not achieved, as well as the positive or negative externalities of a norm
on the actors involved (i.e. the regulates, regulated, beneficiaries).34 Depending,
then, on the regulatory relationship and governance model chosen, the degree of
effectiveness varies. Such calls upon the “interpreters” of effectiveness to inquire
upon the combination of these variables – the type of norm and institutional
framework – as a means to understanding effectiveness, as both an organizationally
and structurally informed metric stick. The very interdependency among all of the
dimensions mentioned above mirrors the “integrated approach” deployed by the
OECD, which attributes an interrelated dimensionality of quality, legitimacy,
enforcement and effectiveness. In other words, the quality of a given scheme
increases its legitimacy, which in turn promotes compliance, that then enables
enforcement, which in turn increases effectiveness, ultimately re-‐confirming its
quality. The causal relationship among the characteristics of TPR allows for a so-‐
called “disentangling” of their interlocked qualities.
HOW GOVERNANCE CAN EFFECT EFFECTIVENSS?
As discussed, the role of the governmental model chosen -‐ as defined by its legal
form and related design -‐ plays an important role in determining the effectiveness of
private regulatory organizations. That said, we have witnessed a causal relationship
between governance and activity, calling into question what are the effects of
governance and, successively, how governance can effect effectiveness? The major
34 idem, p. 61.
23
elements of governance that demonstrate an impact on effectiveness can be related
to eight proxies.
Firstly, the instance of functional separation represents a criterion for good
governance insofar as separation promises the existence of checks and balances
between the main standard-‐setting body (such as the board) and the main decision-‐
making body.35 Without “regulatory space” between standard-‐setting, monitoring of
compliance, and enforcement, the organization runs the risk of covert interests
underlying these processes, undermining the effectiveness of regulation. In such a
way, the lack of separation becomes problematic, especially in the enforcement
stages of compliance, in which a hypothetical organization that does not exhibit
functional separation may be less willing to enforce sanctions or monitoring
compliance on itself given that responsibility becomes an internal “value” rather
than an external indicator to be monitored. Hence, good governance implies
functional separation followed by the coordination of functions separated across the
regulatory process.
The logical progression from functional separation, then, is to cover that of a multi-‐
level structure of the regulatory chain. As the regulatory scheme is made up of multi-‐
boded and multi-‐faceted parts, the role of “separation,” as a functional distinction
(descriptive in nature) and operational one (normative), the degree of separation
both horizontally and vertically impacts the effectiveness of the regulatory scheme.
The trans-‐national nature of TPR implies the structuring of global, regional, and
national levels of standard-‐setting, monitoring and enforcing agents, whereas
functional separation distinguishes among the bodies responsible for these
regulatory processes. Thus, the “degree” and “quality” of coordination among and
across these layers of normative and descriptive separation represent the
dependent variables upon which regulatory performance relies. The blurred
boundaries between governance spheres increase the uncertainty of responsibility
allocation and heighten the degree of structural interdependencies, underlining the
35 Cafaggi, p. 64.
24
necessity to reformulate the unit of analysis. Considering the multi-‐level structure
and complexity of the regulatory chain, the unit of analysis must be reduced to the
individual organization,36 allowing for the measurement and assessment of
regulatory effectiveness in conjunction with the coordination mechanisms employed
across functions and between spheres.
Thirdly, the quality of membership in the organization impacts the assessment of its
effectiveness, depending upon the degree of representation allotted to its members;
that is, the (in)accessibility of a “voice.” The distribution of representative power
among stakeholders directly correlates to the legitimacy of the member-‐based
organization, enabling a system of “checks and balances” by virtue of proportionate
allocation of representation and, theoretically, more “fair” decision-‐making. The
effectiveness of a member-‐based organization relies upon the legal and non-‐legal
tools available to it—the ability to “check” the activity of the board. The normative
variable of inclusiveness, in which all stakeholders relevant to the decision-‐making
and/or standard-‐making process should be present in order to ensure the adoption
of common norms or agreements. The consultation of stakeholders in ex post
evaluation of an organization’s performance plays a key part in assessing
effectiveness.
Another normative variable includes the organization’s choice to regulate for profit
or not-‐for-‐profit, which again influences not only effectiveness but how it should be
evaluated. As Cafaggi and Renda argue, “it affects definition of both means and
objectives” (66). The former may tend to maximize benefits in favor of its members
over the public, whereas the latter tends to maximize public welfare over the
interests of member-‐based interests. This interplay reveals the relationship between
public interest and the relationship between the regulated/beneficiaries.37
Similarly, the source of funding affects the accountability of a private organization.
That is, an organization with multiple financial inputs may be more “free” to pursue
36 Cafaggi, F. and A. Renda, p. 66. 37 idem, p. 66.
25
common and public good objectives, whereas an organization dependent on a single
source of financial support may be notably conditioned by the interests defined by
its funder. In such a way, the heterogeneity of funding represents a crucial factor for
effective governance.
The transparency of the regulatory process – from agenda-‐setting to enforcement –
marks a crucial factor in qualifying effectiveness. That is, the disclosure of financial
and non-‐financial reporting, the communication of decisions, the consultation of
stakeholders, comprehensive ex ante impact assessments, and ex post evaluations
all contribute to the transparency of an organization’s activity, exposing its quality.
The very quality of the regulatory process, thus, is directly related to the
effectiveness of its governance.
Lastly, the notion of accountability emerges in the realm of private regulation,
specifically in reference to those bodies in pursuit of common or public goods.
Adopting the definition of accountability set forth by Cafaggi and Renda, in which an
organization “adequately motivates and explains its regulatory decisions, monitors
results and impacts, and bears responsibility for the failure to achieve its own goals
and objectives,” the level of accountability is proportionally related to the
effectiveness of regulation.
WHAT TO MAKE OF EFFECTIVNESS
The variables of governance that impact the quality of governance – and by
consequence, effectiveness – must, then, be evaluated by virtue of the tools of and
means for measuring that effectiveness. However, the choice of assessment tools
becomes equally as important as the level of effectiveness itself if we understand
the assessment as a means towards higher efficiency and further improvement of
regulation. That said, the importance of stakeholder consultations, ex ante
assessments, and monitoring and ex post evaluation represents not only the
measurement of effectiveness but the very vehicle for achieving it.
26
As previously discussed, the role of stakeholders is evidenced in various steps
throughout the regulatory process, including the early consultations to locate policy
issues, discussion of policy options, and their experience-‐based input as to the
expected effectiveness and potential problematics of a given regulation (i.e.
compliance). However, the legitimacy of stakeholder questions lies in the legitimacy
of the questions asked; that is, only relative questions can expose the relevant policy
issues at stake. In such a way, the consultation exercise can facilitate regulators in
identifying policy issues both ex ante and ex post.
The evaluation of effectiveness occurs most pertinently in the early stages of the
regulatory process; that is, through ex ante impact assessments, which comprises a
qualitative and quantitative based framework for assessing the sustainability of a
given policy proposal whereas ex post evaluation and monitoring allows for policy-‐
makers to consider the quality of legal rules and the accountability of the
regulators.38 This method is best exemplified in the EU’s “Smart Regulation
Framework”39 of 2014, providing for structured guidelines to assess the viability of
policy options by identifying the policy problem, providing the reasoning for
intervention, performing a comparative analysis of policy options (as well as the
“zero change” option), indicating the preferred policy options, and outlining the
criterion used for evaluating its implementation, enforcement, and effectiveness ex
post in order to compare the policy objectives with empirical impacts. The “Smart
Regulation” guidelines suggest, theoretically and linguistically, for the “bettering” of
regulation – that is, the qualitative improvement over time – by virtue of the
evaluation of its effectiveness and the implications on regulatory sustainability.
Hence, the EU’s interpretation of impact assessment reveals a comprehensive
approach that aims not only at the policy-‐making process, but “good” policy-‐making,
underlining the importance of improvement and efficiency through the learning
curve created between ex ante and ex post assessments.
38 idem, p. 69. 39 European Commission (2014), “Regulatory Fitness and Performance Programme (REFIT): State of Play and Outlook,” COM (2014) 368 Final, European Commission Communication, 18 June.
27
This integrated approach towards assessing the effectiveness of the regulatory
process is often referred to as the “policy cycle” approach (Cafaggi and Renda,
2014), calling upon the need of a “complete and sound evaluation toolkit” made up
of ex ante assessment of legislation and legislation measures, interim monitoring
and evaluation of enforcement and compliance, and a pre-‐determined ex post
evaluation for assessing the performance and sustainability of regulation through
cost-‐benefit analysis. Thus, the quality of the “policy cycle” approach marks its merit
as a tool for smart policy-‐making and effective regulation. If, however, the
comprehensive of analysis is compromised – in the instance of the partial
assessment of impacts or if the “right” questions of evaluation are not posed – the
quality of analysis is consequentially unaccountable. In other words, if all the
impacts of the regulatory process are not taken into account, the assessment of
regulatory quality and performance is rendered futile—the impact assessment, de
facto, dismantles itself from within.
THE (MIS)USE OF INDICATORS
The functionality of regulatory impact assessment lies in the choice of appropriate
indicators chosen in relation to the regulatory question under evaluation. Failing to
choose the “right” indicators translates, ultimately, to the failure to assess regulatory
effectiveness. Indicators may be used by a variety of actors and within a variety of
regulatory schemes: (i) for self-‐evaluation in private schemes, (ii) for independent or
contractual evaluation by third parties, or (iii) for the public sector.40 Whereas ex
ante indicators serve to identify the preferred policy choice among alternative
regulatory options, ex post indicators allow for the evaluation of the governance-‐
models chosen, regulatory performance, and to re-‐align the relationship between
instruments and policy objectives. Again, this “policy-‐approach” assessment allows
for complementarity interaction between ex ante indicators (such as governance)
and ex post indicators (such as impact) in order to promote “better” regulation by
virtue of better instruments. The “policy cycle’ approach, which is strongly endorsed
40 Cafaggi, F. and A. Renda, p. 72.
28
by the OECD, relies upon ex ante and ex post indicators as marking the “two most
important appraisal instruments.”41
Furthermore, the European Commission’s endorsement of “fitness checks” in its
“REFIT” communication of 2014 stresses the use of impact indicators, which extends
beyond mere output indicators, in order to assess the impact of regulatory regimes
on widened spheres of policy concern, social welfare for example. In such a way, the
use of indicators – and, most importantly, what they indicate – expands the horizons
of public and private regulatory “checks and balances” into the realms of
transnational private regulatory schemes (such as the ISEAL Alliance), leading to a
hybridization of multiple indicators. Depending on the objective of the regulatory
body, effectives may be measured through “performance” indicators in co-‐
regulatory schemes (public and private regulation), or simply “output” indicators in
technical standardization fields (third party regulation), or through a comprehensive
set of indicators encompassed by “impact” indicators in the trans-‐national arena.
Bearing in mind that output – the material result of a given regulation – is not
synonymous with its impact – the effect, impact indicators transcend the boundaries
of conventional areas of assessment, enabling the scientific-‐based appraisal of
matters such as social welfare, gender equality, and the quality of life. In such a way,
the “policy cycle” approach endorsed by international organizations and the “Smart
Regulation” agenda set forth by the EU demonstrate comprehensive approaches
towards quantifying and qualifying regulatory impact in a revolutionary way,
provoking us to ask what lies at stake in regulation and how is the regulatory process
a socially, economically, and politically informed one.
RE-‐PIECING THE PUZZLE: COMPLIANCE, EFFECTIVNESS, AND THE LEARNING CURVE
Monitoring compliance in relation to the standards, rules, and impacts of the
regulatory scheme marks a fundamental role in the overall evaluation of
effectiveness: the ability to monitor the regulated entity’s compliance (or
incompliance) with rules and the assessment of that entity’s ability to achieve (or not
41 idem, p. 73.
29
achieve) its objectives. Measuring compliance, hence, is representative of a core
concern for regulatory effectiveness (i.e. if the regulated entity meets the
requirements of provisions). Insofar as the notion of compliance concerns both a)
the regulated entities’ compliance with the rules determined by the scheme and b)
the performance and overall effectiveness of the scheme itself42, the former – a –
does not however directly imply the occurrence of the latter – b. Although a typically
leads to b in the above hypothesis, correlation does not necessarily confirm
causation – a post hoc, ergo propter hoc fallacy – in our methodological evaluation of
compliance and effectiveness.
Instead, the actual effectiveness of a regulatory scheme – and its performance
therein – marks the center of our analysis, as an inclusive consequence of the
various “inputs.” Measuring effectiveness in its correlative relation to compliance
allows us to characterize the “collective dimension of compliance,” in which
cooperation between the regulators and regulated enables monitoring in private
regulatory schemes. As Cafaggi and Renda argue, “whether or not regulatory
objectives are met depends on the performance of the regulated as a collective.”
Therefore, the metric of evaluation must account for “interdependencies among
regulated entities’ conducts,” (75) which represents a fundamental concern in the
area of food safety or environmental protection. That is, partial compliance in food
safety or traceability schemes undermines their entire risk management system,
rendering control mechanisms partial and inconclusive and, therefore, ineffective.
Such underlines the need for interdependent cooperation among regulated entities
in order to approach a full account of not only compliance but overall impact. In light
of such argument, Cafaggi and Renda astutely stress:
The focus of effectiveness measurement should thus not be limited to compliance with rules, but rather focus on the achievement of regulatory objectives and the causes for the failures whenever they occur. (75) Such argument draws our attention to the objectives of assessing effectiveness
itself; that is, the analysis of a given scheme’s effectiveness has no value in and of
itself if we do not apply the lessons learned from the weaknesses identified in
42 Cafaggi, F. and A. Renda, p. 74.
30
existing regulatory schemes towards future methods of improvement—the so-‐called
“learning curve.” Fragmenting, here, compliance from effectiveness as no longer
mutually dependent factors, the authors argue that high compliance can be
considered “necessary,” but not always the sufficient condition for the achievement
of regulatory goals. Insofar as compliance represents only one portion for
consideration, the ex post learning process represents an equally important (if not
more) part of the puzzle, as effectiveness measures tangible results. In other words,
the goal – for private regulators – should be that of making private regulation work.
Lastly, the method and quality of enforcement represents a key prerequisite to the
effectiveness of private regulatory organizations. Without enforcement mechanisms
in place, the degree of compliance obviously tends to decrease. The threat of
sanctions or violation mechanisms can only be effective if legitimate enforcement
takes place. That said, the main characteristics for assessing the method and quality
of enforcement are as follows: (i) the scope and quality of enforcement, (ii) the
independence of enforcement (hence, its legitimacy), (iii) the length of procedures,
(iv) the relationship between enforcement by private bodies and public enforcement
bodies (e.g. courts).43 In such a way, the quality of the remedies available to the
regulator determines the quality of enforcement. On the other hand, the
incorporation of non-‐member representation (in addition to member
representativeness) increases the credibility of the enforcement scheme from a
point of view “outside” that of the regulator; the incorporation of due process
requirements, the right to fair trail, impartiality of enforces, access rights, the right
to appeal, and the recognition of third-‐parties rights and the potential impacts on
third-‐parties, transfers a degree of legitimacy to the enforcement activity, ultimately
increasing its effectiveness. Lastly, the use of “reputational effects” – in “public” or
“confidential” settlement of conflict – the effect of staking reputation in public
disputes typically tends to increase the effectiveness and standing of private
enforcement mechanisms from a public perspective.
43 idem, p. 76.
31
THE FIVE Ws OF TPR
As discussed throughout the first part of the paper, transnational private regulatory
schemes can be assessed in terms of quality, legitimacy, enforcement, and
effectiveness. The ultimate role and capacity of a scheme’s effectiveness distills the
other 3 factors into the most determinant factor of consideration: that of
effectiveness, in which the perceived legitimacy of the scheme, quality of rules, and
credibility of enforcement all serve to make a scheme desirable (or undesirable,
depending on one’s objectives) to potential members if indicative of benefits in line
with the given agenda. That said, depending on the policy perspective from which
one stands – public, private, individual, or collective – the perceived effectiveness of
regulatory organizations conditions the appeal of it: the who doing the evaluation,
the what is being evaluated, the how evaluation is organized, the when or where in
the regulatory process does it take place, and the why – in other words, for whom –
all condition and are conditioned by the regulatory scheme chosen.
As argued by Cafaggi and Renda, “what really matters is not whether the evaluation
is performed directly or indirectly by the evaluated entity, but who defines the
purposes and the objectives of the evaluation, and what is made of the results” (77).
Consequently, we can distinguish among transnational private regulators in terms of
who does the evaluating, whether (i) by independent private organizations (usually
for profit or non profit based, or independent organizations in contractual
agreements with the evaluated entity such as the case of certification), (ii) by meta-‐
private regulators (such as ISEAL or NGOs in charge of monitoring on behalf of other
constituencies), or (iii) by public entities.44
The subject or the what should be evaluated, as previously discussed, should inquire
most importantly into the link between the form of governance and the regulatory
process, the inclusiveness of stakeholders, the consideration of not only regulatory
objectives but regulatory impacts as well (and their distributive consequences), and
the selection of indicators and the allocation of their so-‐called “weight” in the
44 idem, p. 78.
32
evaluation process. Although many typologies of TPR exist, we shall narrow our
focus here on to the function of meta-‐regulators, inquiring into the evaluation of
their performance as meta-‐regulators. That is, how should meta-‐regulators evaluate
their effectiveness? Insofar as the objective of meta-‐regulation is the capacity to
change the behavior of the regulators moreover than the regulated, the
effectiveness of the meta-‐regulator’s performance lies in this crux. The ability to
transform the regulator’s relationship in regards to governance and within the
regulatory process evidences, then, the effectiveness of the meta-‐regulator, namely
the capacity to improve regulatory performance over the degree of compliance of
the regulated entity (e.g. ISEAL).45 Meta-‐regulation can be broken down into 2
categories: those with members and those who produce standards and codes to be
used freely by others. Thus, the organizational nature of the meta-‐regulator
influences its effectiveness; that is, the credibility of a member-‐based meta-‐
regulator depends upon its regulatory “independence” from its members (i.e. the
potential influence of stakeholders in agenda-‐setting and the degree of compliance
enforced). In the latter form, such as the GRI scheme, effectiveness can be reflected
by the degree in which those organizations that freely adopt rules and standards
ensure monitoring the compliance with those rules standards. The form of meta-‐
regulation is generally materialized into a set of guidelines to be adopted.
The notion of “benchmarking” comes into play in discussing the where and when
regulation occurs along the regulatory process, specifically in regards to the attempt
to harmonize multiple regulatory schemes along a homogenous axis of comparison.
In such a way, the relationship between the actors in the regulatory process (i.e. if
contractually bound or voluntarily cooperative), affects the institutional set-‐up of the
regulatory scheme. Although benchmarking allows for a direct comparison of
multiple regulatory schemes (such as those carried out in food certification schemes
like that of GlobalGap), the attempt to harmonize according to formal requirements
may, in turn, reduce its ability to actually evaluate “on the basis of real functioning”
(84). In such a way, comparative evaluation becomes problematic if the number of
45 idem, p. 81.
33
standards and implementation practices are too diverse, replacing harmonization via
standard-‐setting with defining common criteria and indicators that create systemic
rather than single organization effectiveness.46 On the contrary, we have observed
an evolution in the attempt to harmonize differences in the area of sustainability,
shifting between the harmonization approach to one more based on coordination,
across sectors, by virtue of drafting indicators to define regulatory performance and
to assess sustainability.47 One of the leading models of these coordination efforts in
meta-‐regulation is illustrated by the SAFA tool set forth by the Food and Agricultural
Organization (FAO) in collaboration with private organizations; its effort located in
the definition of a global Sustainability Assessment of Food and Agricultural Systems
(SAFA), namely through the drafting of comprehensive guidelines.
HOLES IN THE SYSTEM
In their conclusion on the analysis of a selection of private regulatory schemes,
Cafaggi and Renda underline three major issues associated with the appraisal of
their effectiveness. Firstly, ex ante impact analysis “is not very common” and “does
not follow a rigorous methodology,” (99) which undermines the accountability of
governance in the choice of a given standard and, in turn, the ability for internal
members or external stakeholders to quantitatively and qualitatively assess the
performance of the regulatory scheme. Secondly, the authors claim that despite the
prevalence of ex post evaluation, the assessment concentrates on member
compliance, failing to select indicators to measure the achievement of regulatory
objectives and, furthermore, impacts external to the regulatory scheme itself. On a
side note, according to their findings, those who define the indicators are not usually
those who perform the assessment. Lastly, as we have emphasized throughout our
discussion, the exclusion of the regulatory process and the role of governance from
the private regulatory scheme analysis disables the ability to evaluate overall
effectiveness.48 Such criticisms, then, serve as a bridge-‐way between our analysis of
46 idem, p. 86. 47 Although Cafaggi and Renda refer to the strength of SAFA as engaging in the definition of criteria “to define regulatory performance,” (86) I have expanded the tool’s merit to include that of assessing sustainability as well. 48 idem, p. 99.
34
the effectiveness of transnational private regulation – and corporate social
responsibility reporting – to alternative frameworks developed by virtue of private
and public cooperation in the field of sustainability, in which the SAFA tool
represents one of the leading frameworks for assessing sustainability in the food and
agricultural sector. In contrast to the above criticisms plaguing the majority of
transnational private regulatory schemes, SAFA instead represents a comprehensive
tool, subverting TPR’s “holes in the system” into areas of thorough assessment.
Hence, the guidelines comprise an extensive use of ex ante impact assessment and
ex post evaluation, integrating and building upon multiple sustainability frameworks
ranging from corporate social responsibility (CSR) to social and environmental
standards and reporting. In such a way, the FAO performed a sort of “mapping” of
sustainability indicators for the agri-‐food sector, designing a structured framework of
performance-‐based indicators in cooperation with another UN subsidiary body, the
International Trade Centre via the creation of “Standards Maps.”49 The SAFA tool
structurally avoids such methodological flaws, drafting guidelines for the use of
extensive ex ante and ex post impact assessment, the possibility for the regulated
entity to contextualize the default indicators in light of specific circumstances, and
the allocation of “value” in performance-‐based indicators over the mere practice and
target-‐based appraisals. Lastly, the stipulation of governance as one of the 4
overarching dimensions of sustainability in the SAFA guidelines and the underlying
emphasis on external impacts retort the “holes” found in most TPR schemes,
suggesting SAFA to be one of the leading tools in the field of sustainability
assessment.
49 Available at http://www.intracen.org
35
CHAPTER 2: SUSTAINABILITY IN SEARCH OF A DEFINITION
As described in the preface of the Sustainability Assessment of Food and Agricultural
Systems Guidelines (SAFA) of 2013, the principle of sustainable development had
first emerged as a universal concept at the 1992 Earth Summit, after which its
significance continuously mutated in space and time, assuming varied and multiple
meanings. Over the course of nearly 25 years, 200 different types of sustainability
frameworks have developed in thee public and private spheres, national and
international institutions, and domestic and global domains, manifesting in codes of
conduct, CSR obligations, social and environmental standards, and food safety
guidelines. Further, 120 voluntary sustainability standards, including eco-‐labeling,
origins certification schemes, GMO standards, as well as organic certified practices
have emerged, merging and diverging in scope. Despite this notable shift towards
and desire for “sustainability,” a single and comprehensive framework for
sustainability reporting does not yet exist today. Proliferating beyond domestic,
international, and transnational spheres of regulation and between supplier, retailer,
and consumer regulatory relations across the supply-‐chain, the notion of
“sustainability” becomes even more evanescent, slipping between obfuscating
spheres of meaning(lessness). Considering the intricacy (or, perhaps, impossibility) of
compounding multiple frameworks into a single, common language, lens, and set of
rules, the FAO seeks to simultaneously simplify sustainability assessment while
defining a new, ever more encompassing framework for it, establishing a
comprehensive, step-‐by-‐step tool for assessing 4 crucial dimensions of sustainability
– good governance, environmental integrity, economic resilience, and social well-‐
being – along food and agricultural value chains.50 This paper shall argue for the
merit of the SAFA tool in the field of sustainability assessment and reporting, as well
as the implications of it for the future. With that in mind, one must continually call
into question what lies at stake in measuring that which we cannot define; as the
definition of “sustainability” forever escapes the definer, a similar dynamic seems to
50 FAO. 2013. Sustainability Assessment Food and Agricultural Systems Guidelines. Food and Agriculture Organization of the United Nations, Rome (available at http://www.fao.org/fileadmin/templates/nr/sustainability_pathways/docs/SAFA_Guidelines_Final_122013.pdf
36
be taking place in the measurement of it. That is, despite the number of
mechanisms, tools, and metrics for sustainable development perpetually increases
over time, sustainability per se continuously diminishes.
RECONSIDERING SUSTAINABILITY VIS-‐À-‐VIS SAFA
In order to integrate and appropriate the multiple existing frameworks for
sustainability schemes, SAFA offers a “holistic” approach, considering the complex
dimensionality of sustainability by virtue of the incorporation of existing
standardization systems, building upon rather than substituting them. In such a way,
SAFA “reconsiders” existing sustainability schemes and expands upon them,
introducing an umbrella-‐like framework, allowing for a common language for
sustainability measurement. The very making of the tool – as a process of integration
– by the FAO reveals just this, in which the writers “mapped” all sustainability
indicators in the food sector and drafted its goal and scope in 2010, which was
followed by stakeholder surveys targeting field experts from the food and
agricultural industry, public administrations, NGOs, multi-‐stakeholders and
multilateral institutions in mid-‐2011, a cross-‐comparison of standards and indictors
of 44 different systems in late 2011 (i.e. 18 industry standards, 5 farm level systems,
4 systems of multilateral institutions, 7 NGO systems, 5 roundtable standards, and 5
other systems), and the circulation of draft guidelines in January 2012; after a “pilot
phase” of the Test Version 1.0 in late 2012, the SAFA Guidelines Version 3.0 were
eventually approved in 2013.51 Thus, the structure of the current SAFA Guidelines
are, mostly, based upon the existing ISO 140140:2006, the ISEAL Code of Good
Practice and the Sustainability Reporting Guidelines and the Food Sector Supplement
of the GRI.52 By building upon existing schemes rather than replacing them, SAFA
simultaneously expands upon reliable expertise and allows enterprises to utilize
existing data sources, decreasing costs, avoiding duplicity, and increasing efficiency,
attesting to its comprehensiveness and accountability as a tool.
51 FAO. 2012. Sustainability Assessment Food and Agricultural Systems Guidelines (Test Version 1.0). Food and Agriculture Organization of the United Nations, Rome (available at http://www.fao.org/fileadmin/templates/nr/sustainability_pathways/docs/SAFA_Guidelines_12_June_2012_final_v2.pdf 52 Cafaggi, F. and A. Renda, p. 104.
37
The “guiding vision” of SAFA is “that food and agricultural systems worldwide are
characterized by all 4 dimensions of sustainability: good governance, environmental
integrity, economic resilience, and social well-‐being,”53 in which we can observe a
sort of “re-‐consideration” of what it means to be sustainable. The previous dilemma
of TPR and sustainability “in search of a definition,” returns in our account of SAFA,
in which the FAO’s efforts evidence a holistic approach to measuring not only what is
sustainability, but why it is important. Hence, the SAFA “vision” re-‐thinks
conventional notions of “value,” allocating value in terms of performance here,
striving not only for a mere appraisal of a system, enterprise, or even country’s
sustainability, but instead questioning the implications of its current performance for
the future; sustainability reporting becomes the instrument, thus, of improvement, if
implemented within an transparent and aggregated framework, such as that
provided for by the framework in discussion. By breaking down the assessment into
constituent parts, strengths and weaknesses can be identified and, consequently,
improvements can be drafted—the framework itself determines its functionality
(and potentiality).
SUSTAINABLE DEVELOPMENT: WHAT LIES AT STAKE?
Before introducing the assessment framework and how it works, the SAFA
Guidelines first contextualize its vision, approach, the actors involved, and scope. In
light of the 2015 Millennium Development Goals, and in consideration of the goal to
halve the number of chronically undernourished people worldwide vis-‐à-‐vis
sustainable development, such challenge lies in the effective incorporation of the
environmental, economic, and social dimensions of development. As the amount of
the population still suffering from hunger today represents 16% of the global
community,54 1.4 billion people live in extreme poverty (2005), and food security is
“no reality” for 875 million individuals55, and the increasing human demand strips
the ecosystem of its already scarce availability of resources, we are facing a global
53 SAFA (2012), p. 2. 54 SAFA (2013), p. 12. 55 idem, p. 178.
38
crisis of extreme and irreversible magnitude, calling upon the sustainable
management of agricultural land, forestry, water, atmosphere, and biodiversity.
According to the Stockholm Resilience Centre’s report of 2009, humanity has
transgressed the 3 “environmental planetary boundaries within which we can
operate safely,” underscoring the urgent issues we face today as a global
community: climate change, biodiversity loss, and the impact on the global nitrogen
cycle.56 That said, the sustainable management of the earth’s resources marks a
crucial factor in the preservation of it. The purpose of SAFA, thus, seeks to establish
a framework that can systematically address these issues across sectors, nations,
and markets, echoing its vision for “sustainable development:”
The management and conservation of natural resource base, and the orientation of technological and institutional change in such a manner as to ensure the attainment and continued satisfaction of human needs for present and future generations. Such sustainable development (in the agriculture, forestry, and fisheries sectors) conserved land, water, plant and genetic resource, is environmentally non-‐degrading, technologically appropriate, economically viable and socially acceptable. (FAO Council, 1989).
By situating itself along the supply chain, SAFA aims for the long-‐term
transformation of sustainable food systems, which would require the management
of interconnected sustainability issues beyond the supply chain, extending its
analysis onto the environmental impact of a given product through its lifecycle—the
Life Cycle Approach (LCA). Although SAFA incorporates LCA (a product or process-‐
specific assessment) within its analytical framework, SAFA’s approach lies in the
assessment of the enterprises rather than the product. That is, SAFA assesses the
sustainability of an enterprise in terms of economic, environmental, social, and
governance dimensions, allowing the framework to be implemented on a variety of
levels and along different axis of comparison: at a regional, national, or international
level; in relation to the supply chain; or even in regards to an operational unit.57
56 idem, p. 12. 57 idem, p. 7.
39
WHO USES SAFA?
Therefore, SAFA’s multi-‐level application and multi-‐faceted purposes enables a
variety of actors to use its harmonized framework to ensure consistency,
transparency, and applicability. As a voluntary, self-‐evaluation tool for sustainability
assessment, SAFA may be adopted by agri-‐food enterprises for self-‐assessment (to
identify “hot-‐spots” or areas for performance improvement), NGOs and
sustainability standards systems (to monitor impacts, share data, best practices, and
thresholds) and governments, investors, and policy-‐makers (to establish
sustainability goals, implement local planning or procurement, promote investment
or legislation, provide sustainability reporting for informed policy-‐making).58 By
creating a network of sustainability frameworks on which the SAFA Guidelines are
based, actors may adopt the tool for a varied purposes, including i) sustainability
reporting (e.g. CSR), (ii) benchmarking (e.g. standards and codes), (iii) standards
based on production or processing standards (i.e. added-‐value systems vis-‐à-‐vis
transparent and sustainable processes), and (iv) assessment methodologies of the
production unit or supply chain (e.g. LCA).59
As an umbrella-‐like framework, SAFA serves the function of harmonizing a variety of
sustainability tools in order to promote coordination and convergence into a
common framework based on a common language. The guidelines indicate the
methodological and implementation principles (see Figure 2), in which its core
principles (i.e. relevance, simplicity, cost efficiency, goal orientation, performance
orientation, and transparency) echo those cited in the EU protocol for Better
Regulation set forth by the Mandelkern report60 (i.e. necessity, proportionality,
subsidiarity, transparency, accountability, accessibility and simplicity), aligning the
SAFA guidelines with “real” impact assessment guidelines.
58 idem, p. 5. 59 Idem, p. 8. 60 European Commission (2001), “Mandelkern Group on Better Regulation,” European Commission Communication, 13 November.
40
As evidenced, the coinciding and conflicting nature of sustainability assessments
disables the ability to “internationalize” a single metric of sustainability, blurring our
ability to define it. Recalling our discussion of transnational private regulation, in
which the regulatory process transgressed domestic/international and public/private
boundaries in the assessment of an entity’s “sustainability,” the ability to define
what makes a company “sustainable” under the meta-‐regulation perspective, from
which SAFA operates, faces a similar predicament. In order to “fill the gap,” then, in
such absence of the definition of what qualifies sustainability per se, the SAFA
guidelines “methodologize” sustainability performance assessment (by reference to
a set of themes, sub-‐themes and default indicators) in order to attain an overall
report of an entity’s “sustainability.” In other words, SAFA “fills the gap,” by defining
how to measure sustainability, without actually defining it. Reversing conventional
rules of linguistic theory here, SAFA qualifies the means to its significance rather
than qualifying the sign itself.
DEFINING BOUNDARIES: THE SCOPE OF SAFA
Before introducing the procedures and protocol of SAFA, the boundaries of its
framework must first be established. Specifically, SAFA undertakes a “supply chain
scope,” which allows assessment to occur in relation to all entities in the supply
chain, ranging from the input suppliers (e.g. the feed provider), to the site of primary
production (e.g. fisheries), the wholesaler, transporter, and the retailer. However,
the supply chain, under SAFA “jurisprudence,” does not include the
consumer/consumption insofar as the tool cannot be applied to product-‐specific
sustainability. Instead, SAFA applies to all entities in the global supply chain.
According to the guidelines, an entity conducting self-‐evaluation must determine its
“realm of influence” by firstly defining its scope and the boundaries of the
assessment.61
The temporal scope usually represents the activity of a given entity for a period of 1
year in order to serve as the basis to which future assessments can be compared.
61 SAFA (2013), p. 17.
41
However, depending on the specificity of a the subject of assessment (e.g. long-‐term
impacts on an ecosystem) may require a five year temporal scope, which should be
determined according to expert or scientific-‐based knowledge.
The thematic scope of the sustainability context is a condition of its coherent, multi-‐
level structure. The SAFA framework is made up of 4 overarching dimensions of
sustainability (good governance, environmental integrity, economic resilience, and
social well-‐being), which are then broken down into 21 “high-‐level themes” (such as
participation, materials and energy, vulnerability, and labour rights). These high level
themes represent core sustainability issues, which are further distilled into 58 sub-‐
themes related to the food and agricultural sector. Each sub-‐theme is coupled with
sustainability objectives, detailing an expected sustainability performance for that
specific sub-‐theme, allowing supply chain actors in the course of the assessment to
identify “hot-‐spot” areas – risks or gaps in the existing sustainability efforts – to
solicit recommendations for improvement. The SAFA tool includes “default
indicators” for each of the 58 sub-‐themes in order to exemplify what kinds of
questions should be asked in relation to the given issue in question, providing
performance-‐based indicators (with associated criteria and objectives) as a means of
example. However, in the case of small-‐scale enterprises or on indigenous farming
practices, the use of target-‐based or practice-‐based indicators for assessment may
be more appropriate given the limited capacity and resources of evaluating body.
Although SAFA urges for the use of performance-‐based indicators, its flexibility can
be evidenced here, in its use of default indicators, appropriation of alternative
indicators where fit, and varied weighing mechanisms to uphold SAFA’s strict
protocol even when performance may be difficult to measure.
42
Figure 1: SAFA Framework
Figure 2: SAFA sustainability dimensions and themes
43
SAFA STEP-‐BY-‐STEP
The first section of the SAFA Guidelines describes the overall vision, background, and
purpose of the tool, whereas the second section provides for the actual
implementation of it. The step-‐by-‐step approach stipulates 4 main procedures to be
followed (mapping, contextualization, indicators, and reporting), leading to the final
Performance Report: a descriptive and critical sustainability report of the assessed
entity.
Figure 3: SAFA steps and phases
STEP #1: MAPPING MATERIAL MATTERS
The first step – mapping – comprises the definition of goals and purpose, a
description of the assessed entity (size, position, and market power), the definition
of the scope and boundaries of the assessment (as well as the entity’s sphere of
influence and impact), the description of impact allocation criteria, and a visual
representation of the value chain, the interconnected relationships, and
boundaries.62 In order to complete the above, one must “map” the goals and scope
62 SAFA (2013), p. 25.
44
of the assessment and assessed entity. According to the guidelines, the setting of
goals should “unambiguously state the reasons for doing the assessment, the
intended audience and the intended use of the results,” (25) which builds off the
provisions set forth by ISO in 2009. In other words, one must answer the “5 W’s” as
discussed earlier.
The setting of the scope and boundaries, on the other hand, presents a difficult task
to the evaluator insofar as the notion of a borders are perpetually transgressed –
both spatially and temporally – by virtue of the underlying notion of
interconnectedness that both conditions and is conditioned by globalization,
rendering our ability to distinguish boundaries as a problematic feet. The notion of a
“boundary” is compromised spatially – as spheres of influence and direct/indirect
impact extend beyond defined geographical borders – and temporally – as cause and
effect schemes complicate over time. The complication posed by the necessity of
setting boundaries rests precisely in this point, as impacts (and their “weight”) alter
over time in terms of both quantity and quality, varying between distinctions of
direct and indirect, short-‐term and long-‐term, and acute and obtuse. Thus, as no
entity can be considered (nor assessed) in a vacuum, we must consider the role of
externalities while mapping the scope and boundaries of assessment. Building upon
GRI “Boundary Protocol,” SAFA addresses how “activities involving a complex
network of value chain actors, sometimes around the globe, affect and are affected
by an organization’s social, environmental, and economic performance” (28). Thus,
mapping requires an exercise of inclusion and exclusion in order to understand what
should be measured, how the organizational and operational boundaries are
designed, and where interactions occur within the network of production. As noted,
the larger the company, the greater the diversity of supply chain interaction will be,
calling upon the supply chain map to address its complexity and distilling the
assessment to “material matters,” including those potential spheres of influence
and/or influence and excluding the so-‐called “cut-‐off” areas beyond relevance. In
such a way, “mapping” becomes a collection of “material matters”63—from this
63 idem, p. 31.
45
perspective; boundaries can thus be determined by those issue areas, which could
significantly influence the assessment of the intended report user.
PROCEDURAL ISSUE: HOW TO MAP SMALL-‐SCALE PRODUCERS?
Insofar as SAFA was created for the assessment of both small-‐scale and large-‐scale
producers, the amount of resources and capacity needed for a comprehensive,
performance-‐based assessment is not often accessible to small-‐scale producers to
successfully perform a “true” SAFA assessment according to the criteria required. In
light of such limitations – that is, the lack of “material matters” – the SAFA guidelines
foresee and assuage such problematic by including a number of sub-‐themes
specifically concerning small-‐scale producers and related indicators for assessment.
In such a way, despite the potential lack of existing data, resources, and capacity, as
well as the most-‐likely irrelevance of global indicators for small-‐scale producers,
SAFA claims that its inclusion of tailored indicators (practice-‐based or target-‐based
indicators rather than strictly performance-‐based ones) allows for a small-‐scale
entity to nevertheless achieve high-‐sustainability scores. However, we shall review
the actual effectiveness of SAFA for small-‐scale producers in the following chapter,
as such provisions were laid out ex ante and, therefore, require critical ex post
review in order to determine if SAFA is indeed a tool applicable to both large and
small-‐scale entities.64
In defining the boundaries and scope of SAFA for small-‐scale producers, the
objectives underlying such actors to partake in the assessment play an important
role in defining its scope. That is, SAFA acts as an incentivizing tool for small-‐scale
growers to achieve high performance ratings and, thus, bolster their market
position. The emphasis on “sustainable development,” as paralleled in TPR schemes,
whether through self-‐evaluation or delegated assessment may be desired by the
assessed entity insofar as “being sustainable” equates to credibility, accountability,
64 Please note that another version of SAFA for small-‐scale holders is expected to be released in November 2014. However, the newly adapted version should nevertheless be assessed in terms of legitimacy, accessibility, and effectiveness for small-‐scale producers. That is, does it provide access to data—the key weakness in small-‐scale sustainability assessments—in order to correct what was not working in the original “one size fits all” version of SAFA?
46
and transparency. Such qualities incentivize, for example, local procurement or
foreign investors to invest in “sustainable” companies, as our globalized economy
and transnational regulatory processes, increasingly call upon “sustainability” as the
foundation for a smart, green, and inclusive future. Furthermore, those small-‐scale
growers who may evidence their “sustainable” practices and high performance
ratings through SAFA or alternative sustainability reporting frameworks may profit
from compensation provided by local, regional, national, or international
governments and/or organizations (e.g. Payment for Ecosystem Services and the
development of local legislation with potentially direct effects on entities). Thus, the
incentives behind sustainable development are two-‐fold: both the regulator and
regulated, theoretically, benefit.
STEP #2: CONTEXTUALIZING DATA AND OUR READING OF IT
The necessity to adapt the small-‐scale producer to SAFA by virtue of assessing the
relevance of sub-‐themes and indicators, leads us to the second procedural step in
SAFA—that of “contextualization.” In order to refine the measurements and ratings
of a given assessment, the specific context of each entity must be considered; that is,
the geographical, social, and political background – in addition to capacity for the
entity to access data resources for the assessment itself -‐ must be considered. By
means of contextualization, the apt set of sustainability themes, sub-‐themes and
indicators can be determined in light of their relevance, consequently providing for
the most appropriate sustainability ratings. The main procedures in the
contextualization step involve the review of sub themes (based on their stated
boundaries and objectives) and the review of default indicators (based on their
relevance to the sub-‐themes selected). The act of contextualization contrasts the
“one-‐size-‐fits-‐all” approach of sustainability reporting, allowing for each assessment
to consider the pressing issues underlying a specific entity’s circumstance (e.g.
human rights violations, soil degradation, water scarcity in the area). If a certain sub-‐
theme is not considered relevant in determining the overall rating, SAFA requires the
elimination to be justified in the final Performance Report.
47
The act of reviewing default indicators involves a comprehensive collection of data in
order to “contextualize” the findings/ratings for each sub-‐theme. However, some
small enterprises may be faced with the dilemma of limited data and/or the lack of
resources to obtain data in the attempt to contextualize a default indicator, thus
prompting the entity to seek an alternative method for comparing data (for example,
GHG emissions or biodiversity) in relation to other entities in a similar or comparable
“context.” Although, in this hypothesis, GHG emissions data is not feasible for the
entity to obtain, the assessor may decide to compare practice-‐based indicators as a
proxy for performance—whether it performs better or worse in the given context.
Consequently, in the same way performance-‐based indicators yield reportable
results, the assessor can determine numerical values for its overall scoring on the
practice-‐based proxies, which is then visually expressed as ranging between degrees
of red (i.e. “unacceptable”) and dark green (i.e. “best”). Following such logic, the lack
of primary data does not disable sustainability reporting under the functionality of
contextualization in SAFA. Instead, the SAFA tool’s flexibility addresses such issue of
rigidity, favoring the tool’s accessibility.
Lastly, SAFA also provides for the possibility of creating “contextualized indicators,”
in which the assessor may not find the default indicator as fit. In such case, the
assessed enterprise must comprise a customized metric system for determining the
sustainability score in relation to a relevant sub-‐theme. In such a way, SAFA’s
“holistic approach” enables the assessor to identify “critical areas, based on
materiality principles of the context of that entity” (43), extending sustainability
reporting beyond the stock guidelines.
STEP #3: SELECTING APPROPRIATE TOOLS AND INDICATORS
Building upon the contextualized sub-‐themes from the previous step, the selection
of relevant tools and indicators, further distills the assessment of the sustainability
themes as a means of approaching a more “correct” score. The SAFA guidelines
define “tools” as the variety of commonly used measurement systems or assessment
48
techniques for different sustainability topics.65 The selection of the tools used in
assessment should be based on default indicators (or their substitute), the
availability of data on an entity’s performance, the budget for the assessment, and
the required level of compliance review. The selected tools or standards for data
collection and metrics for measurement should be included in the SAFA Performance
Report alongside the Accuracy Score.
As the type of data and method of data collection may vary among assessors, SAFA
codifies two different typologies: primary data (data collected, measured, calculated,
or estimated from the site associated with the unit processes within the system
boundary)66 or secondary data (data derived from an alternative source).67 In some
cases, existing databases or commonly used tools, metrics, and standards may
facilitate the data collection process, integrating already-‐existing data, avoiding the
duplication of results, and facilitating assessment. The Appendix A therein contains a
list of partner initiatives for existing data sources, which allow the SAFA framework
to serve as an “umbrella-‐like” structure, incorporating existing data reporting (such
as CSR audit reports or GHG Protocol Corporate Accounting and Reporting
Standards) while creating a common language and system of equivalencies in both
the data collection and reporting processes. The possibility of using/accessing
secondary data enables small-‐sized entities to reduce overall assessment burdens
and efficiently appraise its sustainability in order to identify potential “hot spots”
and implement improvement measures. Although primary data is certainly more
accurate, it is also more costly. Therefore, a tradeoff may occur as the user is offered
the choice to incorporate strictly primary data, secondary data, or a mix of the two,
as a means of reducing cumbersome data collection and fostering efficiency, while
nevertheless maintaining quality.
65 SAFA (2013), p. 45. 66 Based on the definition of primary data as found in ISO14044: 2006, which SAFA endorses. 67 idem.
49
Figure 4: Assessing data quality in SAFA
The notion of quality becomes prominent in the calculation of the Accuracy Score,
which rests upon the quality of data collected and the type of indicator used.
The quality of data, as set forth by SAFA, can be categorized “high quality” if primary
data is collected for the prime purpose of the SAFA assessment (and, therefore,
geared towards the 4 overarching sustainability dimensions), or if primary data
collected for another sustainability tool; “moderate quality” refers to secondary data
used as a proxy to make an informed assumption regarding enterprise performance;
“low quality” describes estimates made based on general information that are not of
primary or secondary data nature. Hence, depending on the time frame (i.e. when
the data was collected), the quality of data, and the methodology of data collection
(i.e. primary or secondary), the Accuracy Score can then be calculated for the final
SAFA Performance Report, which is attained by averaging the sub-‐theme, theme,
and overall levels.
SUSTAINABILY PERFORMANCE
In order to determine the level of sustainability performance, an entity should use
the “indicators” as measurements or assessments that provide evidence as to
whether or not a certain condition exists to calculate the Performance Report. The
default indicators were established according to what experts consider to be the
most important components of a sub-‐theme to be measured in order to assess
sustainability at that level. In such a way, default indicators provide standardized
metrics or guidelines for the future of sustainability assessment. SAFA provides for
the rating scale framework of the default indicators (i.e. the range between
50
“unacceptable” and “best” levels of sustainability performance), yet it does not
provide for the full rating scale. The full rating scale, hence, must be “contextualized”
by the assessor itself in light of the entity’s sub-‐theme boundaries and indicator
thresholds established.
Determining the effectiveness, or “weight,” of indicators serves to appraise their
accuracy by outlining thresholds in order to calculate the entity’s rating in the overall
Performance Report. The “hierarchy of indicators” (see Figure 5) indicates the
degree of quality in relation to each of the 3 types of indicators cited. That is, the
“performance-‐based indicators” are “focused on the results of compliance with an
objective and can measure the performance of an operation, identify trends, and
communicate results;”68 “practice-‐based indicators,” on the other hand, represent
process over outcome-‐oriented indicators, referring to compliance with certain
procedural requirements and, therefore, serve as proxies for “good” practice;
“target-‐based indicators” represent those which focus on the planning, policies or
monitoring of targets, where ratings depend upon the degree of progress vis-‐à-‐vis
implementation.
Figure 5: Hierarch of Indicators
68 SAFA (2013), p. 56.
51
The necessity of determining the thresholds – that is, the flexibility of sustainability
reporting in light of diverse contexts – directly correlates to the SAFA score, as a
“fair” and “balanced” assessment. In order to account for the varying “weight” of
indicators, ranging between the most accountable and accurate (performance-‐
based) to less accurate yet sound estimates (practice-‐based), allows SAFA to
differentiate among indicator types and allot varying degrees of “weight” to them. In
order to assure that all sub-‐themes are weighted equally (and effectively), the
weight must be distributed evenly among indicators within each sub-‐theme, for each
dimension. For example, the use of multiple indicators maximizes the potential
scoring; that is, the maximum potential score of 3 indicators related to a single sub-‐
theme allows for a “best” or dark-‐green score (i.e. the highest mark), On the other
hand, the use of only 1 indicator per sub-‐theme would not require any weighing
apparatus, which could potentially lead to limited assessment of the overall
sustainability dimension. The guidelines comprehensively detail how the SAFA
Performance Report should be drafted, located in the appendixes therein, enabling
flexibility by virtue of the variety of indicators and accounting for such differences by
virtue of aggregated weighing systems (see Figure 6).
Figure 6: Rating scale and weights of indicators
Thus, the calculation of multiple indicators and types in the final Performance Report
evidences the holistic approach proposed, presenting the SAFA tool as a landmark
advance in sustainability reporting, capable of addressing tradeoffs and
52
equivalencies; externalities and internal gaps; synergies and impacts; a tool for
improvement.
STEP #4: REPORTING PERFORMANCE IN A “SNAPSHOT”
The final step comprises a visual representation of the SAFA results, a complete SAFA
Performance Report, the disclosure of the procedure, and a critical review. The
“visual representation” protocol describes the attempt to condense the mass of data
collected for each of the 21 sustainability themes, 58 sub-‐themes, and hundreds of
related indicators into a single “snapshot,” visualizing the overall sustainability
performance and sustainability gaps in a graphic: a polygon. This process of data
visualization allows for an entity to “read” the conceptual framework and the
interconnected relations between them by identifying all 21 sustainability themes
around the circumference of the SAFA polygon. The degree of color disseminating
from the core establishes varying degree of performance – along the diameter
stemming from the nucleus – between “unacceptable” (at the core) to “best” (in the
outer ring). In such a way, a think black line connects themes, visualizing
performance by virtue of the user-‐friendly, color-‐coded ranges (dark
green/green/orange/yellow/red) as a means of depicting where performance is
good,
Figure 7: SAFA Polygon
53
The final report comprises an overall synthesis of the assessment, detailing the
scope, boundary setting, qualified themes’ ratings (and Accuracy Report), hotspot
issues, irrelevant sub-‐themes (and the justification of their omission), and areas to
be improved. The disclosure of the procedure used in the assessment speaks to the
accuracy of it, requiring a transparent documentation of the procedure and
methodology used. Such disclosure encompasses the selection of system
boundaries, indicators, and thresholds determined, the data sources (primary or
secondary), and the incorporation of any assumptions regarding the entity’s
performance. Such transparency calls upon the assessor to identify areas where lack
of data or resources were encountered, as well as its justification for excluding
certain sub-‐themes or the use of alternative contextualized indicators instead. The
Accuracy Report can serve as a form of self-‐evaluation for mere internal use or for
external sustainability reporting. In the latter, the transparency of all methodologies
– the boundary, threshold, and indicator settings – “accounts” for the
accountability” of the assessment itself.
Mirroring the procedural outlines of already existent sustainability reporting (e.g.
LCA, GRI, ISEAL Alliance), the act of critically reviewing compliance and indicator
selection/performance by a third party renders it ultimately more tangible, practical,
and “real.” That is, a critical review promotes the quality, credibility, and
transparency of the assessment, identifying areas of sustainable performance and
others in need of improvement, staking the SAFA tool as an instrument of and for
socio-‐political, economic, environmental, and regulatory transformation(s).
54
CHAPTER #3: TESTING SAFA VIS-‐À-‐VIS THE PILOT PHASE
The current version 3.0 of the SAFA Guidelines described above represents an
attempt to consolidate a notion of sustainability within a single, coherent
framework, building upon 3 years of interactive collaboration among public and
private actors in the effort to “distill” the meaning of sustainability, specifically, in
relation to the food and agricultural sectors. As multiple frameworks, standards,
indicators, and technical protocols for sustainable production, manufacturing, and
retailing of food and agricultural products exist, the SAFA initiative of 2009 marks the
first international attempt to harmonize such standards—that is, to construct a
conceptual framework of and for measuring sustainability.
The SAFA Guidelines today, hence, stem from the evolution of negotiations, cross-‐
comparisons, and collaboration among a variety of stakeholders, NGOs, public
institutions, corporate enterprises, and scientific institutions, accounting for the
publication of its first version – the Test Version 1.0 of the SAFA Guidelines – in June
2012. In order to “test” the applicability, usefulness, acceptance, and technically
soundness of the instrument, the FAO held a “Pilot Phase” between October 2012
and February 2013. Insofar as the guidelines are built upon the cross-‐comparison of
codes of practice, corporate reporting, standards, and indicators used by enterprises
and organizations that implement sustainability tools, the participants in the SAFA
trial constituted, then, a group of those very actors as a way for SAFA to “test” itself.
In the course of conducting research on the current SAFA tool, I was kindly granted
the opportunity to access the pilot studies, which allowed me to assess the
methodologies used, sub-‐themes and indicators selected, completeness (or
incompleteness) of the tool, degree of compliance (or incompliance), and method of
data collection and aggregation employed. Consequently, the present chapter shall
discuss the overall successes and shortcomings of the tool, as identified both within
the pilots themselves and by virtue of my observations herein. Noting recurring
obstacles and trends within the pilots, I shall set forth a qualitative assessment of the
failures and virtues, gaps and overlaps, potential areas of improvement and overall
55
effectiveness –and, consequential, utility – of the given instrument in assessing
sustainability.
SAFA “LIVE” FROM THEORY TO PRACTICE
As a tool representing an aim to create a collective understanding of the constitutive
elements of sustainability and a common framework for the assessment of it, in
theory, the pilot studies served to uncover the issues at stake, in practice. Such
studies encompass 23 small, medium and large-‐enterprises from 19 different
countries (i.e. Bangladesh, Bolivia, Canada, Costa Rica, Dominican Republic,
Germany, Ireland, Italy, Nepal, New Zealand, Peru, Sao Tomé et Principe, Spain,
Switzerland, Tanzania, Thailand, UK, and USA), spanning across various sectors,
including crops, livestock, forestry, fisheries, wild harvests, cotton, bioenergy,
tobacco, and peat moss commodities and value chains. Some pilots covered a variety
of processes from primary production to packaging, whereas others focused on a
single site or step in the given value chain, depending on the stated scope and
boundaries of the entity’s assessment.
The overall consensus among participants speaks to the tool as a useful instrument
for self-‐evaluation and internal monitoring as a means of identifying sustainability
“hot-‐spots,” in turn allowing for future improvement strategies. Other
commonalities found within the SAFA Performance Report and the feedback
sections underline the SAFA approach as comprehensive (i.e. extensive themes and
sub-‐themes), systemic (i.e. the mapping of an ecological-‐social-‐economic-‐
governance resilience network), and useful for internal sustainability evaluation (e.g.
the polygon enables a simple visualization of sustainability issue areas).
Despite the concurring attitude towards SAFA’s effort to create an overarching
framework for sustainability as a positive step forward, the majority of pilots (with
the exception of one) stressed multiple challenges encountered, highlighting
structural, operational, and conceptual “gaps” in the (then) current Test Version and
offering suggestions for its improvement in light of the exercise. In order to function
as both an effective and inclusive instrument, the tool would have to consider those
56
limitations and integrate their solutions into a new framework. The most prominent
challenges, which surfaced during the pilot phase expose the following
shortcomings: ineffective compliance checks, indicator challenges (boundaries,
thresholds, and types), inappropriate metrics, the issue of data collection, the
implications for small-‐holders, and the future utility of SAFA. Hence, we shall not
only identify the issues presented during the pilot phase but also inquire as to if and
how the writers of SAFA redressed the underpinning issues at stake in the current
version of the tool.69
SAFA “HOT-‐SPOT” #1: INEFFECTIVE COMPLIANCE CHECKS
As an umbrella framework, SAFA includes compliance checks in order to incorporate
existing sustainability schemes as a means of facilitating assessments by virtue of
benchmarking, consequentially avoiding the “double work” entailed in data
collection and sustainability reporting. This step (step 2) comprises 2 parts, in which
the entity must first determine if a sub-‐theme is “relevant” and, consequently, cite
the means by which it “complies” to the sub-‐theme indicator (e.g. listing the
alternative certification or standards scheme to which it adheres). However, such
relevance questions were often criticized as being “very general” and “not specific to
our area of concern” (as stated by a tobacco producer in Bangladesh), presenting the
entity with the choice of eliminating the sub-‐theme on the basis of irrelevance. As a
result, the following SAFA assessment could, potentially, not cover significant
sustainability dimensions by virtue of the justifiable or unjustifiable decision on
behalf of the assessor – a subjective decision – rendering the notion of relevance
immediately problematic. In the above case, despite the possibility of providing
contextualized indicators for compliance, the entity failed to do so, evidencing the
embedded “option” within the SAFA framework to construct alternative indicators
for compliance shall the entity have both the temporal and motivational conditions
to carry out a comprehensive analysis. In such a way, the relevance and compliance
69 One should note that the original Test Version 1.0 of the SAFA Guidelines comprised 6 steps: 1) Goals and Scope; 2) Relevance and Compliance Check; 3-‐4) Indicator Selection and Data Collection; 5a) Aggregation of results of SAFA themes and sub-‐themes; 5b) Visualization of SAFA results; and 6) SAFA Performance Report. The exercise also included a final “Feedback” section.
57
check may exclude sub-‐themes and indicators, which could potentially expose crucial
hot-‐spots and, in SAFA’s test version, run the risk of being overlooked.
Secondly, the notion of “compliance,” or benchmarking, reveals further obstacles, as
pilot feedbacks speak to how “existing schemes do not comply with indicator
structures,” how “some areas are not covered” in existing sustainability frameworks
(e.g. FSC does not include GHG balances), the difficulty of aggregating LCA data, and
the discrepancy between certification schemes, as not all sustainable practices are
certified (e.g. an Argentinean producer of organic pears and apples is “not always
organic,” resulting in a poor, skewed score under the “Agricultural Biodiversity” SAFA
indicator). In such a way, compliance based on un-‐guided benchmarking exposes
ineffectiveness, as the SAFA framework assumes optimal performance of the sub-‐
theme. However, differences in approach, scope, and scoring do not allow for
ultimate equivalency. A Quebec-‐based pork producer ultimately suggests, the
solution here is to “remove [the] compliance step.”
Figure 8: Entity compliance with SAFA sustainability guidelines
The problematic of compliance cannot only be located within the structural makeup
of the tool but also within the operational aspect; that is, how the entity performs
the compliance check. Reviewing the pilots, the majority of entities failed to follow
0
5
10
15
20
25
Step 1 Step 2 Step 3 Step 4 Step 5 Step 6
# of Completed Pilots per Step
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
58
the SAFA guidelines in its compilation of step 2, such as selecting “yes” to
compliance without providing the relevant standards (e.g. all 3 pilots performed by a
New Zealand Agribusiness on its organic dairy farms, wineries, and the Ngai tahu
Maori indigenous food programme selected all sub-‐themes as relevant despite many
of them being inaccessible). In addition to inappropriate selection and,
consequentially, distorted scoring, at times the sub-‐themes were considered
“relevant” in step 2, yet “inactive” in step 3-‐4, such as the sub-‐theme “Food Safety”
for the above-‐mentioned enterprise’s assessment of its wine production. In many
cases, such inconsistencies either led to the “best” score, wrongfully, or eliminated
rightful areas of assessment.
Furthermore, the use of compliance checks may create a structural bias towards
large companies, which have the capacity for adhering to multiple schemes of
sustainability. As an organic small-‐grains producer in North Carolina describes:
relevance check questions are poorly worded and exclude many themes that are relevant to all. Only clearly irrelevant sub-‐themes should be excluded. Compliance check will not capture true performance of the farm and will bias safa toward large users who hold many certifications or claims, regardless of the value or rigor of these claims. (Performance Report)
Insofar as the level of verification and degree of compliance among so-‐called
“compliant” alternative schemes are variable, no objective standard or common
ground for sustainability exists, potentially leading to the opposite and least
desirable result: incompliance. As the compliance check renders the value of SAFA as
a rigorous instrument for assessing sustainability questionable here, the SAFA
working group, which convened post-‐pilot phase, addressed this matter by
suggesting to “integrate the results from existing sustainability tools and certification
schemes into SAFA,”70 serving to maintain the integrity of the SAFA tool.
70 SAFA: A Long-‐awaited Step Forward to Understanding and Acting Upon Sustainability in the Food and Agriculture Sectors, 2013. Web. 14 November 2014. (available at www.sciforum.net/conference/wsf2/paper/959/download/pdf).
59
However, one may ask if this integration occurred and, if so, to what extent. The risk
of integrating unequal benchmarks and mis-‐allocating sustainability rigor comes into
question again, which should be explored within the new SAFA Version 3.0. That is,
were different schemes rated, scored, aggregated, prioritized, and/or disregarded?
How are the necessary boundaries of integration framed in the proposed solution?
And, most importantly, does the new SAFA solve the problem of benchmarking by a
selective approach towards integration? As the stated goal of SAFA is “improved
accuracy of analysis of sustainability for all users,”71 one may ask how the new SAFA
differs from the prior framework. In response to such challenge, the Appendix A of
SAFA Version 3.0 – “Selected Sustainability Tools” – provides for the scope of
selected tools as compared to the SAFA landscape, listing a variety of sustainability
tools (e.g. Footprint Calculators, LCA, VSS, GRI, etc.) and allocating compliance
against the “Steps of the Value Chain Impacts Covered” (i.e. production, processing,
and retail) and the “Sustainability Dimensions Covered” (i.e. environment, economy,
governance, and social). In such a way, the current version of SAFA evidences a more
selective approach to identifying the benchmarks for compliance.
SAFA “HOT-‐SPOT” #2: INDICATOR CHALLENGES
SETTING BOUNDARIES: SPATIAL AND TEMPORAL SPHERES OF INCLUENCE
The function of indicators to serve as metrics for actual performance – whether
appropriate or not to given sub-‐sectors and, consequentially, their utility – marks a
major shortcoming in the pilot SAFA framework. Although step 1 calls for the
identification of setting the material and spatial system boundaries of the SAFA
assessment, such process lacks any sense of guidance therein. That is, when
addressed with the question, “Which is the entity’s sphere of influence?” many pilot
studies struggled with the notion of distinguishing between processes along the
value chain in question, which is a fundamental step foreseen by SAFA protocol in
order to carry out a rigorous and scientifically sound impact assessment of entity
performance. As one of the pilot entity’s reported (a processor of organic fruits and
cereals), the assessor was “not sure where the sphere of influence ends.” Unable to
71 Idem.
60
determine the entity’s “sphere of influence” as allocated along the supply chain,
within the company as a whole, or at a certain processing level, they posed the
question, “Is it a farmer or company being assessed?” In such a way, the lack of
guidance leads to an unclear understanding of setting the very scope of assessment
itself, rendering the ultimate sustainability report as a misguided one. Similarly, as an
organic and fair trade coffee producer in Tanzania and Mexico pointed out, the
necessity to differentiate among each element of the supply chain posed a
“conundrum” to the assessor insofar as the spheres of impact allocation within each
step of the supply chain continually shift.
As a result, the necessity to “draw lines” in absence of SAFA guidance lends itself to
multiple interpretations of what defines operational boundaries. Failing to set
minimum requirements for these boundaries revealed a critical gap in SAFA’s ability
to capture negative externalities—and, therefore, assess their impact(s). As
described by the coffee producer in regards to GHG balances, “specific boundaries
are needed for this indicator,” underlining how “with carbon foot printing,
boundaries make all the difference.” Thus, the inability to capture externalities –
whether positive or negative – disables the tool’s efficiency, failing to account for
what lies at stake in sustainability assessment; that is, impacts and implications.
In a similar way, the issue of temporal boundaries emerged within the pilot studies
as well. As SAFA’s default temporal coverage is that of 1 year (with the potential
extension to 5 years for certain indicators), the attempt to assess ecological
processes however exposed another limitation to boundary-‐setting. That is, the
analysis of many ecological processes extends beyond the 5-‐year boundary (e.g.
forests, wetlands, and grasslands), complicating temporal dimensionality where
impacts cannot be seen for up to 20 years in some cases. In such a way, the arbitrary
setting of boundaries here vis-‐à-‐vis short ecosystem timelines risks to either
overlook potentially detrimental environmental impacts (e.g. land degradation and
biodiversity reduction) and/or fail to acknowledge long-‐term sustainability efforts
(e.g. GHG mitigation). As the Feedback Report from a New Zealand Agribusiness
mentions:
61
The 5-‐year stand down threshold period for production from natural habitat is very short indeed! The downstream ecological consequences are expressed in several decades. This may prevent use of SAFA to discourage habitat destruction (was that your intent), the SAFA releases the producers from sustainability score penalties after very little time, so consequences for the producer are very temporary.
Insofar as the SAFA settings limit ecological processes to low downstream
thresholds, the ultimate sustainability score cannot, de facto, be truly representative
of the actual situation, in turn failing to punish unsustainable practices, exempting
producers from maintaining sustainable performance, and ultimately undermining
the very goal of SAFA itself.
In response to such loophole, the SAFA working group proposed to make certain
indicators “measurable,” suggesting for the use of assessment kits for quantifying
environmental indicators, including reference to existing resources.72 Although the
current SAFA Version 3.0 does not include such “assessment kits,” the re-‐working of
indicators (now step 3) integrates metrics for data collection and dedicates a section
to the “Rating of sub-‐themes for [the] environmental dimension,” which differs from
the other 3 fundamental sustainability dimensions. The treatment of indicators
under the environmental dimension incorporates multiple indicators and indicator
types (whether performance, practice, or target-‐based), allowing for a more
comprehensive approach towards data collection and indicator selection. Lastly, the
omission of certain environmental indicators is allowed on the basis of justified
reasoning (if small-‐scale producers, for example, cannot access necessary data). At
the same time, sanctioning mechanisms (i.e. negative scores) are implemented for
cases, in which “regular enterprises omit relevant performance indicators” (63). Such
modifications speak to the evolution of the SAFA tool in the realm of environmental
indicator mapping; when boundaries are narrowed, SAFA aims to maintain its
integrity by virtue of transparent reporting.
72 idem.
62
BETWEEN FLEXIBILITY AND RIGIDITY: INDICATORS IN SEARCH OF A THRESHOLD
As SAFA seeks to guarantee a fair playing ground for all types of enterprises across
regions and sectors, such aim renders the delineation of thresholds problematic.
That is, flexibility offers the ability to address varied and multiple settings, whereas
rigidity seeks to uphold the rigor of SAFA sustainability reporting. Consequently, the
guidelines must find a balance between these two extremes, which represents a
common issue reported in the pilot studies. For example, a smallholder tobacco
producer in Brazil and Bangladesh called for “indicators to be minimized and
précised,” as a peat moss producer in Canada argued:
While it is relevant and essential to define strictly themes and sub-‐themes to assure a common understanding of the issues to be assessed, the proposed indicators are currently too specific and hence, too restrictive to cope with the different activities / practices / behaviours the organisation might have (step 3-‐4, column F). Similarly, proposed questions are often inappropriate (depending of the size of the organisation) or too specific to provide the flexibility needed to document the organisation's behaviour/pratice. We spent a lot of time trying to adjust the indicators/questions to fit with the existing information -‐ which however was both sufficient and relevant to cover the themes and sub-‐themes. More flexibility is hence needed at the "indicator" and "question" levels. (Step 6)
Hence, an observed tradeoff occurs between the general and the specific, in which
the issue of threshold subjectivity oscillates indicators between the realm of the rigid
(and, therefore, irrelevant or exclusive) and the realm of the flexible (and, therefore,
prone to unregulated openness). Furthering its argument, the peat moss producer
underlined how “most issues are common to all,” whether upstream or downstream
and among sectors (livestock, grain production, etc.), yet some are “specific and
should be treated as such.” Supporting such argument, the entity stressed how its
main activity – wetland management – is not covered by the SAFA indicators as is
and, therefore, recommends “sector specificities.”
The pilots reveal the failure to capture the full picture of sustainability despite the
comprehensive scope of SAFA themes and sub-‐themes, calling upon the need for a
core list of required indicators accompanied by precise guidelines for the
63
customization of sector-‐specific/region-‐specific indicators at different steps along
the value chain or level of production. Although the current version of SAFA provides
for review and contextualization of all 58 sustainability sub-‐themes and relative
default indicators, does this resolve the issue at hand? That is, although the
“contextualization” (current step 2) and “selecting tools and indicators” (current step
3) do provide, once again, for the possibility of contextualizing indicators according
to entity-‐specific operations, a suggested database for regional and sectoral-‐specific
data has yet to be created. Given such, are we not stuck in the same situation as
before? Is the new version of SAFA so different from its original form in this respect?
Although the guidelines provide for a broader explanation of default indicators, in
light of the fact that SAFA users may not have the technical knowledge to develop
the indicators themselves (e.g. a small-‐scale producer’s lack of capacity or
resources), the critical element – data – is not made available via a shared database
for contextualizing sectoral/regional/value-‐chain specific indicators. Hence, the
problem persists in the form of the unknown: user subjectivity, varied degrees of
expertise, and unregulated capture of the sustainability scope. Stuck, then, in the
same trappings, the current SAFA guidelines note that “expert knowledge and/or
technical expertise” (34) will most likely be required in order to properly
contextualize indicators, aiming towards, on one hand, a scientifically sound “use” of
the tool yet simultaneously undermining its stated claim to be a tool for all kinds of
potential users.
BLURRED THRESHOLDS: WHAT TO DO WITH THE SPACE-‐IN-‐BETWEEN
The implications behind the ambiguity of thresholds renders the SAFA tool
vulnerable, as indicators become the subject of interpretation and – in the case of
contextualization – subjective constructions(s). In order to analyze the customization
step in practice, the peat moss pilot exemplifies a case, in which customized
indicators were constructed for the environmental dimension, namely Air Pollution,
Water Quality Eco-‐Efficiency, and Waste Reduction and Disposal. For the given sub-‐
themes, the test version default questions for each were quantitative in nature; for
example, regarding waste reduction, the indicator posed the following question: “By
what percentage has the total amount of waste decreased over the last five years?”
64
The ability to customize such indicator however – whether appropriate or not – is
not conditioned by SAFA guidelines on how that customization should take place.
The entity’s tendency, then, to substitute the percentage-‐based questions for each
of the above sub-‐themes with ex ante indicators (i.e. whether the entity has put
actions plans for future mitigation or improvement in place), ultimately undermines
the aim of SAFA to assess an entity’s sustainability over a certain time period. In such
a way, the entity shifted its sustainability scoring from one based on performance
over time (in the past) to another based on improvement measures for the future,
thus modifying the underlying question at hand and potentially altering its score for
the better. Although the functionality of “customized indicators” is an important one
within the SAFA framework in the name of contextualization, the pilot studies reveal
the ever more important need for guidance in that process.
Similarly, other pilots evidenced blurred regulatory distinctions—that is, the
knowhow of where to draw the line. The feedback from a tobacco leaf producer in
Brazil sheds light onto this issue in reference to the sub-‐theme Community
Investment, in which the indicator poses the question, “What share of revenue is
invested into the maintenance or rehabilitation of common goods like soil, water or
forests at a community level?” Scoring below 20%, the entity challenged whether
this indicator should be based on a benchmarking scheme or market-‐based
reference point insofar as “20% of revenues invested in sustainability looks not that
bad, but is considered poor rate (red). What's the reference for what can be good or
bad?” Posing such question (rightfully so), the feedback underscores the implications
of word choice; default questions, hence, must consider what is implied by such
wording, as default indicators act as mediums by which scoring takes place. Another
example can be found in a New Zealand winery pilot regarding the sub-‐theme
Waste, in which the entity claims to fulfill the “best available technology” standard
by indicating how “very little waste” is produced, their practice being “landfill rather
than burn.” However, one may question whether a landfill should really represent an
appropriate benchmark for high sustainability performance within a framework such
as SAFA. Such example, again, points the necessity of re-‐considering threshold
65
design, which depending on the rigor and ultimate intended use of the SAFA may still
constitute an area of concern in the current version of SAFA today.
INDICATOR TYPES
Another common suggestion found among pilot participants was that of creating a
variety of indicator types, as conditioned by the size, scale, capacity, and resources
of the assessed entity and its ability to report upon performance-‐based indicators.
As the SAFA tool is designed to assess the performance of enterprises, many of those
enterprises – represented by virtue of the pilot as proxies – attested to the difficulty
of obtaining measurements for performance-‐based indicators. Examples include the
suggestion to use “practice” over performance scores since “more suitable at the
environmental level,” (a Canadian peat moss producer), the argument for “best
practices at times” (a pork producer in Quebec), the proposal for “more practice-‐
based indicators” (a small-‐scale fishery in the US), and the use of “general
knowledge” over performance-‐based indicators (an organic dairy farm in New
Zealand). In the attempt, then, to maintain the performance-‐based SAFA approach,
the post-‐pilot working group suggested to include a hierarchy of different indicator
types and to create an aggregated weighing system dependent on the type of
indicator selected. Following up on such recommendations, the current SAFA
Guidelines do indeed incorporate an indicator pyramid, detailing a comprehensive
scoring system based on the “type of data” (high, medium, or low quality) and type
of indicator used (performance, practice, or target-‐based). The current hierarchy and
aggregation of scores based on 5 (rather than the previous 4) sustainability
performance levels evidence the broadened structure of the current SAFA
guidelines, acknowledging the network of complexities in sustainability assessment.
SAFA “HOT-‐SPOT” #3: (IN)APPROPERIATE METRICS
The complexity of sustainability reporting can be observed in the intricate number of
SAFA sub-‐themes and indicators, attempting to “make sense” of what constitutes
sustainable performance. The appropriateness of the indicators, therefore, vary on a
66
case-‐by-‐case basis depending on the entity in question—its size, location, sector, and
scope of assessment. As discussed, SAFA attempts to cope with the issue of
relevance by virtue of its “relevance and compliance check,” (step 2) which unveiled
a number of functional gaps within the SAFA framework, ultimately calling upon a
revised understanding of how “relevance” and “compliance” should be evaluated.
The pilot studies served to uncover these “holes” in the system, as default indicators
often posed problems to the assessor in measuring performance for a given sub-‐
theme. We shall investigate a few of the most common challenges found in the pilot
phase and review how the new version of SAFA deals with these challenges.
Firstly, the question of full-‐cost accounting (G5.3, a governance-‐related indicator)
explores the notion of holistic management—a key component of assessing
sustainability through good governance. However, the inability or mere failure of
many of the pilot studies to complete this indicator question reflects two areas of
concern: the first, a question of feasibility and the other, a matter of effectiveness.
The former speaks to the lack of capacity for certain entities to quantify external
impacts (as faced by a coffee producer in Tanzania and a forestry enterprise in
Bolivia). As full-‐cost accounting accounts for a fundamental dimension of measuring
sustainability practices – as impacts are externalities carrying implications – SAFA,
then, is faced with the question of how to address this issue as one appropriate for
all users, thus speaking to the latter point. The Full-‐Cost Accounting indicator still
plays a part in the new version of SAFA, as representative of an effective metric for
sustainability, calling upon a further investigation of how SAFA addresses (or
overlooks) this issue as a collective one.
Another problematic indicator discussed was that of Greenhouse Gases (E1.1), which
we shall discuss in length in the following section regarding the issue of data
collection. However, the indicator presents not only a matter of data availability but
also one of boundaries. As evidenced by the indicator description stating
“Operations contain greenhouse gas emissions as much as possible,” one may object
as to the subjectivity of such measurement. That is, to reduce GHG “as much as
possible” is highly subject to interpretation, ultimately undermining the aim of
67
rigorous sustainability reporting by virtue of slippery boundaries here. Furthermore,
the test version asks users to identify the percentage share of reduction of emissions
over the past year in order to measure performance under the sub-‐theme Air
Pollution. However, the formulation of the question suggests that expansion (mere
company development) would entail a poor score for the company even if mitigation
programmes were successful and reduced emissions proportionally overall. In other
words, a well-‐performing company could score poorly under such formulation.
Again, wording can make all the difference in indicator appropriateness and, hence,
in the correct measurement of sustainable performance.
A third dimension, which proved extremely challenging throughout the pilot studies
was that of governance—one of the 4 pillars of sustainability. In order to analyze the
governance theme, the pilot study performed by a small-‐scale fishery in North
Carolina allows us to explore a few of the issues encountered. The first sub-‐theme,
Governance Structure (G1) assesses corporate ethics and due diligence in the form
of identifying mission statements, codes of conduct, risk management policies, and
stakeholder forums. However, such prerequisites may not be relevant to small-‐scale
farmers, for example, where such formal governance structures do not exist.
Instead, many pilots called for a second indicator, which was not size-‐based (and,
therefore, not only relative to big businesses) but one based on ethics, management
plans, missions, and/or visions as proxies for governance indicators. However, one
may argue here that “Due Diligence” represents a dimension (unlike “Holistic
Management” which is more demanding for small-‐holders) that is, de facto, always
relevant to all users and, thus, the current indicator question should not be altered.
Insofar as small businesses do have stakeholders and should have plans for
informing them of decisions made or including them within the decision-‐making
process, a potential solution to the “Due Diligence” indicator would be one of mere
modification, allowing for informal agreements to attest to good governance as well.
Another problematic governance indicator for the small-‐holder represented that of
Rule of Law (G4), asking the entity to state whether “mechanisms are in place to
ensure remedy, restoration and prevention of the infringement of national and
68
international law.” Insofar, however, as the majority of large operations would have
formal mechanisms and internal audits in place, the likelihood for small-‐holders is
very small on the other hand. As the small-‐scale fishery claimed, such indicator
therefore is “unfair to weigh,” as “no formal mechanisms [are] in place.” In light of
the inclusive nature of SAFA, including small-‐scale and large-‐scale enterprises alike,
the incorporation, then, of both formal and informal mechanisms, contracts, and
agreements is crucial in order to fully assess an entity’s performance in a fair and
balanced way. The current SAFA Guidelines addresses such challenges by re-‐
structuring G1, removing “Governance Structure” and replacing it with “Corporate
Ethics.” Accordingly, “Due Diligence” sub-‐theme rightfully remains in the current
guidelines whereas the wording under G4 was modified to ask if the company “can
provide evidence” as to the existence of remedy in the case of rights infringement.
Figure 10: Good Governance Themes in SAFA
The realm of governance represents the most unattended dimensions of SAFA
overall, as the majority of pilot studies opted out of governance indicators by virtue
of compliance checks, claiming the varied sub-‐themes as “irrelevant,” which often
remained either unjustified in the SAFA Performance report or considered
inappropriate to the small-‐holder. Considering such, the often overlooked dimension
here challenges the integrity of sustainability assessment overall, as regulatory
choices require that forms of governance be effective—and, therefore, provide for
# of Pilots with "Irrelevant" Governance Themes and Sub-‐
Themes
Checked Relevant
Checked Irrelevant
69
sustainability in the long-‐run. The crucial role, then, of the governance dimension
within SAFA recalls our earlier discussion of the choice of governance structures and
the regulatory process as a fundamental relationship under transnational
governance, implicating the overall effectiveness of the regulatory scheme and,
hence, highlighting the issue at stake in SAFA.
SAFA “HOT-‐SPOT” #4: DATA COLLECTION
As previously touched upon, the issue of data collection represents another “hot-‐
spot” within the SAFA Guidelines pilot version, as different approaches lend
themselves to different results, calling upon data collection guidelines and protocols
to ensure accuracy. Such coordination between existing data and secondary data
(i.e. data collected for previously existing sustainability schemes such as LCA, GRI,
etc.) could establish common taxonomy in the field of sustainability—one of the
main goals of SAFA in its effort to create an international, umbrella-‐like framework
for sustainability reporting. However, as discussed, the new version of SAFA lacks a
shared database for benchmarking locally-‐specific practices and secondary data,
namely sectoral and regional-‐specific data needed for those entities that struggle in
the process of data collection. As a result, small-‐holders for example may find
themselves in the same compromised position as before, lacking the means and
method for data collection. The final review process of SAFA, however, if intended
for an external audience (e.g. business-‐to-‐business or business-‐to-‐consumer
communication) does foresee a review of the SAFA assessment. We may once again,
though, question as to how data collection is provided for? How are the existing
meta-‐initiatives treated? Does coordination take place? Although Annex A of Version
3.0 does provide for a more comprehensive means of benchmarking secondary data
sources and, hence, reducing the issue of false equivalence, what steps have been
taken to resolve the issue of data collection in and of itself? Insofar as the capacity
and resources for data collection have not been facilitated vis-‐à-‐vis the existence of
such database, doesn’t such absence reveal the same shortcoming as before?
The most common issues attributed to the process of data collection as reported in
the pilot studies locate either the impossibility to quantify certain sub-‐theme
70
indicators and/or the mere lack of available resources for improving its sustainability
performance. The majority of pilot studies reported the lack of capacity to provide
for the environmental dimension, such as GHG footprinting. As evidenced in the
table below, 19 out of 23 pilot case studies failed to provide data regarding GHG
calculations, as some pilots noted that a quantitative analysis would be simply “too
expensive” (a Peruvian organic fruit farmer commented) whereas others lacked
“local-‐sectoral data” on GHG measurements (ranging from forestry operations in
Bolivia, a small-‐scale fishery in North Carolina, and to a small-‐holder farmer in
Nepal). In the case of unavailable data, entities can either omit the indicator (and,
therefore, not be penalized if such omission is justified) or “customize” the indicator
according to an alternative method of measuring the GHG balance, such as in the
case of a Vancouver urban enterprise with 3 farming operations. The pilot exerciese
exemplifies a strategic attempt to bypass the challenge of GHG measurement due to
the rigidity of the default question. Given the significant cost and time-‐consuming
nature of collecting primary environmental data (such as GHG inventories, soil tests,
biodiversity assessments, water testing, etc.), the entity developed a Rapid Impact
Assessment Matrix (RIAM) tool. Its function represents that of providing a company
with a better idea of how specific urban farming activities directly impact the
environment, allowing a company with more than one operation to break them
down into their smaller components. The RIAM presented 2 main questions:
(1) which specific farm activities have the greatest environmental impacts (negative, positive, neutral, unknown) and (2) what environmental areas should a specific farm prioritize to improve the sustainability of the enterprise? (ENV-‐RIAM)
The entity customized an alternative method for calculating problematic indicators,
such as GHG footprinting in this case, in a way that allowed for data collection and
comparison as a means for assessing its sustainability performance.
71
Figure 9: Availability of GHG Footprinting Data
Following the complexity of GHG indicators lays that of Energy Supply and,
specifically, the issue of non-‐renewable resources. Given the unequal distribution
and allocation of non-‐renewable resources, many entities could not perform well in
regards to such indicator. As the Bolivian forestry and sawmill operator reported,
“fossil fuels have no substitute in this area,” resulting in a poor/red SAFA score
(although the entity should have omitted the indicator in this case). Similarly, a
small-‐holder farmer in India stated, “no other options” were available. The inability
to achieve the highest level of sustainable performance vis-‐à-‐vis best practices,
often, may be due to the geographical (and underlying socio-‐political) conditions,
conditioning any given entity in space and time. Other challenging indicators include
Traceability, as “no publically available product information” was available for the
Brazilian tobacco smallholder, as well as the lack of sectoral data on employment in
the quantification of the indicator Living Wage for a coffee producer in Mexico.
Other pilot studies simply “guessed” no toxic inputs were used in their winery
operations despite having “no data.” That said, both the lack of data required to
complete the SAFA, as well as the lack of resources, such regionally and sectorally-‐
specific issues speak to the underlying need for flexible indicators and customization,
as well as stress the urgency for a shared database. However, as previously
discussed, such flexibility must be accompanied with specific guidelines in order to
Availability/SufAiciency of GHG Balance Data
Unavailable/InsufCicient
Available/SufCicient
72
upkeep the integrity of SAFA reporting in the wake of subjectivity, while broadening
options for those entities with less resources and capacity for data collection.
SAFA “HOT-‐SPOT” #5: IMPLICATIONS FOR SMALL-‐HOLDERS
Challenging the “inclusive” scope of the SAFA aim to function as a tool applicable to
both large and small-‐scale enterprises, the pilot SAFA version failed to cover the
rights and issues of the small-‐holder insofar as the scope of proposed indicators
were often deemed inapplicable or irrelevant. Additionally, the implied costs and
resources required to attain the data for the assessment itself often exceeded their
capacity, ultimately disconnecting the user from the tool. As an organic and fair
trade cocoa producer in São Tomé and Príncipe noted, many indicators were
“irrelevant for small-‐scale operations,” or “not useful at a farmer level,” according to
a small-‐holder tobacco producer in Brazil.
The concurred difficulty to perform SAFA at a “farmer-‐level” exposed the
incoherency within its claim that a sustainability assessment can be carried out at
the “site of action.”73 In response to such inadequacy, an aquaculture processer of
trout developed its own set of “appropriate” questions in the form of a
questionnaire to be used “on the field.” In such a way, the entity re-‐worked the sub-‐
themes and indicator questions found within SAFA in reference to the local and
sectoral indicators for sustainability reporting. However, if all small-‐holders were
required to undergo such an extensive (and potentially expertise required) task, the
utility of the SAFA tool for the small-‐holder, in effect, dramatically reduces.
Another common shortcoming encountered by small-‐scale enterprises was the lack
of regional-‐sectoral data, representing an overall “gap” within the SAFA framework
for potential users. As a small-‐scale tobacco producer in Brazil noted that “no
publically available product information” for traceability indicators were available,
others reiterated the lack of local data for quantitative monitoring and reporting on
GHG indicators. Others, such as a Swiss meat processing company, simply omitted
73 A statement attributed to a New Zealand based organic dairy farmer.
73
the Land indicator as the “entity is too small to deal with [the] topic.” The lack of
local-‐sectoral data parallels the lack of resources available for the small-‐holder as
well. As noted by a Canadian pork association, the farmer’s use of variety seed (a
locally GMO licensed variety) stemmed from the fact that “no non-‐GMO
conventional options are available!” In such a case, the entity would score poorly
under such dimension due to the lack of resources beyond its feasible reach,
highlighting one of the most common dilemmas for the small-‐holder in sustainability
assessments and calling upon the revision of the SAFA framework in order to
incorporate the issue of resource (and data) availability. Hence, the notion of
benchmarking represents an issue for small-‐holders insofar as an entity’s
sustainability “performance” may score poorly simply due to its inability to attain the
information or resources needed to either attest to its sustainability claims and/or
achieve higher degrees of sustainability performance. Hence, geographical
conditions – as well as the underpinning socio-‐political ones – often “condition” the
so-‐called sustainability of a small-‐scale entity under a system of metrics to which it
has no access (nor jurisdiction), ultimately penalizing it for matters beyond its
tangible control.
Thus, in light of the issues exposed – limited existing data, the inappropriateness of
proposed indicatory the lack of capacity to complete the assessment independently,
and the difficulty of the Excel worksheet interface – the SAFA working group
discussed the necessity of incorporating small-‐holders’ rights and changing the scope
of indicators in order to enable applicability. Considering the large number of small-‐
holders who partook in the pilot phase, as evidenced in Figure 11 for a mere
suggestive/representative sample, the role of the small-‐holder within SAFA cannot
be marginalized. Such leads one to ask, does the current version of SAFA address
these gaps? How does SAFA treat the use of performance-‐based and best practice
indicators in terms of the small-‐holder?
74
Figure 11: Small-‐Holder Sample of Pilot Phase
The current version of SAFA does aim to increase the applicability of the tool to
small-‐holders in a number of ways. Firstly, it defines “small-‐scale producers “by
virtue of 3 criteria conditions and restates its “aim to be applicable to both large and
small-‐scale enterprises,” (34) yet does not explain how the issues, which emerged in
the pilots are actually resolved. The new version includes “some sub-‐themes that
specifically address some of the concerns specific to small-‐scale producers, as well as
different types of indicators,” which as discussed previously provide for the a variety
of indicator types (performance, practice, and target-‐based) and aggregated
weighing systems. In such a way, the new version, theoretically, addresses the
necessity to incorporate alternative metrics for the small-‐holder in assessing its
sustainability. However, the underlying lack of data – in the form of a
shared/accessible best-‐practice and local-‐sectoral global knowledge database –
renders the new version of SAFA victim to the same deficit exposed in the original
one; that is, the crucial dimension of access to data perpetually plagues the ultimate
effectiveness of SAFA, specifically for the small-‐holder.
SAFA “HOT-‐SPOT” #6: FUTURE UTILITY?
Insofar as the stated objective of the test version of SAFA presented a tool to assess
an entity’s sustainability through self-‐evaluation and, consequently, for internal use,
Percentage of Small-‐Holders in Pilot Phase
Small-‐Holders
Other
75
many of the pilot studies raised the question of the added benefit of the given tool.
All but one of the pilot studies confirmed the use of the SAFA assessment to be
merely that of an “internal” one given the noted shortcomings identified throughout
the study and suggested areas for improvement (the only one to claim SAFA “ready
as is” happened to be the only SAFA to not score below good/best despite the
incorrect use of indicators throughout the pilot). That said, nearly all entities
questioned the added benefit – and, therefore, future utility – of SAFA as a tool for
assessing sustainable performance given its purely internal aim, considering that
other existing schemes provide for legitimate business-‐to-‐business (B2B) and
business-‐to-‐consumer (B2C) relationships and certification schemes. An organic
cereal and fruit processer in Germany claimed SAFA “would be good if B2C schemes
were set up,” since otherwise companies “will opt for other approaches.” The notion
of external use provides an incentive for users of the tool, as underlined by the
feedback found in the pilot studies. However, as another entity expressed, “If B2C,
SAFA runs the risk of ‘misusing’ the UN name for its own ends,” as SAFA represents
an “honesty based assessment.” Calling upon the need for an external auditor or a
third-‐party review of the SAFA assessment, most pilots concurred such suggestion in
fear of potential “green-‐washing” strategies. In such a way, good governance
improvement would facilitate the legitimate use of SAFA, providing the opportunity
for B2B sustainability relationships vis-‐à-‐vis transparent reporting.
The limited intended use of the SAFA Guidelines, in turn, restricted the potential
functionality of it, confining its use – and future utility – within a quasi-‐incentiveless
sphere. As one entity noted, the bottom-‐up approach of SAFA risks to “sit unused,”
given its “weak influence on decision-‐makers.” The given entity further called upon
sector-‐specific SAFAs. Considering the question of utility, the new version of SAFA
evidences the incorporation of such suggestions, allowing for two forms of
“reporting” in its final step. The introduction of a “critical review” foresees two
potential uses: Level 1 (internal use) and Level 2 (B2B or B2C). The second level, as
suggested, requires an external verification process, allowing for increased
legitimacy and, in turn, increased utility.
76
However, one may recall the “5W’s” mentioned in our discussion of transnational
regulatory schemes here, inquiring as to who does the monitoring and upon what
grounds are metrics selected for compliancy evaluation. Stressing the role of
legitimacy, the effectiveness of SAFA can largely be located within the dimension of
effective governance. In other words, simply looking at the board is not enough,
whereas it is in the very checks and balances of the board that makes all the
difference. In its current form, SAFA represents a complicated process of folding an
evaluation of effectiveness within an-‐other evaluation of effectiveness, creating a
potentially never-‐ending loop of evaluation(s), with the tendency to disconnect the
legitimacy, transparency, and overall effectiveness of the regulatory scheme from
the regulated entity in question. As SAFA aims to simplify and converge regulatory
schemes, it nevertheless faces a perpetual crisis of inter and intra-‐fragmentation.
The “sustainability,” then, of SAFA will lie in its ability to incorporate governance
within its structural, operational, and functional framework for effective policy-‐
making.
77
CONCLUSION
As discussed, the notion of “sustainability” marks a socially, politically, economically,
culturally, and linguistically charged metric for negotiating what lies at stake in
transnational regulation—a space and place for enacting regulatory
transformation(s). The SAFA tool set forth by the FAO represents a comprehensive
effort to systematically address the issue of sustainable development across sectors,
nations, and markets, paralleling the interdependencies of the supply chain with the
contingencies of transnational regulatory relationships. In such a way, FAO’s efforts
evidence a strategic approach of embedding voluntary standards within the
transnational regulatory process, subverting conventional schemes of competition
into ones of potential cooperation. The convergence of conventional domains of
conflict – such as trade and environment protection, or e-‐commerce and data
privacy – speak to the growing integration between regulatory regimes, enabling an
axis of cooperation over competition within the transnational sphere.
Thus, in a modern world plagued by extreme poverty, exponential population
growth, depleting natural resources, and rising global temperatures, the urgency of
sustainable development becomes ever more important, calling upon effective
regulation across multiple forms of governance and among regulatory relationships.
Insofar as the definition of “sustainability” – a slippery linguistic construction
reflecting a given socio-‐political agenda – perpetually escape the interpreter, it
becomes difficult to say if an entity performs sustainably or not, as one never seems
to “know” what sustainability actually means. Furthermore, we have witnessed an
inverse effect in the field of sustainable development; although the amount of
mechanisms, tools, and metrics for sustainability have increased over time,
sustainability in and of itself continually diminishes. FAO’s SAFA platform represents
an attempt to “make sense” of sustainability by virtue of disentangling its
constituent parts and re-‐constructing a comprehensive framework for its
constitution. Following in the footsteps of the efforts set forth by the FAO (and its
hopeful path ahead), this paper has sought to underscore the need for an integrated
approach towards sustainability reporting, encouraging the use of objective
78
evaluation mechanisms, sound ex ante standard-‐setting, ex post indicator-‐selection,
and comprehensive impact assessments in the evaluation of regulatory
effectiveness. Grounding the viability of sustainable assessment in the ability to
integrate its frameworks within the crucial dimension of governance and along the
axis of regulatory relationships, the future of sustainable development rests, then,
within this very crux—a space to negotiate not only what is sustainability but stake
why it is important.
79
BIBLIOGRAPHY Cafaggi, F. and A. Renda. 2014, Measuring the effectiveness of private regulatory organizations, forthcoming. Cafaggi, F. 2011a, New Foundations of Transnational Private Regulation, Vol. 38, “Journal of Law and Society,” EUI WP. Cafaggi, F. 2012b, The Enforcement of Transnational Regulation. Ensuring Compliance in a Global World, Edward Elgar Publishers. Cafaggi, F. 2013a, A comparative analysis of transnational private regulation: legitimacy, quality, effectiveness and enforcement, HILL Report, 2013. (available at http://www.ssrn.org). European Commission (2001), “Mandelkern Group on Better Regulation,” European Commission Communication, 13 November. European Commission (2014), “Regulatory Fitness and Performance Programme (REFIT): State of Play and Outlook,” COM (2014) 368 Final, European Commission Communication, 18 June.
FAO. 1989. Sustainable Development and Natural Resources Management. Twenty-‐Fifth Conference, Paper C 89/2 -‐ sup. 2. Rome.
FAO. 2012. Sustainability Assessment Food and Agricultural Systems Guidelines (Test Version 1.0). Food and Agriculture Organization of the United Nations, Rome (available at http://www.fao.org/fileadmin/templates/nr/sustainability_pathways/docs/SAFA_Guidelines_12_June_2012_final_v2.pdf). FAO. 2013. Sustainability Assessment Food and Agricultural Systems Guidelines. Food and Agriculture Organization of the United Nations, Rome (available at http://www.fao.org/fileadmin/templates/nr/sustainability_pathways/docs/SAFA_Guidelines_Final_122013.pdf). Koritza, L. 2013, Certification showing it’s worth the investment, Sustainable Brands. Web. 14 November 2014. (available at http://www.sustainablebrands.com). SAFA: A Long-‐awaited Step Forward to Understanding and Acting Upon Sustainability in the Food and Agriculture Sectors, 2013. Web. 14 November 2014. (available at www.sciforum.net/conference/wsf2/paper/959/download/pdf).