1

Sustainability adrift: An evaluation of the credibility of sustainability information disclosed by public organizations Soumaya Chiba, École nationale d’administration publique (ENAP), Québec, Canada David Talbot, École nationale d’administration publique (ENAP), Québec, Canada Olivier Boiral, Département de management, Faculté des sciences de l’administration, Pavillon Palasis-Prince, Université Laval, Québec, Canada Accepté pour publication dans : Accounting Forum, 2018, Volume 42, Numéro 4, pp. 328-340. Abstract This article aims to evaluate the credibility of information disclosed by public organizations in terms of sustainable development. It focuses on an under-studied aspect in the sustainability reporting literature—namely, the factors that may affect the credibility of disclosure practices. The study is based on a qualitative analysis of the sustainable development content of annual management reports disclosed by 113 ministries and public bodies in the province of Quebec, Canada. The findings shed more light on the main factors that affect the credibility of the information disclosed therein, particularly in terms of lack of transparency and flawed monitoring mechanisms. Keywords: Sustainable development; Public administration; Mandatory disclosure; Quality of information 1. Introduction Following the 2002 Johannesburg Conference, 106 governments have defined a national sustainable development strategy and set up structures to coordinate sustainability actions (Rydin, 2008). Public organizations have established targets and indicators in order to assess their achievements in sustainable development (Bell & Morse, 2001; Hák, Janoušková, & Moldan, 2016). To ensure accountability towards their stakeholders, organizations are also increasingly required to disclose information about their sustainable development performance (Niemann & Hoppe, 2018). In general, disclosing these outcomes promotes citizen participation (Farneti & Guthrie, 2009; Greco, Sciulli, & D’Onza, 2015; Sciulli, 2011) and increases employee commitment to achieving sustainability goals. The literature has also highlighted the role of stakeholder pressures, the search for social legitimacy, financial incentives, and the improvement of the organization’s image in the development of sustainability reporting (e.g. Bellringer, Ball, & Craig, 2011; Deegan, 2002; Stubbs, Higgins, & Milne, 2013; Talbot & Boiral, 2018). To enable the analysis and evaluation of an organization’s sustainability performance, the information disclosed in sustainability reports must, insofar as possible, comply with recognized reporting principles. The Global Reporting Initiative (GRI) is by far the most comprehensive and widely used reporting framework (Boiral, 2013, 2016; KPMG, 2017; Marimon, del Mar Alonso-Almeida, del Pilar Rodríguez, & Alejandro, 2012). The GRI has been adopted by the vast majority

2

of large organizations worldwide (KPMG, 2017), and it details a wide range of principles aimed at improving the quality of sustainability reporting. Among other things, reports must be balanced, accurate, and reliable (Global Reporting Initiative, 2004). These principles define the quality of the information disclosed in the report, which “enables stakeholders to make sound and reasonable assessments of performance, and take appropriate action” (Global Reporting Initiative, 2006, p. 13). However, a number of studies have criticized the quality of information in the sustainability reports of public organizations (Cassar, Conrad, Bell, & Morse, 2013; Greiling & Grüb, 2014; Guthrie & Farneti, 2008; Pires & Fidélis, 2012). Among other things, some authors claim that the information disclosed fails to give a balanced account of the three dimensions of sustainable development (Greiling, Traxler, & Stötzer, 2015; Guthrie & Farneti, 2008) or that it does not reflect the extent to which the targets set by certain public organizations are achieved (Del Sordo, Farneti, Guthrie, Pazzi, & Siboni, 2016). Some studies have even questioned the role and relevance of sustainable development indicators and their formulation; other studies have demonstrated that some indicators do not operationalize the dimensions of sustainable development (Cassar et al., 2013; Mazzi, Mason, Mason, & Scipioni, 2012; Pires & Fidélis, 2012; Russell & Thomson, 2009). Such considerations call into question the credibility of the information disclosed by public organizations. Credibility is an important determinant of the relevance of information to decision-making (Cash et al., 2003; Lemos & Morehouse, 2005; White et al., 2010). A lack of credibility may undermine the usefulness (i.e. usability and utility) of this information for public organizations and stakeholders (Cassar et al., 2013; Mazzi et al., 2012). For the purposes of our analysis, we define the credibility of the information provided in sustainability reports as its perceived trustworthiness, particularly in terms of transparency and the monitoring mechanisms employed. The main objective of this study is to evaluate the credibility of information on the sustainable development outcomes of government ministries and agencies in the province of Quebec (Canada) and to analyze the factors undermining its quality. The literature on sustainability disclosures has mainly focused on private companies (e.g. Boiral, 2013; Boiral & Henri, 2017; Talbot & Boiral, 2018). As a result, public organizations have been overlooked (Ball & Bebbington, 2008; Guthrie, Ball, & Farneti, 2010; Hahn & Kühnen, 2013). Moreover, most studies that focus on public organizations have analyzed information published on a voluntary rather than a mandatory basis (e.g. Adams, Muir, & Hoque, 2014; Del Sordo et al., 2016; Farneti & Siboni, 2011; Guthrie & Farneti, 2008; Lynch, 2010). The present study is among the first to examine factors that may affect the credibility of disclosure practices governed by a legislative framework. In particular, it assesses whether the information provided is actually sufficient to measure the degree to which sustainability goals are achieved. Most studies in this field aim to confirm that the information for specific indicators is indeed present (e.g. Del Sordo et al., 2016; Farneti & Siboni, 2011; Romolini, Fissi, & Gori, 2015). It has mainly been taken for granted that the indicators actually make it possible to measure the degree to which the targets have been met (Espeland & Sauder, 2008). In this sense, the present study contributes to the literature on the disclosure practices of public organizations by questioning whether the measurement mechanisms implemented provide stakeholders with credible and useful information on the degree to which the ministries and agencies of Quebec's public administration have achieved their sustainable development objectives.

3

The remainder of this paper is organized as follows. The first section is devoted to a review of the literature on the disclosure of sustainable development outcomes by public organizations. The second section describes the methodological approach of the study. The third section presents the main findings of the research. Finally, the last section of the article is devoted to the discussion of findings, contributions, and avenues for future research. 2. The quality and transparency of information on sustainable development outcomes High-quality information is essential to accurately assess the sustainability performance of organizations, facilitate decision-making, and inform stakeholders about an organization’s actual performance relative to its sustainable development commitments (e.g. Boiral, Heras-Saizarbitoria, & Brotherton, 2017; Diouf & Boiral, 2017; Fernandez-Feijoo, Romero, & Ruiz, 2014; Hahn & Lülfs, 2014; Michelon, Pilonato, & Ricceri, 2015). Information quality is generally associated with its transparency, which can be defined as “the complete disclosure of information on the topics and indicators required to reflect impacts and enable stakeholders to make decisions, and the processes, procedures and assumptions used to prepare those disclosures” (Global Reporting Initiative, 2006, p. 6). Various studies on sustainability reporting have highlighted a lack of quality and transparency, particularly in terms of balance, completeness, and reliability of corporate disclosures (e.g. Boiral et al., 2017; Cho, Laine, Roberts, & Rodrigue, 2015; Lyon & Montgomery, 2015). The information disclosed is often unbalanced and biased, and the reports are uncritical of negative events or outcomes that could undermine corporate image (e.g. Boiral & Henri, 2017; Laufer, 2003; Lyon & Montgomery, 2015). In the literature, the inferior quality of sustainability disclosures has been analyzed through the lens of neo-institutional and impression management theories (Adams & McNicholas, 2007; Cho, Guidry, Hageman, & Patten, 2012; Deegan, 2010; Hahn & Kühnen, 2013). According to these theories, sustainability disclosure is shaped by the search for corporate legitimacy rather than a desire to release reliable information. As a result, sustainability reporting tends to obfuscate negative events (e.g. accidental spills, environmental lawsuits, or conflicts with local populations) and to foreground positive outcomes that are likely to improve corporate image (Boiral, 2016; Cho, Michelon, & Patten, 2012; Hahn & Lülfs, 2014; Talbot & Boiral, 2015). Although the literature on sustainability disclosure practices in public organizations is much less developed than the literature on corporate sustainability reporting, a few studies have described serious shortcomings in the information published by public organizations—particularly poor coherence and incompleteness (Dagilienė & Mykolaitienė, 2015; Del Sordo et al., 2016; Greiling & Grüb, 2014; Guthrie & Farneti, 2008; Lynch, 2010). It seems that public organizations do not provide balanced information on the social, environmental, and economic dimensions of sustainable development in their reports, as most of the studied organizations disclosed information on just one or two dimensions (Greiling et al., 2015; Williams, 2015). Guthrie and Farneti (2008), who reviewed the sustainability reports of seven Australian public organizations, reported that most of the information disclosed pertained to social indicators. Similarly, Greiling et al. (2015) found that reports from German, Austrian, and Swiss public organizations disclosed unbalanced information on sustainable development and that economic indicators were emphasized in these reports. In sum, the reports published by public organizations often fail to

4

provide stakeholders with a balanced account of the degree to which the objectives falling under all three dimensions of sustainable development are met. Moreover, the information on sustainable development outcomes published by public organizations has been criticized for its lack of transparency. A number of studies (e.g. Del Sordo et al., 2016; Farneti & Siboni, 2011; Guthrie & Farneti, 2008; Romolini et al., 2015) have focused on compliance with the GRI, which is commonly used by private as well as public organizations (Domingues, Lozano, Ceulemans, & Ramos, 2017; KPMG, 2017). Their findings demonstrate that the reports of the public organizations studied did not include information on all the elements required by the GRI. A review of sustainability reports published by Italian public universities (Del Sordo et al., 2016) reveals that only 7.8% of the elements recommended by the GRI were present. Guthrie and Farneti (2008) similarly demonstrate that the reports of some public organizations contain only 20 of the 79 elements required by the Global Reporting Initiative (2006) for indicators related to the three dimensions of sustainable development. These studies demonstrate that the information disclosed in the reports of public organizations is incomplete and fails to comply with recognized reporting principles; this does not allow readers to determine the degree to which sustainable development objectives have been met. 2.1. The usefulness of the information provided by the sustainable development indicators The usefulness of sustainability information is related to its relevance and practicality for information users. This usefulness largely depends on both the quality and purpose of the indicators employed to measure sustainability performance. Among other things, the information provided by indicators is supposed to be used to support strategic planning processes (Hezri & Dovers, 2006, 2009; Singh, Murty, Gupta, & Dikshit, 2009), involve stakeholders (Holden, 2011), and serve as a calibration tool. However, a number of studies (e.g. Bell & Morse, 2001; Hezri, 2004; Holman, 2009; Krank, Wallbaum, & Grêt-Regamey, 2010; Lehtonen, 2013; Mazzi et al., 2012; Palme & Tillman, 2008) question the role and usefulness of information provided by sustainable development indicators. These studies demonstrate that indicators and the information they provide tend to be used to a very limited extent (Cassar et al., 2013; Pires & Fidélis, 2012, 2015; Pires, Fidélis, & Ramos, 2014). Cassar et al. (2013) examined the degree to which sustainable development indicators were used in Malta and concluded that the indicators were not used to support strategic planning processes. The information needed for the indicators was collected on an ad hoc basis: it was generated and used only for international projects, mainly for the purpose of Malta’s cooperation with other countries of the European Union. Pires and Fidélis (2012) reached a similar conclusion in their study, which assessed the role of sustainable development indicators in a municipality of Portugal; they found that the indicators were used primarily to assess organizational performance in sustainable development. This information was not used to involve external stakeholders or to stimulate learning experiences. As a result, indicators played a very limited role as tools of coordination and communication. Lehtonen (2013) also examined the role of indicators in the development of the UK energy policy and demonstrated indicators are rarely used to support decision-making processes and are poorly adapted to monitor the priorities set out in the energy policy. The information is instead politically used to legitimize decisions and actions already implemented. Similarly, Mazzi et al. (2012) assessed the role of sustainability indicators in supporting benchmarking processes in Italian municipalities. They

5

concluded that sustainability indicators could not be used to compare the outcomes achieved in different municipalities, largely owing to unclear definitions of the indicators. A number of deficiencies exist in the quality of the monitoring mechanisms used to assess organizations’ sustainable development performance: the indicators as defined lack relevance for local issues (Cassar et al., 2013; Da Silva & Shear, 2010; Mazzi et al., 2012), indicator-related targets are missing (Geng, Fu, Sarkis, & Xue, 2012; Rinne, Lyytimäki, & Kautto, 2013), and indicators are defined in a manner that does not allow them to actually measure the dimensions of sustainable development. Russell and Thomson’s study (2009) of Scotland's National Sustainable Development Strategy demonstrates that the indicators developed for this purpose do not allow the goals of the program to be operationalized and measured. The authors concluded that the indicators essentially make it impossible to evaluate a number of dimensions of sustainable development. A similar observation was made by Mazzi et al. (2012) who concluded that indicators used by Italian municipalities have a number of shortcomings as far as their relevance to local conditions is concerned. As a result, the selected indicators do not allow all of the municipalities’ objectives to be measured, nor do they measure or operationalize certain environmental dimensions. In this light, it appears that sustainable development indicators often do not allow reporting organizations or their stakeholders to measure the degree to which the targeted sustainable development objectives were actually achieved. Overall, the empirical literature on sustainability disclosure of public organizations is consistent with neo-institutional and impression management theories. In this perspective, sustainability disclosure is seen as a public relations tool mostly intended to improve organizational image rather than to transparently measure sustainability outcomes (Boiral, 2016; Cho, Guidry et al., 2012; Deegan, 2010; Hahn & Kühnen, 2013). The uncertain usefulness of indicators to a large extent results from the poor quality of sustainability disclosure from public organizations. This issue is also fueled by the lack of commensurability of sustainability information. Espeland and Stevens (1998) have shown that commensuration (i.e. the existence of common metrics to measure and compare different things) is a key feature of modern societies. Commensuration makes the development of rankings, ratios, comparisons, and performance evaluations possible. Although commensurability tends to be taken for granted, it often requires significant investment, as these authors highlight (Espeland & Stevens, 1998). People and organizations may resist commensuration because of its costs and political implications. In this perspective, the incommensurability of sustainability disclosure may be partly due to impression management strategies intended to avoid comparisons that could undermine the organization’s image and legitimacy. This type of strategy has been evidenced by Boiral and Henri (2017), who showed a lack of comparability of sustainability performance between organizations using the same indicators and reporting framework. 2.2. The credibility of sustainability disclosure for Quebec's public administration Overall, the deficiencies in the quality, usefulness, and commensurability of sustainability outcomes are interdependent and undermine the credibility of the disclosure practices of public organizations, as shown in the literature. However, previous studies have not assessed whether sustainability reports, or the information disclosed therein, make it possible to measure how well public organizations have reached their sustainability targets—which is precisely the aim of this

6

study. Most studies have examined the quality of information voluntarily published by public organizations on their sustainable development outcomes (Goswami & Lodhia, 2014; Guthrie & Farneti, 2008), and, as a result, shortcomings in the quality of information on these outcomes can be attributed to the voluntary nature of their disclosure. Adams et al. (2014) have suggested that public organizations do not sufficiently assess their sustainable development performance because no legislative framework exists to regulate this type of disclosure. However, in the Canadian province of Quebec, the disclosure of sustainable development outcomes is required by law. This context provides an opportunity—if not a need—to identify the factors that may affect the credibility of sustainable development disclosure practices required by legislative standards. Furthermore, most studies quantitatively analyze information on sustainable development outcomes disclosed by public organizations: they focus on the frequency with which information related to sustainable development indicators appears in the reports. Such studies conclude that public organization reports do not allow their performance, in terms of achievement of objectives, to be assessed, as these reports fail to provide information on indicators that measure some dimensions of sustainable development. Existing studies do not provide a qualitative assessment of the usefulness of indicators in measuring and evaluating the extent to which sustainable development objectives are achieved. This has generally been taken for granted: as tools of measurement, indicators are often considered to be mechanisms that generate valid and precise information (Desrosières, 2008; Espeland & Stevens, 1998). However, Porter (1996) has stated that measuring tools are assumed to be objective in a purely mechanical manner. The sustainability disclosure practices of Quebec public organizations make it possible to assess whether indicators actually allow one to measure the degree to which sustainable development objectives are achieved. In Quebec, disclosing such information is required by the Sustainable Development Act of 2006 (SDA), which defines the responsibilities of Quebec’s public administration bodies in terms of sustainable development. In particular, the SDA requires ministries and agencies to draw up sustainable development plans defining their objectives (National Assembly of Quebec, 2006, p. 11). Organizations affected by the SDA must describe their sustainability performance in relation to their sustainability goals in their annual management reports (National Assembly of Quebec, 2006, p. 12). These reports may be verified by the Quebec government’s Commissioner for Sustainable Development, although verification is not systematic and only certain organizations are verified annually. The annual management reports are generally disseminated via the ministries’ and public administration bodies’ websites (Secrétariat du Conseil du trésor, 2002, p. 16). As a result, 116 governmental ministries and agencies are required to publish their sustainable development outcomes in their annual management reports. At the time of the study, the SDA covered the practices of 21 public companies, 19 ministries, 66 governmental agencies, and 10 administrative tribunals (Gouvernement du Québec, 2015). The SDA therefore offers a suitable field to analyze the credibility of sustainability disclosure in public administration, particularly in terms of monitoring mechanisms and transparency. 3. Methodology The present article aims to identify the factors that may affect the credibility of the information disclosed by ministries and agencies of the Quebec provincial government regarding their sustainable development outcomes. This study is based on a qualitative content analysis of annual

7

management reports published by government ministries and agencies. This method of analysis is appropriate for the purposes of this study and is particularly relevant for research with a descriptive or exploratory aim (Hsieh & Shannon, 2005; Thomas, 2006). 3.1. Data collection Data collection was based on the compilation of annual management reports produced by the ministries and agencies of Quebec’s provincial government, which disclose their sustainable development outcomes through these reports (National Assembly of Quebec, 2006). This study includes the reports for the 2014–2015 fiscal year from the 116 public organizations covered by the SDA (see Table 1). As 2014–2015 was the last year of the National Sustainable Development Strategy (2008–2015), these reports illustrate the degree to which the sustainability goals set by the ministries and public bodies for this seven-year period were achieved. However, despite regulatory obligations, three organizations did not provide sustainable development information in their 2014–2015 annual management report. Table 1. Sample descriptiona

3.2. Data analysis The qualitative content analysis method consists of a systematic process of data classification and coding (Hsieh & Shannon, 2005, p. 1278). It is based on an inductive logic that aims to generate categories and themes from raw data (Thomas, 2006). Unlike quantitative content analysis, this method is not intended to determine the statistical significance of the text (Thomas, 2006). Rather, it consists in identifying units of meaning and then proceeding to their classification. This method is appropriate when the purpose of the research is descriptive in nature (Hsieh & Shannon, 2005, p. 1279), and it is also commonly used in studies evaluating the quality of sustainability disclosure (e.g. Boiral, 2013; Cho et al., 2015; Hahn & Lülfs, 2014; Talbot & Boiral, 2018). Qualitative content analysis is thus particularly relevant to assess the credibility of information on the sustainable development outcomes of the ministries and agencies of Quebec’s provincial government. For this study, categories were created on the basis of inductive logic. The codification process was not intended to impose preconceived concepts onto the analyzed text and, in line with most qualitative content analysis approaches (e.g. Boiral, 2013; Cho et al., 2015; Talbot & Boiral, 2018), categories were constructed from raw data through an iterative and inductive process. To facilitate the analysis, the sections devoted to sustainable development outcomes were extracted from the annual reports. Subsequently, these excerpts were exported to QDA Miner software

a In 2014, the Quebec government’s Commissioner for Sustainable Development evaluated the integration of sustainable development in four public organizations. The report notably reveals a lack of monitoring of sustainable development activities (Sustainable Development Commissioner of Quebec, 2014). Despite the serious shortcomings identified, there were no sanctions for these public organizations.

8

(version 5). A total of 1098 passages were coded. At the beginning of the analysis, twenty categories were created according to four factors. These categories were then adjusted, and some of them were merged. The process of creating, merging, or subdividing codes throughout the analysis is essential to ensure that it is grounded in the data; this method is commonly used in qualitative content analysis studies (e.g. Boiral, 2013; Kohlbacher, 2006; Talbot & Boiral, 2018). At the end of this process, five categories were created (inaccurate measurement, theoretical confusion, failure to describe monitoring mechanisms, non-representativeness of the presented outcomes, and failure to provide information; see Table 2) and grouped according to two main factors (flawed monitoring mechanisms and lack of transparency on sustainability outcomes). Two strategies were used to ensure the consistency of the categories created. The first consisted in defining the identified categories (Miles & Huberman, 2003), which makes the coding process more systematic. The category definitions were also discussed by the researchers, and five meetings were held to clarify these definitions. The second strategy aimed to confirm the mutual understanding of the different categories and consisted of double-blind coding (Thomas, 2006). To ensure the reliability of the coding process, double-blind coding was conducted on 30 annual management reports. As comparison of the coding performed by different coders revealed no major differences, the remainder was coded by one of the two coders. Table 2. Category definitions.

4. Findings The analysis of the annual management reports produced by the Quebec ministries and public bodies made it possible to identify a number of shortcomings in terms of the monitoring mechanisms applied and the completeness of the information disclosed. Two main factors that affect the credibility of information on sustainable development outcomes were identified:

• lack of transparency on sustainability outcomes; • flawed monitoring mechanisms.

4.1. The lack of transparency on sustainability outcomes In theory, indicators and targets are tools that can be used to assess sustainability performance. More specifically, targets are used to accurately define the outcomes the organization strives to reach (Government of Canada, 2015). Indicators, in turn, are measurement tools used to assess the degree to which the pursued objectives are achieved (Rey-Valette, Laloë, & Le Fur, 2007). Overall, the quality and transparency of defined indicators depend on various criteria, including

9

accessibility, relevance, measurability, methodological soundness, and focus on outcomes (United Nations Statistics Division, 2015). To be credible, information must also reflect material issues and outcomes. In practice, however, the information disclosed by the Quebec government’s ministries and agencies does not consistently reflect sustainability outcomes. This problem was observed in 70 reports (61.9%). Two main elements are responsible for this lack of transparency: (1) the non-representativeness of the presented outcomes, and (2) the lack of outcomes. The problem of the non-representativeness of the presented outcomes was identified in 59 reports (52.2%). In these cases, organizations submit information that does not allow the outcomes to be assessed. In some cases, information on sustainable development outcomes is presented in a vague and imprecise manner and is not specifically related to the indicators. Instead, organizations provide detailed information of questionable relevance and usefulness. The following excerpt illustrates how Organization 17 (Montreal Museum of Contemporary Art) disclosed information that does not consistently reflect their defined indicators:

The organization did not specify the number of volunteers or trainees involved, nor did it indicate the annual number of volunteer work hours. It only mentions that “a number of trainees” were employed. The information submitted by this organization is thus vague and imprecise, and does not allow the reader to assess whether, and to what extent, its targets have been achieved. Other organizations provided information on new activities unrelated to the previously defined indicators. Describing these new activities does not allow the achievement of the pre-established targets to be measured. In these cases, the results provided do not illustrate the extent to which targets were met, as information specific to the defined indicators is absent. The following excerpt illustrates how Organization 13 (Superior Council of the French Language) provided outcomes unrelated to their previously defined targets and indicators:

10

The organization has presented information on activities not related to their indicators. The pre-defined indicators are the percentage of employees reached in awareness-raising activities and the percentage of employees with knowledge of the organization’s sustainable development approach. At the time of disclosure, Organization 13 (Superior Council of the French Language) presented information on a number of actions implemented that does not inform stakeholders about the extent to which their targets were met. Nor does this information indicate whether awareness levels were increased for 90% of staff or what percentage of employees gained knowledge on their sustainable development approach. The presentation of information that is not specific to their indicators greatly limits the relevance and usefulness of the disclosed outcomes. The second element that demonstrates that the information provided in the reports lacks transparency concerns the failure to provide information on sustainable development outcomes. This problem was identified in 43 reports (38%). In these cases, the organizations decided not to disclose information on the degree to which certain targets were achieved: they defined monitoring mechanisms but did not provide information on their outcomes. In some cases, organizations explained the lack of information on sustainable development outcomes by stating that this specific target was cancelled. Such explanations were provided in 15 reports (13.3%). For example, Organization 27 (Cultural Enterprise Development Corporation - SODEC) did not submit information on one of their targets, explaining that this specific target was dropped: “After consultation with the various stakeholders of SODEC, this action was withdrawn, since it is implicitly a part of all of SODEC’s activities.” In this case, the agency questioned the relevance of its own established target. In other cases, organizations justified the failure to disclose the outcomes by data unavailability. Such justifications were provided in 11 reports (9.7%). For example, Organization 49 (Québec Health Research Fund) justified the lack of outcomes by stating that “information on the number of research personnel is not available, as the current register does not provide this information.” Some organizations did not provide any explanation whatsoever for the failure to submit information on sustainable development outcomes. This problem was identified in 25 reports (22.1%). 4.2. Flawed monitoring mechanisms Monitoring mechanisms are essential to providing useful information on sustainability performance. In most cases, the monitoring mechanisms used by the Quebec ministries and public

11

bodies are unclear and inconsistent. This issue was observed in 98 annual management reports (86.7%) and is related to the following: (1) inaccurate measurement, (2) theoretical confusion, and (3) failure to describe monitoring mechanisms. The first element is related to the numerous inaccuracies observed in the measurement tools. This problem was identified in 73 reports (64.6%). Inaccuracies were found in both the sustainability objectives and the indicators. In some cases, organizations set imprecise targets that did not allow outcomes to be specifically defined. Theoretically, targets are used to facilitate the comparison between the intended outcomes and those actually achieved, which is the basis for evaluating and assessing performance in terms of sustainable development (Government of Canada, 2015), and targets also help to operationalize general objectives (Hák et al., 2016). In this light, the failure to clearly define specific targets in terms of expected outcomes limits the possibility of evaluating the outcomes achieved. For example, Organization 20 (Metropolitan Transportation Agency) has set a target of “optimizing elements favorable to sustainable development in projects.” Such a target does not specify the type and number of elements to be included in these projects, nor the number of projects in which these elements have to be included. It cannot be used to compare the expected outcomes to those achieved. As a result, performance cannot be evaluated, as there is no basis for comparison. Inaccuracies were also identified in the form and definition of the indicators. In some cases, indicator definitions lacked information on the method or units of measurement. Theoretically, indicators are supposed to be used to measure outcomes (United Nations Statistics Division, 2015) so that the degree to which the pre-established targets were attained can be evaluated. Defining indicators in such a manner that they do not allow objectives to be measured hampers the evaluation of the sustainable development outcomes. For example, Organization 63 (Ministry of Education and Higher Education) defined one indicator as follows: “government support for funding sports infrastructure.” This indicator does not include how sports infrastructure funding is to be measured, and it does not require information related to the amounts allocated for financing sports infrastructure. Therefore, this indicator does not mention or necessitate information that would make it possible to evaluate the degree to which the target is achieved. The second element concerns a theoretical confusion regarding indicators and targets. This problem was identified in 14 reports (12.4%). In these cases, organizations confused targets and indicators, mistaking the definitions of both terms. Some indicator definitions are more similar to targets. For example, in some cases, the indicator specified the level of expected outcomes, even though indicators are not supposed to include such information. It is rather targets that are meant to specify the level of performance the organization is aiming for (Government of Canada, 2015). The following excerpt illustrates the theoretical confusion of Organization 29 (Ministry of Employment and Social Solidarity) in terms of some indicator definitions:

Cases of theoretical confusion regarding targets were also identified. In these cases, organizations’ targets look like indicators. These targets make it possible to operationalize the general objective

12

but do not define the level of expected outcomes. For example, the Ministry of Transport (Organization 73) defined a target as “the number of eco-responsible practices.” This does not indicate how many eco-responsible practices the department plans to implement; it only specifies the unit of measurement for the overall goal. Theoretical confusion regarding the definition of certain targets significantly limits the possibility of evaluating the organization’s performance levels. The third element concerns the failure to describe targets and indicators for all activities pursued in the area of sustainable development. This problem was identified in 58 reports (51.3%). In some cases, the organization did not provide a specific target for an activity for which an indicator had already been defined. As the targets are missing, it is not possible to compare the obtained and intended outcomes or the extent to which objectives have been achieved. For example, Organization 110 (Executive Council Office) did not provide a specific target for one of its activities. The following excerpt illustrates how the information provided does not allow the outcomes to be evaluated, because the point of comparison (target) is missing:

The organization did not define the amount of money that it planned to invest in infrastructure development in Quebec’s aboriginal communities or its targeted performance. As a result, even when the information about the invested sums is provided, it is impossible to assess the degree to which the organization achieved its targets for this activity. In other cases, organizations did not provide indicators for activities for which targets had been set. Indicators are generally used to measure the degree to which the pursued objectives are actually achieved and are supposed to require information that allows performance levels to be assessed. Failure to provide an indicator makes it impossible to assess outcomes. The following excerpt illustrates how the information provided by Organization 82 (Office for Consumers) does not allow the extent to which the target was actually achieved to be measured, as the indicator is missing:

13

The information disclosed by Organization 82 (Office for Consumers) does not mention any indicator. Instead, it provides details that do not inform stakeholders about the degree to which the pre-established target was achieved. Given the absence of any indicator, it is impossible to quantify the target. There is limited possibility of evaluating the extent to which the target set by Organization 82 (Office for Consumers) has been achieved. Other organizations did not provide targets and indicators for all their planned activities. In these cases, the information on sustainable development outcomes cannot be assessed because the evaluation criteria are missing. The usefulness of the provided information is limited, as the information relates to achievements that cannot be evaluated. The following excerpt illustrates how Héma-Québec (Organization 92) disclosed information that does not make it possible to assess the obtained outcomes:

The organization does not specify, for example, the number of training activities it plans to organize or the percentage of staff targeted by its awareness campaign. As a result, the presented outcomes cannot be evaluated owing to the failure to define targets and indicators. 4.3. Impact of organization type on problems of credibility One could assume that the credibility of the information presented depends on the type of organization. However, flawed monitoring mechanisms and a lack of transparency on sustainability outcomes were observed regardless of the type of organization considered. In fact, findings show no significant relationship between the type of organization and factors affecting the credibility of the information. For example, these reporting problems are just as significant in the reports of ministries and the reports of public companies. Table 3 presents a summary of the factors observed in relation to the type of organization. Table 3. Presence of factors by type of organization.a

14

5. Conclusion The present study aimed to identify the factors that could affect the credibility of information on the sustainable development outcomes in the reports of Quebec’s provincial government’s ministries and agencies, particularly in terms of transparency and monitoring mechanisms. The factors identified in this study demonstrate that the disclosed information is inconsistent and incomplete. They also point to significant deficiencies in the quality of the monitoring mechanisms used to measure sustainable development outcomes; these deficiencies make it impossible to evaluate the sustainable development objectives outlined in their reports. In a number of cases, the defined indicators did not comprise any means of measurement, and the established targets did not allow the expected outcomes to be defined. These shortcomings significantly decrease the possibility of evaluating and assessing the degree to which sustainable development objectives were actually achieved. The impossibility of evaluating sustainability performance does not solely result from this lack of transparency, but also results from inconsistencies in the quality of monitoring mechanisms. These measurement mechanisms were affected by theoretical inaccuracies and confusion. In general, indicators and mechanisms for monitoring sustainable development outcomes are considered to be tools to streamline and create objectives for evaluation and decision-making processes (Desrosières, 2008; Hezri & Dovers, 2006; Porter, 1996). They are intended to enable the measurement and assessment of targeted sustainability goals. However, the results from the present study demonstrate that the information generated by measurement mechanisms is not necessarily valid and accurate. The use of some measurement mechanisms is not in itself sufficient to ensure that sustainable development outcomes can be evaluated and measured for the ministries and agencies of Quebec’s provincial government. Overall, the credibility of the information disclosed on sustainable development outcomes can be called into question, as necessary information is often missing, inconsistent, or generated by deficient measurement mechanisms. As a result, the sustainability disclosures of Quebec’s ministries and public bodies are deficient and do not inform stakeholders about the organization’s sustainability performance. 5.1. Contributions The present study makes three important contributions to the literature on the sustainable development disclosure practices of public organizations. First, the results of the study question the credibility of the information given for the indicators used to measure performance relative to sustainable development objectives. This confirms what

15

other studies have also found: the information disclosed by public organizations about their sustainable development outcomes lacks transparency (e.g. Del Sordo et al., 2016; Farneti & Siboni, 2011; Guthrie & Farneti, 2008; Romolini et al., 2015). A number of studies have already questioned the relevance and usefulness of sustainable development indicators (Bell & Morse, 2001; Hezri, 2004; Krank et al., 2010), citing the low degree to which these are used to support decision-making processes (Cassar et al., 2013; Pires & Fidélis, 2012, 2015; Pires et al., 2014), and the fact that they do not allow all the dimensions of sustainable development to be operationalized (Mazzi et al., 2012; Russell & Thomson, 2009). The present study demonstrates that the relevance and usefulness of defining sustainable development indicators can also be questioned on other grounds, notably by the fact that the indicators used do not allow sustainable development objectives to be measured and evaluated. Overall, issues related to the quality of monitoring mechanisms used to measure sustainability goals were identified in 86% of the reports. In 50% of the reports, the defined indicators did not include any means of measurement, and thus do not allow the degree to which the pre-established targets were actually achieved to be measured. The present study demonstrates that the use of indicators does not necessarily make it possible to evaluate sustainable development outcomes. It also demonstrates that the credibility of information on outcomes should not be taken for granted. Second, the present study examines the information on sustainable development outcomes disclosed in the context of legislative requirements. Most research in this field has focused on the quality of voluntarily published information, and some authors have attributed the limited use of monitoring mechanisms to the voluntary nature of the disclosure process (Adams et al., 2014). The present research demonstrates that, even when the disclosure of sustainable development outcomes is required by a legislative framework, its practice is characterized by a number of shortcomings in terms of transparency, indicator quality, and monitoring mechanisms. This finding suggests that a legislative framework regulating sustainable development reporting does not necessarily ensure the quality of the information disclosed or the monitoring mechanisms used. In addition, the present study allows us to better understand the nature of the factors affecting the credibility of sustainable development disclosure for the ministries and agencies of Quebec’s provincial government. This study demonstrates that these shortcomings are related to a lack of transparency and the quality of the monitoring mechanisms used. Understanding the nature of the factors affecting the credibility of sustainable development disclosure practices helps us to better understand what improvements are needed. Third, this study contributes to the literature on the neo-institutional and impression management approaches to sustainability disclosure. Most studies in this area have demonstrated that institutional pressures and a search for social legitimacy are the main drivers of sustainability disclosure and that sustainability disclosure is mostly intended to improve organizational image through impression management strategies (e.g. Boiral, 2013; Cho, Michelon et al., 2012; Hahn & Kühnen, 2013; Perego & Kolk, 2012). However, studies with this theoretical framework have mostly focused on private organizations. Furthermore, the role of monitoring mechanisms and measurement tools has clearly been overlooked in the literature. This study suggests that inconsistencies and shortcomings in monitoring mechanisms can be used as a sort of smoke screen to limit or prevent the measurement of sustainability outcomes and their comparison over time or between public organizations. In this perspective, the lack of transparency and rigorous monitoring mechanisms could be used as an impression management strategy to obscure under achievements

16

in terms of sustainability performance. This strategy can explain why organizations have not developed sustainability indicators consistent with their objectives and associated outcomes. By releasing opaque and incommensurable information, public organizations greatly hinder attempts to monitor and compare performance—which could otherwise result in social pressures to implement more substantial and potentially more costly sustainability measures. In many contexts, the establishment of government structures to achieve sustainable development goals has been described as symbolic (Baker, 2007; Happaerts, 2012a, 2012b; Meadowcroft, 2007). Such structures have been considered as instruments that mainly allow governments to demonstrate a political commitment and to respond to external pressures (Meadowcroft, 2007; Newig, 2007). Some authors have argued that sustainable development ministries are unable to truly influence the practices of other public organizations (Russel, 2007; Volkery, Swanson, Jacob, Bregha, & Pintér, 2006). The weak authority of these sustainable development ministries can be partly explained by the low budgets allocated to them (Steurer & Martinuzzi, 2005). In Quebec, the budget devoted to the activities of the Ministry of Sustainable Development represents only 0.2% of all expenditures (Treasury Board of Quebec, 2017). This low budget could arguably demonstrate that achieving sustainable development objectives is not a priority for the Quebec government. 5.2. Limitations and avenues for future research This research has certain limitations. First, the qualitative content analysis method does not allow the identified factors to be generalized to other public organizations in different institutional contexts. Moreover, qualitative approaches are not suitable for measuring tendencies and differences that depend on contextual variables. A quantitative content analysis based on a larger sample could shed more light on possible differences between different types of public organizations (e.g. public companies, ministries, governmental agencies, cities) or between the public organizations of different countries. The findings of the present study are therefore specific to the ministries and agencies of Quebec’s public administration. Second, the data collection was based on the study of annual management reports for the 2014–2015 fiscal year. As a result, this study does not evaluate possible improvements in the quality of information related to sustainable development outcomes. Further research is needed to assess the evolution of the credibility and information quality over a number of financial years. It would also be relevant to study how the users of this information perceive the credibility of sustainability disclosure. More in-depth studies are also necessary to examine the reasons underlying the lack of transparency and the issues related to the quality of monitoring mechanisms. Such studies should be based on methods that capture the perceptions of public managers involved in the development and the use of sustainable development indicators. Finally, it would be interesting to better understand the process of preparing sustainable development plans in public organizations as some facets of this process, such as how issues are identified and prioritized, could influence the relevance and credibility of the indicators used. Likewise, there is a need for further study of the motivations underlying the selection of indicators in sustainable development plans. The inconsistencies and inaccuracies in the indicators used to support the planning process could be related to various factors, including the lack of adequate measurement tools, misunderstanding of existing reporting standards, or deliberate impression management strategies to obscure unfavorable outcomes.

17

References Adams, C. A., & McNicholas, P. (2007). Making a difference: Sustainability reporting, accountability and organisational change. Accounting, Auditing & Accountability Journal, 20, 382–402. Adams, C. A., Muir, S., & Hoque, Z. (2014). Measurement of sustainability performance in the public sector. Sustainability Accounting, Management and Policy Journal, 5, 46–67. Baker, S. (2007). Sustainable development as symbolic commitment: Declaratory politics and the seductive appeal of ecological modernisation in the European Union. Environmental Politics, 16, 297–317. Ball, A., & Bebbington, J. (2008). Accounting and reporting for sustainable development in public service organizations. Public Money and Management, 28, 323–326. Bell, S., & Morse, S. (2001). Breaking through the Glass Ceiling: Who really cares about sustainability indicators? Local Environment, 6, 291–309. Bellringer, A., Ball, A., & Craig, R. (2011). Reasons for sustainability reporting by New Zealand local governments. Sustainability Accounting, Management and Policy Journal, 2, 126–138. Boiral, O. (2013). Sustainability reports as simulacra? A counter-account of A and A+ GRI reports. Accounting, Auditing & Accountability Journal, 26, 1036–1071. Boiral, O. (2016). Accounting for the unaccountable: Biodiversity reporting and impression management. Journal of Business Ethics, 135, 751–768. Boiral, O., & Henri, J.-F. (2017). Is sustainability performance comparable? A study of GRI reports of mining organizations. Business & Society, 56, 283–317. Boiral, O., Heras-Saizarbitoria, I., & Brotherton, M.-C. (2017). Assessing and improving the quality of sustainability reports: The auditors’ perspective. Journal of Business Ethics. https://doi.org/10.1007/s10551-017-3516-4 (in press). Cash, D. W., Clark, W. C., Alcock, F., Dickson, N. M., Eckley, N., Guston, D. H., et al. (2003). Knowledge systems for sustainable development. Proceedings of the National Academy of Sciences, 100, 8086–8091. Cassar, L. F., Conrad, E., Bell, S., & Morse, S. (2013). Assessing the use and influence of sustainability indicators at the European periphery. Ecological Indicators, 35, 52–61. Cho, C. H., Laine, M., Roberts, R. W., & Rodrigue, M. (2015). Organized hypocrisy, organizational façades, and sustainability reporting. Accounting, Organizations and Society, 40, 78–94.

18

Cho, C. H., Guidry, R. P., Hageman, A. M., & Patten, D. M. (2012). Do actions speak louder than words? An empirical investigation of corporate environmental reputation. Accounting, Organizations and Society, 37, 14–25. Cho, C. H., Michelon, G., & Patten, D. M. (2012). Impression management in sustainability reports: An empirical investigation of the use of graphs. Accounting and the Public Interest, 12, 16–37. Da Silva, S. E., & Shear, H. (2010). Great Lakes environmental indicators and state of the environment reporting: Use, needs, and limitations. Local Environment, 15, 699–716. Dagilienė, L., & Mykolaitienė, V. (2015). Disclosure of social responsibility in annual performance reports of universities. Procedia-Social and Behavioral Sciences, 213, 586–592. Deegan, C. (2002). Introduction: The legitimising effect of social and environmental disclosures-a theoretical foundation. Accounting, Auditing & Accountability Journal, 15, 282–311. Deegan, C. (2010). Organizational legitimacy as a motive for sustainability reporting. In J. Unerman, J. Bebbington, & B. O’Dwyer (Eds.). Sustainability accounting and accountability (pp. 146–168). New York: Routledge. Del Sordo, C., Farneti, F., Guthrie, J., Pazzi, S., & Siboni, B. (2016). Social reports in Italian universities: Disclosures and preparers’ perspective. Meditari Accountancy Research, 24, 91–110. Desrosières, A. (2008). Pour une sociologie historique de la quantification. Paris: Presses des Mines. Diouf, D., & Boiral, O. (2017). The quality of sustainability reports and impression management: A stakeholder perspective. Accounting, Auditing & Accountability Journal, 30, 643–667. Domingues, A. R., Lozano, R., Ceulemans, K., & Ramos, T. B. (2017). Sustainability reporting in public sector organisations: Exploring the relation between the reporting process and organisational change management for sustainability. Journal of Environmental Management, 192, 292–301. Espeland, W. N., & Sauder, M. (2008). Rankings and diversity. Southern California Review of Law & Social Justice, 18, 587–608. Espeland, W. N., & Stevens, M. L. (1998). Commensuration as a social process. Annual Review of Sociology, 24, 313–343. Farneti, F., & Guthrie, J. (2009). Sustainability reporting by Australian public sector organisations: Why they report. Accounting Forum, 33, 89–98.

19

Farneti, F., & Siboni, B. (2011). An analysis of the Italian governmental guidelines and of the local governments’ practices for social reports. Sustainability Accounting, Management and Policy Journal, 2, 101–125. Fernandez-Feijoo, B., Romero, S., & Ruiz, S. (2014). Effect of stakeholders’ pressure on transparency of sustainability reports within the GRI framework. Journal of Business Ethics, 122, 53–63. Geng, Y., Fu, J., Sarkis, J., & Xue, B. (2012). Towards a national circular economy indicator system in China: An evaluation and critical analysis. Journal of Cleaner Production, 23, 216–224. Global Reporting Initiative (2004). Public agency sustainability reporting Amsterdam: GRI. Global Reporting Initiative (2006). G3 sustainability reporting guidelines Amsterdam: GRI. Goswami, K., & Lodhia, S. (2014). Sustainability disclosure patterns of South Australian local councils: A case study. Public Money & Management, 34, 273–280. Gouvernement du Québec (2015). Rapport synthèse de mise en œuvre de la Stratégie gouvernementale de développement durable 2008-2013 (prolongée jusqu’en 2015). http://www.mddelcc.gouv.qc.ca/developpement/strategie_gouvernementale/rapport/2013-2014.pdf. Government of Canada (2015). Results-Based management lexicon. https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/centre-excellenceevaluation/results-based-management-lexicon.html. Greco, G., Sciulli, N., & D’Onza, G. (2015). The influence of stakeholder engagement on sustainability reporting: Evidence from Italian local councils. Public Management Review, 17, 465–488. Greiling, D., & Grüb, B. (2014). Sustainability reporting in Austrian and German local public enterprises. Journal of Economic Policy Reform, 17, 209–223. Greiling, D., Traxler, A. A., & Stötzer, S. (2015). Sustainability reporting in the Austrian, German and Swiss public sector. International Journal of Public Sector Management, 28, 404–428. Guthrie, J., & Farneti, F. (2008). GRI sustainability reporting by Australian public sector organizations. Public Money & Management, 28, 361–366. Guthrie, J., Ball, A., & Farneti, F. (2010). Advancing sustainable management of public and not for profit organizations. Public Management Review, 12, 449–459. Hahn, R., & Kühnen, M. (2013). Determinants of sustainability reporting: A review of results, trends, theory, and opportunities in an expanding field of research. Journal of Cleaner Production, 59, 5–21.

20

Hahn, R., & Lülfs, R. (2014). Legitimizing negative aspects in GRI-oriented sustainability reporting: A qualitative analysis of corporate disclosure strategies. Journal of Business Ethics, 123, 401–420. Hák, T., Janoušková, S., & Moldan, B. (2016). Sustainable Development Goals: A need for relevant indicators. Ecological Indicators, 60, 565–573. Happaerts, S. (2012a). Sustainable development and subnational governments: Going beyond symbolic politics? Environmental Development, 4, 2–17. Happaerts, S. (2012b). Sustainable development in Quebec and Flanders: Institutionalizing symbolic politics? Canadian Public Administration, 55, 553–573. Hezri, A. A. (2004). Sustainability indicator system and policy processes in Malaysia: A framework for utilisation and learning. Journal of Environmental Management, 73, 357–371. Hezri, A. A., & Dovers, S. R. (2006). Sustainability indicators, policy and governance: Issues for ecological economics. Ecological Economics, 60, 86–99. Hezri, A. A., & Dovers, S. R. (2009). Australia’s indicator-based sustainability assessments and public policy. Australian Journal of Public Administration, 68, 303–318. Holden, M. (2011). Public participation and local sustainability: Questioning a common agenda in urban governance. International Journal of Urban and Regional Research, 35, 312–329. Holman, N. (2009). Incorporating local sustainability indicators into structures of local governance: A review of the literature. Local Environment, 14, 365–375. Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15, 1277–1288. Kohlbacher, F. (2006). The use of qualitative content analysis in case study research. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 7, 21. KPMG (2017). The road ahead: The KPMG survey of corporate responsibility reporting 2017. KPMG International. Krank, S., Wallbaum, H., & Grêt-Regamey, A. (2010). Constraints to implementation of sustainability indicator systems in five Asian cities. Local Environment, 15, 731–742. Laufer, W. S. (2003). Social accountability and corporate greenwashing. Journal of Business Ethics, 43, 253–261. Lehtonen, M. (2013). The non-use and influence of UK energy sector indicators. Ecological Indicators, 35, 24–34.

21

Lemos, M. C., & Morehouse, B. J. (2005). The co-production of science and policy in integrated climate assessments. Global Environmental Change, 15, 57–68. Lynch, B. (2010). An examination of environmental reporting by Australian state government departments. Accounting Forum, 34, 32–45. Lyon, T. P., & Montgomery, A. W. (2015). The means and end of greenwash. Organization & Environment, 28, 223–249. Marimon, F., del Mar Alonso-Almeida, M., del Pilar Rodríguez, M., & Alejandro, K. A. C. (2012). The worldwide diffusion of the global reporting initiative: What is the point? Journal of Cleaner Production, 33, 132–144. Mazzi, A., Mason, C., Mason, M., & Scipioni, A. (2012). Is it possible to compare environmental performance indicators reported by public administrations? Results from an Italian survey. Ecological Indicators, 23, 653–659. Meadowcroft, J. (2007). Who is in charge here? Governance for sustainable development in a complex world. Journal of Environmental Policy & Planning, 9, 299–314. Michelon, G., Pilonato, S., & Ricceri, F. (2015). CSR reporting practices and the quality of disclosure: An empirical analysis. Critical Perspectives on Accounting, 33, 59–78. Miles, M. B., & Huberman, A. M. (2003). Analyse des données qualitatives. Paris: De Boeck. National Assembly of Quebec (2006). Sustainable development act. Quebec city: Québec Official Publisher. Newig, J. (2007). Does public participation in environmental decisions lead to improved environmental quality?: Towards an analytical framework. Communication, Cooperation, Participation (International Journal of Sustainability Communication), 1, 51–71. Niemann, L., & Hoppe, T. (2018). Sustainability reporting by local governments: A magic tool? Lessons on use and usefulness from European pioneers. Public Management Review, 20, 201–223. Palme, U., & Tillman, A. M. (2008). Sustainable development indicators: How are they used in Swedish water utilities? Journal of Cleaner Production, 16, 1346–1357. Perego, P., & Kolk, A. (2012). Multinationals’ accountability on sustainability: The evolution of third-party assurance of sustainability reports. Journal of Business Ethics, 110, 173–190. Pires, S. M., & Fidélis, T. (2012). A proposal to explore the role of sustainability indicators in local governance contexts: The case of Palmela, Portugal. Ecological Indicators, 23, 608–615.

22

Pires, S. M., & Fidélis, T. (2015). Local sustainability indicators in Portugal: Assessing implementation and use in governance contexts. Journal of Cleaner Production, 86, 289–300. Pires, S. M., Fidélis, T., & Ramos, T. B. (2014). Measuring and comparing local sustainable development through common indicators: Constraints and achievements in practice. Cities, 39, 1–9. Porter, T. M. (1996). Trust in numbers: The pursuit of objectivity in science and public life. Princeton: Princeton University Press. Rey-Valette, H., Laloë, F., & Le Fur, J. (2007). Introduction to the key issue concerning the use of sustainable development indicators. International Journal of Sustainable Development, 10, 4–13. Rinne, J., Lyytimäki, J., & Kautto, P. (2013). From sustainability to well-being: Lessons learned from the use of sustainable development indicators at national and EU level. Ecological Indicators, 35, 35–42. Romolini, A., Fissi, S., & Gori, E. (2015). Quality disclosure in sustainability reporting: Evidence from universities. Transylvanian Review of Administrative Sciences, 11, 196–218. Russel, D. (2007). The United Kingdom’s sustainable development strategies: Leading the way or flattering to deceive? European Environment, 17, 189–200. Russell, S. L., & Thomson, I. (2009). Analysing the role of Sustainable development indicators in accounting for and constructing a Sustainable Scotland. Accounting Forum, 33, 225–244. Rydin, Y. (2008). Sustainable development and governance. Sage Publications. Sciulli, N. (2011). Influences on sustainability reporting within local government. International Review of Business Research Papers, 7, 282–291. Secrétariat du Conseil du trésor (2002). Modernisation de la gestion publique : Guide sur le rapport annuel de gestion. http://www.tresor.gouv.qc.ca/cadredegestion/fileadmin/documents/publications/sct/guide_rag.pdf. Singh, R. K., Murty, H. R., Gupta, S. K., & Dikshit, A. K. (2009). An overview of sustainability assessment methodologies. Ecological Indicators, 9, 189–212. Steurer, R., & Martinuzzi, A. (2005). Towards a new pattern of strategy formation in the public sector: First experiences with national strategies for sustainable development in Europe. Environment and Planning C: Government and Policy, 23, 455–472. Stubbs, W., Higgins, C., & Milne, M. (2013). Why do companies not produce sustainability reports? Business Strategy and the Environment, 22, 456–470.

23

Sustainable Development Commissioner of Quebec (2014). Rapport du Vérificateur général du Québec à l’Assemblée nationale pour l’année 2014-2015: Rapport du commissaire au développement durable. http://www.vgq.gouv.qc.ca/fr/fr_publications/fr_rapport-annuel/fr_2014-2015-CDD/fr_Rapport2014-2015-CDD.pdf. Talbot, D., & Boiral, O. (2015). Strategies for climate change and impression management: A case study among Canada’s large industrial emitters. Journal of Business Ethics, 132, 329–346. Talbot, D., & Boiral, O. (2018). GHG reporting and impression management: An assessment of sustainability reports from the energy sector. Journal of Business Ethics, 147, 367–383. Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data. American Journal of Evaluation, 27, 237–246. Treasury Board of Quebec (2017). Budget de dépenses 2017-2018 : Crédits des ministères et organismes. https://www.tresor.gouv.qc.ca/fileadmin/PDF/budget_depenses/17-18/creditsMinisteresOrganismes.pdf. United Nations Statistics Division (2015). Discussion paper on principles of using quantification to operationalize the SDGs and criteria for indicator selection. New York. Volkery, A., Swanson, D., Jacob, K., Bregha, F., & Pintér, L. (2006). Coordination, challenges, and innovations in 19 national sustainable development strategies. World Development, 34, 2047–2063. White, D. D., Wutich, A., Larson, K. L., Gober, P., Lant, T., & Senneville, C. (2010). Credibility, salience, and legitimacy of boundary objects: Water managers’ assessment of a simulation model in an immersive decision theater. Science and Public Policy, 37, 219–232. Williams, B. (2015). The local government accountants’ perspective on sustainability. Sustainability Accounting, Management and Policy Journal, 6, 267–287.