do procedures matter when communicating assurance… · do procedures matter when communicating...

Do Procedures Matter When Communicating Assurance?

An Experiment Applying New IAASB Standards

Sandra Vera-Muñoz* University of Notre Dame

Notre Dame, Indiana [email protected]

Lisa Gaynor University of South Florida

Tampa, Florida [email protected]

Linda McDaniel University of Kentucky (emeritus)

Lexington, KY [email protected]

William Kinney University of Texas at Austin

Austin, Texas [email protected]

ISAR version - June 6, 2014 - Preliminary

*Corresponding author

The authors gratefully acknowledge the helpful comments of ISAE 3000, ISAE 3410, and ISRE 2400 Task Force members, as well as Roger Martin, Marcy Shepardson and her Indiana University PhD students, Roger Simnett, Elizabeth Carson, and an anonymous reviewer for the AAA Annual Meeting. We also thank Rachel Webster for excellent research assistance.

mailto:[email protected]






ABSTRACT

Can assurance report users reliably assess incremental assurance achieved via an audit firm’s customized description of its work effort? The answer is important because international assurance standards now mandate such procedure descriptions to communicate achieved assurance for most assurance engagements. Also, auditing standards and U.S. assurance standards remain risk-focused and the change may impair comparative credibility assessment by intended users of assurance reports. We conduct a between subjects experiment using ordered alternatives to parse participants’ confidence judgments after they read actual procedure descriptions for greenhouse gas emissions assurance reports. We find that, given a report format, including or explicitly excluding a critical procedure necessary for “reasonable” assurance did not affect user perceptions. Further, varying report formats while holding constant description of “limited” assurance procedures produced median confidence judgments ranging from 60% to 81%. These results suggest that, other things equal, procedures descriptions do not allow users to assess relative assurance risk.

Key Words: Limited Assurance, Perceived Confidence, IAASB Assurance Standards



1. Introduction

Traditionally, U.S. and international standards for audit firms’ independent assurance

reports have stated the risk of material misstatement remaining after applying commonly

understood verification procedures. 1 For example, higher credibility financial statement audits

apply typical audit procedures to achieve “low risk” of material financial misstatement (or

“reasonable assurance”), while lower credibility financial statement reviews apply analytical

procedures and inquiries to achieve “moderate risk” (or “moderate assurance”).2 The stated risk

approach has also been applied to other subject matters such as internal control quality,

greenhouse gas emissions, labor practices, and sustainability assertions.

The IAASB recently eliminated the stated risk approach for assurance engagements that

provide “limited assurance.” That is, “moderate assurance,” such as “about as likely as not,” is

replaced by variable assurance that could range from low to fairly high, depending on the

assurance procedures applied. The new standards allow an assurance practitioner to choose (and

even negotiate) verification procedures to be applied for a particular limited assurance

engagement and then to customize description of the procedures in the assurance report (ISAE

3410 [2011], ISRE 2400 [2012], and ISAE 3000 [2013]).

The IAASB assumes implicitly that limited assurance report users can determine the level

of risk of material misstatement from these customized procedures descriptions. This assumption

raises the fundamental question: Can typical assurance report readers reliably determine the

1 According to the IAASB’s Assurance Framework, an assurance engagement is “an engagement in which a

practitioner aims to obtain sufficient appropriate evidence in order to express a conclusion designed to enhance the degree of confidence of the intended users other than the responsible party about the outcome of the measurement or evaluation of an underlying subject matter against criteria.” (IAASB 2013.10).

2 For example, see ISRE 2400.09 (2006 version).

2

assurance achieved through the practitioner’s particularized descriptions of verification

procedures applied?

Answers are potentially important because if intended users cannot reliably differentiate

the credibility added by a particular assurance report, then lower quality/lower cost limited

assurance engagements will drive out audit firms’ higher quality but higher cost assurance

services. Further, differences between IAASB and U.S. standards would impair credibility

comparisons across jurisdictions as well as focus assurance practitioner attention on describing

procedures rather than achieving a stated assurance.

Historically, assurance standards setters around the world have adopted essentially

equivalent standards for audit firms’ attestations to information quality across potential subject

matters. The AICPA developed such assurance standards in the 1980s that were adopted

verbatim by the PCAOB in 2003 and were a template for the IAASB’s initial standards. U.S.

standards provide for examinations (e.g., financial statement audits) that achieve “high

assurance” (denoted “reasonable assurance”), and reviews that achieve “moderate” assurance.

Prescribed reasonable assurance examination reports for U.S. and international standards

are all based on “examining evidence and such procedures as we considered necessary in the

circumstances” to achieve reasonable assurance (e.g., AT101.66), and moderate assurance

reviews are based primarily on “applying analytical procedures to management’s financial data

and making inquiries of company management” that typically achieve moderate assurance (e.g.,

AR 90.73). Thus, by intent, procedures are limited relative to those needed for reasonable

assurance. As to expressing a conclusion, both U.S. and international standards require a

positive form for examinations -- “In our opinion, the [information] is properly prepared, in all

material respects, based on XYZ criteria” (e.g., ISAE 3000 A.178) and a negative form for a

3

review -- “. . . nothing has come to our attention that causes us to believe that the information is

not properly prepared, in material respects, based on XYZ criteria” (e.g., ISAE 3000 A.180).

With the IAASB’s 2011-13 changes, international and U.S. standards for limited

assurance now differ on the level(s) of assurance achieved and the evidential basis needed.

Rather than prescribing that the procedures are to produce “moderate” assurance, “limited

assurance” can now be any level over a wide range. Specifically, under new IAASB standards,

limited assurance can vary from

“. . . just above assurance that is likely to enhance the intended users’ confidence about the subject matter information to a degree that is clearly more than inconsequential to just below reasonable assurance” (ISAE 3000.A6, emphasis added).

So if reasonable assurance is 95%, then limited assurance might provide 10% to 90% assurance

depending on chosen procedure descriptions, rather than “moderate assurance” of, say 60%.

In developing its new guidance, the IAASB concluded that assurance professionals (i.e.,

audit firm personnel) are unable to make such quantitative judgments and that, at least in theory,

infinite variation in assurance is possible (ISAE 3000. 69(k), .A6, .A173-75). Thus, the IAASB

rejected any quantified characterizations of limited assurance and instead maintained that a

description of the chosen procedures allows each report user to individually assess the

confidence level achieved. As an alternative, the Board decided to require engagement-specific

assurance report descriptions of the procedures chosen for both reasonable and limited assurance

engagements with elaboration as necessary for limited assurance engagements. The IAASB did

not specify how intended users can assess incremental assurance from reading the audit firm’s

procedures description.

To address whether users can reliably assess assurance based on procedure descriptions,

we conduct an experiment that uses 210 undergraduate accounting majors as surrogates for

4

typical non-expert GHG emissions report users. We elicit their confidence judgments about the

credibility of an assured GHG emissions report applying new IAASB standards. We vary

engagement description (limited vs. reasonable assurance) and conclusion form (negative vs.

positive) as well as varying the procedures described. To enhance external validity, our

procedure descriptions use actual GHG emissions assurance reports for a global group that

include five assurance procedures. Some segments received limited assurance reports applying

four stated procedures. Other segments received reasonable assurance reports applying the same

four procedure descriptions, plus an especially important fifth procedure deemed necessary by

the audit firm for reasonable assurance.3

To isolate the effect of assurance procedures per our research question, we vary the

procedures (four vs. five) across three combinations of conditions of report wording (limited vs.

reasonable and negative vs. positive). The five resulting cells are ordered according to our

expectations of report users’ confidence assuming that the IAASB’s presumptions in writing the

new standards are descriptive.

Four cells list four procedures for limited assurance, but vary reference to the omitted

procedure (explicit vs. no mention), stated assurance (limited vs. reasonable), and conclusion

framing (positive vs. negative). A fifth cell includes the four procedures and adds the critical

fifth procedure judged necessary for reasonable assurance. The stated assurance and conclusion

framing manipulations allow us to establish “other things equal” conditions to (a) address our

primary research question about procedure description, and (b) to test the continued relevance of

traditional wording of prescribed assurance reports (Hasan et al. [2003]).

3 The assurance report is based on E&Y’s 2012 Independent Reasonable and Limited Assurance Reports on

Toronto-Dominion Bank’s GHG emissions, available at: http://www.tdcanadatrust.com/easyweb5/crr-2011/pdfs/E&Y%20Assurance.pdf.

http://www.tdcanadatrust.com/easyweb5/crr-2011/pdfs/E&Y%20Assurance.pdf

http://www.tdcanadatrust.com/easyweb5/crr-2011/pdfs/E&Y%20Assurance.pdf

5

Regarding our tests of procedure description effects, we find that participants are not able

to reliably assess confidence using customized procedure descriptions, other things equal. In

particular, participants did not differentiate between assurance via a positive form reasonable

assurance opinion with all five essential procedures applied (reasonable assurance in our real

world engagement) from an identical report with only four described procedures (which did not

achieve reasonable assurance). Also, participants receiving a limited assurance negative frame

conclusion report explicitly noting omission of the essential procedure did not assess

significantly lower confidence compared with the same report without noting the omission.

As to report format, participants did distinguish assurance from a limited assurance

engagement from a reasonable assurance engagement and a negative frame from a positive

frame. Thus, participants seemed to comprehend the warnings inherent in the IAASB’s

engagement descriptions as well as recognize the inherent difference between negative and

positive assurance. Further, our manipulations of engagement type and conclusion frame had

significant effects in our effort to achieve “other things equal” in order to isolate the role of

procedure descriptions.

Overall, these results suggest that customized procedure descriptions alone do not

reliably convey to users the level of assurance attained and that engagement characterizations

and conclusion format convey at least the direction of difference in assurance achieved.

In section 2, we review the assurance standards and the conceptual background for our

predictions of the effect of stated assurance, conclusion form, and procedures, while Section 3

describes our research design. Section 4 provides statistical analyses and robustness tests, while

Section 5 briefly concludes.

6

2. Background and Hypothesis Development

2.1 THE NEW IAASB ASSURANCE STANDARDS

Globally, there is growing demand for a broad range of independent assurance beyond

traditional audits of financial statements that stems from market factors affecting entities, such as

demands of customers, suppliers and workers, investors, as well as regulators and the general

public. 4 Subject matters include greenhouse gas emissions and climate-change risk (Matsumura,

Prakash, and Vera-Muñoz [2014]; Eccles et al. [2011]), sustainability, and integrated reporting,

plus companies’ corporate social responsibility (CSR) reports on their triple bottom lines of

environmental, social, and economic performance (Mock, Rao, and Srivastava [2013]); Brockett

and Rezaee [2012]; Simnett, Nugent, and Huggins [2009]; Simnett, Vanstraelen, and Chua

[2009]).

Most of these reports are widely available to the general public as well as stockholders,

with more than 3,000 business organizations worldwide voluntarily issuing stand-alone reports

(Brockett and Rezaee [2012], 27-35).5 Firms use CSR reports for college student recruiting

(Auger et al. [2013]), as marketing tools (Simnett, Nugent, and Huggins [2009, p. 351]), or for

reputation risk management (Borkowski, Welsh, and Wentzel [2010]).6 To the extent that

entities make public disclosures intended for an external non-expert audience, independent

assurance on these statements helps the entity mitigate stakeholders’ perception that such

4 See, e.g.,

http://www.aicpa.org/InterestAreas/BusinessIndustryAndGovernment/Resources/Sustainability/Pages/SustainabilityFAQs.aspx (accessed 12/9/13).

5 In addition to stand-alone sustainability reports, entities’ voluntary disclosures of GHG emissions information can take many other forms, including press releases, single GHG statements, the Carbon Disclosure Project, or MD&As in the10-K reports filed with the SEC.

6 For instance, firms use CSR reports to claim that their operations are, or are moving toward “carbon neutrality” or reducing their “carbon footprint” or to associate “green” attributes to their products in order to capitalize on the popular trend towards eco-conscience (Brockett and Rezaee [2012]).

http://www.aicpa.org/InterestAreas/BusinessIndustryAndGovernment/Resources/Sustainability/Pages/SustainabilityFAQs.aspx

http://www.aicpa.org/InterestAreas/BusinessIndustryAndGovernment/Resources/Sustainability/Pages/SustainabilityFAQs.aspx

7

statements or disclosures are merely “green-wash,” that is, an attempt to persuade stakeholders

of an entity’s sustainability credentials (Simnett, Nugent, and Huggins [2009, p. 351]).

Collectively, these recent developments provide incentives to publish non-financial

information and to consider whether an independent assurer can also add value by enhancing

user confidence about the information. If assurance is to be provided, then management,

contracting parties, and regulators must decide whether reasonable assurance or limited

assurance adds more value net of cost. The value of audit firms’ assurance report depends in part

on the assurance standards that are intended to promote reliable credibility enhancement

services.

In addition to these new demands for non-financial assurance, many countries outside the

U.S. have recently exempted some smaller entities from statutory financial audit mandates and

many of these entities have decided that audits are not cost-effective and thus reduced demand

from audit services. However, relatively little is known about the cost and value added by

assurance other than financial statement audits.

The IAASB responded to new demands for non-traditional assurance by developing

simultaneously three related and synthesized assurance standards. The results are (a) a revised

ISAE 3000 (issued 2013) to provide independent assurers updated and integrated guidance to

evaluate or measure a wide range of potential subject matter other than financial statements

against suitable criteria, (b) a revised ISRE 2400 (issued 2012) to improve financial statement

reviews (see Figure 1), and (c) a new standard, ISAE 3410 (issued 2011), for assurance on an

important and highly specialized subject matter—GHG emissions.

| Insert Figure 1 about here |

8

Each of the new IAASB standards defines limited assurance as less than reasonable

assurance by design, but without quantifying either in probability terms. 7 Regarding possible

quantification, the IAASB takes the position “The level of assurance the practitioner plans to

obtain is not ordinarily susceptible to quantification . . ..” (ISAE 3000.A4). In the 2013 version

of ISAE 3000, no reference is made to “moderate” risk or “less likely than not.” In fact, when

debating the 2013 revision to ISAE 3000, the IAASB explicitly rejected as impracticable, use of

other standard’s quantification of limited assurance as “moderate” and defining “likely” as “more

likely than not” (per IAASB Board meeting transcript, December 2012).

This presumed inability of audit firm assurance professionals to quantify risk even in

general terms is at least arguable, given that conventional definitions of audit “quality” are

expressed in terms of the probability that financial statements contain no material omissions or

misstatements (e.g., Palmrose [1988]). Also, U.S. and IAASB auditing standards require that

achieved audit risk that the auditor fails to detect financial statements that are materially

misstated be appropriately or acceptably “low” (AU 312.13, ISA 200.17]).

2.2 DETERMINANTS OF PERCEIVED ASSURANCE ATTAINED

While revising its assurance standards, the IAASB considered changes to several factors

suggested as value relevant in communicating to intended users the level of assurance achieved

by applying its new assurance standards, but without using quantitative terms.8 We consider

these factors to quantify their effects in order to assess the effect of procedure choices by limited

7 The original ISAE 3000, issued by the IAASB in 2003, replaced and extended ISAE 100 to provide guidance on

both reasonable (formerly high) and limited (formerly moderate) assurance. Similarly, the original ISRE 2400, issued by the IAASB in 2005, states “A review engagement provides a moderate level of assurance that the information subject to review is free of material misstatement . . ..” (para. 9) (emphasis added).

8 As examples, the IAASB considered the positive opinion form to express review conclusions, whether to require warnings about the higher risk of reviews, and whether noting omission of a procedure commonly applied in an examination was needed. The Board’s discussion was motivated by the need to balance proper warnings about review limitations while not denigrating reviews through cautions or negative framing.

9

assurance practitioners. Specifically, we consider the conceptual basis supporting relevance of

(a) practitioner description of assurance procedures, (b) explicit mention of omitted procedures,

(c) description of the engagement, and (d) the conclusion frame.

2.2.1 Assurer’s Description of Assurance Procedures

As discussed above, ISAE 3000, ISAE 3410, and ISRE 2400 allow the assurer to choose

the verification procedures to be applied to form the basis for assurance and to describe the

chosen procedures in the assurance report. The guidance for describing procedures in ISAE 3000

states that the assurance report must include:

. . . an informative summary of the work performed as the basis for the practitioner’s conclusion. In the case of a limited assurance engagement, an appreciation of the nature, timing, and extent of procedures performed is essential to understanding the practitioner’s conclusion (ISAE 3000.69(k)) (emphasis added).

Such a description is needed, in part, “[b]ecause the level of assurance obtained by the

practitioner in limited assurance engagements varies (ISAE 3000.A6).

In addition, for limited assurance the guidance suggests that the report “is ordinarily more

detailed” regarding procedures than a reasonable assurance report (ISAE 3000.A175). In

practice, this additional detail option could include, for example, dividing an integrated

procedure into a list of several component procedures. Also, as discussed in 2.2.4 below, ISAE

3000 offers optional guidance on reporting procedures “not” performed in a limited assurance

engagement.

Collectively, the guidance for extensive and detailed description of limited assurance

procedures may have the unintended effect on users because an elaborate work effort description

in a limited assurance report may appear to warrant more confidence than does a reasonable

assurance report that describes only a work effort overview.

10

2.2.2 Reporting “Typically Applied” Procedures Omitted

For a limited assurance engagement report, ISAE 3000 guidance for procedure or work

effort descriptions states:

It also may be appropriate to indicate in the summary of the work performed certain procedures that were not performed that would ordinarily be expected to be performed in a reasonable assurance engagement (ISAE 3000.A175).

This guidance could assist the user in seeing what “might not have come to the attention

of the assurer” in a limited assurance engagement and thus lower perceived confidence in a

negative assurance conclusion. In practical application, however, the guidance is problematic for

non-financial information because there is no available description of the procedures “ordinarily

expected to be performed in a reasonable assurance engagement” for, say, GHG emissions or

compliance with some non-financial statement measurement criteria.9 Thus, audit firms typically

rely on teams of GHG subject matter experts and assurance professionals to determine particular

procedures for GHG engagements, resulting in such procedures being unknown and unknowable.

There is, then, a conceptual and practical question of how assurers can determine “typical”

procedures for a reasonable assurance examination and how users can ascertain the relevance of

any procedures that may be explicitly stated as omitted.

Nonetheless, the IAASB decided instead to require the practitioner to list his or her

procedures and when unusual or exceptional verification procedures must be used, the

practitioner lists the alternative procedure. Thus, determining the confidence justified by the

listed procedures depends on the capabilities of the assurance report user.

Psychology research on salience effects shows that known to be salient stimuli (e.g.,

explicit mention of an omitted procedure) that triggers the perceiver’s attention have a

9 There is guidance on expected procedures for integrated audits of internal control effectiveness via the PCAOB’s

AS No. 5.

11

disproportionately large impact on judgment processes (Taylor, Crocker, Fiske, Sprinzen, and

Winkler [1979]; Taylor and Fiske [1978]). Similarly, according to rules-plus-exception theory

(Palmeri and Nosofsky [1995]; Nosofsky and Palmeri [1998]), people assign relatively more

weight to exceptions when evaluating similarities. These theories suggest that report users may

attend more to an explicit statement that singles out a procedure that has been omitted than to

procedures that are listed as performed.

Another body of psychology research provides an explanation for why explicit mention

of an omitted procedure may not lower report users’ confidence judgments (Richey, Koenigs,

Richey, and Fortin [1975]; Pratto and John [1991]; Mummendey et al. [2000]; see Mummendey

and Otten [1998] for a review). In a series of experiments, Richey et al. (1975) provide evidence

that predominantly negative information is weighted heavier when mixed with opposing positive

information, but predominantly positive information is not weighted any lower when combined

with varying amounts of negative information. If these results extrapolate to report users’

confidence assessments, then a list describing applied procedures followed by an explicit

statement of an omitted procedure will not lower report users’ confidence assessments.

2.2.3 Stated Level of Assurance

Cognitive psychology research demonstrates that information presentation and labels

influence how individuals classify information (Sherman, Mackie, and Driscoll [1990],

Kozminsky [1977]). Consistent with psychology research, accounting research documents that

“labeling” matters – examples include “comprehensive income items” vs. “balance sheet items”

(Maines and McDaniel [2000]), interpretation of fair value gains and losses (Gaynor, McDaniel,

and Yohn [2011]), resource allocation based on data labeled as historical accounting earnings-

based vs. future cash flows-based (Vera-Muñoz, Kinney, and Bonner [2001]).

12

The IAASB uses labels to describe the relative assurance provided by reasonable and

limited assurance engagements and offers the following explanation:

“Because the level of assurance obtained in a limited assurance engagement is lower than in a reasonable assurance engagement, the procedures the practitioner performs in a limited assurance engagement vary in nature and timing from, and are less in extent than for, a reasonable assurance engagement” (ISAE 3000.A3).

In practice, this suggests that the purposely less rigorous verification procedures for limited

assurance may not make the practitioner aware of all significant matters that might be identified

in a reasonable assurance engagement, thus reducing achieved assurance.

Cognitive psychology and accounting research discussed above (see also Libby,

Bloomfield, and Nelson [2002]) suggests that defining one service relative to another implies

that the referent is better known or understood. Therefore, communicating assurance depends in

part on user familiarity with the procedures that would typically be applied for a reasonable

assurance examination. This is, of course, problematic for new services such as providing

assurance on GHG emissions or compliance with a new law or process.

2.2.4 Assurer’s Conclusion Frame

Psychology research documents that decision makers respond differently to different, but

objectively equivalent, descriptions of the same problem (Levin et al. [1998]) and that how

information is framed can evoke different emotions or different mental accounts, and thus affect

decisions (Levin et al. [2002]; Levin and Gaeth [1988]). Accounting research documents that

framing affects professional skepticism (Nelson [2009]), audit risk assessments and audit effort

allocations (Fukukawa and Mock [2011]), and budget hours for auditing fair value estimates

(Maksymov, Nelson, and Kinney [2014]).

As to assurance conclusion frame, many commentators on the IAASB’s non-audit

assurance standards projects (particularly practicing auditors of small to medium-sized entities),

13

suggested that the negative form be eliminated for limited assurance to avoid the use of the

double negative (“nothing” and “not”) and the negative connotations of the negative form. These

commentators recommended that the review report be adapted to say “In our opinion, based on

the limited evidence collected, the [information] is properly prepared, in all material respects,

based on XYZ criteria.”

In this study, we vary conclusion frame and stated assurance to assess their incremental

effects on user perceptions of assurance and to isolate the effect of procedures descriptions, other

things equal. To our knowledge, with one notable exception (Hasan et al. [2003]), research has

not explored users’ ability to reliably assess assurance conveyed by assurance reports other than

financial statement audits and reviews. Using early guidance from the IFAC report [2002] and

the ISAE 100 [2001] assurance framework, Hasan et al. develop an experiment on alternative

report formats to communicate “moderate” assurance. Four report formats are labeled as

“moderate” assurance (consistent with ISAE 100 [2001]), and a fifth format, labeled “high

assurance,” is used as the control condition and procedures performed are held constant across

the five conditions. 10 Hasan et al. find significantly higher perceived assurance from the high

assurance report vs. three of the four “moderate” assurance reports and no difference between the

high assurance and the moderate assurance “opinion on procedures paragraph.”

2.3 EXPECTATIONS AND RESEARCH DESIGN

We use the IAASB’s assumptions in adopting ISAE 3000, ISAE 3410, and ISRE 2400 and

the differences between U.S. and international assurance standards discussed as well as

10 The key wordings in the four moderate assurance reports are, respectively: (1) “The procedures for gathering and

processing environmental information have, in our opinion, been planned and implemented in an appropriate way;” (2) “As a result of the procedures performed, nothing has come to our attention that causes us to believe…”; (3) “As a result of the procedures performed, it is our opinion that the Report fairly presents, in all material respects…”; (4) Same as (3), with the caveat “Given the …extent of the work performed on the engagement, we are not in a position to express a high level of assurance.”

14

psychology and accounting research above to assess how new IAASB standards will affect

users’ perceptions of achieved assurance. To be specific, the research and practical arguments

suggest that users’ confidence will be increased by (a) adding an important procedure to the

work effort description, (b) deleting explicit mention of an important procedure omitted, (c)

describing the engagement objective as providing “reasonable assurance,” and (d) presenting the

assurance conclusion in a positive frame (see Figure 2).


We test these predictions across five cells with cell 1 (lowest expected assurance) noting

explicitly that an important procedure has been omitted, expresses limited assurance, uses a

negative frame conclusion, and applies only four procedures. We end with cell 5 (highest

expected assurance) that expresses reasonable assurance, notes no omitted procedures, uses a

positive frame conclusion, and applies five procedures. We change one of the four

characteristics at a time across adjacent cells.

Comparing cell 1 with cell 2 and comparing cell 4 with cell 5 allows assessment of

whether procedures descriptions matter while holding stated assurance and conclusion frame

constant. Specifically, does deletion of explicit notice of omission of a procedure essential for

reasonable assurance and/or including the essential procedure increase users’ confidence

assessments, other things equal? The cell 4 to cell 5 comparison is particularly important

because addition of this procedure allowed valid expression of reasonable assurance and a

positive form opinion. Comparisons within cells 2 through 4 serve to isolate procedures effects

by holding constant wording of the assurance report.

15

3. Research Method

3.1 PARTICIPANTS AND ADMINISTRATION

Two hundred ten junior and senior students enrolled in an accountancy program at a

private university participated in our study and are representing the general third party assurance

report users who are appropriate for our research goals (Libby et al. [2002]). The participants

completed the study outside of class using their own computers to access an internet link that

was emailed by their instructors. We used the Qualtrics software platform to administer the

experiment. To encourage participants to attend to the study, their instructors announced that

they were eligible to receive extra credit points, with the credit magnitude contingent upon their

performance in post-experimental questions.

3.2 RESEARCH DESIGN AND EXPERIMENTAL INSTRUMENT

Participants were provided with a hypothetical global audit firm’s assurance report, based

on guidance provided by ISAE 3000, for a hypothetical company’s Schedule of GHG emissions.

As a measure of the confidence (denoted CONF) associated with the assurance report, we elicit

each participant’s assessment of the assurance (or confidence level) they perceived for the GHG

information based on the independent assurer’s investigation and report.

We vary combinations of four conditions (procedure exclusion noted, conclusion frame,

stated assurance, procedures completeness) across five cells ordered in expected CONF values

based on our discussion in section 2. For all five cells, we hold constant the following bullet-

point list of four assurance procedures descriptions based on E&Y’s reasonable and limited

assurance reports for segments of Toronto-Dominion Bank’s 2012 GHG emissions reports:

“To obtain reasonable (limited) assurance, our procedures included:

• Interviewing select personnel to understand the key corporate responsibility issues related to the data and processes for the reporting of the emissions

16

• Inquiring of management regarding key assumptions and the evidence to support the assumptions

• Validating, on a sample basis, the accuracy of calculations performed primarily through inquiry and analytical procedures

• Performing, where relevant, walkthroughs of systems and processes for data aggregation and reporting.”

Participants in cell 5 (the cell expected to yield the highest CONF) also saw a fifth

procedure description:

• Examining evidence supporting the information in the Schedule of Greenhouse Gas Emissions.

and those in cell 1 (the cell expected to yield the lowest CONF), instead, saw the notice:

“Our procedures did not include examining evidence supporting the information in the Schedule of Greenhouse Gas Emissions.”

Note that the nature of the four procedures common to both limited and reasonable

assurance (denoted LA and RA) engagements is directed to whether the information should be

correct by asking questions about processes and assumptions, and to see if “calculations” seem

reasonable, but there is no indication of verifying whether the particular information produced is

correct. That is, there is no testing of the details supporting the information produced.

Determining that processes could or should produce accurate output is not as persuasive as

evidence that the information produced is, in fact, accurate. Thus, the fifth procedure that E&Y

used for a RA engagement should be important in assessing CONF.

In contrast to those in cells 1 and 5, participants in cells 2, 3, and 4 saw no

mention of the fifth procedure. Thus, cell 1 participants would be explicitly aware of the

omission, while those in cells 2 through 4 could be aware implicitly—if they expected it

to be applied and also noted its omission from the work effort description. Therefore, the

procedure omission would be considered only if the participant expected application of

the procedure and noticed its omission.

17

Finally, following the IAASB’s discussion of alternatives leading to ISAE 3000, we

manipulate between-participants the conclusion paragraph frame as either negative or positive,

and the engagement description as LA or RA.

Dependent Variable

We construct the dependent variable, CONF, based on the participants’ judgments of

their level of assurance provided by an auditor’s report on GHG schedule. We elicited the

judgments as follows: “Given the report you read, how assured (or confident) do you feel that

DuLac Corporation’s Schedule of Greenhouse Gas Emissions is fairly presented in all material

respects?” Participants provided their responses on a 0−100 scale, where 0 = “Not at all assured

(No confidence),” and 100 = “Completely assured (Complete confidence).”

As noted in Figure 2, we predict/assume that CONF will be higher if (a) an important

omitted procedure is not explicitly noted, (b) RA is stated, (c) the conclusion frame is positive,

and (d) an important procedure is included and reported. These predictions are tested by

comparing adjacent cell medians. Factors (a) and (d) relate to procedures and factors (b) and (c)

relate to wording. Finally, if all predictions of the prior research and the IAASB’s presumption

underlying ISAE 3000, ISAE 3410, and ISRE 2400 obtain, then the means of our five cells will

increase with each cell as the cell number increases. Also, as shown in Figure 2, it should be

noted that cells 1 and 2 each reflect an externally valid combination of factors (a) – (d) for a LA

engagement under IAASB standards, and cell (5) is an externally valid combination for RA.

Cells 3 and 4 vary assurance description and frame for experimental purposes only to separate

the effects of factors (b) and (c) on CONF. In contrast, cells 3 and 4 are comprised of

18

hypothetical combinations of procedures, stated assurance, and conclusion format that we

use to assess the effect of alternative wording.11

3.3 MATERIALS

The instrument first introduces the company (DuLac Corporation), whose management

has prepared a Schedule of GHG Emissions and hired an independent assurer (Global Audit &

Assurance, LLP) to report on the Schedule’s compliance with stated GHG emissions criteria.

Next, participants are asked to assume the role of an interested outside reader of the assurance

report along with a brief assurance analogy based on a used automobile. To give participants a

better idea of their role, the case provides an analogy using the role of a prospective buyer for a

particular used car whose seller states is in good condition.12 The participants are then asked to

read Global Audit’s assurance report and asked for their evaluation (CONF) and personal

impressions. Post experimental questions conclude the exercise.

We refined our materials in two sequential steps. First, we pilot-tested our instruments

with a panel of seventeen experts, all of whom had experience with development of ISAE 3000

as a member of the IAASB, or as a Technical Advisor to a Board member, or as a member of the

ISAE 3000 Task Force. We also asked panel members for written comments and suggestions

(e.g., regarding external validity, clarity of exposition, format and content of the auditor’s

report). We revised our instruments as appropriate to incorporate their comments.

3.4 EXPERIMENTAL PROCEDURES

Because of the large number of participants required by our research design, we

staggered administration of our experiment with an administration near the beginning of a 4 ½ 11 Hasan et al. [2003] use the same approach to assessing the effects of alternative language, but without the

procedure description considerations. 12 In the analogy, the prospective buyer feels unsure about potential hidden defects in the car. To reduce the

buyer’s uncertainty, the seller hired MIDAS Auto Service™, an independent and reputable automotive technicians company, to conduct a set of tests. In a signed report, MIDAS Auto Service™ reports that the car passed all tests and appears to be in good condition.

19

month semester and the second near the end of the semester. As protection against possible

information leakage and hypothesis guessing with respect to procedures applied, students for the

earlier administration were randomly assigned using Qualtrics to only cells 1, 2, 3, and 4—all of

which received four procedure descriptions. All students in the later administration were

assigned to cell 5, which is our five procedure RA and positive frame cell and is consistent with

the real world RA report.13

The study is presented via six screens using the Qualtrics platform. Our primary cue

screen shows the full independent assurance report on DuLac Corporation’s Schedule of GHG

Emissions, signed by Global Audit and Assurance, LLP, followed by our request for the

assurance assessment (CONF).14 After entering their judgments, participants are presented with

five forced-choice attention/recall questions, and then four background questions.15

4. Results

4.1 RESULTS

Higher CONF is expected as the cell number increases per the IAASB’s maintained

assumption. Panel A of Figure 3 presents a plot of cell medians and means of our dependent

achieved assurance metric, CONF, and Panel B presents box-and-whisker plots of cell median

CONF. As shown in Panel A, both metrics follow an “S” shape with the first two cells and the

last two cells forming the relatively “horizontal” parts of the S-shape. Cells 2 through 4, which

differ only by their hypothetical combinations of stated assurance and conclusion frame, reflect

13 Twenty-seven of the later students had not participated in the earlier administration and we rely on these for our

primary analyses. Thirty-eight of the later students had also participated in the earlier administration and we do not include them in our primary analysis. However, we conduct limited test/retest analyses as well as robustness tests (discussed below in section 4.2) to assess whether there was apparent recall or information leakage.

14 Consistent with Hasan et al. [2003] we focus on the assurance report and do not present the GHG emissions statement.

15 Across all five experimental conditions, the average time taken to complete the experiment was 9.08 minutes (s.d. = 12.241).

20

an upward slope as the frame and stated assurance move toward the RA opinion. Panel A of

Table 1 shows that the median CONF for each cell ranges from 60% for cells 1 and 2 to 81% and

80% for cells 4 and 5, respectively, and Panel B shows a similar pattern of results using mean

CONF.

| Insert Figure 3 and Table 1 about here |

By construction, our five combinations of conditions (cells) are ordered a priori as to

expectation of report users’ perceptions of achieved assurance based on the IAASB’s maintained

assumptions as to which combinations would yield higher perceived CONF by report users.

Expectations for CONF are increasing from cell 1 through cell 5, but no prediction is made about

the magnitude of any increase. Given our a priori ordering of cells and the fact that differences

between cells is measured on an ordinal scale (and not an interval or ratio scale), we use the

Jonckheere-Terpstra (J-T) independent-samples test for ordered alternatives. The J-T test for

ordered alternatives is suitable when the alternative hypothesis for a k independent-samples

design specifies the rank-order of the k independent samples (Siegel and Castellan [1988]). We

use the J-T test for our primary analyses.

Table 2 presents pairwise J-T tests of difference in medians for our four cell difference

expectations shown in Figure 2. Note that cells 1 and 2 as well as cells 4 and 5 do not differ

significantly (p > 0.10, one-tailed), suggesting that our users did not distinguish achieved

assurance based on the differing procedure descriptions underlying these two pairs of cells.

These results obtain even though the procedure is an important one logically because it involves

the accuracy of the underlying information in the GHG report. And important professionally

because the differences are based on a real world assurance report prepared by a large global

network audit firm and the fifth procedure was essential to meeting the RA criteria regarding

21

material misstatement. The addition of this important/essential procedure does not add

significantly to the CONF for a “four procedure” report labeled as RA and with a positive frame

conclusion. Further, the explicit mention of this important procedure does not significantly

lower CONF for the four procedure LA condition with a negative frame report.

| Insert Table 2 about here |

Table 2 also shows that J-T tests of the median CONF for cells 2 and 3 (i.e., negative vs.

positive conclusion framing, respectively) differ significantly (p < 0.05, one-tailed), as well as

the median CONF for cells 3 and 4 (i.e., LA vs. RA, respectively) (p = 0.00), consistent with the

presumption of those wanting the IAASB to alter its guidance for reporting limited assurance

results. In particular, our results suggest that both stating a limited assurance report’s conclusion

in a positive opinion form (as opposed to negative) and characterizing the level of assurance as

reasonable (as opposed to limited with warnings) result in higher perceived achieved assurance

or CONF among users.

The box-and-whisker plots of median CONF across our five cells (Panel B of Figure 3)

show a wider spread in the lower and upper quartiles as well as the interquartile ranges for cells 1

and 2 than for cells 4 and 5. The overall standardized J-T statistic (6.575) is highly significant (p

= 0.00), which provides support for our expectation of median CONF increasing from cell 1

through cell 5.

As a final analysis of our CONF scores and ordered cells, we converted CONF scores to

ranks assigning 1 to the lowest and 210 for the highest (and substituting mean rank for ties). By

making this transformation, we can determine the expected median rank for each of the ordered

cells if the IAASB’s assumptions were perfectly described by each our users’ CONF judgments.

For example, the 48 participants in cell 1 would have the lowest 48 scores and their median rank

22

would be 24. The 46 participants in cell 2 would have the next lowest scores and their median

rank would be 70, and so on through cell 5. Figure 4 plots the expected median CONF rank

(E[MCR]) conditional on perfect application of the IAASB’s assumption and shows an

approximately straight upward sloping line. Figure 4 also plots the actual cell median CONF

ranks (MCR) and shows that cells 2 through 4 are very close to their theoretical expectation –

thus suggesting that the IAASB’s assumption that stated LA and RA wording and conclusion

frames are important in communicating assurance is correct. On the other hand, cells 1 (and 5)

are considerably above (and below) expectations.16


4.2 ATTENTION CHECKS AND ROBUSTNESS TESTS

We asked three attention check questions to determine whether participants attended to or

noted the independent variable cues as intended. Regarding the stated level of assurance (LA vs.

RA), we asked the forced choice question:

“In the assurance report you read, what was the stated level of assurance?”

with possible responses: (1) Reasonable Assurance; (2) Limited Assurance; (3) I do not recall.17 Regarding explicit vs. implicit excluded procedure manipulation we asked:

“In the assurance report you read, do you recall whether Global Audit and Assurance, LLP identified specific procedures that were not included in the work plan?”

with responses: “Yes – I recall such a statement;” “No – I do not recall such a statement.” Finally

to address the conclusion frame, we asked:

“At the end of the assurance report that you read, there was a stated conclusion. Indicate which of the following two wordings (A or B) was used in that conclusion: “A. “…nothing came to our attention that causes us to conclude that the Schedule…is not

16 We also regressed CONF against our rank-ordered cells and found significantly positive slopes with the slope for

the cell range restricted to cells 2 through 4 significantly steeper and with a higher R-square than for the full cell range 1 through 5.

17 We randomized the order of the first two choices in our three attention check questions.

23

fairly presented, in all material respects…”; B. “In our opinion, the Schedule…is fairly presented, in all material respects…,”

followed by the choices: “(1) The conclusion used the wording in A; (2) The conclusion used the

wording in B; and (3) I do not recall.”

For the first and third attention check questions, 91.9% and 92.7% of participants’

responses, respectively, were consistent with their condition (stated level of assurance and

conclusion frame conditions, respectively). We also noted no significant differences in these

response rates across conditions. For the second attention check question on implicit versus

explicit procedure exclusion, we find that 83.5% of the responses overall were consistent with

their respective condition. However, we also noted that the response rates varied across

conditions, specifically, in cell 1, where the procedure was explicitly excluded. We found that

while the majority (58.3%) of those participants in cell 1 gave responses that were consistent

with the explicit exclusion, a significant minority (41.7%) of the participants gave responses that

were not.18

To test the sensitivity of our full sample to potential lack of participant attention, we

excluded two groups of participants and reran our J-T tests of medians on our restricted samples.

We excluded (a) participants who failed to recall the stated assurance (question 1), and (b)

participants who failed to recall any of our three attention checks. Our results are shown in Table

3 where the two robustness tests are qualitatively similar to our full sample results.

Finally, recall that 38 students from cell 5 also participated in the earlier administration as

a participant in cell 1, 2, 3, or 4, and we excluded these participants from our primary analyses.

We conducted various robustness tests to assess whether there was apparent recall or information

18 Mummendy et al. (2000) suggest that initial categorization upon receiving positive information (e.g., procedures

that were performed) may affect the salience of negative information that follows (e.g., omitted procedure note). Hence, participants may not recall the statement that a procedure was not performed.

24

leakage. The results of our first robustness test are shown in Panel C of Table 3, which combines

these 38 participants with the 210 participants in our sample for our primary analyses. As shown

in Panel C, the results are qualitatively similar to our primary results that do not include these 38

participants. Finally, our second set of robustness tests (untabulated) shows that the cells 1 to 4

medians for these 38 students were very close to the medians of students who participated only

once and also very close to the medians for cell 5 who had not previously participated. We

conclude that there was little information leakage, information recall, or hypothesis guessing by

these participants.

| Insert Table 3 about here |

5. Conclusion

In this study we use key features simultaneously shared by the IAASB’s

new ISAE 3000, ISAE 3410, and ISRE 2400 to investigate whether practitioner-customized work

effort descriptions and assurance report words matter in that they affect users’ perceptions of

assurance achieved. We have two broad findings: (1) whether an important verification

procedure is performed (as in cell 5) or not performed and its omission noted in the report (as in

cell 1) did not significantly affect report users’ perceptions of achieved assurance, but (2)

changing traditional standardized report wording that characterizes assurance attained by

applying identical verification procedures did significantly change perceptions.

Results for our two tests regarding procedures-based standards do not support the

IAASB’s presumption that users can reliably determine or differentiate achieved assurance from

the practitioner’s customized descriptions of assurance procedures chosen and applied. Results

for our two tests regarding traditional report language characterizing assurance do support the

25

IAASB’s decision to retain traditional warnings and conclusion frame for limited assurance

engagements.

These results have implications for assurance report users, information preparers making

assurance purchase decisions, regulators deciding whether to mandate assurance, and audit firm

practitioners as well as assurance standards setters. Because, other than the stated assurance

claim and the conclusion frame, a report user has only the practitioner’s description of

procedures performed to assess confidence, he or she must be diligent regarding whether

descriptions reflect substantial verification or are merely descriptions that are “overstated or

embellished” (ISAE 3000.A177). Information preparers must evaluate whether reasonable

assurance examinations will raise user confidence sufficiently to justify their higher cost and

regulators must make similar evaluations for decisions to mandate particular assurance

reporting. Audit firms applying the new guidance may unwittingly join a “race to the bottom”

by competing with embellished limited assurance procedure descriptions rather than on service

quality necessary for reasonable or “moderate” assurance.

Regarding standards setting for assurance, a significant conceptual and practical

difference now exists between the risk-based standards of the AICPA and the PCAOB vs. the

stated procedures-based approach of the IAASB. An overarching question is whether multiple

limited assurance standards affect the value of and demand for quality-differentiated assurance

for other than financial statement audits. At present, few jurisdictions mandate particular

assurance standards and the largest (the U.S.) has a different conceptual basis for limited

assurance from IAASB standards that, in contrast to financial statement audits, have few

mandates at present. Implications of these differences warrant further study at many levels.

26

REFERENCES

AUGER, P., T.M. DEVINNEY, G.R. DOWLING, C. EXCKERT, and N. LIN. “How much does a company’s reputation matter in recruiting?” MIT Sloan Management Review 54 (2013): 78—88.

BORKOWSKI, S.C., M.J. WELSH, and K. WENTZEL. “Johnson & Johnson – A model for Sustainability Reporting.” Strategic Finance (2010): 29—37.

BROCKETT, A.M., and Z. REZAEE. “Corporate Sustainability—Integrating Performance and

Reporting” (2012) John Wiley & Sons, Inc. Hoboken, N.J. ECCLES, R. G.; M. P. KRZUS; and G. SERAFEIM. “Market Interest in Nonfinancial Information.”

Journal of Applied Corporate Finance 23 (2011): 113−127. FUKUKAWA, H. and T.J. MOCK. “Audit Risk Assessments Using Belief Versus Probability.”

Auditing: A Journal of Practice and Theory 30 (2011): 75−99. GAYNOR, L. M., L. MCDANIEL, AND T. L. YOHN. 2011. “Fair Value Accounting For Liabilities:

The Role of Disclosures in Unraveling the Counterintuitive Income Statement Effect From Credit Risk Changes.” Accounting, Organizations and Society 36 (2011): 125−134.

HASAN, M., P.J. ROEBUCK, AND R. SIMNETT. “An investigation of alternative report

formats for communicating moderate levels of assurance.” Auditing: A journal of Practice & Theory 22 (2003): 171-187.

INTERNATIONAL AUDITING AND ASSURANCE STANDARDS BOARD. “ISAE 3000 (Revised),

Assurance Engagements Other Than Audits or Reviews of Historical Financial Information” (2013). New York, NY: IAASB.

INTERNATIONAL AUDITING AND ASSURANCE STANDARDS BOARD. “ISAE 3410, Assurance

Engagements on Greenhouse Gas Statements.” (2011). New York, NY: IAASB. INTERNATIONAL AUDITING AND ASSURANCE STANDARDS BOARD. “ISRE 2400 (Revised),

Engagements to Review Historical Financial Statements” (2012). New York, NY: IAASB.

INTERNATIONAL FEDERATION OF ACCOUNTANTS (IFAC). The Determination and Communication

of Levels of Assurance other than High.” New York, NY: International Federation of Accountants (2002).

KOZMINSKY, E. “Altering Comprehension: The Effect of Biasing Titles on Text Comprehension.” Memory and Cognition 5 (1977): 482−490.

LEVIN, I.P., and G.J. GAETH. “Framing of Attribute Information Before And After Consuming

The Product.” Journal of Consumer Research 15 (1988): 374−378.

27

LEVIN, I.P., G.J. GAETH, J. SCHREIBER, and M. LAURIOLA. “A New Look at Framing Effects: Distribution of Effect Sizes, Individual Differences, and Independence of Types of Effects.” Organizational Behavior and Human Decision Processes 88 (2002): 411−429.

LEVIN, I.P., S.L. SCHNEIDER, and G.J. GAETH. “All Frames Are Not Created Equal: A Typology

and Critical Analysis of Framing Effects.” Organizational Behavior and Human Decision Processes 76 (1998): 149−188.

LIBBY, R., R. BLOOMFIELD, and M. W. NELSON. “Experimental Research in Financial

Accounting.” Accounting, Organizations and Society 27 (2002): 775−810. MAINES, L. A., and L. S. MCDANIEL. “Effects of Comprehensive-Income Characteristics on

Nonprofessional Investors’ Judgments: The Role of Financial-Statement Presentation Format.” The Accounting Review 75 (2000): 179−207.

MAKSYMOV, E., M.W. NELSON, and W.R., KINNEY, Jr. “Effects of Procedure Frame, Procedure

Verifiability, and Audit Efficiency Pressure on Planning Audits of Fair Values.” Working paper, Cornell University and University of Texas at Austin, 2014.

MATSUMURA, E.M.; R. PRAKASH; and S.C. VERA-MUÑOZ. “Firm-Value Effects of Carbon

Emissions and Carbon Disclosures.” The Accounting Review 89/2 (2014): 695-724. MOCK, T.J.; S.S. RAO; and R.P. SRIVASTAVA. “The Development of Worldwide Sustainability

Reporting Assurance.” Australian Accounting Review 23 (2013): 280-294. MUMMENDEY, A., AND S. OTTEN. “Positive-Negative Asymmetry in Social Discrimination.” In

W. Stroebe and M. Hewstone (Eds.), European Review of Social Psychology 9, pp. 107-143 (1998). Chichester, UK: Wiley.

MUMMENDEY, A., S. OTTEN, U. BERGER, AND T. KESSLER. “Positive-Negative Asymmetry in

Social Discrimination: Valence of Evaluation and Salience of Categorization.” Personality and Social Psychology Bulletin 26 (2000): 1258-1270.

NELSON, M.W. “A Model and Literature Review of Professional Skepticism in Auditing.”

Auditing: A Journal of Practice and Theory 28 (2009): 1−34. NOSOFSKY, R.M, AND T.J. PALMERI. “A Rule-Plus-Exception Model for Classifying Objects in

Continuous-Dimension Spaces. Psychonomic Bulletin & Review 5 (1998): 345−369. PALMERI, T.J., AND R.M. NOSOFSKY. “Recognition Memory for Exceptions to the Category

Rule.” Journal of Experimental Psychology: Learning, Memory, and Cognition 21 (1995): 548−568.

PALMROSE, Z-V. “An Analysis of Auditor Litigation and Audit Service Quality.” The Accounting

Review 63 (1988): 55−73.

28

PRATTO, F., AND O. JOHN. “Automatic vigilance: The attention-grabbing power of negative social information.” Journal of Personality and Social Psychology 61 (1991): 380−391.

RICHEY, M., R.J. KOENIGS, H.W. RICHEY, AND R. FORTIN. “Negative Salience in Impressions of

Character: Effects of Unequal Proportions of Positive and Negative Information.” Journal of Social Psychology 97 (1975): 233-241.

SHERMAN, S.J.; D.M. MACKIE; AND D.M. DRISCOLL. “Priming and the Differential Use of

Dimensions in Evaluation.” Personality and Social Psychology Bulletin 16 (1990): 405−418.

SIEGEL, S., AND N.J. CASTELLAN, JR. “The Case of k Independent Samples (Chapter 8).” In

Nonparametric Statistics for the Behavioral Sciences (2nd Edition), 216−221 (1988). Boston, MA: McGraw-Hill.

SIMNETT, R.; M. NUGENT; and A. L. HUGGINS. “Developing an International Assurance Standard

on Greenhouse Gas Statements.” Accounting Horizons 23 (2009): 347–363. SIMNETT, R.; A. VANSTRAELEN; and W. F. CHUA. “Assurance on Sustainability Reports: An

International Comparison.” The Accounting Review 84 (2009): 937−967. TAYLOR, S.E., J. CROCKER, S.T. FISKE, M. SPRINZEN, AND J.D. WINKLER. “The

Generalizability of Salience Effects.” Journal of Personality and Social Psychology 37 (1979): 357−368.

TAYLOR, S.E., AND S.T. FISKE. “Salience, Attention, and Attribution: Top of the Head

Phenomena.” In L. Berkowitz (Ed.), Advances in Experimental Psychology (Vol. 11). New York: Academic Press (1978).

VERA-MUÑOZ, S.C.; W.R. KINNEY, JR.; AND S.E. BONNER. “The Effects of Domain Experience

and Task Presentation Format on Accountants’ Information Relevance Assurance.” The Accounting Review 76 (2001): 405−429.

29

FIGURE 1 Assurance Level, Risk, Procedures across Standards, and Conclusion Frame

Panel A. Financial Statement Assertions

Reasonable Assurance (Audits/Examinations)

Limited Assurance (Reviews)

PCAOB (AU508)

IAASB (ISA700)

AICPA (AR90)

IAASB (ISRE 2400) (ISRE 2400 revised)

Stated assurance Reasonable Reasonable Limited Moderate (Appendix 3) Limited (> “inconsequential”)3

Target level risk Low Low Reasonable basis1 Moderate (para.9)2 Varies – User assessed Procedures Prescribed Prescribed Analytical proc. & Inq. Analytical proc. & Inq.

(26(d)(ii)) Analytical proc. & Inq.

Conclusion frame Positive Positive Negative Negative Negative Panel B. Other than Financial Statement Assertions

Reasonable Assurance (Audits/Examinations)

Limited Assurance (Reviews)

PCAOB (AT101)

IAASB (ISAE 3000 revised)

PCAOB (AT101)

IAASB (ISAE 3000) (ISAE 3000 revised)

Stated assurance Reasonable Reasonable Limited Limited (> “inconsequential”)

Limited (> “inconsequential”)

Target level risk Low Low Moderate Varies – User assessed Varies – User assessed Procedures Vary – Aud.

choice, broad summary

Vary – Aud. choice, broad summary

Analytical proc. & Inq.

Vary – Aud. choice, detailed summary

Vary – Aud. choice, detailed summary

Conclusion frame Positive Positive Negative Negative Negative 1 A “reasonable basis” is that “ordinarily attained by applying appropriately designed and implemented analytical procedures and inquiries of management.” 2 Practitioner is required to “obtain moderate assurance as to whether the financial statements are free of material misstatements.” (para. 9) 3 Limited assurance can vary from “just above assurance that is likely to enhance the intended users’ confidence about the subject matter information to a degree that is clearly

more than inconsequential to just below reasonable assurance. (ISAE 3000.A6, emphasis added)

30

FIGURE 2 Expectations and Research Design

Conditions: Cell 1 Cell 2 Cell 3 Cell 4 Cell 5 Excluded Procedure Noted Noted No Excluded Procedure Noted

Conclusion Frame

Negative

Positive Opinion

Stated Level of Assurance Limited Reasonable

Procedures Performed 4 Procedures 5 Procedures

Perceived Confidence

+

+ +

+ + +

+ + + +

+ + + + + Perceived Assurance (CONF) Increasing →→→→→

Description of Condition

EXCL / NEG / LA /

PROC4 /

NEX / NEG / LA /

PROC4 /

NEX / POS / LA /

PROC4

NEX / POS / RA /

PROC4

INC / POS / RA /

PROC5 Legend: Manipulated Variable: Excluded Procedure EXCL = Excluded procedure noted;

NEX = Excluded procedure not noted INC = Procedure Included and noted

Conclusion Frame NEG = Negative conclusion; POS = Positive Opinion Procedures Performed PROC4 = 4 Procedures;

PROC5 = 5 Procedures

Stated Level of Assurance LA = Limited Assurance; RA = Reasonable Assurance Predictions and tests:

Prediction Test Exclusion matters CONF (cell 1) < CONF (cell 2), i.e., CONF1 < CONF2 Conclusion frame matters CONF (cell 2) < CONF (cell3) Stated assurance matters CONF (cell 3) < CONF (cell4) Procedures matter CONF (cell 4) < CONF (cell5) All factors matter Correlation of CONF with cell rank order per the IAASB’s

assumptions and expected rank analysis

31

FIGURE 3 Plots of Median and Mean CONFIDENCE Assessments

(n = 210) Panel A. Mean and Median CONFIDENCE Assessments, by Rank-Ordered Cells

For expectations and legend see Figure 2.

32

FIGURE 3 Plots of Median and Mean CONFIDENCE Assessments

(n = 210) Panel B. Box-and-Whisker Plots of Median CONFIDENCE Assessments, by Rank-Ordered Cells


33

FIGURE 4 Plots of Median CONF Ranks (MCR) and Expected Median CONF Ranks (E[MCR]) per IAASB


34

TABLE 1 Median and Mean CONFIDENCE Assessments [n = 210]

Panel A. Median CONFIDENCE Assessments by Rank-Ordered Cells [# of participants]

(1)

EXC/LA/NEG/ PR4

(2) NEX/LA/NEG/

PR4

(3) NEX/LA/POS/

PR4

(4) NEX/RA/POS/

PR4

(5) INC/RA/POS/

PR5 60.00

[48] 60.00 [46]

70.00 [46]

81.00 [43]

80.00 [27]

Panel B. Mean CONFIDENCE1 Assessments by Rank-Ordered Cells2

(Standard Deviation) [# of participants]

(1) EXC/LA/NEG/

PR4

(2) NEX/LA/NEG/

PR4

(3) NEX/LA/POS/

PR4

(4) NEX/RA/POS/

PR4

(5) INC/RA/POS/

PR5 56.17

(24.12) [48]

58.83 (19.26)

[46]

64.20 (19.99)

[46]

78.56 (17.03)

[43]

78. 00 (13.61)

[27] 1 Participants were asked, “Given the report you read, how assured (or confident) do you feel that DuLac

Corporation’s Schedule of Greenhouse Gas Emissions is fairly presented in all material respects?” Participants provided their responses on a 0−100 scale, where 0 = “Not at all assured (No confidence),” and 100 = “Completely assured (Complete confidence).”

2 The five combinations of conditions (cells) are ranked-ordered as to expectation of report users’ perceptions of

achieved assurance, from cell 1 increasing to cell 5 representing the expected increasing amount of CONFIDENCE. See legend in Figure 2.

35

TABLE 2 Pairwise Independent-Samples Jonckheere-Terpstra Tests1 for Ordered Cell Medians – Full

Sample [N = 210]

Prediction (CONFi < CONFi+1) Cell Medians Std. J-T-statistic2

p-value (one-tailed)

CONF1 < CONF2 (Does EXCL vs. NEX matter?)

60.00 [48]

60.00 [46]

0.338 0.368

CONF2 < CONF3 (Does NEG vs. POS matter?)

60.00 [46]

70.00 [46]

1.658 0.049

CONF3 < CONF4 (Does LA vs. RA matter?)

70.00 [46]

81.00 [43]

4.291 0.000

CONF4 < CONF5 (Does NEX vs. INC. matter?)

81.00 [43]

80.00 [27]

-0.589 0.722

1 The Jonckheere-Terpstra (J-T) test for ordered alternatives is suitable when the alternative hypothesis for a k independent-samples design specifies the rank-order of the k independent samples (Siegel and Castellan [1988]). Our five combinations of conditions (cells) are ranked-ordered as to expectation of report users’ perceptions of achieved assurance, from cell 1 increasing to cell 5 representing the expected increasing amount of CONFIDENCE. See legend in Figure 2.

2 The overall standardized J-T statistic is 6.575 (p = 0.000), which allows to reject the null hypothesis of no trend in the values of the CONF ratings across our five experimental conditions.

36

TABLE 3 Pairwise Independent-Samples Jonckheere-Terpstra Tests1 for Ordered Cell Medians –

Robustness Tests Panel A. Robustness Test 1 – Excluding participants who failed the stated level of assurance

attention check [N = 190]




60.00 [35]

60.00 [42]

1.196 0.116


60.00 [42]

69.00 [43]

2.109 0.018


69.00 [43]

81.00 [43]

4.193 0.000


81.00 [43]

80.00 [27]

-0.589 0.722

Panel B. Robustness Test 2 – Excluding participants who failed any of the three attention checks

(stated level of assurance, statement of omitted procedure, conclusion paragraph format) [N = 150]




60.00 [21]

60.00 [31]

0.291 0.386


60.00 [31]

70.00 [35]

2.163 0.016


70.00 [35]

82.00 [38]

3.586 0.000


82.00 [38]

80.00 [25]

-0.297 0.617

Panel C. Robustness Test 3 – Including thirty-eight cell 5 participants who also participated in

the earlier administration of the study [N = 248]




60.00 [48]

60.00 [46]

0.338 0.368


60.00 [46]

69.50 [46]

1.658 0.049


69.50 [46]

81.00 [43]

4.291 0.000


81.00 [43]

80.00 [65]

-0.262 0.603

1 Overall standardized J-T statistic is 7.624 (p = 0.000). 2 Overall standardized J-T statistic is 6.559 (p = 0.000). 3 Overall standardized J-T statistic is 7.550 (p = 0.000).

do procedures matter when communicating assurance… · do procedures matter when communicating...

Documents