1 abolaureen maines 2008 choosing participants for financial accounting experiments october 10, 2008...
TRANSCRIPT
![Page 1: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/1.jpg)
1ABO Laureen Maines 2008
Choosing Participants for Financial Accounting Experiments
October 10, 2008
ABO Doctoral Consortium
![Page 2: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/2.jpg)
222ABO Laureen Maines 2008
Objectives
• Identify participants in financial accounting experiments
• Identify characteristics of participants
• Discuss factors to consider when choosing participants
• Provide advice for finding participants
![Page 3: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/3.jpg)
333ABO Laureen Maines 2008
Financial Reporting Process
Standard Setters
Preparers Financial Reports
Analysts Stock Market Reaction
Auditors Investors
Regulators Creditors
![Page 4: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/4.jpg)
444ABO Laureen Maines 2008
Research Process
Research Question (Theory)
Experimental Design and Task
Participants (Theory)
![Page 5: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/5.jpg)
555ABO Laureen Maines 2008
Variables Influencing Judgments and Decisions
• Decision-Maker Characteristics
• Task Characteristics
• Environmental Characteristics
(Sarah Bonner, Judgment and Decision Making in Accounting, Prentice-Hall, 2007)
![Page 6: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/6.jpg)
666ABO Laureen Maines 2008
Participant (Decision-Maker) Characteristics
• External / Easily observable or measured• Gender, Age, Professional Certifications, Work Experience
• Internal / Need to be measured via test or possibly manipulated
• Abilities
• Knowledge
• Preferences
![Page 7: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/7.jpg)
777ABO Laureen Maines 2008
Participant Characteristics - Abilities
• Abilities• Verbal abilities – reading comprehension
• Reasoning abilities - interpret data, reason from analogy, reason analytically
• Spatial abilities – identify and comprehend patterns
• Measurement• Typically through standardized college admission tests – SAT and GMAT
• Manipulation • Difficult in adults
![Page 8: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/8.jpg)
888ABO Laureen Maines 2008
Participant Characteristics - Knowledge
• Knowledge• Semantic
• General world knowledge, domain-specific knowledge, sub-specialty knowledge
• Episodic knowledge/beliefs based on personal experience
• Incentives, reactions of other parties, empirical regularities
• Measurement• Knowledge-specific tests (e.g., CPA exam questions)
• Post-experimental questions about beliefs
• Manipulation • Possible, but need to be careful
![Page 9: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/9.jpg)
999ABO Laureen Maines 2008
Participant Characteristics - Preferences
• Preferences (Utility)• Money (Risk Aversion)
• Effort
• Honesty, ethical behavior, welfare of others
• Measurement• Validated tests from economics, psychology, sociology
• Manipulation• Induced preferences for money (Berg et al. QJE 1986)
![Page 10: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/10.jpg)
101010ABO Laureen Maines 2008
Role of Participant Characteristics
• Independent Variable• Decision-maker characteristic directly affects dependent variable
• McDaniel et al. (TAR 2002)
• Moderating Variable• Decision-maker characteristic changes the relation between a task or environmental variable and the dependent variable
• Elliott (TAR 2006)
• Neither of above – just choosing participants for your task
• Think in terms of a moderating role
![Page 11: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/11.jpg)
111111ABO Laureen Maines 2008
Participant Characteristics as Independent Variable (McDaniel et al.)
X Y
Experience(Accounting)(Business)
Judgment(Reporting Issue Identified – recurring vs. nonrecurring)
Causal relation
![Page 12: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/12.jpg)
121212ABO Laureen Maines 2008
Participant Characteristics as Independent Variable
• Not typical in financial accounting research
• Typically requires measuring participant characteristics – external/internal
• May be hard to avoid confounding • External characteristics (e.g., years of experience) typically represent a number of internal characteristics (e.g., knowledge, ability)
• For example, knowledge and ability are rarely independent (Link 2 in Libby and Luft AOS 1993)
• A problem if you want to establish causality for internal characteristics (need manipulation + random assignment, not measurement)
![Page 13: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/13.jpg)
131313ABO Laureen Maines 2008
Participant Characteristics as Moderating Variables (Elliott 2006)
Y
U
GAAP – ProForma Reconciliation(Yes/No)
Decision
(Investment Decision)
Knowledge
X
![Page 14: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/14.jpg)
141414ABO Laureen Maines 2008
Participant Characteristics as Moderating Variables
• Use this approach even if you aren’t planning on a moderating variable approach
• Does theory suggest that judgments and decisions will depend on characteristics (abilities, knowledge, preferences) of participants? Will the JDM process or outcomes change?
• Does the level of relevant characteristics vary among relevant decision-makers or participant groups?
![Page 15: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/15.jpg)
151515ABO Laureen Maines 2008
Participant Characteristics as Moderating Variables
• If answers to prior questions are yes, how do I proceed?
• Embed theory in paper and test multiple groups that should differ based on theory – true moderating study
• Examine only one group
• What group is most important in terms of the research question?
• External validity limitations
![Page 16: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/16.jpg)
161616ABO Laureen Maines 2008
Participant Characteristics – Professionals versus Students
• Some generalizations
• Basic reasoning abilities typically don’t change much after adulthood
• College students likely have similar abilities to most adults
• Be careful – there can be cross-sectional variation - top people in any field have survived tournaments and are likely to have higher ability
![Page 17: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/17.jpg)
171717ABO Laureen Maines 2008
Participant Characteristics – Professionals versus Students
• Some generalizations
• Knowledge is a big factor in accounting experiments
• Semantic knowledge – world knowledge and sub-specialty knowledge increases with experience
• Episodic Knowledge / Beliefs also change with experience – personal experiences with incentives, behavior of other people
![Page 18: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/18.jpg)
181818ABO Laureen Maines 2008
Participant Characteristics – Professionals versus Students
• Some generalizations
• Preferences
• Reaction to economic incentives is basic human reaction
• Experimental research based on economic theory typically can use abstract tasks and college students
![Page 19: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/19.jpg)
191919ABO Laureen Maines 2008
Issues – Working with Professional Participants
• Negative externalities of using professionals when not needed: limited resource (Libby, Bloomfield & Nelson AOS 2002)
• Places high demands on researcher – good research question requiring professionals, good design, good instrument
• Personal experiences of professionals lead to expectations that will create problems if not met
• Need to run materials by professionals and pilot test
![Page 20: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/20.jpg)
202020ABO Laureen Maines 2008
Issues – Working with Professional Participants
• Selection and non-response bias may be stronger with professionals than with students (Peecher and Solomon 2001)
• End up using one firm – how general are your results?
• Professionals are busy people which leads to non-response
• Need to get them interested and committed
• Work around their schedule
![Page 21: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/21.jpg)
212121ABO Laureen Maines 2008
Source of Participants
• Preparers
• Alumni directory – controllers, CFO’s
• Financial Executive Institute
• Institute of Management Accountants
![Page 22: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/22.jpg)
222222ABO Laureen Maines 2008
Source of Participants
• Investors - Nonprofessional
• College Students
• Think carefully about research question and task
• Insights on using MBA students as proxies for investors – Elliott et al. (TAR 2007)
• National Association of Investors Corporation – 400,000 members
• See Hodge (AH 2003)
![Page 23: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/23.jpg)
232323ABO Laureen Maines 2008
Source of Participants
• Financial Analysts (sell and buy side)
• Alumni Directory
• Individual investment banks, financial management firms, pensions funds, etc.
• Association for Investment Management and Research
• Nelson’s Directory of Investment Managers
![Page 24: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/24.jpg)
242424ABO Laureen Maines 2008
Source of Participants
• Standard setters and regulators
• Good luck!
![Page 25: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/25.jpg)
252525ABO Laureen Maines 2008
Conclusions
• Appropriate choice of participants requires a good research question, a good design and task, and careful consideration of how personal characteristics affect JDM
• How does theory suggest different levels of a characteristic would change JDM?
![Page 26: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/26.jpg)
262626ABO Laureen Maines 2008
Conclusions
• Be enthusiastic and able to explain why people should care about your research
• Practice describing your research to your accountant and non-accountant friends
• BE PERSISTENT !!! (but not annoying!)
![Page 27: 1 ABOLaureen Maines 2008 Choosing Participants for Financial Accounting Experiments October 10, 2008 ABO Doctoral Consortium](https://reader035.vdocuments.net/reader035/viewer/2022070305/5514c2b6550346b0338b4983/html5/thumbnails/27.jpg)
272727ABO Laureen Maines 2008
References
• Berg, J.E., Daley, L.A., Dickhaut, J.W., and J. O’Brien. 1986. Controlling preferences for lotteries on units of experimental exchange. Quarterly Journal of Economics 101 (May): 281-306.
• Elliott, W. B. 2006. Are investors influenced by pro forma emphasis and reconciliations in earnings announcements? The Accounting Review 81 (January): 113-133.
• Elliott, W. B., F. Hodge, J. Kennedy, and M. Pronk. 2007. Are MBA students a good proxy for nonprofessional investors? The Accounting Review (January):139-168.
• Hodge, F. D. 2003. Investors’ perceptions of earnings quality, auditor independence, and the usefulness of audited financial statements. Accounting Horizons 17(Supplement): 37-48.
• Libby, R., Bloomfield, R., and M. Nelson. 2002. Experimental research in financial accounting. Accounting, Organizations and Society 27 (November): 775-810.
• Libby, R. and J. Luft. 1993. Determinants of judgment performance in accounting settings: Ability, knowledge, motivation, and environment. Accounting Organizations and Society 18 (July):425-450.
• McDaniel, L.S., Martin, R. D., and Maines, L.A. (2002). Evaluating financial reporting quality: The effects of financial expertise vs. financial literacy. The Accounting Review 77 (Supplement): 139-167.
• Peecher, M. and I. Solomon. (2001). Theory and experimentation in studies of audit judgments and decisions: Avoiding common research traps. International Journal of Auditing (November): 193-203.