a systems approach to establishing scientific integrity in evidence based policy making
DESCRIPTION
A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making. Prof. Dr. Wijnand J. Swart Centre for Plant Health Management (CePHMa) Faculty of Natural and Agricultural Sciences University of the Free State Bloemfontein South Africa [email protected]. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/1.jpg)
A Systems Approach to Establishing Scientific Integrity in
Evidence Based Policy Making
Prof. Dr. Wijnand J. SwartCentre for Plant Health Management (CePHMa)
Faculty of Natural and Agricultural SciencesUniversity of the Free State
Bloemfontein South Africa
![Page 2: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/2.jpg)
EVIDENCE in Policy Context • Sound, credible and robust evidence, whether :
– quantitative– qualitative– statistical– economic– attitudinal / behavioural– anecdotal– social– opinion based – or review based .....
......is an essential and necessary part of the enabling environment for formulating policies that are coherent and effective in terms of their outcomes.
![Page 3: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/3.jpg)
Use GOOD data & information….
Source: DEFRA, UK
Use POOR data & information….
…and use it POORLY…
There are policies that:…and use it WELL…
The Policy Making Process• Techniques, analyses and judgements used to evaluate and
formulate data and information into knowledge / evidence for making effective policies are critical.
![Page 4: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/4.jpg)
What is ‘Sound Evidence?’• Concept of “sound and credible evidence”
is very complex.
• Dependant on inter alia:– types– sources – generating techniques– context – ‘understanding’
...... of data / information / knowledge
![Page 5: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/5.jpg)
DATA
KNOWLEDGE
INFORMATION
WISDOM
CO
NTE
XT
IND
EPEN
DEN
CE
UNDERSTANDING
understanding relations
understanding patterns
understanding principles
• Data are facts (e.g. numbers, names, symbols) and have little value in themselves.
• Information relates to description, definition, or perspective (what, who, when, where).
• Knowledge comprises strategy, practice, method, or approach (how).
• Wisdom embodies principle, insight, morals, or archetypes (why).
• Absolute Truth ?
TRUTH?
‘Sound Evidence’: Context & Understanding
![Page 6: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/6.jpg)
Knowledge vs Science • Science is organized knowledge. Herbert Spencer
• Science: a knowledge of principles and causes.(Webster's Revised Unabridged Dictionary)
• Science is a way of thinking much more than it is a body of facts. Carl Sagan
• “Good Science” could be defined as those practices which contribute most to advances in understanding.
Sindermann: The Joy of Science 1985.
• Science is to see what everyone else has seen but think what no one else has thought. Albert Szent-Gyorgyi
![Page 7: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/7.jpg)
Science and Policy Making• Creative and innovative contributions of
scientists and policy makers and the trust engendered in them by the public, to whom they are accountable is of paramount importance.
• Follows therefore that fostering an environment that promotes research or scientific integrity is an integral part of that accountability and the pursuit of new knowledge.
![Page 8: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/8.jpg)
‘Scientific Integrity’• “Integrity” by definition:
– “honesty” – “a state of being entire or whole”
• Perspectives of ‘scientific integrity’ :
1. Ethical issues relating to misconduct (fraud) or manipulation, suppression, or distortion of facts.
2. Striving towards “wholeness” or excellence in the search for knowledge.
![Page 9: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/9.jpg)
• Essential for maintaining scientific excellence and for keeping the public’s trust.
• Research integrity characterizes :
b. Institutional integrity: creating an environment that promotes responsible conduct and
high levels of integrity embracing standards of excellence, trustworthiness, and
lawfulness
c. Individual integrity: Scientist’s commitment to intellectual honesty and personal responsibility and is an aspect of moral character and experience. A good scientist must:
communicate well obtain research grants excel in teaching and mentoring engage in ethical decision making use knowledge ‘wisely’ to plan and execute research.
Research Integrity
![Page 10: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/10.jpg)
Institutional Integrity:Politics vs Science
• Politicization of science as old as science itself e.g Galileo's theory that the Earth revolves around the sun perceived as a challenge to the authority of the Catholic church.
• Political interference threatens the integrity of government science and policy making all over the world.
• Manipulation, suppression, and distortion of government science misinforms public and leads to poor policy decisions.
• Especially rife in developing countries, e.g. the assertion of the South African president Thabo Mbeki that AIDS is not caused by HIV flew in the face of decades of research and threatened to undermine proper treatment of the disease.
![Page 11: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/11.jpg)
Bush Administration's Misuse of Science • On February 18, 2004, 62 pre-eminent
scientists AND researchers released a statement titled Restoring Scientific Integrity in Policy Making in the USA.
• Scientists charged the Bush administration with widespread and unprecedented “manipulation of the process through which science enters into its decisions.”
• Scientists accused Bush administration of :
1. Epidemic altering and concealing of scientific information by senior officials in various federal agencies.
2. Active censorship of scientific information that the administration considered threatening to its own philosophies
3. Restriction of the ability of government-supported scientists to freely communicate scientific ideas related to "sensitive" issues .
![Page 12: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/12.jpg)
Integrity of Research Institutions• The organizational structure and processes that typify the
mission and activities of a research institution can either promote or detract from the responsible conduct of research.
• These process are in part determined by the external environment and are influenced by the dynamics between and among organizational members.
• Any element or part of an organization can be viewed as a system in and of itself.
• External conditions influence the inputs into an organization, affect the reception of outputs from an organization’s activities, and directly affect an organization’s internal operations.
![Page 13: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/13.jpg)
Source: National Academy of Sciences - http://www.nap.edu
Open systems model of internal environmental elements of a research organization showing: Inputs / resources for organizational functions Structures and processes that define an organization’s operation Outputs / outcomes of activities carried out by individual scientists, research
groups or teams, and other research-related programs.
![Page 14: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/14.jpg)
• Interrelatedness between research organizations and the various external influences that have an impact on integrity in research.
• Systems and subsystems of the external-task environment are embedded within the general sociocultural, political, and economic environment.
• Relationships also exist between and among the elements within the external environment.
Source: National Academy of Sciences - http://www.nap.edu
![Page 15: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/15.jpg)
Holistic Knowledge / Evidence• Rather than focus on the ethical or moral aspects of scientific
integrity, focus here is on the process of generating data and information and integrating it into sound knowledge (sound evidence) for decision-making.
• “Integrity” by definition:– “honesty” – “a state of being entire or whole”
• “Integrate” by definition:– “to combine parts into a whole”
“A whole.. is more than the sum of its parts.”
Jan Christian Smuts, Holism and Evolution
1926
![Page 16: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/16.jpg)
• A collection of data is not information. • A collection of information is not knowledge. • A collection of knowledge is not wisdom.
• Information, knowledge, and wisdom are more than simply collections
• Each concept represents more than the sum of its parts and has a synergy of its own.
‘Sound Evidence’: A Holistic View
![Page 17: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/17.jpg)
A “WHOLE” AS A SYSTEM• “ A system is defined as a set of interacting units with
relationships among them. The properties (or behaviour) of the system as a whole emerge out of the interaction of the components comprising the system.”
• The interactions of the parts become more relevant to understanding the system than understanding the parts.
• This definition of a system implies something beyond cause and effect.
![Page 18: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/18.jpg)
CO
MP
LEX
ITY
NUMBER OF SUB-SYSTEMS
particleatom
moleculecell
organperson
communitystate
nationworldsolar system
galaxyuniverse
• In truth only one system, the “Universe"
• All systems are sub-systems of a larger system. ?
The Ultimate System
![Page 19: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/19.jpg)
Systems Thinking and Policy
• Science is a way of thinking much more than it is a body of facts. Carl Sagan
• “Systems thinking” offers a conceptual framework or model for ‘thinking differently’.
• Systems thinking has permeated many scientific
fields including: education, business management, human development, sociology, psychology, agriculture, ecology and biology, earth sciences.
![Page 20: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/20.jpg)
• Adopting a systemic perspective on solving policy problems therefore appear to offer a useful way of correcting these deficiencies.
• In 1960s, a ‘hard’ (quantitative) systems approach was touted as the policy science.
• However, hopes not realized for variety of reasons; its comprehensive modelling too information-intensive and mathematical.
• The ‘soft’ (qualitative) systems approach of systems thinking has increasingly been used since the 1990’s as a paradigm in policy planning and implementation.
• Soft systems methods stress the self-organizing and adaptive capacities of appropriately designed systems.
Hard vs Soft Systems
![Page 21: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/21.jpg)
Soft systems methods
• Subjective (interpretive) philosophy• Systems + sociological theory base• Flexible methodology• Organizational problem-solving
focus• Creative / intuitive• Analyst is facilitator• Participative• Organizational learning outcomes• Several ambiguous outcomes
Hard systems methods
• Objective philosophy• Computer science + systems theory• Rigid method• Data, process, database technical
focus• Scientifically analytical• Analyst is expert• Analyst dominated• Computer design outcomes• One ‘correct’ solution
Hard vs Soft Systems
![Page 22: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/22.jpg)
A hard systems view of a farming system; a biological network suitable for mathematical solution.
A soft systems view of a farming system; an arena for gaining experience and increased understanding.
Source: Robinson, B. 2003. 11th Australian Agronomy Conference)
Agro-ecosystem
![Page 23: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/23.jpg)
Soft Systems Methodology (SSM)• Can be used both for general problem solving and in
the management of change.
• Used in the analysis of complex situations where there are divergent views about the definition of the problem — "soft problems" or policy options (e.g. How to improve health services delivery; How to manage disaster planning).
• At the heart of SSM is a comparison between the world as it is, and some models of the world as it might be.
• Out of this comparison comes a better understanding of the world ("research"), and some ideas for improvement ("action").
![Page 24: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/24.jpg)
SSM for Problem Solving
• Differences between models and reality become the basis for planning and policy making process.
reality
conceptual models
understanding and
improvement
• ‘Classic form’ of SSM consists of seven steps:– Problem unstructured by researchers – Problem situation expressed to capture “rich picture” – Create root definitions of relevant systems (i.e. social, political &
environmental) – Making and testing conceptual models based upon world views – Comparing conceptual models with reality – Identifying feasible and desirable changes – Acting to improve the problem situation
![Page 25: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/25.jpg)
Multi-agent systems (MAS)• Policy increasingly has to address topics that have to do with
disequilibrium, dynamics, and locality.
• The overwhelming complexity of biophysical and socio-economic constraints that increasingly characterize rural areas in developing countries necessitates the development of more sophisticated tools to support policy making in these areas.
• Multi-agent systems (MAS) are a relatively new field in computer science that have been proposed as a modelling approach for establishing even higher levels of scientific integrity in the generation and evaluation of evidence for making policies.
• Analogous to artificial intelligence.
![Page 26: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/26.jpg)
• Multi-agent models might be the preferred choice when heterogeneity and interactions of agents and environments are significant and policy responses cannot be aggregated linearly.
• MAS can thus complement bio-economic simulation models which cannot fully capture the heterogeneity in biophysical and socio-economic constraints and the interactions between them.
• There are several policy questions in the context of agricultural development of rural areas where MAS simulations may generate useful information for decision making on public investments in R&D and the targeting of policy interventions.
• Examples of such policy questions:– Should funds be spent on crop breeding for stress resistance
or in research for improved crop management?– Should micro-finance be promoted or should agricultural
inputs be subsidized?
MAS for Policy Making
![Page 27: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/27.jpg)
• “An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors”
• Agents may be persons, farms, markets, computer programmes or anything that is reactive, autonomous, and goal-oriented.
• Agents may have the ability to communicate with other agents, learning, mobility, and flexibility. May even have personality and show emotions!
AGENT
ENVIRONMENTINPUTOUTPUT
Agents
SYSTEM
SENSOREFFECTOR
![Page 28: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/28.jpg)
Agent Flexibility• An intelligent agent is capable of flexible autonomous action.
• FLEXIBLE:– Reactive: A reactive system is one that maintains an
ongoing interaction with its environment, and responds to changes that occur in it (in time for the response to be useful)
– Pro-active: Generating and attempting to achieve goals; not driven solely by events; taking the initiative i.e. goal directed behavior; recognizing opportunities
– Social: Ability to interact with other agents via some kind of agent-communication language, and perhaps even cooperate with others.
![Page 29: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/29.jpg)
Identifying a problem
“Improving” problem
MANAGEMENT
ANALYSISUnderstanding the
problem
Definition of the system
Maintaining the system
MANIPULATION
ANALYSIS
Understanding the system
REACTIVE PROACTIVE
POLICY POLICY
PROBLEM SOLVING
![Page 30: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/30.jpg)
BIBLIOGRAPHY1. Balmann, A. 2000. Modeling land use with multi-agent systems: perspectives
for the analysis of agricultural policies.2. Berger, T. and Ringler, C. 2002. Trade-offs, efficiency gains and technical
change – Modeling water management and land use within a multiple-agent framework, Quarterly Journal of International Agriculture 41:119–144.
3. Berger, T., Schreinemachers, p., and Woelcke, J., 2006. Multi-agent simulation for the targeting of development policies in less-favored areas. Agricultural Syatems 88:28-43.
4. Checkland, P. 1981 Systems thinking, systems practice. Chichester: Wiley. 5. Checkland, P., and Holwell, S. 1998 Information, systems, and information
systems: making sense of the field. Chichester, UK: Wiley. 6. Checkland, P. and Scholes, J. 1991 Soft systems methodology in action.
Chichester: Wiley. 7. Harrison MI. 1994. Diagnosing Organizations: Methods, Models, and
Processes, 2nd ed. Thousand Oaks, CA: Sage.8. Schreinemachers, P. Berger, T. and Aune, J.B., 2007. Simulating soil fertility
and poverty dynamics in Uganda: A bio-economic multi-agent systems approach. Ecological Economics 64:387-401
9. Union of Concerned Scientists. 2004. Scientific integrity in policy making: An investigation into the Bush administration's misuse of science. Cambridge (Massachusetts): Union of Concerned Scientists; 49 pp.
10. Woolridge, M. 2002. An Introduction to Multiagent Systems by John Wiley & Sons (Chichester, England).
![Page 31: A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making](https://reader035.vdocuments.net/reader035/viewer/2022062521/56813a92550346895da28dc8/html5/thumbnails/31.jpg)
THANK YOU!