the science of conceptual systems: its history and...

© Meaningful Evidence, 2014. Reprint with permission only | www.meaningfulevidence.com 1

In this paper, we briefly present the emerging science of conceptual systems:

■ Its roots in history

■ Its decades of development

■ Its recent advances

■ Its usefulness today for evaluating and improving strategic plans, policies, and programs

Conceptual systems is a general term for theories, policies, strategic plans, and models. In general, we like to think of these as maps because they serve as guides for planning, decision-making, and communication.

More effective conceptual systems, or maps, help us navigate to greater success. They help us to better understand the world around us, more effectively manage change, and communicate key insights to internal and external stakeholders.

Historic Roots

Much of philosophy, in one way or another, has worked towards greater understanding of data and logic because those are the tools we use to understand reality and find truth. In ancient times, great philosophers such as Aristotle, Plato, and Confucius

White Paper

The Science of Conceptual Systems: Its History and Usefulness for Improved Decision-Making and Organizational Success Dr. Steven E. Wallis & Dr. Bernadette Wright, September 2014

were focused on making careful observations and gathering data. While they made some inroads into understanding logic, it wasn’t until the Dark Ages when the focus shifted. Instead of searching for observable data, members of the church delved into more spiritual matters.

One great difficulty with religious arguments is the difficulty of supporting claims with proof in the form of objective data. Without much in the way of data that they could see and test, those theologian-philosophers developed new rules for what counted as a valid argument. It was at this stage when scholars such as Thomas Aquinas and William of Ockham advanced philosophy in Europe by developing more rigorous rules of logic.

Modern science began to emerge in the mid-1500s in part as a response to theology’s lack of reliable data. To these scientists, evidence was the new king. The works of Galileo and Newton showed how careful observations of repeatable phenomena could lead to new understandings.

New research suggests that it was the

combination of data and logic that fueled

the scientific revolution and led to rapid

advances in chemistry, physics, biology,

and other fields.


The Science of Conceptual Systems: Its History and Usefulness for Improved Decision-Making and Organizational Success

Importantly, although these scientists focused on finding data, they worked within a tradition of rigorous logic. That combination of data and logic fueled the scientific revolution and led to rapid advances in chemistry, physics, biology, and other fields of study and practice.

The theoretical maps developed during that revolution led to amazing advancements in technology. Today, we have computers, cell phones, nuclear power, and space travel (just to name a few). Although we have made great progress with our physical technology, our understanding of social systems has lagged far behind.

Today, we face massive problems. Nations struggle with war, famine, pollution, crime, and injustice. Corporations and community organizations face huge challenges as they strive to achieve success. On a personal level, our ability to understand ourselves and each other is very limited. Those limitations lead to misunderstandings and frustration. Ultimately, these are social problems. They are understood and engaged with our conceptual systems – our mental models, theories, policies, plans, and maps.

As individuals, and in organizations, our understandings of these problems are relatively primitive. That is why they persist as problems. We have been unable to achieve reliable success because we have not had a “scientific revolution” in the social sciences. Our best efforts at psychology, business, sociology, policy, and economics lead more often to failure than success.

Of course, the social sciences have been trying to advance – trying to find new maps to solve problems and achieve greater success. Drawing on one lesson from the scientific revolution, social scientists have been relying more on “data” to build their maps.

This focus on data, however, has not led to much success. This has been surprising to some because we live in the “information age” and so we have all the data in the world to draw from.

What happened? With all that data, why haven’t we seen revolutionary improvement in the social sciences? Where are the amazing improvements in business and policy?

One important piece of the puzzle emerged with a new perspective on the scientific revolution [1]. Research showed that the

success of theories in the scientific revolutions was due to the logic as

much as the data. For the social sciences, therefore,

we conclude that the lack of a successful revolution has been due to the building of

theories from data alone, without the use of effective logics.

Decades of Developing Logics

Let us take a moment to explain what we mean by “logic” because the term is often misunderstood and misused. From the scientific revolution to the present day, scholars in the natural sciences (such as physics and biology) have essentially relied on what we now think of as Toulmin’s logic – a structure of argument used to find “truth.”

They compare claims, counter claims, and supporting evidence. Then, they accept the side with the best claim (and the most evidence) as true and reject the other side as false.

Although this may have been a reasonable logic for the physical sciences, it has been insufficient for dealing with the complexities of the social sciences.

The social sciences’ lack of success is

due to building theories from data alone,

without the necessary effective logics.



Instead of limiting ourselves to Toulmin’s logic, research has identified five “structures of logic” that a claim can take [2]:

■ Atomistic (e.g. C is true/real/important)

■ Linear (e.g. More A causes more C)

■ Circular (e.g. More A causes more B, which causes more C, which causes more A)

■ Branching (e.g. More C causes both more A and more B)

■ Concatenated (Both more A and more B cause more C).

Some of these logics are more useful than others for developing effective maps such as theories, policies, and plans. Very briefly:

■ Atomistic logic structures seem useful only for claiming what something “is.”

■ Linear logics fail to reflect the dynamic complexity of our world.

■ Circular logics are generally understood to be “tautological.” Therefore, they are of limited usefulness.

■ Branching structures create difficulty in distinguishing what is really happening. This is because, “coincidences may be explained by a common cause, but not by a common effect” [3, p, 185].

■ The concatenated structure seems most closely associated with effectively useful conceptual systems [1].

Only through the use of logics do we understand the meaning behind the data. If we cannot differentiate effectively between our logics, how can we differentiate between our understandings of the world? As we will discuss below, concatenated structures are associated with more effective understandings and the ability to enable useful change.

Philosophical Base: Concatenated Logic

The work of Bateson [4] provides an example of the effectiveness or usefulness of concatenated logics. He refers to “dual description” where two views of something are better than one. We also see the concatenated logic in the classic Hegelian dialectic [e.g. 5]. There, both thesis (A) and antithesis (B) lead to synthesis (C). The idea of a concatenated aspect is similar to the idea of “emergence” (where something new may be seen or understood).

For an example of emergence, both hydrogen and oxygen are gasses. Yet, when combined, they become water which is a liquid. The water has properties that allow it to be used for many things that are not suitable for hydrogen or oxygen alone!

Concatenated logics provide a useful alternative to Toulmin’s logic. This is important because (for example) evaluating policy claims from a Toulminian approach means endless clashes of claims and counter claims. In the world of social interaction, that approach has historically caused divisive partisan arguments. The focus on advancing arguments (instead of advancing understanding) has reduced our ability to solve complex social problems, create a better world, and achieve greater success.

Today, researchers and business leaders

generally accept systems thinking as a

useful approach.



Systems Thinking

Our ability to create more effective conceptual systems, or maps, has its roots in the study of other systems. These include physical, chemical, biological, and social systems such as management. For a brief overview, see [6]; for greater depth, see [7].

To provide some orienting literature, it is worth noting that Hammond [8] provides a history of systems theory, while Daneke [9] shows the breadth of the theory. Stacey, Griffin, and Shaw [10] apply systems theory to management. Steier [11] draws on an understanding of cybernetics (closely related to systems thinking) to explore reflexive research and social construction. Yolles [12] shows how advances in complexity theory, systems theory, and cybernetics suggest that a systemic perspective provides a useful lens for viewing management theory.

In the world of business, systems thinking gained prominence in the works of Peter Senge (The Fifth Discipline), Margaret Wheatley (Leadership and the New Science), and a growing list of others.

Today, researchers and business leaders generally accept systems thinking as a useful approach. At research facilities such as the Santa Fe Institute and at universities around the world (for example, see: http://www.systemdynamics.org/courses), researchers are applying the systems perspective to better understand our world.

It is reasonable to apply the systems perspective to a conceptual system because a theory indicates changes among multiple interrelated events [1] (p. 156).

Integrative Complexity

The study of conceptual systems began with the idea that concepts in our minds are connected in some kind of interrelated structure [13, 14] – a kind of mental map.

Building on the idea that we hold maps in our minds, a small group of dedicated scientists set out to

measure those structures. They found maps reflected in various texts – including correspondence, speeches, and declarations. Scholars developed Integrative Complexity (IC) to analyze those texts. They rated the relationship between concepts within a text on a scale of one to seven. Simpler statements (having fewer concepts and fewer relationships between them) have a lower score while more complex texts have a higher score. Studies have been conducted with students, managers, revolutionaries, and world leaders. Importantly, this research quantified the relationship between the systemic complexity of the statements and their ability to achieve success.

For example, Suedfeld and Rank [15] used IC to investigate decision making by leaders of successful revolutions. Their analyses of speeches and personal correspondence showed that the thinking of leaders during military conflict reflected a fairly low level of systemic structure. However, those leaders who became successful politicians showed a significant increase in the complexity of their communication. In contrast, leaders who could not adjust their thinking to the greater complexity of political life (could not develop more complex maps) generally had very short careers. In short, they failed to adapt because they failed to create more complex maps.



Raphael [16] investigated statements made by the US and the USSR during the Berlin crisis between 1946 and 1962. That study found statements became simpler as the sides moved toward crisis. In contrast, the statements became more complex as the sides explored options for resolution. One might say that the national-level mind map became more complex. This research suggests how we may analyze texts to predict crises and even predict surprise attacks [17].

Importantly, the IC research stream shows that maps having higher levels of systemic structure (maps that include more ideas with more connections between them) reflect increased ability to understand the world, to lead, and to make effective decisions to achieve desired results.

Propositional Analysis (PA) & IPA

More recently, Wallis [18] developed Propositional Analysis (PA) by building on insights from systems thinking and grounded theory. PA is a tool for rigorously evaluating maps based on their systemic structure.

Following years of development and refinement, PA studies showed how maps of greater systemic structure would be more effective for practical application and prediction [1]. More recently, PA came to be known as Integrative Propositional Analysis (IPA) based on its usefulness for rigorously integrating maps such as theories, policies, and strategic plans.

In some ways, IPA is more useful than IC because IPA is purposefully made to analyze maps that are more “formal.” These include theories, policies, and strategic plans.

Briefly, IPA provides a useful and innovative approach to evaluating conceptual systems. Instead of the traditional assumption that better data alone leads to better maps, IPA works on the idea that two things are needed: more concatenated logic structures and better data. More procedurally, participation and collaboration among stakeholders provides the third leg supporting this methodology.

Based on this new transdisciplinary approach, IPA has proven useful for evaluating theory and generating insights in a variety of fields. These include physics [1], management [19], social entrepreneurship [20], ethics [21], organizations [22], psychology[23], sociology[24], entrepreneurship [25], policy [2, 26-28], and more.

Importantly, IPA is a unique method for rigorously evaluating conceptual systems (maps) such as strategic plans and policies. That means leaders can use IPA

Better data

(including increased quantity and quality of data)

More concatenated

logics

(higher percentage of concatenated

concepts)

Better maps

(conceptual systems including theories, models, policies, and strategic plans)

Integrative Propositional Analysis (IPA)

is a new method that leaders can use

to assess the potential effectiveness of

policies and strategic plans.



to assess how effective their plans and policies will be in practical application. That also means leaders can create better maps and use those maps to make better decisions and increase their effectiveness.

This is an especially useful approach for making plans that will involve large expenditures and will affect many lives. These plans (along with their related difficulties and decisions) are often part of major change efforts in programs or systems in health care, education, and business.

In creating a policy or plan, all data is (to some extent) useful. However, the most important data can only be found after the plan is initiated. This creates a chicken-and-egg problem. We want to create the best plan for implementation, but can only get the best data after implementation.

IPA is unique because it can be used to evaluate plans before they are put into action.

Using Conceptual Systems for More Effective Research and Action

Conceptual systems are the theories, policies, models, strategic plans, and mental models that we use as maps. Those maps help us to understand the world and to make effective decisions.

Maps are made up of concepts. And, those concepts are connected by logical propositions. Propositions are statements that authors make about what is considered to be true in the present and what may be expected in the future.

Using those propositions for data, we apply IPA to integrate and evaluate maps according to a rigorous process. Those maps with a better logical structure are expected to be more useful in practical application – for understanding and engaging the world around us.

For a brief overview of this emerging “science of conceptual systems” please see: [6]. For a recent paper showing how researchers may use IPA for integrating maps within and between disciplines, please see: [29].

IPA involves six steps:

1. Identify the propositions (statements about what is true or what causes lead to what effects) within one or more conceptual systems (e.g. theories, policies, models, strategic plans).

2. Diagram the causal relationships between the concepts within the propositions (one box for each concept and arrows representing causal relationships).

3. Combine those smaller diagrams (where the concepts overlap) to create a larger, more integrated diagram (map).

4. Identify and count the concatenated concepts (those concepts resulting from two or more causal concepts).

5. Identify and count the total number of concepts to determine the Complexity of the larger, integrated map.

6. Calculate the Systemicity (Robustness, interrelatedness, systemic structure) of the integrated map by dividing the number of concatenated concepts by the total number of concepts. This results in a number between 0 and 1, where 1 means fully systemic (all concepts are concatenated) and 0 means not systemic at all (no concatenated concepts).

A high Complexity score indicates that the conceptual system has more “breadth.” A high level of Complexity means (among other things) that you are less likely to be caught by surprise. That means fewer unwanted side-effects or unanticipated consequences. It also means that more work is required to monitor and

A higher percentage of concatenated logic

structures within a conceptual system

means a greater depth of understanding.

More concepts within a conceptual system

means a greater breadth of understanding.



engage the many organizational systems and sub-systems. So, a higher level of Complexity can be good and useful, but only if the organization has sufficient intellectual capital and other resources to cover all those bases.

A high Systemicity score indicates that the conceptual system has more “depth.” This is an indicator of how well the concepts are understood and how effective the system may be when it is applied in the real world.

A high level of Systemicity also provides the opportunity to identify “leverage points.” By finding structures such as positive and negative feedback loops[30] the organization can reap the greatest benefit with the least effort.

From another perspective, we might think of a low Systemicity score as identifying a kind of system “pathology” [12]. This systems pathology perspective is similar to understanding pathologies in biological and social systems. In these kinds of situations, individuals, teams, and organizations are using defective maps.

Whether the system is conceptual, social, or biological, if the parts are not connected, the system will not operate at its full potential.

IPA is flexible, yet rigorous. Also, it is relatively easy to learn and apply. It is a “paper and pencil” approach rather than a computer modeling approach. With guidance, leaders and teams can learn and apply it to gain useful insights in less than a day.

IPA is an emerging methodology. And, while the number of studies using IPA is small, that number is growing. The overall validity of IPA will be advanced as researchers perform more studies comparing the Systemicity and usefulness of maps across the many

sciences. Another area for validation is to test IPA in real-world planning and decision-making activities.

It is also worth mentioning that the IPA evaluation is focused strictly on the causal relationships between concepts within a conceptual system. This is based on the philosophical insights of Pearl and others [31-34]. While there are other kinds of causation, and other approaches to categorizing information, a pragmatic view suggests that we can’t get much to happen in the world if it is not somehow caused to happen. As managers, planners, and actors, we aim to understand and make purposeful changes in the world.

The world is a complex and uncertain place. The “butterfly effect” suggests that small causes may lead to large and unpredictable effects because of complex and interrelated feedback loops. When creating useful conceptual systems, we must strive to identify and understand complex causality using IPA to evaluate maps based on concatenated logics (including data and causal relationships). This structural approach, based on well-known standards and decades of supporting research, provides a new and powerful tool for creating and evaluating maps such as policies and strategic plans.

Policy

A standard practice among larger businesses, nonprofits, and nations is to conduct high quality research leading to effective policy that will enable such organizations to reach their goals.

Although this is seen as a necessary activity, many studies show that policies are rarely as successful as we would like [35-42]. New innovations continue to emerge including collaborative planning[43], computer modelling [44], and the use of various forms of systems thinking[38, 45-48]. However, to date, none of these has proven highly effective.

Whether the system is conceptual,

social, or biological, if the parts are not

connected, the system will not operate

at its full potential.

We can’t get much to happen in the world

if it is not somehow caused to happen.



The science of conceptual systems perspective suggests a new approach. Instead of blaming policy failure on a lack of data, IPA suggests that the reason for policy failure lies primarily in the lack of structure for that knowledge. Creating an effective map offers a new and useful way to evaluate policy and improve policy, integrate multiple perspectives, and predict the likelihood of success for policies [2, 26-28, 49].

Strategic Planning

Strategic plans are necessary for strategic management. They are maps for navigating business, nonprofit, and government organizations through

change. They are also useful for coordinating operations within and between departments. Strategic

planning emerged in the second half of the 20th century and has become a standard practice for larger firms, though it

is often considered to be too expensive and time-

consuming for smaller organizations. For more detailed information, please see our white paper titled “Strategic Planning 3.0.”

Within firms of any size, leaders operate in an environment of significant ambiguity and even chaos. They are often overwhelmed by data. Strategic planning is primarily about making sense of the data – which is what leaders must do to enable organizational success [50]. In one form or another, this involves creating a kind of map. Leaders often make such maps through a SWOT analysis (Strengths, Weaknesses, Opportunities, and Threats).

However, after decades of experience, SWOT seems to be of very limited usefulness [51]. Even though

it is often used, strategic planning has lost much of its vaunted status [52]. Key to this failure, and the opportunity for engaging in more effective forms of strategic planning, is understanding the difference between structured and unstructured data.

Metaphorically, it is like choosing between two maps for a cross-country trip. One map might consist of a truck-load of phone books. The amount of data is huge – but the collection of data is not well-structured – there is no way to connect the dots. The other map, in comparison, might be a single sheet of paper. That sheet, however, includes dots for cities and interconnected lines for roads. The road map will be much more useful – even though the quantity and quality of data are much less than the truck full of highly accurate addresses and phone numbers. Using IPA, a strategic plan can be made much more effective by starting with a knowledge map. For more information, please see our White Paper titled: “Strategic Knowledge Mapping for Improved Policy & Strategic Planning.”

Strategic Knowledge Mapping

Managerial cognition (along with entrepreneurial cognition) is the process of making sense of information and applying that sense to make effective decisions. It is critical for the success of organizations and entrepreneurs. The study of the field grew rapidly in the second half of the 20th century with insights from Herbert Simon, Karl Weick, Bill Starbuck, Anne Huff, and others.

Those scholars focused on developing more effective maps using a variety of methods. Approaches include identifying causal relationships, using reasonable arguments, paying attention to events in the business environment, categorization of knowledge, and cognitive schema [53].

An important tool in the cognitive process is the creation of a knowledge map. Knowledge maps are very useful for communicating organizational understanding and the coordination of activities across departments.

Policies for achieving goals are rarely

as successful as we would like.

http://meaningfulevidence.com/wp-content/uploads/2014/12/Strategic-Planning-3.0-October-27.pdf

http://www.meaningfulevidence.com/images/strategic_knowledge_mapping.pdf

http://www.meaningfulevidence.com/images/strategic_knowledge_mapping.pdf



Several problems limit the usefulness of traditional approaches to knowledge mapping, however. First, much of the needed information is tacit. It is hidden in the minds of leaders who have organizational “savvy” but have not explained their understanding in a way that others can fully grasp. Second, much of the information is isolated in organizational “silos.” Different departments don’t talk to one another as often as they might, so they do not understand one another.

These issues may be alleviated by providing a venue for disparate groups to share their understandings. A facilitator or coach can help a group to surface and crystalize their shared map.

However, another level of difficulty remains. Even if all the information is “on the table,” the group may not understand how the parts of information relate to one another. That is, they might not see the usefulness or importance of the many perspectives. Those problems are reduced by creating a knowledge map in which concepts are understood as causally interconnected.

By understanding connections, information is more amenable to validation. This approach also supports improved collaboration by showing how all parties contribute to achieving interrelated goals.

There is a final issue which few scholars or practitioners have understood. Previously, we lacked a way to objectively compare two maps – to quantify which of the two would provide a more effective guide for collaboration and navigation. In this breakthrough

science of conceptual systems, IPA provides a method to rigorously and rapidly evaluate maps, so we can measurably improve our maps. By comparing the Complexity and Systemicity of multiple maps, leaders can more effectively choose what map will be the best guide for the organizational journey. Further, they can combine the best pieces from multiple maps to create their own improved map.

Conclusion

From ancient times, philosophers learned to gather data. During the Dark Ages, philosophers developed rigorous logics. That combination of data and logic was sufficient for instigating a revolution in the natural sciences. However, if we are to significantly improve the social and behavioral sciences (and create more useful theories, policies, and strategic plans), we need to use our new understandings of logic and new tools for collaboration.

Research shows how maps that are more Complex and more Systemic will be more effective in practical application. This fits an underlying idea that we live in a world of systems; so, we would expect that a systemic map would be a more useful representation of that world. For example, the very useful laws of physics (e.g. Ohm’s Law) are fully systemic and are very useful in practical application.

Therefore, in analyzing conceptual systems such as knowledge maps, policies, and strategic plans, we are looking for a higher level of Complexity and Systemicity among the interconnected logic claims that are supported by data. Using IPA to rate maps provides an innovative and highly effective way to evaluate our maps. For managers, leaders, and coalitions, IPA provides a pathway for improved decision making and greater organizational success.

A strategic plan can be made much more

effective with a knowledge map.



REFERENCES

1. Wallis, S.E., The structure of theory and the structure of scientific revolutions: What constitutes an advance in theory?, in Cybernetics and systems theory in management: Views, tools, and advancements, S.E. Wallis, Editor. 2010, IGI Global: Hershey, PA. p. 151-174.

2. Wallis, S.E., Structures of logic in policy and theory: Identifying sub-systemic bricks for investigating, building, and understanding conceptual systems. Foundations of Science, 2014. In press.

3. Waldmann, M.R., K. Holyoak, and A. Fratianne, Causal models and the acquisition of category structure. Journal of Empirical Sociology, 1995. 124(2): p. 181-206.

4. Bateson, G., Mind in nature: A necessary unity. 1979, Dutton: New York.

5. Appelbaum, R.P., Karl Marx. Masters of social theory, J.H. Turner, Editor. Vol. 7. 1988, Sage: Thousand Oaks, CA.

6. Wallis, S.E., Propositional analysis for evaluating explanations through their conceptual structures, in Modes of Explanation: Affordances for Action and Prediction, M. Lissack and A. Graber, Editors. 2014, Palgrave MacMillan: New York.

7. Wallis, S.E., Propositional analysis for evaluating explanations through their conceptual structures, in International Society for Complexity and Emergence (ISCE) “Modes of Explanation” 2013: Paris.

8. Hammond, D., The science of synthesis: Exploring the social implications of General Systems Theory. 2003, University Press: Boulder, CO.

9. Daneke, G., A., Systemic choices: Nonlinear dynamics and practical management. 1999. The University of Michigan Press: Ann Arbor, MI.

10. Stacey, R.D., D. Griffin, and P. Shaw, Complexity and management: Fad or radical challenge to systems thinking. Complexity and emergence in organizations, R.D. Stacey, D. Griffin, and P. Shaw, Editors. 2000, Routledge: New York.

11. Steier, F., Research and reflexivity. 1991, London: Sage.

12. Yolles, M., Knowledge cybernetics: A new metaphor for social collectives. Organizational Transformation and Social Change, 2006. 3(1): p. 19-49.

13. Kelly, G.A., The psychology of personal constructs. 1955, Norton: New York.

14. Cronbach, L.J., and Paul E. Meehl, Construct validity in psychological tests. Psychological Bulletin, 1955. 52(4): p. 281-.

15. Suedfeld, P. and A.D. Rank, Revolutionary leaders: Long-term success as a function of changes in conceptual complexity. Journal of Personality and Social Psychology, 1976. 34(2): p. 169-178.



16. Raphael, T.D., Integrative complexity theory and forecasting international crises: Berlin 1946-1962. The Journal of Conflict Resolution, 1982. 26(3).

17. Suedfeld, P. and S. Bluck, Changes in integrative complexity prior to surprise attacks. Journal of Conflict Resolution 1988. 32(4): p. 626-635.

18. Wallis, S.E., From reductive to robust: Seeking the core of complex adaptive systems theory, in Intelligent complex adaptive systems, A. Yang and Y. Shan, Editors. 2008, IGI Publishing: Hershey, PA. p. 1-25.

19. Wallis, S.E., The right tool for the job: Philosophy’s evolving role in advancing management theory. Philosophy of Management - Special Issue (Guest Editors: Stephen Sheard, Mark Dibben, 2012. 11(3): p. 67-99.

20. Wallis, S.E., Seeking the robust core of social entrepreneurship theory, in Social entrepreneurship & complexity, J.A. Goldstein, J.K. Hazy, and J. Silberstang, Editors. 2009, ISCE Publishing: Litchfield Park, AZ.

21. Wallis, S.E., Towards developing effective ethics for effective behavior. Social Responsibility Journal, 2010. 6(4): p. 536-550.

22. Wallis, S.E., Seeking the robust core of organisational learning theory. International Journal of Collaborative Enterprise, 2009. 1(2): p. 180-193.

23. Wallis, S.E. Theories of psychology: Evolving towards greater effectiveness or wandering, lost in the jungle, without a guide? in 30th International Congress of Psychology: Psychology Serving Humanity. 2012, Cape Town, South Africa.

24. Wallis, S.E., Are theories of conflict improving? Using propositional analysis to determine the structure of conflict theories over the course of a century. Under submission.

25. Wright, B. and S.E. Wallis, A revolutionary method to advance entrepreneurship theories. Under submission.

26. Wallis, S.E., Towards the development of more robust policy models. Integral Review, 2010. 6(1): p. 153-160.

27. Wallis, S.E., Avoiding policy failure: A workable approach. 2011, Emergent Publications: Litchfield Park, AZ.

28. Wallis, S.E., How to choose between policy proposals: A simple tool based on systems thinking and complexity theory. E:CO - Emergence: Complexity & Organization, 2013. 15(3): p. 94-120.

29. Wallis, S.E., Existing and emerging methods for integrating theories within and between disciplines. Organisational Transformation and Social Change, 2014. 11(1): p. 3-24.

30. Meadows, D. Leverage points: Places to intervene in a system. 1999; Available from: http://www.donellameadows.org/archives/leverage-points-places-to-intervene-in-a-system/.

31. Pearl, J., Causality: Models, reasoning, and inference. 2000, Cambridge University Press: New York.

32. Ris, D., Causal inference in lead research: introduction to the special section on the neurobehavioral effects of environmental lead. Child Neuropsychology, 2003. 9(1): p. 1-9.

http://www.donellameadows.org/archives/leverage-points-places-to-intervene-in-a-system/

http://www.donellameadows.org/archives/leverage-points-places-to-intervene-in-a-system/



33. Sloman, S.A. and Y. Hagmayer, The causal psycho-logic of choice. Trends in Cognitive Science, 2006. 10(9): p. 407-412.

34. Sprites, P., C. Glymour, and R. Scheines, Causation, prediction and search. First, online ed. 1993, MIT Press: Cambridge, MA.

35. Bozeman, B. and D. Landsbergen, Truth and credibility in sincere policy analysis: Alternative approaches for the production of policy-relevant knowledge. Evaluation Review, 1989. 13(4): p. 355-379.

36. deLeon, P., The stages approach to the policy process: What has it done? Where is it going?, in Theories of the policy process, P.A. Sabatier, Editor. 1999, Westview Press: Boulder, CO. p. 19-32.

37. John, P., Is there life after policy streams, advocacy coalitions, and punctuations: Using evolutionary theory to explain policy change? Policy Studies Journal, 2003. 31(4): p. 481-498.

38. Morçöl, G., Issues in reconceptualizing public policy from the perspective of complexity theory. E:CO, 2010. 12(1): p. 52-60.

39. Ramsey, E. and D. Bond. Evaluating public policy formation and support mechanisms for technological innovation. 2006. Available from: http://mpra.ub.uni-muenchen.de/253/.

40. Rittel, H.W.J. and M.M. Webber, Dilemmas in a general theory of planning. Policy Sciences, 1973. 4(2): p. 155-169.

41. Sabatier, P.A., Theories of the policy process. Theoretical lenses on public policy, P.A. Sabatier, Editor. Vol. 1. 1999, Westview Press: Boulder, CO.

42. Smyth, K.F. and L.B. Schorr, A lot to lose: A call to rethink what constitutes ‘evidence’ in finding social interventions that work, in Working Paper Series. 2009, Harvard Kennedy School, Malcolm Wiener Center for Social Policy: Cambridge, MA.

43. Walters, L.C., J. Aydelotte, and J. Miller, Putting more public in policy analysis. Public Administration Review, 2000. 60(4): p. 349-359.

44. Rhodes, M.L., Agent-based modeling for public service policy development: A new framework for policy development, in Complexity and policy analysis: Tools and concepts for designing robust policies in a complex world, L. Dennard, K.A. Richardson, and G. Morçöl, Editors. 2008, ISCE Publishing: Goodyear, AZ.

45. Elliott, E. and L.D. Kiel, Editors. Nonlinear dynamics, complexity and public policy. 1999, Nova Science: Hauppauge, NY.

46. Richardson, K.A., Complex systems thinking and its implications for policy analysis, in Handbook of decision making, G. Morçöl, Editor. 2007, ISCE Publishing: Goodyear, AZ. p. 189-221.

47. Roe, E., Taking complexity seriously: Policy analysis, triangulation and sustainable development. 1998, Kluwer Academic: New York.

http://mpra.ub.uni-muenchen.de/253/



48. Runhaar, H.A.C., C. Dieperink, and P.P.J. Driessen, Policy analysis for sustainable development: Complexities and methodological responses, in Complexity and policy analysis: Tools and concepts for designing robust policies in a complex world, L. Dennard, K.A. Richardson, and G. Morçöl, Editors. 2008, ISCE Publishing: Goodyear, AZ. p. 197-213.

49. Trochim, W.M.K. and D. Cabrera, The complexity of concept mapping for policy analysis. E:CO - Emergence, Complexity, and Organizations, 2005. 7(1): p. 11-22.

50. Dutton, j.E., L. Fahey, and V.K. Narayanan, Toward understanding strategic issue diagnosis. Strategic Management Journal, 1983. 4: p. 307-323.

51. Hill, T. and R. Westbrook, SWOT analysis: It’s time for a product recall. Long Range Planning, 1997. 30(1): p. 46-52.

52. Mintzberg, H., Rise and fall of strategic planning. 1994, Simon and Schuster: New York.

53. Huff, A.S., Managerial and organizational cognition: Islands of coherence, in Great minds in management: The process of theory development, K.G. Smith and M.A. Hitt, Editors. 2005, Oxford University Press: New York.



Meaningful Evidence, LLC1069 West Broad Street #141Falls Church, VA 22046703.348.0061 | [email protected]

Meaningful Evidence, LLC1656 Wynoochee WayPetaluma, CA 94954707.338.3133 | [email protected]

For information on how to become a Certified SKM Coach, please contact:

mailto:info%40meaningfulevidence.com?subject=

mailto:ASKMATT%40meaningfulevidence.com?subject=

the science of conceptual systems: its history and...

Documents