educational efficacy of environmental education
TRANSCRIPT
State Education & Environment Roundtable
The Educational Effi cacy of
Environmental Education
Linda L. Hoody
© 1996 State Education and Environment Roundtable
TABLE OF CONTENTS
INTRODUCTION.................................................................................... 1
PURPOSE OF STUDY........................................................................... 1
RESEARCH METHODS AND INFORMATION SOURCES.................... 2
Terminology ................................................................................. 3
RESEARCH RESULTS .......................................................................... 4
Organization of Results................................................................ 4
Studies to Measure Knowledge and Attitudes
(Cognitive/Affective Domains) ................................................ 5
Studies to Measure Attitudes (Affective Domain)......................... 7
Studies to Measure Behavior/Action (Cognitive Domain) ............ 9
Other Related Findings ................................................................ 12
Research Methodology in EE............................................ 12
NAAEE Research Commission Survey of Priorities .......... 13
DEFINING THE NEED ........................................................................... 14
REFERENCES ....................................................................................... 20
The Educational Efficacy of Environmental Education
INTRODUCTION
A study of environmental education (EE) efforts by federal and state education
agencies, school districts, universities and environmental organizations was conducted
in 1994. As part of the study nine directors from successful state EE programs
assembled to identify the actions needed to build a stronger knowledge base for the
field of environmental education. In the subsequent document, Proposal for a State
Environmental Education Network (Lieberman, 1995), "collection and analysis of
evidence of the pedagogical efficacy of EE methodologies and content" were identified
as high priority critical components for the successful implementation of state-based EE
programs.
The study also included a national survey of 43 organizations that identified
"evaluation and assessment research to gauge student achievement and the
effectiveness of different programmatic approaches to EE" as one of the seven
categories needed to increase the impact and effectiveness of EE.
A primary reason for establishing the State Education and Environment
Roundtable (Roundtable) was to motivate education systems toward incorporating both
methods and content modeled by EE. Additionally, the Roundtable serves to promote
improved learning through systemic education reform efforts. Environmental education
has much to offer the education reform movement. Fundamental to EE are pedagogical
methods that include: hands-on activities; relevant subject matter; and topics that
engage students, encouraging their active participation. Education reformers recognize
EE as an effective tool in capturing students' enthusiasm for learning in subject areas
ranging from science and math to literature (Lieberman, 1994).
As proponents of education reform and educational systems that fully integrate
environmental education, the Roundtable recognizes the need for a strong foundation of
research to generate and direct activities. This document is a preliminary report of the
first of four major studies that the Roundtable will be conducting.
PURPOSE OF STUDY
The purpose of this study was to locate research that assesses the educational
efficacy of EE; specifically, studies that analyze the influence of EE methods and
content on students' ability to learn in subject areas throughout the curriculum.
The Educational Efficacy of EE Page 2
Additionally, the study was designed to find research that evaluates educational
programs in EE that use decision-making and problem-solving processes to develop
higher level thinking skills.
The following report summarizes the research studies that have been identified
and reviewed thus far in the process. It is the intention of the Roundtable staff that this
interim report will generate additional professional contacts and lead to discovery of
further research projects in the areas outlined within this summation.
If you are aware of any other references or resources for related research please
contact: Ms. Linda Hoody, State Education and Environment Roundtable, 16486
Bernardo Center Drive, Suite 328, San Diego, California, 92128; telephone (619)
676-0272, fax (619) 676-1088, e-mail ([email protected]).
RESEARCH METHODS AND INFORMATION SOURCES
Initial information resources were generated by performing a literature search
through the ERIC database. Phone interviews were conducted over the course of
several months in 1995. Bibliographies were collected from a number of leading
national and international educators. Subscriptions to several research forums
sponsored by the American Educational Research Association (AERA) on their
ListServer system elicited additional responses to requests for applicable studies.
Over 150 documents were examined in the process of this study. Professional
journals were the primary source of articles. The ERIC educational microfiche library
was a valuable source for some conference presentations and unpublished documents.
It should be noted that the literature search focused on studies measuring the
impact of EE on K-12 student experiences. Research describing teacher in-servicing,
college level courses and community education programs were not included in this
review.
A survey was developed to identify research that has been conducted in the
Roundtable's areas of interest. Almost 500 survey forms were mailed to registrants of
the North American Association of Environmental Education (NAAEE) conference held
in Portland, Maine in September 1995. Additional surveys were sent to prominent
educational contacts generated by telephone conversations. Resources available on
the Internet were also explored.
At present (11/14/95), approximately 95 survey forms have been returned.
Hopefully, surveys yet to be received will identify more research projects.
The Educational Efficacy of EE Page 3
All research reports were not readily available. Some masters theses and
doctoral dissertations were difficult to locate and some research reports ordered from
universities have not arrived as of the date of this report.
Terminology
Some distinctions need to be made concerning key terms. Most often
"interdisciplinary" is used in the literature to indicate the use of two or more educational
disciplines. "Transdisciplinary" signifies theme-based learning that emphasizes use of a
wide range of knowledge and skills, not coming from a particular discipline (Hug, 1995)1.
The term "interdisciplinary" is used in this report since it is the predominant term in the
surveyed literature.
In recent years the term "environmental literacy" (EL) has come into widespread
use. EE practitioners have worked under the assumption that EL would join the ranks
of the more firmly established literacy definitions for English, geography, science and
math.
Scientific literacy, for instance, has been defined as,
"...the knowledge you need to understand public issues. It is a mix of
facts, vocabulary, concepts, history and philosophy. It is not the
specialized stuff of the experts, but the more general, less precise
knowledge used in political discourse. If you can understand the news of
the day as it relates to science, if you can take articles with headlines
about genetic engineering and the ozone hole and put them in a
meaningful context...you are scientifically literate."
(Hazen and Trefic, 1991)
A number of sources identified the need to establish EL standards for students
(Brouillet, et. al. 1986, Hungerford 1987, Roth 1992). Measurement of literacy would
then presumably focus on environmental knowledge including understanding the basics
in areas such as ecological concepts, biogeochemical cycling, biodiversity, etc.
Leading environmental educators have, however, defined EL as a multi-
dimensional guideline. As outlined by the Environmental Literacy Framework,
developed by the Environmental Literacy Assessment Consortium (ELAC), EL has:
cognitive dimensions (knowledge and skills); affective dimensions; additional
determinants of environmentally responsible behavior and personal and/or group
involvement in environmentally responsible behaviors. C. Roth (1992) similarly defines
1 Appendix B is a collection of definitions prepared by John Hug (1995).
The Educational Efficacy of EE Page 4
EL as “essentially the capacity to perceive and interpret the relative health of
environmental systems and take appropriate action to maintain, restore, or improve the
health of those systems." While Hungerford and Tomera describe "environmentally
literate" citizens as an individuals competent and willing to take action.
Roth’s definition exemplifies a trend in the EE community to define EL as going
several steps beyond acquisition of knowledge. He stresses the students’ ability to
interpret and act to maintain or improve the environment. This goal for EL emphasizes
the need for complex strategies and action based on higher level procedural thinking. It
more accurately approximates the advanced abilities of experts in the field.
The various definitions of environmental literacy call for well-developed skills in
perception, interpretation and environmentally-based actions. Most of the assessments
for levels of EL, however, appear to be content-driven rather than measuring action, as
promoted by proponents of environmental literacy.
RESEARCH RESULTS
Organization of Results
This report was designed to review research publications in a systematic effort to
locate studies that address the influence of EE on learning. During the literature review,
distinct classifications of research projects emerged as characterized by different
research and evaluation goals. The three major groupings of research include studies
directed at measuring: knowledge and attitudes, attitudes only and behavior.
Affective and cognitive domains have long been recognized by educators as
important fields of study. Research projects that measure cognitive outcomes range
from analysis of basic knowledge acquisition (e.g., recall) to higher order thinking skills
(e.g., synthesis and application). Studies that examine the affective domain explore the
students’ emotional reactions to a given subject.
The research projects reviewed here are divided into three subsections: affective
domain, cognitive domain and a combination of the two. Studies of knowledge and
attitude generally use instruments that combine measurement of environmental content
with opinions or feelings regarding environmental issues (cognitive and affective
domain). The attitudinal studies assess the students’ environmentally-centered
emotional responses (affective domain). The behavioral studies attempt to measure the
students’ actions in response to environmental concerns (reported as higher level
cognitive domain).
The Educational Efficacy of EE Page 5
Representative studies are cited as examples of each subcategory of research.
This process helps to clarify the types of studies that do not address the research needs
and objectives of the Roundtable and helps define the types of studies that are
necessary to meet our objectives.
The conclusions and recommendation’s section, focuses on postulation of
reasons for the lack of research in the desired areas and recommendations for future
study. Further discussion identifies the challenges of locating studies that meet the
Roundtable's research objectives.
Studies to Measure Knowledge and Attitudes (Cognitive/Affective Domains)
Statements concerning attitudes and knowledge are often found in the objectives
listed for EE programs. Iozzi (1989) points out that the cognitive and affective domains
function cooperatively. Many research projects have used instruments to collect data
that combine knowledge and attitude questions (Bennett, 1974, R. Roth, 1976, Fortner,
1978, Leeming, Dwyer, Porter and Cobern, 1993).
Studies in this research subcategory are characterized by instruments designed
to assess knowledge and attitudes. These assessments combine questions measuring
basic environmental and ecological knowledge with items to document students’
feelings toward environmental issues and concerns.
Case 1
Hungerford, Bluhm, Volk and McBeth (1995) designed an
evaluation instrument based on the Environmental Literacy Framework to
measure the EL of middle school-age students. The instrument was
designed to measure knowledge, perceived skill and action (relying on
self-reported behavior). Their Middle School Environmental Literacy
Instrument (MSELI) was used to collect field study data from six sites in
five states.
The MSELI data were generated only during the design phase of
their research work, lack of funding limited its broader usage. The project
developers recommended conducting a comparative study between
students receiving in-depth EE curricula and those learners not receiving
such instruction. This research has not yet been conducted.
Case 2
Bryant and Hungerford (1979) analyzed the effects of
environmental instruction on two classes of kindergartners. Bryant
simultaneously taught two classes a one week introductory unit on basic
The Educational Efficacy of EE Page 6
environmental concepts. For three weeks thereafter, the experimental
group received activity-oriented instruction on pollution and solid waste.
The conventional curriculum taught to the control group did not involve
environmental issues. The treatments were then reversed. Each child
participated in an interview consisting of four knowledge and opinion
questions asked before and after the treatment. The evaluators reported
a significant change and suggested that kindergarten children are capable
of forming concepts concerning environmental subjects.
Case 3
Ramsey and Rickson (cited in Iozzi, 1984) investigated knowledge
and attitudes associated with pollution issues. Their study included a
stratified sample of 482 twelfth grade students.
Results indicated a skewed pattern of responses on the student's
knowledge of ecological concepts. These researchers suggested that
increased knowledge about the cause of pollution generates more positive
attitudes toward corrective measures.
Ramsey and Rickson further commented on the nature of gaining
knowledge and attitudes stating a belief that knowledge may lead to the
beginning formation of attitudes which in turn leads to further knowledge
acquisition. (This appears to be a common, but unproven theme in EE
literature.)
Case 4
Fortner (1978) developed the Survey of Oceanic Attitudes and
Knowledge (SOAK), to measure knowledge and attitudes of tenth grade
students and relate those attributes to the students' marine experiences.
She used three types of surveys covering 63 items to measure both
knowledge and attitudes. Statewide, the 787 students who participated in
the study demonstrated a knowledge level of fifty percent (50%). The
results also indicated that their attitudes toward marine issues were
moderately positive.
Discussion
With the exception of oral interviews of kindergartners in the Bryant and
Hungerford study, questionnaires are generally used for data collection in this class of
research. The research projects in this subsection do not immediately address our
research objective.
The Educational Efficacy of EE Page 7
Although the knowledge/attitude class of research includes measurement of EE
content, it does not address instruction that is a marriage between content and
methodology. These instruments strictly measure recall of facts combined with
emotional reactions to environmental topics.
Studies to Measure Attitudes (Affective Domain)
The studies reviewed in this subsection sought to measure attitudes toward
environmental subject matter. Many researchers express a desire to record
development of positive affective outcomes following environmental instruction.
Attitudinal studies are characterized by questionnaires using Likert scales and semantic
differentials.
In A Summary of Research in Environmental Education, 1971-1982 (Iozzi et al.,
1984), the greatest number of EE studies dealt with the affective domain. The point is
then made that in other disciplines researchers tend to focus more heavily on the
cognitive domain. Affective studies deal with the domain of emotions, expression of
opinions and feelings. Extensive studies were conducted in the 1970's that primarily
used instruments with semantic differentials and scales to measure the subjects'
feelings about environmental issues.
Case 5
In one notable 1976 project by Bohl (cited in R. Roth, 1976), a
national inventory was distributed to over 15,000 students . The outcome
indicated that at equivalent grade levels, students generally have a poor
grasp of factual knowledge, although they tended to express positive
environmental attitudes in response to affective questions. These findings
were reinforced by two similar national EE assessments completed during
the same period.
Case 6
In two studies by Blum (cited in Iozzi, 1984), an open-ended inquiry
instructional method was found to have a positive impact on attitude
changes. In a 1978 study by Hepburn (cited in Iozzi, 1984), attitude
changes appeared to be greatest when interdisciplinary approaches were
used. These studies appear to be examining methodologies of interest to
our project. However, they focused strictly on attitudinal outcomes. (The
original studies were not reviewed, only brief summaries could be found.)
Case 7
The Educational Efficacy of EE Page 8
Jaus (1982) reported on an experiment with 53 fifth grade students.
One class was given 40 minutes of EE instruction on 15 consecutive
school days. Each lesson included lecture, discussion, laboratory
activities and homework. The control class of fifth graders at another
school did not receive any environmental instruction. A 20-item, 5-point
Likert scale instrument, developed by the investigator, was used to
measure environmental attitudes. The experimental group expressed
22% more “positive” environmental attitudes than the control group.
Case 8
In 1980, Wilson and Tomera conducted a study on the effects of
simulations combined with case studies on attitudes toward environmental
issues (cited in Leeming, Dwyer, Porter and Cobern, 1993). Two classes
of 10th graders were used for the experiment. For a three day period, the
general biology class was supplemented with simulation and
environmental case study exercises. The control class received the
standard curriculum. The treatment and control groups were then
reversed. A significant effect of treatment was reported for the first issue
following analysis of pre- and post-measures. However, the effect with the
second measure failed to demonstrate statistical significance.
Wilson and Tomera indicated that since the students in the control
group had received the treatment earlier during the course, their
environmental attitudes had already been raised. Leeming et al., point
out that the treatment group received three additional days on the target
subject, a significant factor that possibly influenced results.
Case 9
One of the few reports to measure a program's longitudinal impact
on attitudes was conducted for the outdoor science school of the Orange
County (California) Department of Education. During the 1975-76 school
year 3,278 sixth graders attended the Forest Home Outdoor School. The
researcher located seniors at local high schools who had attended the
outdoor school during 1975-76. Surveys were received from 449 students
(13% of the total enrollment). Ten topics were represented by questions
on the instrument including: interest in natural sciences, appreciation of
the environment, peer and cabin leader relationships, interest in camping,
impact on attitude toward home responsibilities, feelings about
The Educational Efficacy of EE Page 9
conservation/preservation and potential value of outdoor school for other
students.
Overall the Orange County data indicated a positive impact on the
students. Pie charts displayed students' responses that indicated: 59%
increased interest in natural sciences; 80% increased appreciation for the
environment; increased positive feelings about conservation and
preservation of wilderness and national forest areas were developed by
77% of the students; 76% felt that the experience raised their interest in
camping and other outdoor activities; 95% thought that the experience is
one that every sixth grader should have.
Discussion
This class of studies differs from the research we are seeking in that attitudinal
research studies concentrate on the affective domain. This type of research does not
assess the cognitive influence of EE methods and content on learning.
Studies to Measure Behavior/Action (Cognitive Domain)
There is growing effort among environmental educators to develop a citizenry
that both behaves responsibly and is actively working to protect the environment. Roth,
Stapp, Hungerford and Peyton (cited in Hines, Hungerford and Tomera, 1986) state that
the development of environmentally responsible behavior (ERB) and active citizens has
become the ultimate goal of environmental education. Overt environmental behavior
and development of action skills are underlying themes in many recent EE articles.
Assessments of these types of EE programs assert their foundation in measurement of
higher order cognitive skills.
Case 10
A program called issue investigation and action training (IIAT) was
designed to focus on development of responsible environmental behavior.
IIAT directs instruction to address specific variables that have been
empirically related to ERB. The principal elements of IIAT include:
knowledge of environmental issues; values related to the environment;
individual and group locus of control; environmental sensitivity; knowledge
of and skills in environmental action strategies; and knowledge of
ecological concepts (Ramsey, 1993).
IIAT was reported to promote ERB, in populations of eighth
graders, when EE instruction focused on the predictor variables of
The Educational Efficacy of EE Page 10
responsible environmental behavior [Ramsey, (1981 and 1989) and
Klinger, (1980), both cited in Ramsey, (1993)]. However, Ramsey (1993)
reports that in both prior studies the instruction was delivered by the
evaluators, thereby possibly influencing the research results.
Case 11
Another study was designed by Ramsey (1993) to measure the
influence of the IIAT program. Six interdisciplinary modules provided
training in investigation and action skills. Students were introduced to
environmental issues, skills needed to analyze issues, skills to investigate
issues, skills for information processing and skills needed by responsible
citizens to apply action.
Eight heterogeneously grouped eighth grade classes participated in
the study using a modified pre-test/post-test design. Four classes
received 18 weeks of the experimental instruction, the other four control
classes received standard physical science instruction. The control
groups were taught by teachers trained in IIAT methodology. Data
focused on dependent variables: overt environmental behavior; individual
locus of control; group locus of control; environmental action knowledge;
perceived knowledge of environmental action skills; perceived skill in the
use of environmental action skills; and environmental sensitivity.
Data were collected from five instruments that included Likert-scale
items, self-reporting on overt behavior and naming/defining a solution to
an environmental issue. A scoring protocol was developed to assess
appropriate responses that met the criteria of directness, action and focus.
The post-test analysis of variance indicated that at least a 0.05
significance level was achieved for five variables: overt environmental
behavior; group locus of control; knowledge of environmental action skills;
perceived knowledge of environmental action skills; and perceived skill in
the use of environmental action skills. These findings nearly paralleled the
results reported by Ramsey and Hungerford (1989) in a similar study with
seventh graders following IIAT instruction.
Case 12
Culen (1994) reported on an evaluation using the issue
investigation/evaluation and action skills training model to assess the
effects of an extended case study on the subject of wetland issues. A
The Educational Efficacy of EE Page 11
modified pre-test/post-test nonequivalent control group design was used
with fifteen intact seventh and eighth grade classes.
Experimental Treatment I (four seventh grade and two eighth grade
classes), including four levels of instruction, ran 10-14 weeks.
Experimental Treatment II (two seventh grade and two eighth grade
classes) was completed in four to six weeks and only included two levels
of instruction. The control group (two seventh grade and three eighth
grade classes) received 12 weeks of traditional science instruction.
Experimental treatments were presented by four teachers who had
participated in investigation and evaluation of environmental issues and
action (IEEIA) in-service training.
Five different instruments were used to collect data. Post-test data
were collected to measure the variables of: overt environmental behavior;
knowledge of ecological foundations; individual locus of control; group
locus of control; knowledge of citizenship action skills; and perceived skill
in the use of citizenship action skills. Covariance analysis compared
means of treatment groups and the control group. Statistically significant
differences were found for the overt environmental behavior variable.
Results indicated that the two treatments were more effective than the
control. Full treatment was more influential than the partial treatment in
increasing overt environmental behavior.
Discussion
A concern identified throughout the literature is the “moderator variable” of self-
reported measures of behavior versus relying on observance of actual behavior. As
reported by Brickman, Deutscher and Wicker (cited in Hines, Hungerford and Tomera,
1986), what people indicate on a questionnaire is often inconsistent with their actual
behaviors.
Robottom and Hart (1995) question the individualist ideology of the
behaviorist/applied science approach used in the research of responsible environmental
behavior. They maintain that this type of research focuses on personal variables of the
individual to shape ERB. They believe the historical, social and political contexts within
which individual and group actions take place are key factors being overlooked in this
type of research effort.
This research subcategory is closest to the body of research we are seeking.
Behavior-based research is, however, oriented toward EE programs that focus on the
The Educational Efficacy of EE Page 12
application of "environmental knowledge and skills." The desired outcome is to
measure the students’ "environmentally-related action" in response to "environmental"
issues.
Higher-level thinking strategies are emphasized and environmental content is
applied during instruction. However, this type of research focuses on the behavioral
outcomes as a result of environmental advocacy-driven instruction, not learning
outcomes per se. This does not meet our need to find research that analyzes the
impact of EE methodologies and content in areas throughout the curriculum.
Other Related Findings
Research Methodology in EE
"Paper and pencil" instruments (e.g., questionnaires) were the method of
measurement used most often in the EE evaluation research surveyed in this study.
Presumably, ease of distribution, administration and data analysis made this the
preferred research method. Consideration of which evaluation design to use in
assessing the impact of EE has recently led to controversy in the field. Quantitative
methods that use standardized measures to fit diverse information into predetermined
response categories have been traditionally used in EE analysis.
There is growing interest in the benefits of qualitative methods that permit the
evaluator to study selected issues and events in more depth and detail. Direct
quotation, careful description of program situations, events, interactions and observed
behaviors provide for the open-ended narrative of qualitative measurement (Patton,
1987).
Consistently, research articles described studies that: failed to evaluate long-term
treatment effects or follow-up measures; measured variables immediately following
intervention; utilized small sample sizes (often a single class); provided inconclusive
results; were poorly designed; lacked appropriate controls; failed to clearly describe
administration of the instrument; lacked validity and/or reliability; used weak statistical
analysis; were subject to confounding factors; or may have been influenced by effects of
the researcher's expectations.
In the early eighties, Lewis (1981-82) reviewed studies of instructional strategies
and classroom methodologies. He concluded that a majority of the reports had
instruments of questionable validity and lacked sufficient methodological detail.
Leeming, Dwyer, Porter and Cobern (1993) reported that few changes have occurred in
the years since Lewis made his statements concerning the weakness of EE research
methodology.
The Educational Efficacy of EE Page 13
Leeming, Dwyer and Bracken (1995) discussed the need for research
instruments with sound psychometric properties. They raised the concern that
oftentimes the individuals who develop EE materials and programs also design the
evaluation instruments to assess their own projects. This "conflict of interest" brings
into question the validity and reliability of many of these instruments. Another concern
about these evaluations is that few of the EE program designers have had in-depth
training or experience in the development or implementation of evaluation instruments.
The lack of validated instruments also makes it impractical or meaningless to
make comparisons among different studies. Leeming, Dwyer and Bracken also
criticized the development and selection of instruments by a limited number of
knowledgeable "experts" and proposed that researchers use other forms of scale
validation. Robertson (1994) maintained that "research methodologies and resulting
knowledge claims are consequences of the epistemological stances adopted by the
researchers."
NAAEE Research Commission Survey of Priorities
The Research Commission of the NAAEE and personnel from the National
Science Foundation conducted a Delphi survey to rank research priorities in the EE
field. The survey was distributed to a random sample of 200 NAAEE members and all
members of NAAEE's Board of Directors. Fifteen of the 26 members of the Board of
Directors and 61 of the 200 NAAEE members returned the survey.
Three of the highest ranking research topics identified by the survey are: a
national longitudinal study of changes in K-12 student behavior as a function of formal
EE instruction; a national cross-sectional study of the status of environmental literacy
among K-12 students; and an analysis of K-12 EE curricular models and the
environmental literacy-related outcomes associated with each of these national surveys.
The results of the survey are interesting but may only indicate that research
priorities reflect the current philosophical structure of EE. The predominant thrust of EE
appears to have gone from its historical "educational" perspective to a role in
environmental advocacy, subsequently narrowing the focus of most EE assessments.
Therefore, identifying "environmental literacy," "environmental attitudes" and
"environmentally responsible behavior" among the highest research priorities simply
mirrors the advocacy philosophy of the EE community, not a pedagogical philosophy.
Robertson (1994) states, "emphasis is often placed on the 'environment' interest
of environmental education, with little regard for educational considerations .... Unless
environmental education researchers attend to the educative aspects of their practice,
The Educational Efficacy of EE Page 14
in terms of both underlying epistemology and theory of learning, one is at a loss to
frame a study within a particular epistemology."
DEFINING THE NEED
Ultimately, education should focus on the development of lifelong learning skills:
communication, problem-solving, decision-making, critical-thinking, evaluation,
participatory citizenship, valuing, aesthetic appreciation and a sense of ethics (Hug,
1995). Each student's environmental behavior will be a result of his or her progression
through development of these skills.
"To be consistent with democratic principles students should be exposed to the
plurality of environmental ideologies, and that through a process of inquiry, critique and
reflection they can be assisted to develop and defend their own set of environmental
beliefs and values....it should be each student's choice to pursue actions deemed
necessary and justifiable for achieving environmental reform in accordance with the
ideological position he or she supports." (Stevenson, 1987)
Education reformers recognize the need for greater strides to be made in
educational systems that provide students with the skills and knowledge they need to
formulate responsible choices concerning their relationship to the world around them
(Lieberman, 1995). Those involved in the reform process see the potential value of EE
methodologies that incorporate problem-solving, hands-on learning approaches,
scientific method, complex thinking strategies, cooperative learning, relevant subject
matter and topics that engage students in the educational process. There is also
support within the environmental education community for an interdisciplinary approach
to EE (R. Roth, 1976, Stapp, 1976, Voelker, 1973, Bennett, 1974, Brogdon and
Rowsey, 1977, Disinger, 1981).
Throughout the course of this study, no compelling research was located that
measured the effectiveness of interdisciplinary EE methods. Although some studies
analyzed projects containing educational components of interest to our search, none
satisfactorily measured the cross-curricular impact of EE content and methodology.
Case 13
Only one group of studies approximated the desired conditions for
the Roundtable's research analysis, four evaluations by Seever (1991)
analyzing environmental science magnet schools for the Kansas City
School District, Missouri. Trailwoods Environmental Science Magnet
Elementary School, Nowlin Environmental Science Magnet Middle School,
The Educational Efficacy of EE Page 15
Knotts Environmental Science Magnet Elementary School and East
Environmental Science Magnet High School were the subjects of these
studies.
Three of the evaluations were formative, following the first or
second year of implementation, only Knotts School had a summative
evaluation after the third year of operation. Demographic data indicated
that each school came close to the district's desired 60/40 minority to non-
minority split. Iowa Tests of Basic Skills (ITBS), the Missouri Mastery and
Achievement Test (MMAT), Degrees of Reading Power Test (DRP) and
Tests of Achievement and Proficiency (TAP) were used to produce
achievement scores.
The Missouri magnet schools' standardized test scores were
consistently strong across all grade levels in science and social studies.
In most other subject areas the students scored near the national norm.
When compared to their district and state-wide counterparts, grade level
scores demonstrated varying degrees of weakness in language arts and
reading. The measurement stressing successful infusion of the
environmental science magnet theme took the form of observations of
hallways and classrooms to document the number of visible displays
(aquariums, live or preserved animals, posters and art work), the presence
of theme related activities (e.g., Earth Day celebrations) and associated
field trips.
During classroom visits the evaluator observed hands-on learning
and activities designed to improve problem-solving and critical-thinking
skills. The research indicated that students were demonstrating changes
in behavior, attitudes and knowledge based on impact of the
environmental science theme. Although, it was unclear how the evaluator
arrived at these conclusions.
Hale and Levine (cited in Seever, 1991) caution in the Long Range
Magnet Plan, "a school's effectiveness should not be determined during its
first year of operation ... during its early years (evaluation) should be
designed with the intent of improving, modifying and revising program
implementation."
Keeping in mind the infancy of these school operations, it was
encouraging to find at least some formal evaluation to examine the
effectiveness of an interdisciplinary EE program. While statistical
The Educational Efficacy of EE Page 16
information was contradictory, the evaluator's interpretation indicates that
program implementation is demonstrating steady positive impact.
The summative evaluation of the Knotts School reported test
scores that were significantly higher than the scores of schools with fewer
years of theme implementation. Of the 221 randomly selected Nowlin
School students polled, two-thirds said that they had learned problem-
solving and decision-making skills that they had been able to apply in their
lives outside school. A high percentage (98% in one study) of the 200+
parents polled believed their child was acting more environmentally
responsible and that the family had thus been encouraged to behave more
responsibly. Over two-thirds also said that they might want a career in the
field of environmental science.
It would be beneficial to explore assessing the impact of
environmental methods on instruction. High test scores in science and
social studies may be indicators of positive influence of EE methods.
Although these methodologies were observed, no direct assessment of
their effectiveness was measured. In-depth qualitative evaluations,
involving students and faculty, would complement more detailed studies
using control groups from demographically comparable traditional schools
in determining program efficacy.
Discussion
Our search failed to find evidence that directly points to the positive influence of
EE content and methodology. Some of the behavior-based research discusses
application of lifelong learning skills. The reason these studies did not match our
research objective is that they went beyond the instructional level to measurement of
environmentally-related actions. The higher level thinking skills entertained in the
behavior-based research were strictly in an environmental context, not across
disciplines. Ideally, these critical-thinking skills and strategies need to demonstrate
transferability into all curricular areas.
According to Monroe and Kaplan (1988) other elements important in solving
environmental problems may include: knowledge of the environment and of issues;
knowledge of action strategies that help resolve issues; locus of control and
empowerment, a sense of responsibility and communication skills. Historically however,
"...schools were not intended to develop critical thinkers, social inquirers and problem-
The Educational Efficacy of EE Page 17
solvers, or active participants in environmental and political (or even educational)
decision-making." (Stevenson cited in Robottom, 1987)
The role of EE when knowledge-based, can be comfortably accommodated by
the structural organization of traditional schools. Methods that involve critical-thinking
strategies or action-oriented approaches, on the other hand, represent a much greater
challenge for orthodox educational systems.
The present challenge is to define the classification of research that would
address the original research goal. That task is to locate studies that analyze the cross-
curricular impact of EE methods and content.
SUMMARY AND CONCLUSIONS
Ideally, schools, as one of the societal systems responsible for the development
of citizenry, should be charged with developing cognitive skills, across a variety of
disciplines, to equip students with the ability to make responsible decisions. Business
and industrial sources report the inability of many employees to think critically in job
situations. The complexities of the modern world and our democratic society
necessitate effective processing of information to determine a course of action. The
ability to think critically is essential in a student's capacity to make choices and
judgments of appropriate actions.
As stated over twenty years ago, "... we are at that point in time when rhetoric
and opinion must be substantiated by consolidating existing research efforts and
focusing future efforts ... we must now be about the business of validating the
assumptions and utilizing a research base if EE is to continue to advance" (Voelker,
1973).
R. Roth (1976) expressed the need for continued development and strengthening
of environmental education programs through evaluation of such programs. He
recognized the need for increased sophistication of methods and techniques used in
empirical research involving EE.
Our review found that interdisciplinary-based research in EE is poorly
represented in the literature. Some members of the national EE community alluded to
the scarcity of such studies during phone interviews and e-mail correspondence.
Candid comments by those interviewed included: "I'm doubtful you'll find anything like
that out there," and "In my research, in a related area, I have not come across anything
of this nature, despite looking for it."
The Educational Efficacy of EE Page 18
There are several possible explanations for a lack of research that reinforces the
pedagogical strengths of EE. Among the possible reasons are: lack of funding and/or
planning for program evaluation; difficulties incorporating assessments of problem-
solving and critical-thinking skills into traditional school structures; lack of relevant case
examples of interdisciplinary model programs; and most EE researchers are evaluating
program outcomes related to environmental attitudes and behaviors rather than
assessing general educational impacts of EE.
Regarding funding and program planning, one of the comments heard repeatedly
during phone interviews of EE program directors was, "I don't have the funding for
evaluation." Just as "effective assessment should be integrated into instruction"
(Jonassen, cited in Klein and Merritt, 1994), so should effective evaluation be an
integral part of any educational program. The decision-maker bases his actions on the
unambiguous scientific findings provided by the researcher (Stapp, 1976). Evaluation
results drive decisions about on-going revision and necessitate examination of
instructional design, curricular goals and the effectiveness of instructional methods.
Measurement of student learning within interdisciplinary EE curriculum presents
complications for traditional educators. Even with the recent movement toward
authentic assessment it is difficult to evaluate students’ abilities to investigate, evaluate,
make decisions or demonstrate environmental action through standardized testing
(Jonassen, cited in Klein and Merritt, 1994).
At the same time, organizational conditions that engage students in critical
analyses of environmental issues are generally not encouraged within traditional
education systems. Teachers and administrators expect to maintain order and a
competitive atmosphere rather than emphasizing the types of learning methods used in
EE.
Another reason it is difficult to find studies analyzing the use of cross-disciplinary
EE content and methods may be that the educational structures in which they would
successfully be integrated are virtually nonexistent. Until the educational systems are
restructured to incorporate learning modeled by EE methods (e.g., critical-thinking,
problem-solving, hands-on activities and use of relevant subject matter), evaluation of
its effectiveness can't take place.
There are apparently few pilot programs that have incorporated EE methods and
content into traditional educational systems. At least, there is a scarcity of case
examples describing these model programs, such as “environmental magnet schools”.
The focus of most EE researchers has not been on assessing the all-inclusive
educational importance of EE methodology. Primarily, the evaluation projects in the
The Educational Efficacy of EE Page 19
field have centered on assessing student outcomes relating to environmental
knowledge, attitudes and behaviors.
We set out to find research evidence related to the effectiveness of EE as an
educational tool. The motivation for this search was to locate documentation that
demonstrated the educational efficacy of EE. We believe this research could then be
used by the EE community to begin building a more effective dialog with the education
community. This is the basis for helping to advance EE into a parallel position with
other efforts in education reform.
We had hoped to locate studies that demonstrated the influence of using EE
content and/or methods on student learning in math, language arts, social studies, etc.
Documentation focusing on this type of research would help answer questions such as:
Do students learn to be better readers when the topic is animals, plants or other
environmental content? Are students more motivated to learn about science while
studying chemistry, math or physics, in pursuit of answers related to the environment?
Answers to these and other questions, if responded to affirmatively, might help to
promote the full integration of environmental content and methods across the
curriculum. Perhaps, capitalizing on a child’s "intrinsic interest" in the natural world and
actively engaging that child in the educational process would create an enthusiastic
learner for a lifetime.
- - - - - - - - - - - -
The Educational Efficacy of EE Page 20
REFERENCES
Bennett, D. B. (1974). A report on research and development in environmental
education. Yarmouth, NJ: New Jersey State Council for Environmental education. A paper presented at the annual meeting of the National Association for Research in Science Teaching (47th, Chicago, IL, April 1974).
Bennett, D. B. (1989). Four steps to evaluating environmental education learning
experiences. Journal of Environmental Education, 20(2), 14-21. Borden, R. J. & Schettino, A. P. (1979). Determinants of environmentally responsible
behavior. Journal of Environmental Education, 10(4), 35-39. Bowman, M. L., & Disinger, J. F., Editors. (1980). Environmental education in action-
IV: Case studies of teacher education programs for environmental education. Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Ohio State University.
Brogdon. R., & Rowsey, R. (1977). Some effects of an interdisciplinary environmental
education effort. Journal of Environmental Education, 8(3), 26-31. Brouillet, F. B., Bailey, M., Bumgarner, K., & Kennedy, D. (1986, January). Status and
needs assessment of environmental education in Washington. A report by the Washington Superintendent of Public Instruction. Olympia, Washington.
Bryant, C. K. & Hungerford, H. R. (1977). An analysis of strategies for teaching
environmental concepts and values clarification in kindergarten. Cohen, M. R. (1977). Using student-determined needs to evaluate environmental
education programs. Journal of Environmental Education, 8(4), 27-30. Chen, Pei-Jen. (1995). Environmental Education: The perfect tool to integrate across
curriculum. Paper presented at the 24th annual conference of the North American Association for Environmental Education, Portland, Maine.
Childress, R. B. (1978). Public school environmental education curricula: A national
profile. Journal of Environmental Education, 9(3), 2-10. Cowan, M E., & Stapp, W. B., Editors. (1982). Environmental education in action-V:
International case studies in environmental education. Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Ohio State University.
Culen, G.R. (1994, September). The effects of an extended case study on
environmental behavior and associated variables in seventh and eighth grade
The Educational Efficacy of EE Page 21
students. Paper presented at the North American Association for Environmental Education, Cancun, Mexico.
Disinger, J. F. (1981). Environmental education for the eighties. Journal of Soil and Water Conservation, 36(1) 9-12.
Disinger, J. F., Editor. (1982, December). Environmental education in action VI:
change agents in and for environmental education. Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Ohio State University.
Disinger, J. F., Editor. (1986). Current research in environmental education. Columbus OH: ERIC/SMEAC. EE Digest No. 1 Disinger, J. F. (1985-86 Winter). Current trends in environmental education. Journal of
Environmental Education, 17(2), 1-3. Disinger, J. F., Editor. (1987). Trends and issues in environmental education: EE in
school curricula: Reports of a symposium and a survey. Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Ohio State University.
Disinger, J. F. (1990). Environmental education for sustainable development? Journal
of Environmental Education, 21(4), 3-6. Disinger, J. F. & Roth, C. E. (1992). Environmental literacy. Columbus, OH: ERIC
Clearinghouse for Science, Mathematics, and Environmental Education, Ohio State University.
Doran, R. L. (1977). "State of the art" for measurement and evaluation of
environmental objectives. Journal of Environmental Education, 9(1), 50-63. Field, K. S., Editor. (1981). Recent master's thesis work in environmental education
and communications. Troy, OH: National Association for Environmental Education.
Fortner, R. W. (1978). Experiences related to oceanic knowledge and attitudes of tenth
grade students in Virginia. Ed.D. Dissertation, Virginia Polytechnic Inst. and State University.
Gigliotti, L. M. (1990). Environmental education: What went wrong? What can be
done? Journal of Environmental Education, 22(1), 9-12. Guillierie, R, & Schoenfeld, A. C., Editors. (1979). An annotated bibliography of
environmental communication research and commentary: 1969-1979. Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Ohio State University.
The Educational Efficacy of EE Page 22
Hanselman, D. & Yuen, C., Editors. (1978). Recent master's thesis work in environmental education and communications. Troy, OH: National Association for Environmental Education.
Hazen, R. & Trefic, J. (1991). Science Matters: Achieving Scientific Literacy. Anchor
Books. Hines, J. M., Hungerford, H. R., & Tomera, N. T. (1986). Analysis and synthesis of
research on responsible environmental behavior: A meta-analysis. Journal of Environmental Education, 18(2), 1-8
Hug, J. (1977). Two hats. From a report of the North American Regional Seminar on
Environmental Education, organized by The Alliance for Environmental Education. Columbus OH: ERIC/SMESC.
Hug, J. (1995). Definitions. A paper prepared to define key educational terms. Columbus OH: Ohio Department of Education. Hungerford, H.R. & Volk, T. L. (1990). Changing learner behavior through
environmental education. Journal of Environmental Education, 21(3), 8-21. Hungerford, H. R., & Peyton, R. B. (1977). A paradigm of environmental action.
Carbondale, IL: Science Education Center, Southern Illinois University. Hungerford, H. R., & Peyton, R. B. (1986). Procedures for developing an environmental
education curriculum. A discussion guide for UNESCO training seminars on environmental education. Environmental educational series 22.
Iozzi, L. A. (1984). Summary of research in environmental education, 1971-1982.
Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Ohio State University.
Iozzi, L. A. (1989). What research says to the educator. Part one: Environmental
education and the affective domain. Journal of Environmental Education, 20(3), 3-9.
Iozzi, L. A. (1989. What research says to the educator. Part two: Environmental
education and the affective domain. Journal of Environmental Education, 20(4), 6-13.
Jaus, H. H. (1982). The effect of environmental education instruction on children’s
attitudes toward the environment. Science Education, 66(5), 689-692. Jickling, B. (1994). Why I don't want my children to be educated for sustainable
development. Journal of Environmental Education, 11(3), 114-16. Kauchak, D., Krall, F. & Heimsath, K. (1978). The need for education, not
indoctrination. Journal of Environmental Education, 10(1), 19-22.
The Educational Efficacy of EE Page 23
Klein, E. S., & Merritt, E. (1994). Environmental education as a model for constructivist
teaching. Journal of Environmental Education, 25(3), 14-21. Kogut, B. H., Editor. (1982). Recent master's thesis work in environmental education
and communications. Volume V. Troy, OH: National Association for Environmental Education.
LaHart, D. E., & Barnes, L. W. ( ). A holistic scheme for environmental education
research. Journal of Environmental Education, Lane, J., Wilke, R., Champeau, R., Sivek, D. (1994). Environmental education in
Wisconsin: A teacher survey. Journal of Environmental Education, 25(4), 9-17. Leeming, F. C., Dwyer, W. O., Porter, B. E. & Cobern, M. K. (1993). Outcome
research in environmental education: a critical review. Journal of Environmental Education, 24(4), 8-21.
Leeming, F. C., Dwyer, W. O., & Bracken, B. A. (1995). Children's environmental
attitude and knowledge scale: Construction and validation. Journal of Environmental Education, 26(3), 22-31.
Lewis, G. E. (1981-82). A review of classroom methodologies for environmental
education. Journal of Environmental Education, 13, 12-15. Lieberman, G. A. (1994). Proposal for a state environmental education network.
Report prepared for the Pew Charitable Trusts. Lieberman, G. A. (1995). Pieces of a puzzle: An overview of the status of
environmental education in the United States. Report prepared for the Pew Charitable Trusts.
Marcinkowski, T. J. (1992 ). America 2000 and reform in science education: Where
does EE fit? Journal of Environmental Education, 23(22), 4-9. McCabe, R. H., Editor. (1977). Current issues in environmental education - III:
Selected papers from the 6th annual conference of the National Association for Environmental Education. Columbus, OH: ERIC Information Analysis Center for Science, Mathematics, and Environmental Education.
Milton, B. Cleveland, E., & Bennett-Gates, D. (1995). Changing perceptions of nature,
self, and others: A report on a park/school program. Journal of Environmental Education, 26(3), 32-39.
Monroe, M.C., & Kaplan, S. (1988). When words speak louder than actions:
Environmental problem solving in the classroom. Journal of Environmental Education, 19(3), 38-41.
The Educational Efficacy of EE Page 24
Morris, L. L., Fitz-Gibbon, C. T., & Freeman, M. E. (1987). How to communicate evaluation findings. Newbury Park, CA: SAGE Publications Inc.
Olson, L. (1992). Teachers' guide to school reform: quality is job one. Teacher
Magazine. May/June, 26-44. Palmer, J. A. (1992). Connoisseurship and creativity: Monitoring creative thinking
abilities in primary school children in England. Journal of Environmental Education, 23(3), 17-27.
Patton, M. (1987). How to use qualitative methods in evaluation. Newbury Park, CA:
SAGE Publications Inc. Perkins, J. H., Editor. (1986). Monographs in environmental education and
environmental studies, volume III: International aspects of environmental education. Troy, OH: The North American Association for Environmental Education.
Peters, R. (1981). Putting global 2000 to work in the school. Environmental Education Report.
Ramsey, J. M. (1993). The effects of issue investigation and action training on eighth-
grade students' environmental behavior. Journal of Environmental Education, 24(3), 31-36.
Ramsey, J.M., Hungerford, H.R., & Volk, R.L. (1992). Environmental education in the
K-12 curriculum: Finding a niche. Journal of Environmental Education, 23(2), 35-45.
Ramsey, J.M., & Hungerford, H.R. (1989). The effects of issue investigation and action
training on environmental behavior in seventh grade students. Journal of Environmental Education, 20(4), 29-34.
Robertson, A. (1994). Toward constructivist research in environmental education.
Journal of Environmental Education, 25(2), 21-31. Robottom, I. (1985, July). Contestation and continuity in educational reform: a critical
study of innovations in environmental education. A doctoral dissertation. Deakin University, Australia.
Robottom, I. (1985). Evaluation in environmental education: Time for a change in
perspective? Journal of Environmental Education, 17(1), 31-36. Robottom, I., Editor. (1987). Environmental education: Practice and possibility.
Victoria, Australia: Deakin University Press. Robottom, I., & Hart, P. (1995). Behaviorist EE research: Environmentalism as
individualism. Journal of Environmental Education, 26(2), 5-9.
The Educational Efficacy of EE Page 25
Roth, C. (1992). Environmental literacy: its roots, evolution, and directions in the
1990's. Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Ohio State University.
Roth, R. E. & Helgeson, S. L. (1972). A review of the research related to
environmental education. Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Ohio State University.
Roth, R. E. (1976). A review of research related to environmental education, 1973-
1976. Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Ohio State University.
Schlechty, P. C. (1993, Fall). On the frontier of school reform with trailblazers, pioneers,
and settlers. Journal of Staff Development, 14 (4), 46-51. Seever, M. (1991). Trailwoods environmental science magnet elementary school.
Formative evaluation. Kansas City School District, MO. Seever, M. (1991). Nowlin environmental science magnet middle school: 1990-1991.
Formative evaluation. Kansas City School District, MO. Seever, M. (1991). Knotts environmental science magnet elementary school: 1988-
1989, 1989-1990, 1990-1991. Summative evaluation. Kansas City School District, MO.
Seever, M. (1991). East environmental science magnet high school: 1990-1991.
Formative evaluation. Kansas City School District, MO. Sia, A. P., Hungerford, H. J., & Tomera, A. N. (1986). Selected predictors of
responsible environmental behavior: An analysis. Journal of Environmental Education, 17(2), 31-40.
Simmons, D. A. (1989). More infusion confusion: A look at environmental education
curriculum materials. Journal of Environmental Education, 20(4), 15-18. Simmons, D. A., (1991). Are we meeting the goal of responsible environmental
behavior?: An examination of nature and environmental education center goals. Journal of Environmental Education, 22(3), 16-21.
Simpson, B. (1982). Six years later. A report analyzing the influence of the outdoor
science school experience six years following attendance. Orange County Department of Education. Mentone, CA.
Singletary, T.J. (1992). Case studies of selected high school environmental education
classes. Journal of Environmental Education, 23(4), 35-40.
The Educational Efficacy of EE Page 26
Smith-Sebasto, N. J. (1995). The effects of an environmental studies course on selected variables related to environmentally responsible behavior. Journal of Environmental Education, 26(4), 30-34.
Stapp, W. B. (1976). International environmental education: The UNESCO-UNEP
programme. Journal of Environmental Education, 8(2), 19-25. Thomas, I. G. (1989). Evaluating environmental education programs using case
studies. Journal of Environmental Education, 21(2), 3-8. Voelker, A. M. (1973). Environmental education -- Related research, 1969-72. An
Annotated bibliography. Madison, WI: Wisconsin University Center for Environmental Communications and Education Studies.
__________ (1980). Recent master's thesis work in environmental education and
communications. Troy, OH: National Association for Environmental Education.