an evaluation management model by james b. …...an evaluation management model for environmental...
TRANSCRIPT
AN EVALUATION MANAGEMENT MODEL
FOR ENVIRONMENTAL EDUCATION PROGRAMS
by
James B. Armstrong
Dissertation submitted to the Graduate Faculty of the
Virginia Polytechnic Institute and State University
in partial fulfillment of the requirements
APPROVED:
for the degree of
DOCTOR OF PHILOSOPHY
in
Educational Research and Evaluation
J~mes C. Impara, Co-chairman Daniel E. v/()gler, Co-chairman
Lawrence H. Cross Jimmie c. Fortune
Peter ~Omiey
December, 1989
Blacksburg, Virginia
ACKNOWLEDGMENTS
An endeavor of this magnitude is seldom accomplished
alone, and this study is certainly no exception. There are
many to whom I am grateful for contributing to this study
and I apologize to those who have inadvertently been
omitted.
To my wife, Shaliah, I am forever indebted for her
contributions to this effort. Without her constant and
unwavering support the final product would not have been
possible.
To my daughter, Jami, I am thankful for her cooperation
during the study. She gave up many hours of playtime for me
to go to the library or work quietly.
To my parents and Shaliah's parents, I am grateful for
their encouragement and support.
Each member of my Graduate Committee has contributed to
this study and my overall education in special ways. To my
co-chairman, James C. Impara and Daniel E. Vogler, my
heartfelt thanks for the numerous reviews and revisions to
this document, as well as, encouragement throughout my stay
at Virginia Tech.
To Peter T. Bromley, my cognate professor, my sincere
thanks for his efforts in guiding my growth as a researcher
i i
in environmental education and his assistance in my job
pursuits.
To Jimmie c. Fortune and Lawrence H. Cross, thanks for
their continued guidance and friendship.
Finally, thank you and congratulations to many of my
fellow graduate students who have traveled this same road
and shared the joys and pains of graduate school. We
finally made it.
i i i
ACKNOWLEDGMENTS
LIST OF TABLES
LIST OF FIGURES
CHAPTER
ONE
TWO
TABLE OF CONTENTS
INTRODUCTION
Environmental Threats
Environmental Education
Conceptual Framework
Need for Study
Problem Statement
Purposes
Research Questions
Assumptions
Delimitations .
Limitations
Definitions
Organization of Study
REVIEW OF RELATED LITERATURE .
Evaluation Management
Evaluation Theory
iv
Page
ii
x
xii
1
1
2
5
9
12
13
13
16
17
17
20
22
23
23
26
Environmental Education 30
Knowledge and Attitudes . 30
Interdisciplinary . 32
Supplemental 33
Evaluation of Environmental Education Programs 35
Measures of Environmental Knowledge 39
Environmental Knowledge . 39
Research Designs Using Environmental Knowledge Measures . 40
Measures of Environmental Attitudes 41
Environmental Attitudes . 41
Uses of Environmental Attitude Measures . 42
Research Designs Using Environmental Attitude Measures 43
The Relationship Between Knowledge and Attitudes . 44
Knowledge and Attitudes . 44
Methods of Establishing Relationship 48
Teacher Impressions . 49
Importance of teacher Impressions . 49
Collection Techniques for Teacher Impressions 49
Summary 50
v
THREE METHODOLOGY
Evaluation Management Processes
Client Relationships
Proposal Development
School System Relationships
Evaluation Methodology .
Selection of NatureScope Activity Guides
Population/Sample
Knowledge Tests
Content Validity .
Instructional Validity .
Attitude Scale
Teacher Handbook .
Pilot Study
Treatment Implementation
Distribution of Materials .
Final Report
Evaluation Management Model
vi
54
54
55
57
57
57
59
59
62
64
65
66
67
68
69
71
72
72
Evaluation Outcomes 72
Response Rates 72
Pretest 73
Post test 74
Attitude Scale 75
Analysis 75
Knowledge Test 75
Attitude Scale 76
Teacher Questionnaire 76
FOUR RESULTS . 77
Introduction . 77
Analysis of Research Questions . 77
Section A . 77
Question 1 . 77
Question 2 . 82
Question 3 . 85
Section B . 87
Question 4 . 88
Question 5 . 94
Question 6 . 97
Question 7 . 98
Question 8 . 101
Summary 111
vii
FIVE CONCLUSIONS AND RECOMMENDATIONS . 113
Conclusions Drawn From the Data 115
Conclusions Concerning Evaluation Management Practices. 116
Conclusions Concerning Effects on Environmental Knowledge. 119
Conclusions Concerning Effects on Environmental Attitudes. 121
Conclusions Concerning the Correlation of Knowledge and Attitudes. 122
Conclusions from Teacher Questionnaires 123
Serendipitous Findings. 123
The Effects of Testing Order On Attitude Scores. 123
NatureScope Objectives . 126
The Effect of Teacher Interest On Test Results 127
Recommendations 129
Evaluation Management 129
NatureScope Evaluation . 132
Recommendations for Further Research . 134
Summary 135
REFERENCES 138
APPENDIX 146
A NatureScope Activity Guides by topic groupings
viii
146
B
c D
E
F
G
H
I
J
K
VITA
ABSTRACT
NatureScope Knowledge Tests .
Naturescope Objectives
Revised NatureScope Objectives .
Attitude Scale
Teacher Survey
Sample Logbook and Forms .
NatureScope's Effect On Environmental Attitudes Using Students as the Unit
Qualitative Comments by Teachers
Tests for Homogeneity of Regression
Cell Means for Knowledge Tests .
ix
147
171
181
188
191
195
199
203
205
206
208
Table 1.
2.
3.
4.
5.
6.
7.
LIST OF TABLES
Page Number of Teachers Assigned to
Each NatureScope Issue 60
Reliability Estimates for the Five Instruments . 63
Number of Treatment and Control Classes by Grade Level . 74
ANCOVA for Let's Hear It for Herps . 89
Treatment and Control Group Means for Knowledge Tests 90
ANCOVA for Wild About Weather 91
ANCOVA for Discovering Deserts 91
8. ANCOVA for Endangered Species: Wild & Rare 92
9. ANOVA for Pretest Means for Endangered Species: Wild & Rare . 93
10. ANOVA for Attitude Scale Using Teachers as Unit 95
11. Attitude Means for NatureScope and Control Groups . 96
12. PPM Correlations for Knowledge and Attitudes 97
13. Adjusted Treatment Means Based on Grade 99
14. ANCOVA for Endangered Species: Wild & Rare by Grade 100
15. Participant Impressions of Naturescope 106
16. ANOVA Summary for Knowledge Scores by Activities 128
17. ANOVA Summary for Attitude Scores by Activities 129
x
18. Attitude Means for Each Unit of Analysis 202
19. ANOVA for Student Attitude Means by Knowledge Test .
xi
202
LIST OF FIGURES
Figure Page
1. General evaluation Management Model . 25
2. Management Model for the NWF Evaluation 56
3. Summary of Experimental/Control Groups for Knowledge Tests 61
4. Activities Used in Study . 103
5. Revised Management Model for the NWF Evaluation . . 120
xii
CHAPTER ONE
INTRODUCTION
Environmental Threats
There is an inextricable link between the environment
and human activity. The care taken with our environment
today will be manifested in the quality of life for future
generations. Despite this link, people continue to alter
the environment in ways that jeopardize their very
existence.
In an effort to curtail these tendencies, legislation
is frequently passed mandating actions to clean up and
preserve a quality environment that will continue to support
life. However, attempting to legislate compliance with
environmental standards is a short-term solution to a long-
term problem. To deal effectively with these environmental
threats there is an immediate and continued need for an
environmentally-informed citizenry. As noted by Hawkins and
Vinton (1973) ''the solution to the environmental crises
rests neither with scientists nor with government officials
but with a citizenry educated in environmental problem
solving."
Evaluations should be conducted to assist organizations
that develop and distribute environmental education
materials. The way in which such evaluations are managed
1
2
can affect the success of the evaluation process and
influence the findings.
Environmental Education
The Belgrade Charter (UNESCO, 1976) outlined the goal
of environmental education as "develop[ing] a world
population that is aware of and concerned about the
environment and its associated problems, and which has the
knowledge, skills, attitudes, motivations and commitment to
work individually and collectively towards solutions of
current problems and prevention of new ones~'' This goal, or
a slight variation, has been espoused by most environmental
educators.
Many environmental-interest groups, both private and
public, have addressed the need to educate people. Their
efforts run the gamut from outdoor camps to classroom
supplements. Such programs come under the umbrella of
environmental education.
Development of interdisciplinary educational
supplements that stress environmental topics is a popular
environmental education format. These materials are used by
educators in conjunction with the general curriculum to
combine environmental education with a variety of
traditional classroom topics. It is tempting to those
3
developing such materials to distribute them among potential
users and assume that the materials are both effective and
used effectively. However, these assumptions of
effectiveness are essentially untested and the materials may
not be producing the outcomes desired by the developer. For
example, the content and format of the materials may be such
that they are incompatible with classroom instruction and,
consequently, receive little use. Another possible outcome
of poor content, perhaps more detrimental, is that the
materials are not effective in transmitting information or
lead the students to erroneous conclusions.
Even though the developers of such programs may have a
sincere interest in evaluation of their materials, they may
not be familiar with the processes of working with an
evaluator. This lack of familiarity may result in a
communication breakdown between the evaluator and the
client. If the evaluation effort is to be successful, these
lines of communication must be kept open. The evaluator
must educate the client as to what to expect during the
evaluation process.
Moreover, the management processes involved in an
evaluation study are dependent on the client's role in and
expectations from the evaluation. Many of the groups that
develop and distribute environmental education supplements
4
are nonprofit and politically-active. These factors and
other characteristics need to be considered in developing a
management plan for the evaluation.
Complex educational programs, such as interdisciplinary
environmental education supplements, require multi-faceted
evaluations to determine overall effectiveness. This study
modeled quantitative and qualitative techniques for
evaluating three components of an extant environmental
education program; (a) its impact on participant's
knowledge, (b) its impact on participant's attitudes
concerning the environment, and (c) teacher's impressions of
the materials.
Positive knowledge and attitude outcomes and positive
feedback from teachers are assumed to be measures of
effective content and effective use. Ideally, evaluation of
these program outcomes should be done early in program
development. If evaluative efforts do not indicate that
desired outcomes are being attained, modifications can be
made to the materials.
5
CONCEPTUAL FRAMEWORK
This study had two foci: (1) the development of a model
for evaluation management by modifying the generic
Stufflebeam (1973) model of planning an evaluation; and (2)
the examination of evaluation outcomes for NatureScope.
Evaluation Management
This study was grounded in the concept that evaluation
as a process influences the outcomes of the evaluation
study. To produce the outcome information, Stufflebeam's
(1973) model served as the basis for the study. His model
incorporates the following steps: (a) focusing the
evaluation; (b) collecting information; (c) organizing
information; (d) analyzing information; (e) reporting
information; and (f) administering the evaluation.
categories a through e relate to management of the
evaluation, while f addresses evaluation outcomes.
Evaluation models differ in the amount of interaction
between the evaluator and the evaluation audience (i.e.
client, school administrators, data providers) . Loosely-
structured case studies require a close interaction between
the evaluator and those individuals who provide the data
(e.g. classroom teachers), but may require little client
6
interaction. The more highly-structured an evaluation, the
greater the necessary interactions between the client and
the evaluator to ensure that the structured techniques are
addressing the client's needs.
The evaluation management component of this study
focused on the relationship between the evaluator and the
different audiences associated with the evaluation: (a) the
client who wanted the evaluation results and funded the
study; (b) the school administrators who controlled access
into the classrooms where interdisciplinary environmental
education supplements are used; and (c) the data providers,
specifically the classroom teachers who incorporated the
materials into their lesson plans. The interactions between
each of these groups and the evaluator were highly variable
and a strong determinant in the evaluation's success.
Additional processes associated with developing and
managing an evaluation were also considered. These included
developing the proposal, contracting with the client,
designing the instruments, analyzing the data, and reporting
of results.
Evaluation Process
The outcomes evaluation component of this study used
quantitative and qualitative techniques to assess the
7
outcomes of an environmental education curriculum
supplement. Specifically, outcomes related to environmental
knowledge, environmental attitudes, and an effective format
were examined. These are basic goals for most environmental
education program.
The outcomes component focused on NatureScope, an
interdisciplinary environmental education supplement
developed by the National Wildlife Federation (NWF) . To aid
NWF in assessing NatureScope's impact, and to provide future
direction, a sample of NatureScope issues was evaluated. To
perform the evaluation, instruments were designed, pilot
tested, revised, and administered; school systems were asked
for cooperation and volunteer teachers solicited;
instructional workshops were planned and conducted for
participating teachers; quantitative and qualitative data
were collected and analyzed; and recommendations made to the
NWF.
Evaluation Model
Just as the management of an evaluation can take many
forms, so can the specific techniques for determining a
program's value. Evaluation is recognized as a necessary
and appropriate activity and has led to the development of
several evaluation models. These range from the unstructured
8
case study, employing primarily qualitative methods, to the
highly structured systems analysis, using the quantifiable
tools of measurement theory (House,1980).
The underlying structure of the outcomes evaluation
study is based in a goal-oriented evaluation model (Isaac &
Michael, 1985). Provus (1971) saw the goal-oriented
evaluator as someone who operated independently of the
program to assess student progress in an educational
program.
The social, economic and, political factors involved in
the development and implementation of any program results in
a need to tailor an evaluation to address programs goals and
objectives. While the goal-oriented model provides a
blueprint for evaluating environmental education programs,
it does not provide the details for addressing every issue
of such a complex educational entity. Consequently, the
NatureScope evaluation used a modification of the goal-
oriented evaluation strategy.
9
NEED FOR THE STUDY
The study had two foci. First, the study tested a
model for evaluation management for environmental education
programs and, second, provided the NWF with an evaluation of
NatureScope's effectiveness in influencing environmental
knowledge and attitudes, and its compatibility for classroom
implementation.
The need to address the first focus is based in
evaluation methodology. Organizations, such as NWF, are
nonprofit special interest groups that maintain a high
political profile. This high level of interest in the
environment and political activism make these groups
concerned that their materials are effective, yet sensitive
to negative evaluation outcomes. In addition, many of these
groups are not familiar with evaluation processes.
Therefore, there is a need to identify the critical factors
related to designing and conducting an effective evaluation
of materials developed by these organizations.
Second, the study measured evaluation outcomes related
to (a) the effects of NatureScope on environmental knowledge
and attitudes and (b) teacher's impressions of the
materials. Environmental education programs often operate
under the tacit assumption that the programs are effective
10
in transmitting knowledge and improving attitudes. As noted
by Stout and Peyton (1988), this is not always the safe
course of action. Program evaluation models provide a
framework adaptable to specific programs to provide
decision-makers with information on knowledge and attitude
changes. The measurement strategies for knowledge and
attitudes must be specific to the program under evaluation
if the strengths and weaknesses of that program are to be
identified.
In addition, teachers participating in the study can
provide valuable feedback regarding the usefulness of the
materials. Disinger (1989) observed that the key to success
in environmental education is the classroom teacher. There
is a need to develop and refine the techniques necessary for
measuring program effects on environmental knowledge and
attitudes, as well as, provide teacher input on the
usefulness of the materials.
To test the feasibility of using these measures of
program effectiveness, an ongoing program was used.
NWF has been producing NatureScope since 1984 and continues
to develop and distribute these environmental education
materials. To date, 17 issues of NatureScope have been
published (Appendix A), each focusing on a different
environmental subject. Since its beginnings, NatureScope
11
has had the stated goal of "inspiring in children an
understanding and appreciation of the natural world while
developing the skills they will need to make responsible
decisions about the environment." This goal makes
NatureScope ideal for testing an evaluation model for value-
sensitiv~ topics such as environmental education.
A Naturescope subscriber survey (NWF, 1988) showed the
materials to be overwhelmingly popular, however, the report
did not address issues related to its effects on student
learning or attitudes. A need existed for data on
NatureScope's impact on environmental knowledge and
attitudes of students. This need was satisfied by
collecting knowledge test data and attitude scale data on
experimental and control groups. This involved the
development and use of environmental knowledge and attitude
scales to measure students prior to and following exposure
to NatureScope.
Information to supplement this pretest/posttest data
was collected via surveys of participating teachers
regarding their perceptions of the NatureScope materials.
Teachers who use the materials in the classroom will benefit
from a clear, concise format. The combination of
qualitative and quantitative information provided answers to
the outcomes questions that guided this study.
12
PROBLEM STATEMENT
There are many dimensions in a complete evaluation of
environmental education programs: e.g. planning the
evaluation; assessing program impacts on student knowledge
and attitudes; obtaining teacher's reactions; and describing
educational policy issues associated with the materials.
The components of such an evaluation include: (a) the
processes involved in managing an evaluation of an
environmental education program; (b) the effects of an
environmental education program on student knowledge about
the environment; (c) the impact of an environmental
education program on student attitudes about environmental
issues; and (d) the usefulness of teacher input concerning
the quality and usefulness of specific environmental
education materials.
Therefore, the problem was to examine these components
in the context of managing an evaluation for politically-
active organizations that are not familiar with evaluation
activities. The vehicle for addressing this problem was
NWF's NatureScope.
13
PURPOSES
The study was guided by two overriding purposes: (a) to
develop an evaluation management model for environmental
education programs, including dimensions related to client
relationships, instrument selection, data analysis, and
final reports; and (b) to provide evaluative information to
NWF regarding NatureScope.
RESEARCH QUESTIONS
To attain the purposes stated above, the following
research questions were addressed. Questions 1-3 deal with
the evaluation process and attendant management. Questions
4-8 relate to the outcomes of the model evaluation.
1. How do the client and evaluator interact in
developing and conducting evaluations of interdisciplinary
environmental education supplements?
2. What are the interactions between evaluators and
school administrators which affect evaluations of
environmental education supplements?
14
3. What evaluator/teacher interactions are relevant to
the evaluation of interdisciplinary environmental education
supplements?
4. How do students exposed to NatureScope compare to
those not exposed on a measure of environmental knowledge?
A. To what extent do students exposed to Let's
Hear It for Herps differ from a control group on a
knowledge measure of the content of Let's Hear It for
Herps?
B. To what extent do students exposed to Wild
About Weather differ from a control group on a
knowledge measure of the content of Wild About Weather?
C. To what extent do students exposed to
Discovering Deserts differ from a control group on a
knowledge measure of the content of Discovering
Deserts?
D. To what extent do students exposed to
Endangered Species: Wild & Rare differ from a control
group on a knowledge measure of the content of
Endangered Species: Wild & Rare?
5. How do students exposed to NatureScope differ in
environmental attitudes compared to those not exposed?
15
A. To what extent do students exposed to one issue
of NatureScope differ from students exposed to another
issue on a measure of general environmental attitudes?
B. To what extent do students exposed to the four
issues of NatureScope collectively differ from a
control group of students not exposed to NatureScope on
a general measure of environmental attitudes?
6. What is the relationship between student scores on
the knowledge tests and on the attitude measure?
A. In general, what is the relationship between
knowledge and attitude scores for students exposed to
NatureScope?
B. What is the relationship between knowledge and
attitude scores for each of the four issues of NatureScope?
7. How do fifth and seventh grade students compare on
the NatureScope knowledge and attitude tests?
8. What are teacher impressions about the NatureScope
issue used with their students.
16
ASSUMPTIONS
It was assumed that the evaluation of environmental
education supplements developed by special interest groups
or agencies poses a unique situation requiring careful
management of the processes. These organizations or
agencies may lack the experience with evaluation activities
and may require unique techniques of interaction and
management.
It was assumed that all volunteer teachers in the study
shared some interest in environmental education, even though
the intensity of interest varied. Different levels of
teaching experience and expertise may have affected the
success of the NatureScope curriculum. However, it was
assumed that the sampling design controlled for these
differences, thus precluding selection as a threat to
internal validity.
The use of an objectives-oriented model assumes that
NatureScope's objectives can be measured with quantitative
techniques. Use of this model also raises the assumption
that decision-makers want to measure the objectives stated
in Naturescope.
17
DELIMITATIONS
The outcomes portion of the study considered only three
aspects of evaluation of environmental education programs:
knowledge, attitudes, and teacher impressions. In addition,
financial and time constraints resulted in the following
delimitations:
1. While NatureScope is nationally oriented, the
study was restricted to selected Virginia and
Georgia teachers.
2. The Naturescope program is designed for use
with children from kindergarten through the
seventh grade however, the study included only
fifth and seventh grade students.
The complexity of the program negated the feasibility
of a complete evaluation during the time alloted and with
the funds available.
LIMITATIONS
The management processes involved in conducting an
evaluation are dependent on the client and purposes of the
evaluation. This study was concerned with the evaluation
management processes for a nonprofit politically-active
18
client. Therefore, while the findings may relate to other
groups, the results should only be generalized narrowly to
nonprofit politically-active environmental education groups.
The outcomes model considered only three aspects of
evaluation and therefore, cannot be viewed as a
comprehensive evaluation of all aspects of an environmental
education program. Results of the study generalize only to
those aspects of evaluation dealing with knowledge
measurement, attitude measurement, and the teacher
questionnaires.
Delimiting the study to fifth and seventh grade
students in selected Virginia and Georgia school systems
reduces the generalizability to only those populations.
Fifth and seventh grade classes differed in format.
Some schools were departmentalized at both levels, with
students moving to various classrooms and teachers. Others,
especially at grade five, used a self-contained classroom
format, where one teacher was responsible for all or most
subjects. NatureScope is not restricted to any special
classroom/school organizational format, thus, the study did
not take this variable into account.
From the perspective of Cook and Campbell (1979),
internal validity is threatened when causal relationships
between variables cannot be established. A complexity of
19
factors affect one's knowledge and attitudes making it
impossible to attribute observed changes entirely to the
influence of NatureScope.
There was also a threat to validity from the
unreliability of treatment implementation (Cook & Campbell,
1979); no controls were exercised over the actual exposure
time of each student. Consequently, the amount of time and
effort invested by the teacher in using NatureScope was a
possible influence on student scores for both the knowledge
and attitude measures.
Prior exposure of the students to the topic, either
through NatureScope or other curricular materials, may
mitigate against finding differences in either knowledge or
attitudes. This limits the ability to attribute differences
to the treatment.
The close association of the evaluator with the NWF
limits to ability to operate solely as an external
evaluator. In some aspects, this study is an internal
evaluation since the evaluator worked for the NWF during the
initial phases of proposal development. However, during the
actual design and implementation of the analysis, the
evaluator was external to the NWF. This external
characteristic was enhanced by an external evaluation review
20
committee of nationally known experts in wildlife and
evaluation.
DEFINITIONS
ADVANCED STUDENTS - sixth and seventh grade students, as
designated by NatureScope.
ATTITUDE CONTROL GROUP - participants in the study who took
the measure of environmental attitude without
being exposed to any of the four issues of NatureScope.
ENVIRONMENTAL ATTITUDES - a score on an attitude scale
designed to measure a manner of acting, feeling or
thinkirig that shows one~s opinions or feelings
concerning the protection and improvement of the
environment.
ENVIRONMENTAL KNOWLEDGE - a score on one of four
NatureScope tests designed to reflect an awareness and
basic understanding of the environment specific to that
issue of NatureScope.
EXPERIMENTAL GROUP - participants in the study who were
exposed to a particular NatureScope issue in addition
to taking the knowledge test on that issue.
INTERMEDIATE STUDENT - third grade through fifth grade
students, as designated by NatureScope.
21
KNOWLEDGE CONTROL GROUP - participants in the study who took
the test on an issue of NatureScope to which they were
not exposed during the study.
NATURESCOPE - an environmental education program developed
and distributed by the NWF. The materials are designed
as an interdisciplinary supplement suitable for
kindergarten through seventh grade.
22
ORGANIZATION OF THE STUDY
The dissertation is organized as follows:
Chapter I - a discussion of the problems related to
evaluation of environmental education programs and a general
overview of the proposed research.
Chapter II - a review of the pertinent literature
related to evaluation of environmental education and
specifically, effects on environmental knowledge and
attitudes.
Chapter III - a discussion of the methodology related
to managing and conducting the NatureScope evaluation.
This includes client and audience relationships, proposal
development, sample structure, psychometric procedures for
developing the knowledge and attitude tests, and proposed
analytical techniques.
Chapter IV - a report of the findings of the evaluation
management processes, and Naturescope's effect on
environmental knowledge and attitudes.
Chapter V - a summary of the findings and
recommendations for the evaluation of environmental
education programs, and report and recommendation to NWF as
to the effectiveness of NatureScope in the cognitive and
affective domains.
CHAPTER TWO
REVIEW OF RELATED LITERATURE
This chapter provides an overview and synthesis of the
literature pertinent to the development of an evaluation
management model and evaluation of environmental education
programs relative to knowledge, attitudes, and teacher
impressions. The following questions are addressed: 1. What precedent has been established for the
management of evaluations of environmental education programs?
2. What constitutes a model for environmental education, and how does NatureScope fit within this model?
3. What recent research is relevant to the measurement of environmental knowledge?
4. What recent research is relevant to the measurement of environmental attitudes?
5. How have teacher impressions of materials been incorporated into evaluations of environmental education curriculum supplements?
Evaluation Management
Stufflebeam (1973) proposed a checklist which, in many
ways, is suitable for developing evaluations of
environmental education supplements. An initial assessment
of the client's and audiences' needs must be made prior to
23
24
developing an evaluation proposal and contract. The
evaluation must meet the needs of the interest groups if it
is to be used.
It is critical that strong lines of communication be
maintained between the evaluator and the client (Worthen &
Sanders, 1987). Clients who may not be familiar with
evaluation procedures should be informed of what to expect
and the possibility of negative results. Wherever possible,
client input should be incorporated into the evaluation.
Not only does this help keep them informed, it also provides
the evaluator with feedback from the client concerning their
desires for the evaluation.
Through close association with the client, the
evaluator can identify the variables of interest and design
an evaluation that considers those variables. The evaluator
should help the client understand the potential positive and
negative implications from evaluating those variables.
After a general plan for the evaluation has been
developed and agreed on by the parties involved, a contract
should be drawn up and signed. The contract should contain
details of the timetable, financial resources, personnel,
and supplies that will be needed for the evaluation.
The evaluator should then begin the instrument
development process. This should involve early client
25
reviews, reliability estimates, and establishing validity
through the use of pilot tests and expert review. The
refined instruments can be used to collect the data
necessary for the evaluation.
Following collection of the data, analytical procedures
can be applied to draw conclusions about the strengths and
weaknesses of the program. If the data collection
procedures were carefully scrutinized prior to
implementation, the data should produce relevant results
that can lead to program improvement.
A summary report of the findings should be prepared and
distributed to the client and pre-determined audiences.
Depending on the audience, the evaluator may wish to stress
differing aspects of the evaluation.
Guidelines for managing evaluation have been proposed
by Stufflebeam (1973), Joint Committee on Standards for
Educational Evaluation (1981), Isaac and Michael (1985),
Worthen and Sanders (1987), and Worthen and White (1987).
Their recommendations differ very little from one another
and can be adapted to most evaluations with minor
modifications depending on the scope of the evaluation and
client needs. Evaluation management should include some
form of the following activities: work scope; client
responsibilities; contract provisions (legality of
26
evaluation, confidentiality, publication rights,
negotiation, arbitration); audience definition; instrument
development; data analysis; and final reporting.
Figure 1 provides a general model for the management of
an evaluation based on Stufflebeam's (1973) recommendations.
The NatureScope evaluation will operate from this model,
incorporating revisions and modifications where appropriate.
Focusing the Evaluation (client needs, audience)
\11 Collection of Information
(Proposal development)
xi Analysis of Information
\11 Reporting of Information
\11 Administration of the Evaluation
Figure 1. General Evaluation Management Model
Evaluation Theory
Hastings (in Gronlund, 1968) said one outcome of
curriculum evaluation was the collection of information
useful to program developers for revision of materials.
Worthen and Sanders (1987) define evaluation as a process
27
for determining worth or merit. It is often the case that
selective components of programs are evaluated, e.g.
specific outcomes, implementation, planning (Patton, 1980).
All of this implies that program evaluation is designed to
determine the effectiveness of one or a number of components
within a program. Specific to environmental education,
Staley (in Chenery & Hammerman, 1985) suggests that
evaluation should provide information to decision-makers to
improve programs.
Evaluators do not always agree on the best techniques
to arrive at information for decision-makers, in fact, there
are almost as many evaluation ~trategies as their are
evaluators. Several authors (Isaac & Michael, 1978; House,
1980; Worthen & Sanders, 1987) have provided overviews of
various evaluation strategies that may serve as guides for
developing an evaluation. These designs fall along a
spectrum from a highly-structured utilitarian model, such as
the objectives-oriented, to the less-structured naturalistic
and participant-oriented models. Models of this type relate
to the actual data collection procedures and results; they
should not be confused with models for managing an
evaluation. Evaluation management models may actually
incorporate one or more of these evaluation strategies.
28
The uses to be made of the evaluation results influence
what strategies are used for the evaluation (Isaac &
Michael, 1978). These uses are often dictated by the needs
of the client. Most evaluations, if not exact fits, are
modifications or mixtures of existing strategies.
Specific to this study, the client (NWF) desired
information on the effects of NatureScope on environmental
knowledge and attitudes. The objectives-oriented evaluation
design focuses on specified goals, in this case measures of
student progress on some established criteria. The
evaluator attempted to determine the extent to which the
criteria have been met (Worthen & Sanders, 1987). This
design provided the closest match for the NatureScope
evaluation; instruments were developed to determine if the
program was effective in addressing the stated goals related
to environmental knowledge and attitudes.
Student outcomes were not the only concern in the
NatureScope study. Because teachers are the pivotal point
in using NatureScope, their impressions of the materials
were also useful. To glean this information, a form of
naturalistic qualitative inquiry was employed.
Patton (1980) suggests that objectives-oriented
evaluations are not compatible with a pure qualitative
study, but he did concede that the two designs could take
29
place simultaneously. When this occurs, neither approach is
used in the purest sense. In some respects, two evaluations
are being conducted and the results fused into one report;
such is the case with the NatureScope study. The
dichotomy between quantitative and qualitative methods of
evaluation, and the merits of each, have been debated by
advocates of each (Patton, 1980; Lincoln & Guba, 1985;
Worthen & Sanders, 1987). The actual distinction between
qualitative and quantitative evaluation may lie on a
continuum with basic epistemological differences serving as
the anchor points. At one end is the objectivist, requiring
quantitative data that can be replicated by other
researchers or evaluators. The other end is anchored by the
subjectivist who relies on an experiential base to interpret
implied information (Worthen & Sanders, 1987). As one moves
toward the center of the continuum, the distinctions become
more vague, such as the expertise-oriented model which
depends on professional experience to judge the quality of a
program.
Another distinction in evaluation concerns the role of
the evaluator in the evaluation process (Worthen & Sanders,
1987). Internal evaluations are conducted by program
personnel who may or may not have been involved in designing
the program but nonetheless, have a vested interest in the
30
results of the evaluation. External evaluations are
conducted by individuals not affiliated with the program to
be evaluated. Both the internal and external evaluator may
have expertise in evaluation methodology. However, because
the internal evaluator may have a vested interest in the
outcome of the evaluation, human bias may result in the
selection of evaluation variables that indicate program
success. While not foolproof, the use of external
evaluators reduces the likelihood of this type of variable
selection and therefore, may result in an evaluation which
identifies subtle weaknesses in the program.
Environmental Education
The environmental education model proposed for this
study consists of an interdisciplinary multi-grade
supplement designed to influence student's knowledge and
attitudes about some environmentally-oriented topic. This
section reviews pertinent research for each of these
components.
Knowledge and Attitudes
Stapp (1978) proposed that environmental education be
based on helping students understand their connection with
31
the environment and their ability to influence that
environment. This view encompasses the knowledge component
of environmental education, but not the attitude component,
because one may understand their ability to influence the
environment and still maintain an attitude that does not
bring about action. A basic goal of environmental education
is that students exposed to the programs will develop
attitudes about the environment that will lead to mature
decision-making skills and consideration of the
environmental consequences of their activities.
The desired long-term effect has received contradicting
support from research. Byford and Munsey (1984) found that
adults who participated in 4-H activities as students,
tended to remain active in outdoor activities, such as
hunting, fishing, and nature study, as adults. students who
participate in volunteer organizations, such as 4-H, are
predisposed to an interest in outdoor activities. They
differ from the general population targeted by most
environmental education supplements. In contrast, adults
who had membership in environmentally-active organizations
indicated that the greatest influence on their activism was
spending many hours alone or with close friends in a
pristine environment (Tanner, 1982; Larson, 1989). Thus,
the findings of Tanner (1982) and Larson (1989) raise
32
questions about the effectiveness of exposure to activities
from environmental education supplements for producing
environmentally-aware citizens.
Interdisciplinary
The ways to affect student knowledge of the environment
include outdoor education, a traditional single discipline
approach, and interdisciplinary activities. Weisenmayer, et
al. (in Disinger, 1986) found the interdisciplinary approach
to be effective in teaching environmental concepts.
Environmental education supplements have followed an
interdisciplinary format (Hug, 1977; Troy & Schwab, 1982).
This format suggests that environmental education should go
beyond the realm of a separate classroom subject, to one
that permeates every part of the curriculum (Lucke,
Disinger, & Roth, 1982).
If environmental education is considered an aid in the
development of an environmental ethic, it also becomes a
gradual process (Kupchella & Hyland, 1976), providing
students with increasing amounts of information about
environmental topics which can be attached to a growing
experiential base. Several writers have suggested that this
can be accomplished by crossing disciplinary and grade
boundaries.
33
Supplemental
How can environmental education best influence students
knowledge and attitudes about the environment? Disinger
(1989) felt that the key to a successful environmental
education program was to work with educators, not in place
of them. Teachers should be'provided with the materials,
and some form of formal or informal training in their use,
and allowed to be the principal facilitator for the
environmental education program. Using teachers to install
the materials allows their use in a variety of subjects.
The concept of providing supplemental environmental
education materials to teachers may increase the use of the
materials and ultimately, the program's effect on knowledge
and attitudes. However, the program's effects on knowledge
and attitudes should still be measured to determine the
direction and magnitude of the effects.
Even though environmental education may be addressed
through a variety of subjects, the principal paradigm for
all environmental education programs is ecology (Kupchella &
Hyland, 1976). Environmental education programs may use a
core subject, such as mathematics or language arts to teach
a concept, but the basic environmental concept being taught
has its roots in ecology. When used in this way,
environmental education activities become supplementary to
34
the core subject (ecology) and the subject which serves as a
medium for dissemination (e.g. mathematics).
Two of the most successful of these interdisciplinary
environmental education supplements are Project Learning
Tree, promoted by the American Forest Institute, and the
Western Association of Fish and Wildlife Agencies' Project
WILD (Disinger, 1989). NatureScope was developed as an
interdisciplinary environmental education supplement.
The long-term effects of these and other environmental
education programs have not been established. Nonetheless,
if environmental education supplements are to establish
credibility within schools, their effectiveness as
educational tools must be evaluated (Bennett, 1989). Stout
and Peyton (1988) summed up the need for evaluation of
environmental education programs:
"The importance of evaluating curricula cannot be overstressed"
35
Evaluation of Environmental Education Programs
The evaluation of environmental education has received
much attention in the literature. An ERIC search produced
423 documents published between 1976 and 1986 that included
environmental education and evaluation in their list of
descriptors. A review of a random sample of these documents
made it apparent that while evaluation of environmental
education programs has been addressed, there is a lack of
consensus on the best e~aluation techniques.
A survey of 264 environmental education directors
indicated that evaluation was a pressing need (Disinger,
1981). Forty-three percent of the respondents used some
form of informal evaluation, 7.4% used a structured
evaluation, and 49.6% employed no evaluation strategies.
The lack of time, money, or expertise may hinder many
environmental education programs from undertaking evaluation
(Nowak, 1984). Whatever the reason, evaluation does not
seem to be a high priority item for most environmental
education programs.
A national survey of 536 environmental education
directors (Childress, 1977) provides insight into evaluation
methods employed in environmental education. External
evaluators were used by 27.9% of the respondents; 47% never
36
used standardized tests for measuring program effects on
environmental knowledge; and 34.6% never used attitude
scales to measure program effects on environmental
attitudes. Standardized tests of environmental knowledge
and attitudes have not proven to be popular. This may be
due, in part, to the great variation in topics addressed by
individual environmental education programs. The lack of
existing measurement instruments suitable for a variety of
environmental education programs, and the lack of expertise
in developing these instruments on a case-by-case basis, has
contributed to the reluctance of program directors to employ
cognitive and affective measures.
Disinger (1981) suggests that the heavy emphasis on
informal evaluation of environmental education reflects a
tendency toward implied program goals. This makes it
difficult to establish evaluation criteria. Cohen (1977)
suggested a solution to the lack of stated goals by allowing
program participants to determine program needs and
outcomes. This would ensure that at least the participant's
desires were being evaluated. The failure to establish
evaluation criteria suggests that program developers lack
experience with substantive evaluations.
Childress (1977) concluded that, if programs were
evaluated at all, they used loosely-structured, goal-
37
oriented internal evaluations. If the percentage of
respondents in the Childress study is indicative of the
emphasis placed on various evaluative components, affective
impacts were deemed more important than the cognitive
effects of a program. These could not be interpreted as
pure goal-oriented evaluations because qualitative methods
of observation were the principal measurement tool.
While Childress (1977) criticized the use of
quantitative techniques, Doran (1977) and Stapp (1978)
expressed the need for objective, reliable, valid test
instruments for use in evaluating environmental education
programs. Extending the attack on quantitative techniques
for evaluating environmental education programs, O'Hearn
(1982) advised not to use statistical significance on test
instruments as the sole basis for deciding the fate of a
program. This advice reinforces the need for expertise in
measurement in the evaluation of environmental education
programs. A quality program evaluation should never be
based on the results of one criterion.
Chenery and Hammerman (1985) expressed concern that
quantitative methods did not lead to positive useful
findings. However, they cite observation as the primary
evaluation tool of environmental education. If observation
is interpreted as a qualitative technique, their complaint
38
of too much emphasis on quantitative techniques does not
seem justified. According to Robottom (1983), the
scientific/analytic paradigm, using the pretest/posttest
format to determine effectiveness, has dominated the
evaluation of environmental education programs. Consistent
with the scientific/analytic approach, summative evaluations
are often employed to gain an overall measure of program
worth. Recent evaluations of environmental education
programs (Cantrell, in Charles, 1988; Zosel, 1987) have
demonstrated a shift from quantitative to qualitative
techniques.
Environmental education has revolved around (a)
transmission of knowledge, and (b) changes in the affective
domain. Many current programs combine these concepts into
their goal statements, as evidenced by CLASS Project (1982),
Project WILD (Charles, Meyer, & Rensel, 1984), and
NatureScope (1984). Because these concepts appear in most
environmental education programs, it is not surprising that
some programs have attempted to incorporate cognitive and
affective components in evaluation ( Fleming, 1983; Fleming,
1985; Stout & Peyton, 1988).
A precedent has been established for using quantitative
measures (knowledge and attitude instruments) and
qualitative measures (teacher impressions) in the evaluation
39
of environmental education programs. Specific information
on the role of each of these components helps establish the
rationale for the NatureScope evaluation methodology.
Measures of Environmental Knowledge
Environmental Knowledge
Cognitive psychologists have paid increasing attention
to the role of knowledge in various cognitive tasks such as
comprehension and problem solving (Gradwohl & Schumacher,
1989). This increased attention can also be observed in the
domain of science education. Townsend (in Lucko, Disinger,
& Roth, 1982) identified four underlying dimensions for the
concepts taught in environmental education; (a) ecological,
stressing the interdependence of living things, (b)
cultural, concerning human interactions with environmental
considerations, (c) natural resources, related to the
management and use of natural resources, and (d) population,
dealing with human interactions with environmental
conservation. These dimensions represent one framework for
environmental knowledge.
Previous studies addressed the role of environmental
education with respect to knowledge as a mechanism to help
individuals and groups gain experience with the total
40
environment and acquire a basic understanding of the
environment, its problems, and humanity's role in the
environment (Stapp, 1978; Hungerford, Peyton, & Wilke,
1980) .
Research Designs Using Environmental Knowledge Measures
Pretest/posttest/control group designs have been used
extensively in measuring the effects of an environmental
education program on environmental knowledge (Bennett, 1982;
Fleming, 1983, Fleming, 1985; Stout & Peyton, 1988). Use of
these designs implies that group differences in the
cognitive domain are seen by decision-makers as an important
component of an environmental education program.
Similarities in design can be seen between the
NatureScope study and the studies of Fleming (1985) and
Stout and Peyton (1988). All three studies used a
pretest/posttest control group design with analysis of
covariance of posttest scores. One instrument was developed
and used by each study for pretesting and posttesting.
41
Measures of Environmental Attitudes
Environmental Attitudes
Attitudes, interests, beliefs, and opinions are often
placed under the more generic heading of attitudes. When
reviewing literature in the affective domain, a careful
analysis of the source must be conducted to determine what
construct the author(s) are actually addressing.
A number of dimensions have been measured in
environmental education that are placed under the broad
heading of attitudes. "Environmental responsibility'', a
construct implied in environmental attitudes, was measured
by Horvat and Voelker (1976). Kellert (1986) measured
children's attitudes about the environment. Passineau
(1976) developed an "Environmental Awareness Inventory" that
measured interests.
The role of environmental education in attitude
formation is to help students acquire social values related
to the environment, strong feelings of concern for the
environment, and motivation to actively participate in
protection of the environment (Stapp, 1978; Hungerford, et
al., 1980). This role seems to encompass attitudes,
beliefs, and values.
42
Lucke, et al. (1982) raised concerns over the ethical
dimensions of environmental education trying to shape
student attitµdes. Despite these concerns, affective
outcomes remain an element of many environmental education
programs; an element that can serve as a measurable outcome
of program effectiveness.
Uses of Environmental Attitudes Measures
Within the domain of science education, researchers
have criticized attitude measures for being too vague,
inconsistent or ambiguous (Blosser, 1984; Germann, 1988).
Munby (1983a) felt that many attitude measures in science
were too content-specific and therefore, could not be used
in a variety of evaluation settings. Within the affective
domain of environmental education, criticisms of instrument
vagueness and specificity may be a function of the attempt
to evaluate the specific goals of a specific program.
Munby's (1983b) meta-analysis of 204 science attitude
instruments revealed only twenty-one that used multiple
measures. Most had poor or non-existent reliability
estimates and highly inconsistent attempts at establishing
validity. Munby concluded that, with few exceptions,
reliability and validity was lacking or poorly done in
science attitude studies. Specific to environmental
43
education, these concerns over instrument reliability and
validity may result from a lack of experience with
measurement concepts and measurement theory.
Despite these problems, attitudes have been measured
using instruments ranging from a dichotomous choice
(Passineau, 1976) to a semantic differential (Trama &
Newman, 1988). Thus, the form of measurement is highly
variable.
Environmental educators have measured attitudes, both
for gaining general information about segments of the
population and for evaluation of programs. Passineau {1976)
developed the Environmental Awareness Inventory to be a
general measure of environmental attitude; one not
associated with an environmental education program. Jaus
(1984) and Pomerantz (1985) used a Likert-type scale to
measure the environmental attitudes of school children
following exposure to an environmental education program.
Fleming (1983) and Fortner and Lyon (1985) used a variation
of the Likert scale to measure environmental attitudes
following exposure to a program.
Research Designs Using Environmental Attitude Measures
Many of the attitude measures are administered in
conjunction with knowledge measures, so the same design is
44
used. Pretest/posttest control group designs have been used
to measure the effects of an environmental education program
on environmental attitudes (Aird & Tomera, 1977; Fleming,
1983; Fleming, 1985). Mason and Kahle (1988) used a
posttest-only attitude study to guard against pretest
contamination. James and Potts (1981) used a survey to
measure the attitudes of administrators and teachers
regarding environmental education. Even though many
research designs have been used to measure environmental
attitudes, there is still no consensus on the best
techniques. Perhaps this is due to a failure to attribute
affective gains to environmental education programs. Thus,
clients and associated evaluators are still searching for
the best methods. Armstrong and Impara (1989) observed that
testing order could affect attitude scores; students who
took a knowledge test followed by an attitude measure scored
lower on the attitude measure than those who took the tests
in reverse order.
The Relationship Between Knowledge and Attitudes
Knowledge and Attitudes
No conclusive relationship between knowledge and
attitude has been established. Even though attitudes have
45
been included in the domain of science education, and
instruments have been developed to measure attitudes, this
should not be interpreted as an endorsement of the
effectiveness of science education in altering science
attitudes. Mason and Kahle (1988) concluded that it was
unlikely that additional mandatory science courses would
improve negative attitudes regarding science. They
advocated nurturing the students interests, curiosity, and
confidence thereby, improving student motivation. Thus, it
would seem that developers of environmental education
programs should not expect attitude improvements as a result
of exposure to activities. Iozzi (1989) reviewed recent
literature and concluded that the relationship between
knowledge and attitudes was unclear. Mueller (1986) points
out that attitudes are not always good predictors of
behavior.
Baker (1985) found negative science attitudes among
high achieving students and offered two plausible
explanations: (a) high achievers were bored with subject
content; (b) test instruments used in the study were not
valid for measuring attitudes toward science. If materials
are developed for average students, groups at the high and
low extremes may not receive maximum benefit. Perhaps the
multi-grade supplementary format of many environmental
46
education programs should be modified to target specific
groups. Baker's second explanation is relevant to the
criticisms of measurement instruments in environmental
education. If the instruments are not valid for measuring
either environmental knowledge or attitudes, conclusions
about the relationship between these constructs will not be
valid.
The relationship between attitude and achievement is
affected by a host of situational variables that may result
in student behavior being at variance with attitudes
(Germann, 1988). The complexity of this relationship may
result in a low correspondence between attitude and behavior
(Ajzen & Fishbein, 1977).
Ramsey and Rickson (1976) suggest a circularity in the
knowledge/attitudes relationship; greater knowledge improves
one's attitude and the improved attitude stimulates one's
desire to study the topic, thereby increasing knowledge.
Quinn and Jadav {1987) found no predominant evidence of
a causal relationship between math achievement and math
attitudes. Alaimo and Doran (1980) measured student's
perceptions about environmental problems, likelihood of
personal involvement in solutions, knowledge of problems,
and sources of environmental information. Findings
indicated that providing instruction or information in these
47
areas did little to modify environmental values. However,
researchers have also found that an increase in knowledge of
environmental terminology may result in some changes in
environmental behavior (Brown, 1979).
Showers (1987) tested the effects of knowledge
acquisition on attitude formation and change. Causality was
not established; results indicated that knowledge can be
increased without increasing favorable attitudes and
attitudes can be made more favorable without increasing
knowledge. Additionally, Ramsey and Rickson (1976)
identified a leveling effect where greater knowledge did not
result in an increase in positive attitudes about the
environment. Conversely, increases in knowledge resulted in
a moderation of activist views.
While environmental knowledge and attitudes can be
altered by exposure to an environmental education program,
changes in one variable do not necessarily account for
changes in the other variable. Burrus-Bammel (1978) found
an increase in both student knowledge and student attitudes
following a treatment. However, the coefficient of
determination (R2) between knowledge and attitudes was very
low. Birch and Schwab (1978) found that knowledge and
attitudes could be changed, but found no evidence to
establish a causal relationship.
48
Despite the caveats and contradictions, environmental
education programs continue to use attitude measures as an
evaluation tool. Iozzi's (1989) meta-analysis indicated
that environmental education can be effective in teaching
positive environmental attitudes when activities are
designed specifically for that purpose. The key seems to be
in the design of the activities; program developers should
not expect positive affective results if the activities are
strictly knowledge oriented.
Methods of Establishing the Relationship
How to accurately measure the relationship between
knowledge and attitudes is confounded by the numerous
variables that influence these constructs. Most studies
wisely have only measured the relationship. Several
analytical techniques have been used to determine the
relationship between knowledge and attitudes. Trama and
Newman (1988) used a Pearson product-moment correlation to
test the relationship between knowledge and attitudes
measured on a semantic differential scale. Analysis of
covariance, with interest serving as the covariate, was used
by Gradwohl and Schumacher (1989) to determine if the
effects of knowledge measures were independent of interests.
Similarly, pretest knowledge scores could be used as a
49
covariate with attitude scores. In an attempt to establish
causality, cross-lagged analysis (Cook & Campbell, 1979) was
used to measure math achievement and math attitude (Quinn &
Jadav, 1987). In summary, many studies have addressed the
relationship between environmental knowledge and attitudes
using a variety of techniques, but no consensus has been
reached on the efficacy of these studies.
Teacher Impressions
Importance of Teacher Impressions
To ensure that materials may be useful to teachers, and
thus may be used by teachers, working with, not in place of,
educators is a key to effective implementation of
environmental education program supplements (Disinger,
1989). To work effectively with teachers, programs must be
in a format suited to classroom use. Information collected
from teachers who have used the materials can aid in
modifying and improving programs for greater efficiency.
Collection Techniques for Teacher Impressions
In one of the earliest evaluations of environmental
education, Stapp (1965) used qualitative techniques to
gather teacher and administrator opinions of a program's
50
success in approaching instructional goals. Armstrong
(1984) used teacher surveys in conjunction with student
tests to measure program effectiveness. Qualitative
measures, such as interviews, observations, and surveys,
were used by Fleming (1983) in an evaluation of Project
WILD. These studies suggest that qualitative techniques
have been used effectively to collect information from
teachers about programs they have used.
Summary
Literature related to evaluation management stresses
the need for the evaluator to identify client needs prior to
developing the evaluation proposal. Evaluators must develop
proposals that realistically address the needs of the
client. Clients should be kept informed regarding
evaluation progress. The characteristics associated with
the evaluation experience and decision-making roles of the
NWF make it important that an evaluation management plan be
devised specific to this study.
A synthesis of the literature suggests a model for
interdisciplinary environmental education supplements.
Activities should be provided to teachers across
51
disciplines, and directed toward gradually improving student
knowledge and attitudes about the environment.
Unfortunately, research has not provided evidence that
environmental education programs promote the development of
an environmental ethic. Nonetheless, if environmental
education programs are to be concerned with promoting
environmental knowledge and attitudes, they should be
evaluated with respect to those components.
Evaluations of environmental education programs have
often focused ori program impacts on knowledge and attitudes,
implying an objectives-oriented evaluation model. This
heavy emphasis on the objectives-oriented model and its
quantitative techniques has been criticized for not yielding
useful findings. Qualitative and quantitative methods,
singularly and in conjunction, have been used in the formal
and informal evaluation of environmental education programs.
One factor that confounds the ability to draw generic
conclusions about the effects of an environmental education
program on knowledge is the broadness of the environmental
education domain. Tests have usually been developed
specific to an environmental education program and not used
with multiple programs. Thus, while many programs have
demonstrated knowledge impacts, blanket statements about the
effects of environmental education on environmental
52
knowledge must be made in the context of a specific program.
Evaluations of specific environmental education programs
have considered the effects of grade and teacher motivation,
and found these variables to be important in the
interpretation of evaluation findings.
The construct of attitudes has been defined vaguely in
environmental education, and not accepted unanimously as an
ethical outcome of education efforts. This lack of
consensus has carried over into the development of attitude
measures. If attitudes are to be measured, scale items
should be designed to reflect a predisposition to accept or
reject in a consistent manner. As with measures of
environmental knowledge, measures of environmental attitudes
are generally developed for specific programs and not used
in multiple situations.
Various environmental education programs have
demonstrated effects on knowledge and attitudes, but they
have shown no consensus on the relationship between these
variables. Most researchers report results with respect to
program effects on knowledge and attitudes and omit
discussing the absence of a causal relationship. When
studies have attempted to establish causality, results were
negative or inconclusive.
53
Program effects in the cognitive and affective domain
do not constitute a complete evaluation of an environmental
education program. One of many additional components
involves the impressions of teachers who have used the
materials. This cohort can provide valuable insight into
the compatibility of the program for classroom use.
Qualitative measures have been used to identify variables
that may contribute to the effectiveness of an environmental
education program.
Knowledge and attitude effects and teacher impressions
are necessary components of environmental education
evaluation. The intricacies of the objectives and relevant
material for each specific environmental education program
make it impossible to develop measurement tools suitable for
all programs. Therefore, specific instruments must be
developed that address the specifics of NatureScope.
CHAPTER THREE
METHODOLOGY
The methodology section describes the procedures
involved in: (a) the development of the evaluation
management model for interdisciplinary environmental
education supplements and (b) the study of NatureScope's
impact on students' knowledge and attitudes about the
environment. Implementation of these methods resulted in
(a) information on the feasibility of using this model in
evaluating environmental education supplements and (b)
decision making information to NWF.
Evaluation Management Processes
The methodological steps of this evaluation are
consistent with those suggested by the Joint Committee on
Standards for Educational Evaluation (1981) and Stufflebeam
(in Worthen & Sanders, 1987). The steps provided guidelines
for managing and collecting data for the evaluation of
NatureScope.
54
55
Evaluation Management Model
Figure 2 illustrates the management model used in the
evaluation of Naturescope. Using an Activity Network (see
Figure 2), the model was designed to guide the evaluator
through the critical steps in developing and completing the
evaluation, beginning with identification of client needs
and expectations and culminating in a final report. Many of
the steps took place simultaneously.
Client Relationships
To gain an understanding of the NWF's expectations from
the evaluation, a three month internship was completed at
the NWF headquarters. During the internship, numerous
meetings with the NatureScope staff were held. These
meetings were designed to produce a general evaluation
framework including the scope of the evaluation, variables
to be measured, materials that would be provided by NWF and
by the evaluator, and a proposed progress review system for
the evaluation. Through close association with the
NatureScope staff, the evaluator hoped to identify hidden
agendas that might affect the success of the evaluation.
56
Identify NWF's needs and expectations
\11 Select NatureScopes to be evaluated
\11 Develop Proposal
\11 Develop Evaluation Contract
\II Design Sample Plan
\II \II Gain Cooperation of School
Administrators Develop evaluation
instruments
\11 \11 Solicit volunteer teachers Conduct Pilot tests and
revise instruments
\ I Develop treatment/
control groups I
\II
\11 Print final instruments
I \II
Meet with teachers to distribute materials
vi Collect data from pretest
\11 Collect data from posttest, attitude scale,
and teacher questionnaire
\11 Analyze data
\11 Draft Report
\11 Submit Final report to NWF
Figure 2. Activity Network of the NWF Evaluation
57
Proposal Development
A series of steps was undertaken in the development of
a proposal: the need for the evaluation was identified;
evaluation goals were identified; techniques were outlined
to address those goals; and a proposed budget was outlined.
A contract was established with the NWF specifying the
costs, timeline, and method of reporting for the evaluation.
Specific to the NatureScope evaluation, this contract
consisted of a standard NWF Research Fellowship agreement.
School System Relationships
Following agreement on the focus and methods of the
evaluation, contacts were made with school system
administrators for the pilot study and the actual
NatureScope evaluation. These contacts were made by mail,
telephone, and personal visit, with the goal of obtaining
permission to approach teachers within that school district.
Ways of minimizing bureaucratic conflicts were observed and
implemented where possible.
Evaluation Methodology
The proposal and contract limited the scope of the
evaluation and specified the responsibilities of the client
and evaluator. The methods of collecting the evaluation
58
data were outlined in general in the proposal. Quantitative
data were collected on the program's effects on
environmental knowledge and attitudes; qualitative
information regarding the impressions of teachers about the
materials was also collected.
Data on the knowledge aspects of the evaluation were
collected using a pretest/posttest/control group design.
Knowledge measures were taken for treatment and control
groups for each issue prior to exposure to the treatment
(pretest) and following exposure (posttest) . The pretest
and posttest measures were taken with the same test
instrument.
Unlike traditional control group designs where the
control group does not participate in any phase of the
study; this design employed a modification where the
treatment group for one issue served as the control group
for another issue. This could be done because (a) there was
no reason to expect that the groups differed in
environmental knowledge at the outset and (b) there was no
reason to expect that exposure to one issue would result in
increased knowledge of another issue.
Attitudes were measured only at the posttest. Unlike
the knowledge tests, which were content specific, the
attitude measure was a general measure of environmental
59
attitudes. Thus, there was no reason to expect prior
differences in attitude among the groups.
Selection of NatureScope Activity Guides
Each of the 15 issues of NatureScope available at the
time of the study was placed into one of the following four
categories: animals, habitats, earth and space, and
miscellaneous. One issue was randomly selected from each
category. The 4 issues selected were: Let's Hear It for
Herps, Endangered Species: Wild & Rare, Wild About Weather,
and Discovering Deserts. The content of these four issues
constitute the knowledge dimension of the study.
Population/Sample
The population from which the sample was drawn
consisted of fifth and seventh grade teachers throughout
Virginia and Georgia. A non-random selection process was
used based on the ability to gain access to school districts
and classrooms. Two additional school districts were
selected to pilot the instruments. Volunteer fifth and
seventh grade teachers from each cooperating school district
rank ordered the four NatureScope issues they would most
prefer using. Volunteer teachers have been used in other
environmental education studies (Stout & Peyton, 1988) .
60
Mason and Kahle (1988) warned that teachers, required to
participate by superintendents or principals, may be
uncooperative. Teachers were assigned to their most-
preferred issues and experimental or control groups on a
first come-first served basis. Table 1 contains a breakdown
of the original sample by state, grade, and NatureScope
issue. Three teachers dropped out of the study prior to
picking up their materials.
Table 1.
Number of Teachers Assigned to each NatureScope Issue
Issue Virginia Georgia
Fifth Seventh Fifth Seventh
Let's Hear It for Herps 15 3 7 3
Discovering Deserts
Wild About Weather
6
9
Endangered Species: Wild & Rare 5
Totals 35
2
0
10
15
7
5
7
26
4
4
1
12
Teachers were assigned to one of four experimental
groups, each using a different issue of NatureScope (Figure
3). Each experimental group class also served as a control
class for a different NatureScope issue. Group A, served as
the experimental group for "Let's Hear It For Herps" and as
61
the control group for "Wild About Weather"; Group B served
as the experimental group for "Discovering Deserts" and
control group for "Endangered Species: Wild & Rare"; Group
C served as the experimental group for "Wild About Weather"
and control group for "Let's Hear It For Herps"; Finally,
Group D served as the experimental group for "Endangered
Species: Wild & Rare" and the control group for "Discovering
Deserts". Thus, each group took a pretest on two issues,
was exposed to the materials, posttested on their issue and
on the issue for which they served as a control group (with
no exposure to another set of material~) .
Group Experimental Control
A Let's Hear It for Herps Wild About Weather
B Discovering Deserts Endangered Species
c Wild About Weather Let's Hear It for Herps
D Endangered Species Discovering Deserts
Figure 3. summary of Experimental/Control Groups for Knowledge Tests
A pure control group was designated to compare
differences in attitude scores. Because all the students in
the experimental and control groups for the knowledge test
were exposed to some issue of NatureScope, and therefore
62
might exhibit attitude changes, the attitude scale was
administered to a group of students never exposed to the
NatureScope materials. Six teachers per grade were selected
to serve as pure controls.
Teachers who had volunteered to participate in the
study were given the option of serving as a treatment or
pure control group. These classes served as the control for
all four issues of NatureScope. All experimental groups
took the attitude measure.
Knowledge Test
A knowledge test was developed by the researcher for
each of the four issues. A list of general concepts for
each activity guide was developed in conjunction with NWF
staff and test items designed to address those concepts.
Items were developed according to guidelines set forth by
Gronlund (1981). Discovering Deserts, Wild About Weather,
and Endangered Species: Wild & Rare each contained 34
multiple choice, true/false, and matching items, with the
first 3 items serving as examples for completing the scoring
sheet. Let's Hear It for Herps contained 32 matching and
true/false items, with the first 2 items serving as
examples. Appendix B contains copies of the four knowledge
tests.
63
For each of the 4 knowledge tests, reliability
estimates were obtained during the pilot test, pretest, and
posttest. Coefficient alpha (Crocker & Algina, 1986) was
used to estimate score reliability (Table 2).
Table 2. Reliability Estimates for the Five Instruments
Test Pilot test Pretest Post test
Let's Hear It for Herps .62 .68 .81
Discovering Deserts .78 .64 .82
Wild About Weather .65 .79 .77
Endangered Species: Wild & Rare .56 .65 .89
Attitude Scale .62 NA .66
NatureScope is structured for primary (K-2),
intermediate (3-5), and advanced (6-7). Activities within
each issue are designated for the appropriate grade levels
by indicating one or more of these classifications.
Measures at the fifth and seventh grade produced data
pertinent to both intermediate and advanced categories. A
knowledge test was made for each of the four issues and the
same test used at both fifth and seventh grade levels. No
unique fifth or seventh grade activities were identified
for teachers in those groups. Most NatureScope
activities are relevant to either category.
64
All tests were written approximately at the fifth
grade level. The determination of reading level was made by
having fifth grade teachers review the tests and by
reviewing comments made on the tests by students during the
pilot studies. Additionally, elementary and middle school
science text books were reviewed for key words related to
the topics addressed in each of the four tests (Berger, et
al, 1979; Bendick & Gallant, 1980; Barufaldi et al., 1984).
Many of the words and concepts included in the tests were
used in the science books. Thus, the tests were assumed to
be at an appropriate reading level. No standard readability
index was applied.
Content Validity
The content validity of the knowledge tests was judged
by a panel of experts consisting of educational researchers,
environmental education specialists, and topic professionals
in wildlife and weather. Using copies of the NatureScope
activity guides, they assessed the extent that the tests
measured the domains of interest, in this case, knowledge of
the topics contained in each of the four NatureScopes
selected for the evaluation. The four knowledge tests were
initially developed and submitted for review to NWF. Based
on their recommendations, modifications were made. The use
65
of expert review, within and outside of NWF, and the list of
general concepts for each issue aided in producing content
valid instruments for the NatureScope evaluation.
Appendix C contains a listing of the objectives for
each intermediate or advanced activity in the four selected
issues of NatureScope. These objectives are often
duplicated from one activity to another. Some of the
original single objectives were actually multiple
objectives, when viewed from a measurement perspective.
Appendix D contains a set of suggested revised objectives
with duplicate objectives combined into one weighted
objective. The multiple objectives were expanded by the
researcher into several single objectives. The revised
objectives served as an initial blueprint for test
development. In addition, the revised objectives helped
control the distribution of topics from the domain and
provide appropriate emphasis to various topics.
Instructional Validity
Instructional validity focuses on the match between
content covered in the classroom and content covered on the
test (Crocker & Algina, 1986). The study design did not
provide teachers with specific details of classroom conduct
for teaching an' issue. Rather, the design allowed the
66
teacher to implement each activity in a manner most
compatible with their lesson objectives. This made it
difficult to control and ensure instructional validity.
However, a measure of instructional validity was gained
through activity reports completed by the teachers.
Attitude Scale
The domain of environmental education was broadly
sampled for the attitude scale instead of drawing on items
specifically from the four NatureScope issues involved in
the study. This allowed the same instrument to be used with
all groups and provided an overall view of the effect of
NatureScope on the general environmental attitudes of study
participants.
Existing tests of environmental attitudes (Fortner &
Lyon, 1985; Jaus, 1984) were used in conjunction with
original items to form a pool of 23 items. The items from
existing measures were adapted to make them more approp~iate
for the study. Pilot tests were conducted on the attitude
scale using a sample of 40 students. Item analysis was used
to modify or remove items that did not contribute to
reliability. The final measure of environmental attitudes
was a 16 item Likert-type scale. The response scale ranged
from 1 (strongly disagree) to 4 (strongly agree). Likert-
67
type scales have been found to yield high re~iabilities with
few items and are designed to measure the strength of an
individuals attitudes (Germann, 1988). The attitude scale
was administered to .students following exposure to the
Naturescope materials. Appendix E contains a copy of the
attitude scale. Reliabilities are reported in Table 2.
Response choices were summed to obtain a total score
for attitude. The possible score range was sixteen (a
negative attitude) to sixty-four (a positive attitude) .
Teacher Handbook
The handbook provided teachers with summary forms for
activities suitable for fifth and seventh grade students.
Because many of the activities are designated by NWF as
suitable for both grade levels, any activity designated as
intermediate or advanced was included in the handbook. Each
teacher was asked to keep a record of when they used an
activity, and to indicate the depth of study and amount of
time spent on an activity.
Smith and Glass (1987) stressed the importance of
surveying teachers on the readability, content coverage,
accuracy, and aesthetics of curricular materials. The
NatureScope handbook contained a teacher questionnaire,
designed in accordance with the guidelines of Converse and
68
Presser (1986), and summary forms for each activity used in
the study. Questionnaires were completed by teachers after
using the materials and used to determine their reactions
NatureScope's format and usefulness (Appendix F}.
Pilot study
Following development and panel review, the four
knowledge tests and the attitude scale were pilot tested to
obtain data for item analysis and to obtain reliability
estimates on the measures. Information about the length of
time required to complete the knowledge and attitude tests
was also obtained. This information was provided to
teachers participating in the study to assist them in
allocating time. Four classes (2 fifth and 2 seventh)
participated in the pilot testing the knowledge test. Two
additional classes were used to pilot the attitude scale.
students who participated in the pilot test also
provided feedback on question ambiguity. Items that lacked
clarity for more than 3 students were modified or deleted.
Procedures planned for data analysis were tested using
data collected in the pilot studies. Answers were placed on
optical scanning sheets (opscans) and scored. These efforts
ensured that the identification information provided by
69
participants was adequate for classifying the scores into
categories for analysis.
The teacher questionnaire was pilot tested to assess
usefulness and accuracy. This pilot study was done at the
same time as the pilot tests related to the attitude scale
and knowledge tests. Teachers reviewed the questionnaires
and handbook and commented on any statements or questions
that lacked clarity. Minor modifications were made prior to
printing the final version.
Treatment Implementation
Grades for the study were selected to allow a single
measure to be used for both groups. This simplified the
process of test development and allowed for comparisons
across grades of the test results.
Teachers participating in the study were required to
attend an orientation workshop. This was done to ensure
that teachers were familiar with the procedures for the
study and understood the treatment format. Treatment
materials were also given to the teachers at the workshop.
Each teacher participating in a treatment group was
asked to complete a logbook of the activities they used and
a survey of their general impressions of the materials. The
70
activity forms and the survey were provided in a teacher
handbook (Appendix G) .
The handbook was developed by reviewing each activity
guide and identifying the activities suggested for fifth or
seventh grade. Each handbook was issue-specific and forms
were provided for only those activities applicable to the
study.
Materials were to be used in the classrooms for a four
to eight week period. This provided sufficient time for the
teacher to complete an issue of NatureScope (J. Hickey,
personal communication, August 2, 1988). Aird and Tomera
(1977) found significant shifts in students' values
concerning wise use of water resources after only a two week
exposure to a water conservation instructional unit.
The rationale for this amount of exposure to the
materials is twofold. First, this time frame seems to be a
reasonable commitment from the teachers involved without
requiring an unrealistic reallocation of time and effort
from the standard curriculum. Second, research has shown
that intense exposure to a topic over a brief period may
actually produce negative attitudes concerning that topic
(Block, 1970). The Naturescope evaluation was designed to
minimize these possible deleterious effects. Students
received exposure to only one NatureScope issue.
71
In order to simulate a classroom setting, no controls
were placed on the teachers regarding how many activities
they were required to use. The exact method of
implementation was not specified; teachers were allowed to
modify the activities as they wished.
Distribution of Materials
Packets containing a NatureScope activity guide, copies
of necessary tests, handbook, opscans, and a return envelope
with the appropriate postage were prepared for each teacher
participating in the study. These packets were distributed
at an orientation meeting lasting approximately one hour.
The basic purpose and design of the study was explained.
Teachers were given time to review the materials and ask
questions.
Each teacher was assigned a 5-digit pref ix to be used
in conjunction with a 2-digit student identification number
assigned by the teacher. The student used this 7-digit
identification number on their pre, post, and attitude
forms.
Approximately five weeks after distributing the packets
a second packet was mailed to each teacher. This packet
contained another pre-addressed stamped envelope for each
teacher to return the posttest forms. The attitude scale
72
was mailed directly to the teachers following printing at
the NWF offices. Each participating teacher was given the
opportunity to receive, free of charge, 6 additional issues
of NatureScope or a one-year subscription to International
Wildlife, National Wildlife, and Ranger Rick magazines. A
form was provided in this second packet for the teachers to
indicate which of these two choices they would prefer.
Evaluation Outcomes
Response Rates
Thirty-seven teacher activity packets were distributed
in Georgia and forty eight packets in Virginia (N=85).
Twenty usable packets (54%) were returned from Georgia and
28 were returned fro~ Virginia (58.3%), yielding an overall
return rate of 56.5 percent. Teachers were to return the
pretest, posttest, and attitude scale opscans for each
student, the activity handbook indicating which of the
activities they used and their comments on those activities,
and the teacher survey. Twenty-seven teachers (31.8%)
returned the complete packages, ten (11.8%) returned only
the opscans, six (.07%) returned only the activity records,
and five (.06%) returned only the teacher survey.
73
Of the eighty-five teachers participating in the study,
fifty-nine were at the fifth grade level (69%). Thirty-
eight percent (n=32) of the fifth grade teachers returned
complete packets. Nineteen fifth grade teachers (22.4%)
returned none of the information and were classified as
having dropped out of the study. Seven (.08%) returned
either the pretest or posttest, but not both.
The remaining twenty-six teachers (30.6%) participating
in the study were at the seventh grade level. Fourteen
seventh grade teachers (16.5%) returned activity packets
useful for complete analysis. Eight seventh grade teachers
(.09%) returned none of the materials and were classified as
having dropped out of the study. Five (.06%) returned
either the pretest or posttest, but not both. Table 3
provides a summary of the number of treatment and control
classes for each issue.
Pretest
Teachers were instructed to give the treatment and
control pretests prior to using any of the NatureScope
activities. The opscans for the pretest were returned to
Virginia Tech for analysis. Following scoring of individual
tests, class means were calculated.
74
Table 3.
Number of Treatment and Control Classes by Grade Level
Issue Treatment Control
Fifth Seventh Fifth Seventh
Let's Hear It for Herps 10 4 6 1
Discovering Deserts 7 4 7 7
Wild About Weather 4 2 10 3
Endangered Species: Wild & Rare 9 3 10 1
Some teacher attrition was observed following the
pretest, with fifty-nine teachers (69.4%) returning the
pretest. This became the working sample for final analysis
of the knowledge tests.
Posttest
Teachers in the experimental groups were free to begin
using the NatureScope activities after giving the pretest.
After completion of the unit a posttest was administered
using the same instrument as the pretest. Exposure time
between the pretest and posttest varied from four to six
weeks. Classroom activity was not monitored during that
time. The opscans for the posttest were returned to
Virginia Tech for scoring. Only forty-five teachers (52.9%
of the original sample) returned the pretest and posttest.
75
This constituted 76.3% of the sample of the pretest returns.
One teacher returned the posttest even though they did not
send in the pretest.
Attitude Scale
At the same time the posttest was administered, each
student was administered an attitude scale. Mean attitude
scores were calculated for each class across the four issues
of NatureScope and for each class in the pure control
groups. This information was used to make comparisons of
the overall impact of NatureScope and the impact of the
individual issues on environmental attitudes.
Individual student posttest scores and scores on the
attitude scale were correlated using a Pearson Product
Moment correlation coefficient.
Analysis
Knowledge Tests
An a priori alpha of .10 was established for
statistical significance. Four separate 2 X 2 Analysis of
Covariance (ANCOVA) procedures with 2 treatment levels
(experimental and control), and two grade levels (fifth and
seventh) were used to compare means on the knowledge tests.
76
Adjusted class means were the unit of analysis. Using
pretest means as the covariate, adjusted posttest means were
calculated for each teacher.
Attitude Scale
Analysis for the attitude scale consists of a 5 X 2
ANOVA with five treatment levels (four issues of NatureScope
and one control group), and 2 grade levels (fifth and
seventh). This analysis allowed for comparisons of attitude
effects among the various issues of NatureScope and the
control group. Individual student scores on the attitude
measure served as the unit of analysis for determining the
correlation between knowledge and attitudes.
Teacher Questionnaire
Thirty-eight teachers (44.7%) responded to the survey.
Excerpts from teacher surveys and handbooks provided insight
about the effectiveness of the material's design. This
provided the information necessary for the qualitative
question concerning the teachers impressions of the
materials.
CHAPTER FOUR
RESULTS OF THE DATA ANALYSIS
Introduction
This Chapter is divided into two sections. Section A
addresses the first 3 research questions related to the
processes involved in an evaluation management model for
environmental education supplements. Section B considers
the effectiveness of NatureScope in influencing the
environmental knowledge and attitudes of students as related
to questions four through eight.
Analysis of Research Questions
Section A
Question 1.
What client/evaluator relationships are associated with
the evaluation of interdisciplinary environmental education
supplements?
77
78
Introduction
Many organizations that develop and distribute
environmental education materials are nonprofit and
politically-active. If the evaluator is to conduct a
successful evaluation, the special needs of these clients
must be identified and addressed.
Analysis and Results
An internship was conducted with NWF prior to and
during development of the evaluation proposal. This made it
possible to develop a realistic evaluation which better met
the needs of the client. The internship also provided the
evaluator with an internal assessment of the NWF and it's
political nature; factors which might affect the use of
evaluation results. As the evaluation progressed, hidden
agendas resulting from these political factors surfaced that
required modification of the evaluation plan to meet the
needs of the NWF.
An evaluation outcome of the Naturescope study
concerned the arrangement and utility of the objectives
stated for each activity. Many duplicate objectives were
noted as the item specification matrix was being developed
to aid in writing knowledge test items. If the number of
times an objective is mentioned is used to weight the
79
emphasis that objective should receive on the test, there is
not a match between the weighted objectives and the desired
objectives from the NWF. Following exposure to the
materials, program developers desired that students possess
a general knowledge of the ecological concepts expressed in
each issue. This lack of match between the stated and
desired objectives could be misleading to teachers. In
addition, hidden agendas of this type cause complications in
the development of evaluations and evaluation instruments.
The selection and ultimate development of the attitude
measure was influenced by the political nature of the
client. The Environmental Awareness Inventory (EAI),
developed by Passineau (1976), was originally selected as
the measure of NatureScope's effects on environmental
attitudes. This instrument was selected due to it's general
environmental focus and acceptable reliability estimates.
Pilot tests during the NatureScope study supported these
reliability estimates. However, the NWF rejected the
instrument on the basis that it did not measure attitudes in
a way they felt was appropriate. The client consulted
outside evaluators and solicited their support in rejecting
the instrument. This was done despite approval of the
instrument by evaluation experts on the proposal review
team. In an effort to maintain rapport with the client and
80
continue the evaluation, the EAI was rejected and a new
environmental attitude scale developed.
Before the evaluation plan was formulated, meetings
were held with NWF decision-makers to gain insight into
their expectations for the Naturescope evaluation. The
group was not experienced in evaluation or the analytical
processes involved. For example, the concept of test
reliability was not understood and had to be explained at
one of the meetings.
Formal and informal meeting about the evaluation made
it clear that the NWF was very concerned about the
possibility of a negative evaluation. This fact was
manifested in the selection process for the Naturescope
issues to be evaluated. NatureScope program staff had clear
opinions of the relative merits of each of the available
issues and expressed concern that issues be selected which
might produce positive cognitive and affective results.
This resulted in loss of a truly random selection process.
81
A final proposal was developed and submitted to the NWF
for approval. The proposal specified that the evaluator
would conduct an evaluation of the effects of selected
issues of NatureScope on the environmental knowledge and
attitudes of fifth and sixth grade students in Virginia.
This was later modified, as discussed under question 2. The
proposal indicated the general methods to be used in
collecting the data, the expert review team to be employed,
and a proposed timeline and budget. The timeline was
flexible and allowed for the numerous modifications that
were necessary due to evaluator and NWF delays.
Throughout the evaluation, the need for frequent and
direct communication with the client was reinforced. Lack
of communication resulted in a misunderstanding of the NWF's
desired emphasis for the knowledge and attitude tests. This
resulted in delays which threatened the completion of the
evaluation before school adjourned. To minimize the impact
of these delays, the printing and distribution of some
teacher materials was done directly from the NWF offices.
Had communication been more frequent, the last-minute
problems associated with the attitude scale may have been
avoided.
82
Summary
Even though the concept of a NatureScope evaluation was
supported, the decision-makers at the NWF proved to be very
concerned about the possibility of a negative results.
Efforts were made to minimize these concerns by working with
NatureScope staff in the issue selection process and
detailing the evaluation process.
The contract did not specify the liberties that the NWF
could take with instrument modification and approval. This
resulted in client/evaluator conflicts concerning the
knowledge and attitude measures. Closer communications
might have minimized these conflicts.
Question 2.
How do school administrators react to participating in
evaluations of environmental education supplements?
Introduction
To gain access to teachers who might volunteer to
participate in the evaluation, it was necessary to follow
the chain of command and go through the central school
administration. This produced some difficulties, as well as
some enlightenment to the evaluator regarding the nuances of
83
working within the political structure of school
administrations.
Analysis and Results
The original proposal called for volunteer teachers
from Virginia. It was hoped that previous professional
contacts with many Virginia school administrators would
reduce the difficulties in gaining access to the school
systems: such was not the case. Administrators were
protective of their property (i.e. teachers) and expressed
reluctance in granting teachers the opportunity to use the
materials, even on a volunteer basis. The general mind set
was that teachers were too busy to add something else to
their schedule. Even the promise of free materials for
participating teachers did not provide enough incentive.
This difficulty in gaining access to Virginia school
systems made it necessary to expand the boundaries of the
evaluation to include Georgia. This move was approved by
the evaluation review committee. Science coordinators for
the Regional Educational Services Agencies (RESA} were
contacted by letter and telephone. These coordinators were
personal and professional acquaintances of the evaluator and
provided easier access. Some opposition was still met due
to the busy schedules of the teachers. Some administrators
84
who allowed teachers to participate resisted the concept of
an orientation meeting as being too time-consuming; they
encouraged direct mailing of the materials and instructions
to each teacher.
Another concern expressed by Virginia and Georgia
school administrators was the fit of the NatureScope
activities within the fifth and sixth grade curriculum.
After discussions with several administrators, it was
decided to modify the evaluation to include fifth and
seventh grade classes. This change was approved by the
evaluation review committee.
The curriculum for fifth and seventh grades was more
compatible with the four NatureScope issues selected. This
seemed to make administrators more willing to permit
teachers to participate in the evaluation.
By far, the greatest success was experienced with those
administrators who were personal and professional
acquaintances of the evaluation team. This raises the issue
of the political aspect of producing successful evaluations
within the school systems and reinforces the need for
evaluators to work closely and cooperatively with
educational professionals to develop a network of contacts.
85
Summary
Many administrators seemed to refuse to participate in
the evaluation simply because it is easier to decline than
try to cooperate. Others rejected the evaluation as a way
of protecting their teachers from an increased workload,
even though teachers were to volunteer for the study. The
evaluation was also rejected on the basis that the selected
NatureScope issues did not fit with the existing curriculum.
The greatest success was achieved with administrators who
were personal and professional contacts prior to the
NatureScope evaluation.
Question 3.
What evaluator/teacher interactions are relevant to the
evaluation of interdisciplinary environmental education
supplements?
Introduction
The evaluator worked closely with the volunteer
teachers at the beginning of the study. This association
was done to distribute the materials and clarify the
teacher's responsibilities.
86
Analysis and Results
Most of the teachers who signed up for the study
actually attended the orientation meeting. Some could not
attend for legitimate reasons and requested that another
teacher pick up their packets.
Some teachers who attended the orientation meeting
expressed concerns about their lack of knowledge about the
environmental topics in the study. However, others seemed
quite confident and had taught the subjects before. The
evaluator served as a source of information on the topics
and study methodology, as many of the teachers were not
familiar with evaluation processes.
Once the materials were distributed and teachers began
to implement the study, they could call the evaluator for
clarification of activities or responsibilities. This was
done by some teachers, who seemed genuinely concerned that
they do a good job.
There was a wide range of quality in the opscans and
teacher surveys that were returned at the end of the study.
Teachers who contacted the evaluator during the study
generally had forms that were more usable and complete.
The free gifts from the NWF to participating teachers
seemed to promote participation. All teachers who
participated in the study completed and returned the form
87
indicating which of the free materials they would like to
receive. In addition, several teachers contacted the
evaluator to inquire about the materials and when they might
expect to receive them.
Teachers became busier at the close of the school year
and some were forced to abandon the study. Many of these
expressed regrets to the evaluator and wished that they
could have done more.
summary
Approximately 10% of the teachers in the NatureScope
study maintained close contact with the evaluator for
direction in the study. These teachers generally did a
better job of completing the necessary evaluation forms.
The evaluator provided information to teachers on all
aspects of the evaluation including the subject and the
evaluation methods. This ability to answer questions in
both areas proved beneficial at the orientation meeting.
Section B
Relative to answering the following questions on
NatureScope's effects on environmental knowledge, a test for
homogeneity of regression was conducted. The results are
88
contained in Appendix J. Appendix K contains a summary
table for the cell means by grade and group for the pretest
and posttest.
Question 4.
How do students exposed to NatureScope compare to those
not exposed on a measure of environmental knowledge?
Introduction
Results of the knowledge tests for the four NatureScope
issues were analyzed using analysis of covariance, with the
pretest scores serving as the covariate; class means were
the unit of 'analysis. Information from the series of ANCOVA
tests was used to answer the subquestions under research
question 1.
Let's Hear It for Herps showed a statistically
significant difference between treatment and control groups.
Discovering Deserts, while not statistically significant,
did have some practical significance. The other two issues
(Wild About Weather, Endangered Species: Wild & Rare)
exhibited little, if any, differences between the treatment
and control group means.
The planned 2-way ANCOVA for the Endangered Species:
Wild & Rare knowledge test could not be done due to teacher
89
attrition at grade seven. Instead two 1-way ANCOVA's were
done.
Analysis and Results
The main effect for classes that participated in
activities from Let's Hear It for Herps was significantly
different from those in the control group (F=4.291, p=.056).
Table 4 provides a summary of the ANCOVA results for Let's
Hear It for Herps.
A difference of 3.21 points between the adjusted
treatment (19.22) and control (16.01) group means for the
main effect translates to an average score difference on the
Herps knowledge test of 10.7 percent (Table 5). The
practical importance of this finding indicates that exposure
to the activities can increase knowledge.
Table 4. ANCOVA for Let's Hear It for He rps
Source DF MS F F prob.
Pretest (covariate) 1 98.728 17.259 .001*
Treatment/Control 1 24.546 4.291 .056*
Residual 15 5.721
Total 17
90
Table 5.
Adjusted Treatment and Control Group Means for Knowledge
Tests
Issue Treatment Control
Let's Hear It for Herps 19.22 16.01
Discovering Deserts 14.51 12.02
Wild About Weather 18.69 18.41
Endangered Species: Wild & Rare 15.07 15.77
The treatment group for Wild About Weather did not show
a significant difference from the control group (F=.014,
p=.910). Details of this analysis are provided in Table 6.
The difference between treatment and control groups for the
weather knowledge test was .28 points (see Table 5).
Weather was a fifth grade curriculum topic in Virginia and
Georgia thus, many of the students had already been exposed
to the concepts and gained very little additional knowledge.
If the covariate is to be justified, it should
correlate with the dependent variable. The covariate for
the Wild About Weather knowledge test was not significant
(F=.859, p=.390). Thus, controlling for differences in
posttest scores due to variations in pretest knowledge
resulted in a loss of one degree of freedom from the error
term while reducing statistical power.
91
Table 6. ANCOVA for Wild About Weather
Source DF MS F F prob.
Pretest (covariate) 1 8.686 .859 .390
Treatment/Control 1 .142 .014 .910
Grade 1 10.949 1. 083 .338
Interaction 1 .758 .075 .793
Residual 6 10.109
Total 10
The treatment group for Discovering Deserts had an
adjusted knowledge test mean of 14.51 compared to 12.02 for
the control group (F=2.969, p=.113).
are provided in Table 7.
The ANCOVA results
Table 7. ANCOVA for Discovering Deserts
Source DF MS F F prob.
Pretest (covariate) 1 117.952 17.115 .002*
Treatment/Control 1 20.460 2.969 .113
Grade 1 .908 .132 .723
Interaction 1 1. 344 .195 .667
Residual 11 6.892
Total 15 14.475
92
Students in the treatment group had knowledge test scores
that averaged 2.49 points higher (8.3%) than the control
group (see Table 5).
In all but one case, the treatment group outscored the
control group on the knowledge tests. Classes that
participated in activities from Endangered Species: Wild &
Rare had an-adjusted knowledge test mean of 15.07 (N=lO).
The three control classes for Endangered Species: Wild &
Rare had an adjusted mean of 15.77 (see Table 5). This
apparent anomaly (Table 8) was not statistically significant
(F=.120 p=.736). An examination of the pretest means
reveals that the control group mean (13.07) was
significantly lower (F=4.015, p=.060) than the treatment
mean (14.52) at the outset (Table 9). When the posttest
means for "Endangered Species" were adjusted based on
pretest means, the resulting adjusted mean for the control
group was higher.
Table 8. AN COVA for Endangered Species: Wild & Rare
Source DF MS F F prob.
Pretest (covariate) 1 118.927 14.671 .003*
Treatment/Control 1 ~972 .120 .736
Residual 10 8.106
Total 12 16.747
93
Summary
Only the Let's Hear It for Herps sample exhibited
significant differences between the treatment and control
groups. However, the Discovering Deserts sample showed some
gains in knowledge scores and would have been significant
given a larger sample size. Based on teacher comments and
the nature of the differences noted in both raw scores and
adjusted means, there appears to be a real difference
between groups for Discovering Deserts that was suppressed
due to low power. The Wild About Weather groups were very
close in score, possibly resulting from the topic of weather
Table 9. ANO VA for Pretest Means for Endangered Species
Source OF MS F F prob.
Treatment/Control 1 6.681 4.015 .060*
Grade 1 15.250 9.165 .007*
Interaction 1 .506 .304 .588
Residual 19 1.664
Total 22 2.704
being part of the fifth grade curriculum in Virginia and
Georgia. The failure of the Endangered Species: Wild & Rare
treatment group to outscore the control group on adjusted
means can be attributed to significant differences in the
94
treatment and control group pretest means. Qualitative
information related to the number of activities used by the
teachers proved helpful in further clarification of these
findings.
More evidence of treatment effects in the cognitive
domain may have been observed if classroom activities had
been monitored. Requiring each class to participate in the
same activities, or a minimum number of activities, would
have provided a more equal basis on which to compare
results. However, it would also shift the study from a
typical field or naturalistic setting to a "laboratory-like"
setting.
Question 5.
How do students exposed to NatureScope differ in
environmental attitudes compared to those not exposed?
Introduction
A 16-item Likert-type scale was administered to each
student. Unlike the four knowledge tests, the same attitude
scale was used for all students in the study. Analysis of
variance was used to determine if students exposed to one
issue of NatureScope differed in environmental attitudes
95
from students exposed to a different issue of NatureScope or
from the control group. The mean across the four groups of
NatureScope was 50.43 (minimum 16 points, maximum 64 points)
on the environmental attitudes scale. The control group had
a mean of 48.89.
Analysis and Results
Class means were used as the unit of analysis for
question 5. Appendix H discusses the statistical analysis
of student scores on the attitude scale. Table 10 is a
summary of the analysis of variance for class means. No
significant differences were found between the attitude
means for any of the five groups: the four Naturescope
issues and the control group (F=l.006, p=.417). Table 11
provides the attitude means for each of the four NatureScope
issues and the control group.
Table 10. ANOVA for Attitude Scale using teachers as unit
Source DF MS F F prob.
Between 4 7.057 1.006 .417
Within 35 7.012
Total 39
96
Table 11.
Attitude Means for NatureScope and the Control Groups
Issue Class Average SD N
Let's Hear It for Herps 49.95 2.64 13
Discovering Deserts 51.51 3.25 10
Wild About Weather 49.42 2.62 5
Endangered Species: 50.82 2.13 9 Wild & Rare
Control Group 48.89 1.16 3
Grand Mean 50.39 2.76 41
Question 6.
What is the correlation between student scores on the
four knowledge tests and their scbres on the attitude
measure?
Introduction
Correlation coefficients were calculated to estimate
the relationship between student knowledge scores on the
posttest and student attitude scores. A correlation was
calculated between each knowledge test and the attitude
scale.
97
Analysis and Results
A Pearson product moment coefficient (PPM) was
calculated to estimate the correlation between students'
knowledge posttest scores and their scores on the 16-item
attitude scale. Table 12 lists the knowledge and attitude
score correlations for each of the four NatureScope issues.
The highest correlation between knowledge test score
and attitude scale score was "Herps" (.51). "Endangered
Species" had the lowest knowledge/attitude correlation
(.41).
Each of the four knowledge tests was positively
correlated with scores on the attitude scale. Correlations
were converted to a Fisher's Z and the average correlation
calculated for all four NatureScope issues; the average
correlation was .46.
Table 12.
PPM Correlations for Knowledge and Attitudes
Issue PPM R£
Let's Hear It for Herps .51 .26
Discovering Deserts
Wild About Weather
Endangered Species: Wild & Rare
Average
.42
.47
.41
.46
.17
.22
. 17
-----
98
Question 7.
How do intermediate and advanced students compare on
the NatureScope knowledge and attitude tests?
Introduction
There is a possibility that the NatureScope materials
may be more effective at one grade level than the other.
Environmental education supplements are usually designed to
be used at multiple grades. NatureScope is divided into
activities for Primary (grades K-2), Intermediate (grades 3-
5), and Advanced (grades 6-7). This study used students in
the Intermediate (fifth grade) and Advanced (seventh grade)
categories. However, many of the activities included in the
activity guides are listed as being suitable for both
groups. Because the same knowledge test was used for both
groups a comparison of group means for the different issues
is possible (Table 13).
Analysis and Results
Using .pretest as the covariate, none of the comparisons
between grades for the four NatureScope knowledge tests
achieved statistical significance. No grade comparisons
were made for Let's Hear It for Herps because one of the
cells contained only one observation.
99
The adjusted mean for fifth grade classes in the
Discovering Deserts group (N=lO) was 14.75 compared to a
seventh grade mean (N=6) of 13.84 (F=.132, p=.723).
Table 13. Adjusted Treatment Means based on Grade
Issue Fifth Seventh
Let's Hear It for Herps 17.98 13.98
Discovering Deserts 14.75 13.84
Wild About Weather 19.61 16.90
Endangered Species: Wild & Rare 15.67 14.08
Attitude Scale (unadjusted) 50.33 50.25
The largest difference between grades was found in the
analysis of the Wild About Weather group. Here, fifth grade
classes had.an adjusted mean of 19.61 (N=7), while seventh
grade classes (N=4) had a mean of 16.90 (F=l.083, p=.338).
Because weather topics are incorporated as part of the
fifth grade curriculum it is not surprising that fifth
graders scored higher on the knowledge test. However, the
use of the pretest as a covariate should have resulted in
adjusted means that reflected the effects of NatureScope.
The covariate did not contribute (F= .859, p=.390) to an
explanation of the variance in posttest scores.
100
The fifth grade classes exposed to Endangered Species:
Wild & Rare had an adjusted mean of 15.67 compared to a
seventh grade mean of 14.08. Table 14 provides a summary of
the ANCOVA for the "Endangered Species" issue (F=.605,
p=.455).
Table 14.
ANCOVA for Endangered Species: Wild & Rare by Grade
Source DF MS F
Pretest (covariate) 1
Grade
Residual
Total
1
10
12
118.927
4.680
7.736
16.747
15.374
.605
F prob.
.003*
.455
Grade was not a significant factor in attitude scores.
While fifth grade classes had a slightly higher mean, the
attitude effects of NatureScope do not differ across the two
grade levels.
Summary
Fifth grade students scored higher on the knowledge
tests and attitude scale than seventh grade students.
However, none of the differences were statistically
significant.
101
Question 8.
What are teacher impressions about the four issues of
NatureScope after using the materials?
Introduction
This information was collected from comments in the
handbooks and teacher surveys, concerning the activities
teachers used with their classes. Some of these comments
provide valuable insight into the strengths and weaknesses
of the program. Thirty-eight teachers made comments on the
questionnaire. Appendix I compiles teacher comments.
In addition to qualitative comments, the questionnaire
contained a series of Likert-type items related to the
teachers impressions of NatureScope. This information was
to be completed at the close of the study. Teacher comments
and scale means were used to identify strengths and
weaknesses in the program. Teachers were asked to maintain
a logbook (see Appendix G) of the NatureScope activities
they used during the study.
Analysis and Results
Figure 4 provides a summary of the activities used by
each of the responding teachers. Each of the four knowledge
102
tests contained items from the activities used by the
teachers.
The teachers past participation in environmental
education activities, use of NWF materials, and perceptions
of their own environmental awareness provided insight into
the characteristics of study participants. Twenty of the
teachers responding to the questionnaire (52.6%) indicated
that they had conducted some form of environmental education
activities with their classes prior to participating in the
NatureScope study. However, only nine (23.6%) had attended
some type of formal environmental education workshop.
Project Learning Tree, Project WILD and Aquatic WILD
were the environmental education workshops most often
mentioned. In addition, some teachers had participated in
local workshops specific to their geographic areas.
On a scale ranging from one (low awareness) to six
(high awareness), the average teacher participating in the
study rated themselves at 4.387 (median= 4.0). Thus most
of the teachers in the study felt that they had a high level
of environmental awareness. This corresponds to the
assumption that teachers who volunteered for the study
represent a population that might normally purchase
NatureScope.
Issue Let's Hear It for Herps
Discovering Deserts
103
Activity Herps Around the World Herps On Stage Far out Frogs Project Reptile People and Herps Hands-on Herps Hot 'n' Cool Herps Race to the Sea What Would You Do? Call of the Wild Close-up on Cobras Watchers At the Pond Living Like a Reptile Hooray for Herps Herps, Past and Present
Total Average
Lost In the Desert Animal Match-up Desert Landforms Come
to Life Desert Graphics Grit, the Sand Grain Ice Plants and Elephant
Trees Desert Plants In Trouble Evaporation Tricks Community Squares Grow'em On Your Own Desert Relay Race Desert Quest Sunrise, Sunset A Desert Variety Show The Desert Scramble The Desert Daily Times Close-Up Comparisons Desert Culture Day The Spreading Desert
Total Average
N 10
9 9 9 3 9 7 5 3 4 3 1 2 1 2
77 5.13
7 8
8 7 4
7 6 4 3 2 3 3 3 2 2 2 1 1 1
71 3.94
Figure 4 (cont.)
Wild About Weather
Endangered Species: Wild & Rare
104
Balloons Around the Sun 3 The Pressure Is On 3 Water On the Move 2 The Weather Game 2 Follow the Front 3 Crazy Hot Air Balloons 1 Weather On the Job 1 The Weather Master Myth 2 Windblown Poetry 1 A Fold-Out Cloudbook 1 Make A Weather Wheel 1 Lightning on the Loose 1 Feathered and Furry
Forecasters 1 Say It With Symbols 1 Fishy Forecasts 1 First With the Forecast 1 Weather Scavenger Hunt ~1~-
Total 26 Average
Get the Connection The Rare Scare A Timely Activity The People Factor All Around the World Sizing Up Reserves Paradise Lost Recovery Short Takes Wildlife for Sale People Power Wheel of Trouble Giant Panda Facts Meet The Press
Total Average
1. 53
8 8 3 6 4 4 3 2 5 3 2 2 1 1
52 3.71
Figure 4. Activities Used in the study
105
Items related to the participant's past use of NWF
material showed Ranger Rick to be the most used (n=22,
57.9%), followed by National Wildlife (n=20, 52.6%),
Wildlife Week (n=5, 13.2%), and International Wildlife (n=5,
13.2%). Only four teachers (10.5%) indicated that they had
previously used none of the NWF materials. Twenty-seven
teachers (71.1%) had never used Naturescope prior to
participating in the study. Of the NWF materials used in
educational settings, NatureScope was the least used.
Extrapolation of the qualitative comments suggests that this
may be due to the NatureScope format requiring teacher
preparation and participation, as opposed to the relatively
passive use of NWF magazines.
The sample of teachers who volunteered for the study
were experienced in environmental education, either from
participating in workshops, doing environmental units with
their classes, or using NWF materials. This evidence,
combined with a positive self-rating of environmental
awareness, indicates that the sample represented a
population that might use Naturescope.
Likert-type (l=low to 6=high) statements concerning
participants' impressions of NatureScope were to be
completed following use of the materials. Table 15 provides
a summary for the means and medians for each of these items.
106
Table 15. Participant Impressions of NatureScope
statement Mean
Adequate background information 4.813
Match between activities and objectives 5.097
Benefits of NatureScope illustrations 4.781
Practicality of crafts 4.387
Benefits of crafts
Interdisciplinary use
Match with school objectives
4.419
4.625
4.625
Median
5.0
5.0
5.0
4.0
5.0
5.0
5.0
NatureScope received generally high ratings on all areas.
The match between the objectives stated for each activity
and the contents of that activity had the highest mean
(5.097) on the six-point scale. The lowest rating (4.387)
concerned the practicality of the crafts. Teachers found
the materials to be well designed for classroom use and
matched with educational objectives. Illustrations and
crafts, while given a favorable rating on the Likert-scale,
did not receive favorable comments as to practicality for
classroom use.
Interdisciplinary effectiveness was seen favorably by
many teachers. Comments such as "good correlation with
geography" and "good math correlation" indicate that many of
the participants found the NatureScope materials useful with
107
a variety of subjects. These comments seem to be supported
by a mean Likert-rating for interdisciplinary use of 4.625.
This evidence suggests that NatureScope is compatible with a
variety of subjects and can be used to reinforce educational
concepts.
Participants' comments concerning the activities
usefulness in developing higher order thinking skills were
varied. Some teachers felt the materials "helped students
expand their imaginations" or practice ''problem-solving
skills". One participant stated that the ''students engaged
in higher levels of thinking." However, this viewpoint was
not unanimous among participants. One seventh grade honors
teacher felt the materials were too elementary for students
at that level ("[Students] knew the outcome and therefore
most of the activities were unnecessary"). NatureScope was
viewed by teachers as a tool for promoting higher order
thinking skills but, it seemed to be less effective with
high achieving students.
Skills in group discussion and participation were also
seen as a benefit of NatureScope. Teachers felt that many
of the activities "promote[d] discussion skills" and that it
was "nice for the students to work together in groups".
Comments were received, both pro and con, concerning
the requirements of the NatureScope activities. One
108
participant felt the "demonstrations were very effective"
and several teachers commented that the "hands-on
activities" were effective. Negative comments ranged from
an inability to acquire the suggested materials ("recordings
of frog calls not available") to crafts not working properly
("sunflower seeds would not stick to the clay"). These
negative comments are attributable, in part, to the limited
time frame for the study. Teachers did not have adequate
time to obtain reference materials that were not readily
available. Nonetheless, most of the activities were viewed
as being effective for accomplishing their stated
objectives.
Contradictory comments were received from participants
concerning the layout of the materials. Many teachers felt
that if the activities were taken in order students were
prepared for the next activity or had previous activities
reinforced. One participant felt that the information was
"too spread out", while another found the materials to be
"very well organized". Some found the pictures to be "very
helpful" yet another participant wanted "more realistic
pictures". Despite the occurrence of some negative
comments, the illustrations and crafts still received an
above average rating on the Likert-type questions (4.419).
Comments concerning the layout of the materials were highly
109
subjective. Teacher preference for a particular format
influenced their opinions of NatureScope's format.
Each of the four issues of NatureScope contained a
reference list of supplemental materials. Only six (15.8%)
of the respondents indicated that they made use of this
resource. Some teachers used reference materials not listed
in the activity guide that were available in the school
library. Others used the U.S. Fish and Wildlife Service
List of Endangered Species, Project WILD activities and
available references suggested in NatureScope to reinforce
the activities. According to several teachers, time did not
permit the use of references. Another frequent complaint
concerned the failure of many school libraries to have
reference materials. Reference materials and crafts were
not as effective as they might have been, due to the study
being conducted at the end of the school year when time is
limited. Had teachers been afforded more time, they may
have been more successful in locating necessary materials.
The remaining items on the participant questionnaire
were designed to collect demographic information about the
sample. Twenty-four (77.4%) of the 31 teachers responding
to the question concerning grade taught indicated that they
taught primarily fifth grade. The remaining seven teachers
(22.5%) taught primarily seventh grade. Some teachers
110
taught multiple grades in an honors-type program. The
NatureScope materials received positive and negative
comments from teachers at both grades.
The sample provided a good base for measuring the
effectiveness of NatureScope in an interdisciplinary
setting. Of the thirty-three teachers who responded to the
item concerning subjects taught, twenty-eight (84.8%) taught
more than one subject. This is not surprising, as the
majority of the teachers were at fifth grade where they were
not departmentalized. Subjects taught by participants were
social studies, mathematics, language arts, history,
reading, art, and science. Five seventh grade teachers
(15.2%) indicated that they specifically taught science.
Teachers in the sample were experienced educators. The
mean number of years teaching experience at the fifth or
seventh grade level for the 31 respondents was 8.9. The
minimum years of experience was one (n=7,22.6%) while the
maximum was 26 years (n=l,3.2%).
A serendipitous finding, discussed in Chapter 5, was
based on the number of activities used. Teachers
participating in Let's Hear It for Herps used the most
activities (77) and had the highest average number of
activities used (5.13). The Discovering Deserts group
ranked second in total activities (71) and average (3.94),
111
followed by Endangered Species: Wild & Rare (52 total, 3.71
average), and Wild About Weather (26 total, 1.53 average).
While some teachers used several activities from the weather
issue, the low average certainly makes it difficult to draw
any meaningful conclusions. However, the information may
help in explaining the results of the Wild About Weather
knowledge tests.
Chapter Summary
Management of environmental education evaluations for
nonprofit politically-active organizations requires careful
attention to the audiences involved. Hidden agendas on the
part of the client may result in a failure of the evaluation
to address the true desires of the client. Maintaining
close communication with the client can reduce delays and
frustrations in the evaluation process.
Contacts within the school administration should be
developed and maintained prior to the evaluation. This can
ease access into school systems when an evaluation need
arises.
Teachers serving as data providers should be contacted
throughout the study to answer questions which might arise.
This can help minimize teacher attrition.
112
Evaluation of educational programs is far too complex
to base decisions only on one type of information. When
properly allied, qualitative and quantitative data can be
used to influence decisions concerning the effects of a
program.
The information contained in the figures and tables of
this chapter provide measures of the effectiveness or
ineffectiveness of NatureScope based on this evaluation.
While not all research questions produced statistically
significant results they still may provide decision-makers
with information to answer design questions for future
NatureScope issues. The ramifications of
these results to evaluation of environmental education
programs, and Naturescope in particular, will be discussed
in Chapter 5.
Introduction
CHAPTER FIVE
CONCLUSIONS AND RECOMMENDATIONS
The study was designed to test an evaluation management
strategy for environmental education programs and to examine
outcomes related to environmental knowledge, attitudes and
teacher perceptions of program effectiveness. The tool for
modeling this strategy was NatureScope, an interdisciplinary
environmental education supplement developed and distributed
by the National Wildlife Federation (NWF) . Four issues of
Naturescope were selected for the study; Let's Hear It for
Herps, Discovering Deserts, Wild About Weather, and
Endangered Species: Wild & Rare.
The processes involved in managing the NatureScope
evaluation began with identification of client needs and
ended in the production of a final report to the NWF. A
three month internship with the NWF, developers and
distributors of NatureScope, helped the evaluator identify
the client's needs and expectations from the evaluation.
Information collected during this time helped focus the
evaluation proposal. Additional evaluation audiences were
school administrators and classroom teachers. School
administrators were contacted to gain legal access to a
113
114
sample of teachers. Once permission had been granted by the
school administrators to solicit volunteer teachers, the
evaluator/teacher relationship became critical to the
outcome evaluation's success. The evaluation culminated in
a final report to the NWF.
Test questions were generated from the information
contained in the activities and structured to match the
general ecological concepts stressed in each issue. A
separate knowledge test was developed and pilot tested for
each of the four issues. A 16-item Likert-type scale
(strongly agree to strongly disagree) was developed and
pilot tested for measuring environmental attitudes. The
same attitude test was taken by all students, regardless of
the Naturescope issue used.
Volunteer fifth and seventh grade teachers from
Virginia and Georgia participated. Teachers administered a
pretest, used the Naturescope materials, administered a
posttest and attitude scale, and completed a teacher
questionnaire.
Test data were analyzed using analysis of covariance
(ANCOVA) and Pearson product moment correlation coefficients
(PPM). Qualitative data from the teacher questionnaires was
used to reinforce or question quantitative findings. In
115
addition, qualit'ative findings helped to identify potential
new variables that might affect interpretation of results.
Measurement of environmental knowledge, environmental
attitudes, the correlation between knowledge and attitudes,
and teacher impressions of the NatureScope materials are
discussed as separate constructs and in conjunction with one
another. The ramifications of the findings for evaluation
of environmental education programs are discussed within the
context of an evaluation of NatureScope.
Conclusions Drawn from the Data
Conclusions drawn from the management of the evaluation
of an interdisciplinary environmental education supplement
relate to the processes involved in planning and conducting
evaluations for nonprofit politically-active groups, such as
the NWF. These processes pertain to (a) client/evaluator
~elationships, (b) evaluator/school administrator
relationships, and (c) evaluator/teacher relationships.
These conclusions can help facilitate smoother proceedings
in future evaluations of environmental education programs.
The conclusions drawn from the test results and the
questionnaire revolve around (a) NatureScope's effects on
environmental knowledge, (b) NatureScope's effects on
environmental attitudes, (c) the relationship between the
116
knowledge and attitude scores, and (d) information from
questionnaires completed by participating teachers. High
attrition resulted in a loss of statistical power;
consequently, few statistically significant differences were
attained. Moreover, the lack of random assignment hampers
attributing posttest differences, where they exist, to
NatureScope.
Conclusions Concerning Evaluation Management Processes
The political associations of nonprofit special
interest groups, such as the NWF, make them very sensitive
to the possibilities of negative evaluation results. This
was manifested in difficulties in selecting the NatureScope
issues for the evaluation and problems associated with
instrument selection and development.
The NatureScope staff at NWF has had little previous
experience working with evaluators and evaluation
techniques. This led to extreme caution in approving
evaluation activities which had the potential of producing
negative findings about NatureScope. This caution
highlighted the need to maintain close communication with
the NatureScope staff; keeping them informed of the purposes
and consequences of evaluation activities.
117
Due to the difficulties experienced in getting the
Naturescope staff to approve test instruments and acquiring
an adequate sample, the evaluation proposal and timeline
needed to be flexible. These delays coupled with a late
start in the evaluation process, resulted in problems
associated with school district cooperation and teacher
attrition.
There is a political aspect involved in producing
evaluations of this type which involve some form of
classroom intervention. The evaluator found many school
-administrators very protective of teachers and classroom
instructional time. Despite the fact that the NatureScope
activities are supplementary, administrators expressed the
view that teachers were too busy to use the materials. By
fitting the evaluation materials to match the planned
curriculum of the school system, greater cooperation may be
obtained. School administrators who were already contacts
of the evaluator were more willing to accept and support the
NatureScope evaluation. If evaluations of supplementary
environmental education programs are to be made easier,
evaluators and their clients should cultivate a network of
contacts within the school administration.
118
The evaluator should provide support in the form of
written and oral communication to teachers participating in
studies of this nature. Teachers expressed two major
concerns at the outset of the evaluation. First, there was
concern over a lack of experience and knowledge about the
topics involved. Second, there was a concern over a lack of
understanding and experience about the evaluation procedure.
Teachers seemed sincere in their desire to do an effective
job on the evaluation, but were uncertain about teaching and
testing procedures.
Throughout the evaluation the evaluator should maintain
close contact with teachers, either directly or through an
appointed liaison. Those teachers in the Naturescope
evaluation who took the initiative to call the evaluator
with questions had fewer procedural mistakes relative to
opscans and questionnaires. Some teachers may not feel
comfortable in contacting the evaluator with questions, thus
the evaluator should initiate contact.
The New Evaluation Management Model
Figure 5 shows a revised NWF evaluation management
model. Whereas the original model was linear, proceeding
from one task to the next; this model depicts the loops
required to complete the evaluation. These loops were
119
necessary when the original attempt at an evaluation process
had to be modified for some reason. Evaluators should be
prepared to duplicate steps in the evaluation process when
working with groups that are not familiar with evaluation
processes. The first time through a step may be necessary
to educate the client.
Conclusions Concerning Effects on Environmental Knowledge
Only one of the four Naturescope issues, Let's Hear It
for Herps, showed a significant difference between treatment
and control group means on the measure of environmental
knowledge.
Even though NatureScope is marketed for kindergarten
through seventh grade, the evaluation addressed only fifth
and seventh grades. Statistically, none of the four issues
performed better at one grade than the other. However,
fifth grade means were systematically higher than seventh
grade on the knowledge measure for all issues.
One of the serendipitous findings of the study was that
classes exposed to a higher number of activities had
consistently higher means on the knowledge tests.
While no significant differences were -found between the
treatment and control groups for Wild About Weather, the
120
Identify Client Needs and Expectations<---/I\ \ I Select NatureScopes
\11 / I\ \ I
to be evaluated
Develop Proposal<----------------
\ I/ Develop Evaluation Contract
I \I/ \I/
Work with School Develop Evaluation Administrators /I\ \ I
Instruments /I\ \ I Solicit Volunteer Teachers Conduct Pilot Tests and
I \ Revise
\ I 11\ \ I
Develop Treatment/Control Groups Print Final Instruments I \ I I
\II ->-Hold Meetings with teachers to distribute materials
\11 Maintain Communication with Teachers<--
\ I/ Collect Data from Pretest
\11 \ I Collect data from Posttest, attitude scale,
and Teacher Questionnaire
\11 Analyze Data
\11 Draft Report
\11 Submit Final Report to Client
Figure 5. Revised Evaluation Management Plan for the NWF
121
qualitative data revealed that students were already
operating at a level of knowledge where exposure to
NatureScope activities was not sufficient to significantly
increase knowledge scores.
Conclusions Concerning Effects on Environmental Attitudes
None of the four NatureScope activity guides differed
in attitude scores. While all groups displayed positive
attitudes, none differed significantly from the pure control
group.
The more activities a student was exposed to, the
higher their environmental attitude score. Groups exposed
to seven or more activities had 2.9% higher scores on the
attitude scale than groups exposed to six or less
activities. Two previous studies (Block, 1970; Ramsey and
Rickson, 1976) found that students who received intensive
exposure to a topic actually had depressed attitudes about
that topic. Judging from the number of activities performed
by each teacher in the Naturescope study, intensity of
exposure was not a threat.
The groups grade in school was not a significant factor
in affecting environmental attitude scores. Seventh grade
means were slightly higher (less than 1%) and may have been
higher prior to the study. Horvat and Voelker (1976) found
122
eighth grade students to be more environmentally responsible
than fifth grade students. While environmental
responsibility may be linked to environmental attitudes, it
suggests an active stance; whereas, attitudes may be
passive.
Attitude scales have been used in the evaluation of
environmental education programs (Ramsey & Rickson, 1976;
Fleming, 1983). The lack of consistent findings between
this study and others, coupled with research on the long-
term effects bf environmental education would seem to raise
a caution of the efficacy of objectives related to attitude
change. If changes are observed, they cannot be attributed
solely to the effects of an environmental education program.
conclusions Concerning the Correlation of Knowledge and
Attitudes
Students exposed to NatureScope had an average
correlation of .46 between attitude scores and posttest
knowledge scores. Let's Hear It for Herps had the highest
PPM (.51), Endangered Species: Wild & Rare and Discovering
Deserts had the lowest (.41). This moderate correlation
provides little evidence of a relationship between knowledge
and attitudes.
123
Conclusions from Teacher Questionnaires
Most teachers in the study had a positive feeling about
their level of environmental awareness. This opinion is
supported by reports of use of the additional NWF materials
in the past by most teachers in the study.
Based on the information provided by teachers who used
the materials, NatureScope is designed in a way that is
compatible with classroom use. Teachers in the treatment
groups were asked to rate the NatureScope materials on
criteria ranging from background information to utility of
the crafts. On all criteria, teachers rated NatureScope
favorably. Qualitative comments supported these ratings.
The evaluation should have been conducted earlier in
the school year. The study was conducted near the end of
the school year, limiting the amount of time many teachers
could devote to the activities. In addition, the time
factor also kept many teachers from using the supplemental
references suggested in the activity guides.
Quantitative and qualitative findings indicate that the
materials are most suited for students in the mid-range of
achievement. The activities were seen as too difficult for
low achievers and too easy for high achievers.
Based on the evidence, Naturescope is better suited for
teachers in self-contained classrooms who teach multiple
124
subjects. This allows for the use of a maximum number of
activities.
Serendipitous Findings
An additional benefit of research is that it often
raises or answers more questions than were originally asked.
In the course of research, the Naturescope study addressed
three such additional questions.
The Effect of Order of Administration on Attitude Scores
During pilot.testing, two fifth and two sixth grade
classes (N=lOl) were given the NatureScope knowledge tests.
Each student was also administered the Environmental
Awareness Inventory (EAI) (Passineau,1976). The EAI was
developed to assess the "affective environmental awareness"
of upper elementary and junior high.students. While
independent scales were constructed for a variety of
environmental topics, the EAI allows evaluators to use only
those scales deemed relevant.
The EAI scale selected for this study is designed to
assess an individual's general concern for environmental
125
issues in contrast to a concern for other issues of societal
importance such as war, drugs, crime, and poverty. In this
scale, general environmental concern is viewed as a
composite of various attributes collectively known as
environmentalism. As such, the scale measures interests in,
attitudes towards, preferences for, and motivations towards
a variety of environmental topics, problems and solutions.
The scale does not differentiate between these attributes or
topics.
One class at each grade level was given the knowledge
test followed by the attitude test. The other class in each
grade level was given the attitude test followed by the
knowledge test. The four knowledge tests were randomly
distributed within the classes to reduce the chances of a
particular knowledge test influencing attitude scores.
A 2 x 4 Analysis of Variance was computed where score
on the attitude test was the dependent variable, order of
testing (cognitive measure first or second), and knowledge
test (herps, deserts, weather, endangered species) served as
independent variables. Cell sizes were unequal, thus Type
III Sums of Squares were calculated and used in computing
the F statistics.
EAI scores differed significantly depending on the
order of administration (F=7.61,p=.0070). Those students
126
who took an environmental knowledge test before taking the
EAI scored significantly lower on the EAI than did those
students who took the tests in reverse order. The results
of this study suggest that students who have not been
exposed to the content of a knowledge test, thus are unable
to answer many of the questions correctly, have lower
attitudes scores on that topic.
Interactions between these variables were not
significant (F=.60,p=.6135). This serves to strengthen the
argument for order of testing being the primary source of
variance in affective scores by eliminating the possibility
of a confounding effect between cognitive and affective
measures.
Evaluation Implications
When considering the effectiveness of educational
programs, student attitudes are often measured in
conjunction with student knowledge on that subject. A
student who knows little about a topic may become frustrated
by taking a cognitive test on that topic, thus depressing
their score on the affective measure. After exposure to the
topic the student should be more knowledgeable and less
frustrated following testing, thereby resulting in an
increase in affective scores over the first testing. This
127
could result in attributing affective gains to a program
when, in actuality, the students scores had only returned to
their normal range prior to pretesting. Further research is
planned to determine the impact of order of testing in the
affective domain as it relates to subject knowledge.
The Effect of Teacher Interest On Test Results
Using information from the teacher handbooks, group
scores on the knowledge and attitude tests were analyzed to
determine if the number of activities used by the teacher
influenced group scores. Results of this finding have been
incorporated in the discussions of environmental knowledge
and attitudes. This section provides an overview of the
analysis related to this finding.
The number of NatureScope activities used by
participant ranged from zero to nineteen. The mean number
of activities used was 6.73 with a median of six. Teachers
who used zero to six activities were classified as low users
and teachers who used seven to nineteen activities were
categorized as high users. These two categories were then
paired with class means on the knowledge and attitude tests
to determine if the number of activities used were related
128
to scores on these two variables. Table 16 provides a
summary of these results for the knowledge scores.
Table 16. ANO VA Summary for knowledge scores by activities
source DF MS F F prob.
Pretest (covariate) 1 186.867 15.971 .001
Activity Use 1 8.922 .763 .392
Residual 23 11. 701
Total 25 18.596
Using teachers as the unit of analysis, knowledge
scores were not found to differ significantly between high
and low activity groups (F=.763, p=.392). Groups who used
six or less activities had a mean on the knowledge test of
16.45, while groups who used seven or more activities had a
mean of 17.69. Regression analysis of this data resulted in
an R2 of only .01.
Consistent with findings by Fleming (1983), attitude
scores were found to be different for high and low activity
groups (F=4.622, p=.039). Table 17 provides a summary of
the ANOVA. Groups exposed to seven or more activities had a
mean of 51.79, while low exposure groups had a mean of
49.64. Regression analysis for attitude score and the
number of activities used resulted in a R2 of only .02.
129
Table 17. ANO VA Summary for Attitude scores by activity use
Source DF MS F F prob.
Activity Use 1 36.595 4.662 .039
Residual 30 7.850
Total 31 8.77
Recommendations
Evaluation Management
Clients who develop and distribute interdisciplinary
environmental education supplements may be (a) nonprofit,
(b) politically-active and sensitive, and (c) inexperienced
in the evaluation process. These factors necessitate
certain actions by evaluators to bring about a successful
evaluation which meets the clients needs. Relative to the
first 2 factors, groups that operate primarily from
donations may be particularly sensitive to negative
evaluation feedback which they perceive may result in
possible reductions in donations because of negative
publicity. This could weaken the political clout of the
organization. Such organizations will monitor evaluation
activities very closely in an effort to reduce the
likelihood of a negative evaluation. To reduce the
likelihood of negative publicity from negative evaluations,
130
formative evaluation should be used. This would help the
organization in marketing a high-quality product.
The third factor, lack of evaluation experience, makes
it necessary that the evaluator spend time with the client
explaining the details of the evaluation. If the evaluator
works closely with the client in the planning phases, during
instrument development, and throughout other aspects of the
evaluation, many potential problems may be avoided.
Since many evaluations of interdisciplinary
environmental education supplements take place in schools
where the programs are not already in use, evaluators and
their prospective clients must establish a rapport with
school administrators to gain access to school systems.
Evaluators should work on developing contacts within the
school administration prior to evaluation; a political
activity that can facilitate access when actual evaluation
needs arise. A second recommendation to make
evaluator/school administrator interactions more successful
is to focus the evaluation so that it meshes within the
existing curriculum. This involves timing contacts with
administrators to occur when they are establishing the
agenda for the year and timing the, evaluation to coincide
with convenient portions of the school year.
131
Evaluations which involve some form of treatment or
exposure to materials should be conducted earlier in the
school year when teachers have sufficient time to plan,
organize, and conduct the treatment. If this is not done,
study results will be confounded by the teachers inability
to locate references or use the materials. Earlier
implementation may also result in greater cooperation from
administrators.
Evaluators should maintain frequent contact with
teachers who serve as data providers. The initiative must
be taken by the evaluator for three reasons. First, the
teachers are providing a service to the evaluator and,
therefore any inconvenience to the teachers should be
minimized. Second, teachers may not feel comfortable asking
questions about evaluation procedures, therefore providing
less than optimum data. This problem may be eliminated or
reduced by the evaluator periodically checking with the
teachers. Third, frequent contact with teachers should
improve the reliability of the treatment.
The use of volunteer participants in this study and the
study by Stout and Peyton (1988) is a suitable approach to
evaluating environmental education programs. Supplemental
environmental education materials are usually not mandated
by the school board and are used at the discretion of the
132
teacher. Teachers who volunteered for this study expressed
an interest in environmental education; a trait that would
be expected from teachers who might routinely use the
materials with their classes.
NatureScope Evaluation
Qualitative and quantitative information supports
NatureScope being more suited for fifth grade classes than
seventh grade. The materials consistently performed better
with fifth grade groups than seventh grade. If NWF wishes
to improve the usefulness of NatureScope at the seventh
grade level, qualitative findings would indicate that more
advanced activities should be developed. This is made
particularly clear by the seventh grade honors teacher who
felt that her students knew the answers before doing the
activities. Barrington and Hendricks (1988) suggested
differentiated curriculum for gifted and average students.
Teacher attrition during the NatureScope study resulted
in a loss of statistical power and introduction of possible
non-response bias. When evaluating non-mandatory classroom
supplements, such as NatureScope, evaluators should
oversample to minimize some of the deleterious effects of
high mortality. Another possibility for reducing teacher
attrition is to monitor the programs activity during the
133
treatment. However, this decreases the chance of a
naturalistic study, since teachers may attempt to do what
they perceive is expected.
Since weather was a part of the fifth grade curriculum,
NWF might wish to actively market this issue to the fifth
grade and emphasize the supplemental nature of the
materials. Qualitative comments indicate that the materials
would mesh with the objectives for studying weather at the
fifth grade level. However, any actions concerning the Wild
About Weather issue should not be on the basis of knowledge
and attitude scores, since the covariate analysis was not
significant and the homogeneity of regression assumption was
not retained.
When evaluating the effect of an environmental topic on
environmental attitudes, the theme of that topic must be
considered. Some topics are value-sensitive, and can be
expected to affect attitudes more than a topic that is not
value-sensitive. To draw an illustration from this study, a
topic such as endangered species has received extensive
coverage in the media concerning the plight of these species
and what can be done to save them; at the other extreme,
while the media discusses weather on a regular basis, with
few exceptions, it is not from an attitude-altering
perspective.
134
The extent to which environmental attitudes are
influenced seems to be specific to the theme of a particular
NatureScope issue. If NWF wants to increase the attitudinal
effect for less value-sensitive issues, more activities with
a focus on environmental quality should be included.
The positive correlation between knowledge and
attitudes provides some indication that the two variables
work together, i.e. high knowledge scores tend to be
associated with high attitude scores.
Supplementai references listed in each activity guide
should be readily available in most school libraries. The
difficulty in obtaining reference material was probably more
a function of time than scarcity of materials. However, if
teachers are to use the activities as supplements to their
lesson plans, necessary materials must be available quickly.
This could be accomplished by NWF making the materials
available.
Recommendations for Further Research
When conducting research, many additional questions are
generated that may not be within the scope of the original
study. The topics below provide suggestions for additional
research.
135
1. The effects of testing order on environmental
attitude scores certainly needs further research. The
evaluation methodology implications of this conundrum entail
the attributing of gains in the affective domain to a
program when it may actually be an effect of testing.
2. The impact of dissemination techniques on the
successful management of an evaluation. For example,
teachers who receive the materials during a workshop may
differ in from teachers who receive the materials through
the mail.
Summary of the Evaluation Management Study
Conducting evaluations of environmental education
supplements produced by nonprofit special interest
organizations raises special problems for the evaluator.
Since many of these groups lack experience in the evaluation
process, the evaluator must maintain close contact with the
client prior to, and during, the evaluation to clarify
activities.
School systems that do not have active environmental
education programs may be reluctant to participate in these
evaluations. Maintaining a network of contacts and focusing
136
the evaluation within existing curriculum and time
constraints may ease access.
Teachers who use the materials and serve as data
providers during the evaluation must be contacted frequently
to answer questions that may arise. The evaluator must take
this initiative.
Some significant differences were found for some
NatureScope issues whereas, no such differences were found
for other issues. Qualitative and quantitative measures
were useful in the evaluation of environmental education
supplements. Research should be designed so that the
information received from these two techniques can be used
to support each other.
Interpretation of results from knowledge and attitude
measures must be taken in context. Evaluators should look
at the effect size, as well as statistical significance,
when determining the impact of a treatment. If this total
picture is taken into consideration, results with practical
significance can be reported.
Questionnaires completed by teachers participating in
the study can provide insight into program strengths and
weaknesses. In addition, their statements may identify new
variables which may be influencing the results of the
evaluation.
137
The serendipitous finding concerning testing orders
effects on environmental attitudes deserves further
research. Evaluators should consider this phenomenon when
using pretest/posttest designs to measure environmental
knowledge and attitudes.
REFFERENCES
Aird, A. & Tomera. A. (1977). The effects of a water conservation unit on the values held by sixth grade students. Journal of Environmental Education 9:1 pp. 31-43.
Ajzen, I. & Fishbein, M. (1977). Attitude-behavior relations: a theoretical analysis and review of empirical research. Psychological Bulletin 84 pp.888-918.
Alaimo, S.J. & Doran, R.L. (1980). A longitudinal study of factors influencing value oreference in environmental problems. in Sacks, A.B. et al. (eds.) Current Issues IV: The Yearbook of Environmental Education and Environmental Studies. Columbus, OH: ERIC Clearinghouse for Science, Mathematics and Environmental Education. pp. 311-326.
Arcury, T.A. & Johnson; T.P. (1987). Public environmental knowledge: a statewide survey. Journal of Environmental Education 18:4 pp. 31-38.
Armstrong, J.B. (1984). [Wildlife knowledge of elementary and secondary students in Georgia]. Unpublished raw data.
& Impara, J.C. (1989). The Effects of Testing Order On Environmental Attitudes. Unpublished manuscript.
Baker, D.R. (1985). Predictive value of attitude, cognitive ability, and personality to science acheivement in the middle school. Journal of Research in Science Teaching 22:2 pp.103-113.
Barrington, B.L. & Hendricks, B. (1988). Attitudes toward science and science knowledge of intellectually gifted and average students in third, seventh, and eleventh grades. Journal of Research in Science Teaching 25:8 pp.679-687.
Barufaldi,J.P., Ladd, G.T., & Moses,A.J. (1984). Heath Science. D.C. Heath & Co., Lexington, Mass.
Bendick, J. & Gallant, R.A. (1980). Elementary Science 5. Ginn and Co., Lexington,Mass.
138
139
Bennett, D.B. (1982), Evaluating environmental education in the context of a junior high school state studies program. Journal of Environmental Education 13:4 pp. 13-18.
(1989). Four steps to evaluating environmental education learning experiences. Journal of Environmental Education 20:2 pp. 14-21.
Berger, C.F., Berkheimer, G.D., Lewis, L.E., & Neuberger, H.T. (1979). Houghton-Mifflin Science. Moughton-Mifflin Co., Boston,Mass.
Birch, S.K. & Schwaab, K.E. (1983). The effects of water conservation instruction on seventh grade students. Journal of Environmental Education 14:4 pp. 26-31.
Block, J.H. (1970). The effects of various levels of performance on selected cognitive, affective, and time variables. In Torshen, K.P. (1977). Mastery Approach to Competency-Based Education. New York: Academic Press.
Blosser, P.E. (1984). Attitude Research in science education. ERIC Clearinghouse of Science, Mathematics, and Environmental Education Information Bulletin, No.l, Ohio State University, Columbus, Ohio.
Brown, J.D. (1979). Parent perceptions of their oroqenv's observable environmental/energy behavior resulting from increased cognitive knowledge of selected environmental/energy terms. Unpublished Ph.D. Dissertation, Indiana State University. Dissertation Abstracts International 41:7:2538-B, UMI Order No. 8029728.
Burrus-Bammel, L.L. (1978). Information's effect on attitude: a longitudinal study. Journal of Environmental Education 9:4 pp. 41-50.
Byford, J.L. & Munsey, s. (1984). Attitudes on wildlife and knowledge retained by 4-H alumni. Trans. 49th North American Wildlife and Natural Resources Conf. pp. 410-420.
Charles, c., Meyers, J., & Rensel, J. (1984). Project WILD Workshop Handbook. Boulder: Western Regional Environmental Education Council.
140
~~~~~· (1988). Summary of research findings on Project WILD. Trans. 53rd N.A. Wildl. & Nat. Res. Conf. pp. 531-541.
Chase, C.I. (1978). Measurement for Educational Evaluation. (2nd ed.) Reading, Massachusetts: Addison-Wesley.
Chenery, M.F. & Hammerman, W. (1985). Current practice in the evaluation of resident outdoor education programs: report of a national survey. Journal of Environmental Education 16:2 pp. 35-42.
Childress, R.B. (1977). Evaluation strategies and methodologies utilized in public school environmental education programs and projects - a report from a national survey. School Science and Mathematics 77:2 pp.105-116.
CLASS Project (1982). Washington, D.C.: National Wildlife Federation.
Cohen,M.R. (1977). Using student-determined needs to evaluate environmental education programs. Journal of Environmental Education 8:4 pp.27-30.
Converse, J.M. & Presser, s. (1986). Survey Questions: Handcrafting the standardized questionnaire. Beverly Hills: Sage Publishing.
Cook, T.D. & Campbell, D.T. (1979). Quasi-experimentation: Design and Analysis Issues for Field Settings. Houghton-Mifflin Co.: Boston.
Crocker, L. & Algina, J. (1986). Introduction to Classical and Modern Test Theory. New York: Holt, Rinehart & Winston.
Disinger, J.F. (1981). Environmental education in the K12 Schools:a national survey. in Sacks, A.B. et al. (eds.) Current Issues VII: The Yearbook of Environmental Education and Environmental Studies. Columbus, OH: ERIC Clearinghouse for science, Mathematics and Environmental Education. pp. 141-156.
(1986). What research says: Instruction in awareness of environmental issues. Science and Mathematics 86:3 pp. 251-256.
141
. (1989) .Environmental education: a movement in transition. Journal of Soil and Water Conservation May-June pp. 188-191.
Doran, R.L. {1977). "State of the art" for measurement and evaluation of environmental objectives. Journal of Environmental Education 9:1 pp. 50-63.
' Fishbein, M. & Ajzen, I. (1985). Belief, Attitude, Intention
and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley.
Fleming, M.L. {1985). Project WILD evaluation, final results of the field test. Arlington, VA:ERIC, ED 245-890. 43 pages.
~~~-· {1985) Evaluation report of the 1985 implementation of Project WILD in the elementary schools of Lee County, Florida. unpublished report to Western Regional Environmental Education Council, Boulder, co. 12 pages.
Fortner, R.W. & Lyon, A.E. {1985). Effects of a Cousteau television program on viewer knowledge and attitudes. Journal of Environmental Education 16:3: pp.12-20.
Gradwohl, J.M. & Shumacher, G.M. (1989). The relationship between c9ntent knowledge and topic choice in writing. Written Communication 6:2 pp.181-195.
Germann, P.J. {1988). Development of the Attitude Toward Science In School Assessment and its use to investigate the relationship between science acheivement and attitude toward science in school. Journal of Research in Science Education 25:8 pp.689-703.
Gronlund, N.E. {1981) Measurement and Evaluation in Teaching. MacMillan Publishing Co., Inc.:New York
Hawkins, D.E. & Vinton, D.A. (1973). The Environmental Classroom Englewood Cliffs, NJ: Prentice-Hall, Inc.
Horvat, R.E. & Voelker, A.M. (1976). Using a likert scale to measure "environmental responsibility". Journal of Environmental Education 8:1 pp. 36-47.
House, E.R. (1980). Evaluating with Validity. Beverly Hills: Sage.
142
Hug, J. (1977). Interdisciplinary environmental education course: a dream or a reality?. School Science and Mathematics 77:5 pp.397-401.
Hungerford, H.; Peyton, R.B. & Wilke, R.J. (1980). Goals for curriculum development in environmental education. Journal of Environmental Education 11:3 pp.42-47.
Isaac, s. & Michael, W.B. Handbook in Evaluation and Research for Education and the Behavioral Sciences. (2nd ed.) San Diego: EdITS.
Iozzi, L.A. (1978). The Environmental Issues Test (EIT): A new assessment instrument for environmental education. Current Issues IV: The Yearbook of Environmental Education and Environmental Studies. Columbus, OH: ERIC Clearinghouse for Science, Mathematics and Environmental Education., pp. 200-206.
(1989). What research says to the educator - part one: environmental education and the affective domain. Journal of Environmental Education 20:3 pp.3-9.
James, R.K. & Potts, G. (1981). Assessing environmental education in the public schools. School Science and Mathematics 81:2 pp. 103-114.
Jaus, H.H. (1984). The development and retention of environmental attitudes in school children. Journal of Environmental Education 15:3 pp.33-36.
Joint Committee on Standards for Educational Evaluation (1981). Standards for Evaluation of Educational Programs, Projects, and Materials. New York: McGraw-Hill.
Kellert, S.R. (1986). Attitudes toward animals: age-related development among children. Journal of Environmental Education 16:3 pp.29-39.
Kupchella, C.E. & Hyland, M.C. (1976). Essential curriculum components in environmental education. Journal of Environmental Education 8:2 pp. 11-16.
143
Larson, M.A. (1989). Today's college youth - a generation at rest? in Grunig, L.A., editor, Environmental Activism Revisited: The Changing Nature of Communication Through Organizational Public Relations. Special Interest Groups and the Mass Media. NAEE Monographs 5 pp. 40-49.
Lawrenz, F. (1983). student knowledge of energy issues. School Science and Mathematics 83:7 pp. 587-595.
Lincoln, Y.S. and Guba, E.G. (1985). Naturalistic Inquiry. Beverly Hills: Sage
Lucko, B.J., Disinger, J.F., & Roth, R.E. (1982). Evaluation of environmental education programs at the elementary and secondary school levels. Journal of Environmental Education 13:4 pp. 7-12.
Mason, C.L. & Kahle, J .. B. (1988). Student's attitudes toward science and science-related careers: a program designed to promaote a stimulating gender-free learning environment. Journal of Research in Science Teaching 26:1 pp.25-39.
Mueller, D.J. (1986). Measuring Social Attitudes: A Handbook for Researchers and Practitioners. New York: Teachers College Press.
Munby, H. (1983a). Thirty studies involving the "Scientific Attitude Inventory": what confidence can we have in this instrument? Journal of Research in Science Teaching 20:2 pp.141-162.
~~~~-· (1983b). An Investigation into the measurement of attitudes in science education. Columbus, OH: SMEAC Information Reference Center, Ohio State University. (ERIC Document Reproduction Service No. ED 237-437).
National Wildlife Federation (1988). NatureScope Report: A national survey of the publication and its use. (unpublished report). Washington, D.C.
NatureScope (1984). Washington, D.C. National Wildlife Federation.
Nowak, P.F. (1984). Direct evaluation: a management tool for program justification, evolution, and modification. Journal of Environmental Education 15:4 pp. 27-31.
144
O'Hearn G.T. (1982). What is the purpose of evaluation?. Journal of Environmental Education 13:4 pp. 1-3.
Passineau, J.F. (1976). The development of the Environmental Awareness Inventory (EAI)--an assessment instrument for the evaluation of environmental education. Dissertation Abstracts 36(9) :5654-A. (UMI 76-6241; 387pp.)
Patton, M.Q. (1980). Qualitative Evaluation Methods. Newbury Park,California: Sage.
Pomerantz, G.A. (1985). The Influence of "Ranger Rick" Magazine on Children's Perceptions of Natural Resource Issues. Unpublished doctoral dissertation, North Carolina State University, Raleigh .
. (1986). Environmental education tools for elementary schoolchildren: the use of a popular children's magazine. Journal of Environmental Education 17:4 pp. 17-22.
Provus, M. (1971). Discrepancy Evaluation for Educational Program Improvement and Assessment. Berkely, CA: Mccutchan.
Quinn, B. & Jadav, A.O. (1987). Causal relationship between attitude and achievement for elementary grade · mathematics and reading. Journal of Educational Research 80:6. pp. 366-372.
Ramsey, C.E. & Rickson, R.E. (1976). Environmental knowledge and attitudes. Journal of Environmental Education 8:1 pp. 10-18.
Robottom, I. (1983). Evaluation in environmental education: Time for a change in perspective?. Journal of Environmental Education 17:1 pp. 31-36.
Roth, R.E. & Perez, J. (1989). Twelfth grade student knowledge and attitudes toward the environment in the Dominican Republic: an assessment. Journal of Environmental Education 20:3 pp.10-14.
Showers, D.E. (1987). The effect of persuasion on nuclear attitudes of secondary school students. paper presented at the Annual Meeting of tne National Association for Research in Science Teaching.
145
Smith, M.L. & Glass, G.V. (1987). Research and Evaluation in Education and the Social Sciences. Englewood Cliffs, NJ:Prentice-Hall, Inc.
Stapp, W.B. (1965). Integrating Conservation and outdoor Education Into the Curriculum (K-12). Minneapolis,MN: Burgess Publishing co.
~~~~· (1978). An instructional model for environmental education. Prospect 8:4 pp.495-507.
Stout, R.J. and Peyton, B. (1988). The need for wildlife education program evaluation. Trans. 53rd Annu. Proc. N. Am. Conf. Fish and Wildl. Agencies.
Stufflebeam, D.L. (1973). Evaluation as enlightenment for decision making. in B.R. Worthen & J.R. Sanders, Educational Evaluation: Theory and Practice. Belmont, CA: Wadsworth.
Tanner, T. (1982). Huck Finn Elementary School: the making of environmentalists. Phi Delta Kappan 63:10 p. 706.
Trama, J.A. and Newman, B.M. (1988). A comparison of the impact of an alcohol education program with Al-anon on knowledge and attitudes about alcoholism. Journal of Alcohol and Drug Education 34:1 pp. 1-16.
Troy, T.D. & Schwaab, K.E. (1982). A decade of environmental education. School Science and Mathematics 81:3 pp.209-216.
UNESCO. (1976). The Belgrade Charter: A Global Framework for Environmental Education. Paris, France: Unesco.
Worthen, B.R. & sanders, J.R. (1987). Educational Evaluation: Alternative Approaches and Practical Guidelines. New York: Longman.
~~~~r & White, K.R. (1987). Evaluating Educational and Social Programs: Guidelines for Proposal Review, Onsite Evaluation, Evaluation Contracts, and Technical Assistance. Boston: Kluwer-Nijhoff.
Zosel, D. (1987). Factors Affectina Teacher Use of Project WILD. Research Management Findings 19:February 1989. Wisconsin Department of Natural Resources, Madison, Wisc.
Appendix A
Naturescope Groupings
Animals * Let's Hear It for Herps
Birds, Birds, Birds Amazing Mammals I Amazing Mammals II Incredible Insects
Habitats * Discovering Deserts
Wading into Wetlands Diving into Oceans Rain Forests
Earth & Space Astronomy Adventures Geology: The Active Earth
* Wild About Weather
Miscellaneous Digging Into Dinosaurs
* Endangered Species: Wild & Rare Trees Are Terrific
* - issue selected for study
146
Appendix B
Let's Hear It for Herps
General Instructions for completing Answer Sheet:
You are· to enter all answers on the on the Answer Sheet according to the Directions given for each set of questions. Circles must be completely filled in using a number 2 pencil. Make your marks dark enough so you cannot read the number or letter inside the circle. Stray pencil marks may be interpreted as answers so mark only the circle containing the desired answer and erase mistakes carefully.
The top portion of the Answer Sheet should be filled in as follows:
NAME - print your name. COURSE - leave blank. DATE - print the date. ID NUMBER - print the number given by your
teacher. Place only 1 digit in each box and blacken the corresponding circle under that box.
FORM - print 11 D11 in the box and blacken the circle containing the letter 11 D11 •
SEAT NO. - print the number given by your teacher and blacken the corresponding circle under that box.
GROUP - print the number 11 5 11 if you are in the fifth grade and the number 11 7 11 if you are in the seventh grade. Blacken .the corresponding circle under the box.
The following section will provide examples of how to complete each type of question on the test.
(go to next page)
147
148
Multiple Choice Please select the one answer that best answers the question or completes the statement. On the line of the answer sheet corresponding to the question number, please color in the circle containing the number of the correct answer.
Example 1. The earth's only natural satelite is
1. the moon. 2. the sun. 3. Jupiter. 4. Mars.
(Since the correct answer is moon, color in the circle containing the number 11 1 11 on line 1 of the coding sheet.)
True or False On the answer sheet, color in the circle containing the number "1" for each correct or true statement and color in the circle containing the number 11 2 11 for each incorrect or false statement.
Example 2. The earth's only natural satelite is the sun.
(Because this is false, color in the circle containing the number 11 2 11 on line 2 of the coding sheet.)
3.
4.
Together, 1. 2. 3. 4.
By moving 1. 2. 3. 4.
(Begin test)
reptiles and amphibians are known as he rps lizards herbs frogs
to the shade herps shed their skin. warm their body temperatures. cool their body temperatures. see more clearly
5. Endothermic means: 1. warm-blooded. 2. predatory. 3. light-sensitive. 4. aquatic
?
149
6. If you found soft flexible-shelled about one inch long buried in the soft dirt underneath a fallen log, the eggs most likely were laid by a
1. pine barrens tree frog. 2. black rat snake. 3. dusky salamander. 4. garden spider.
7. Which of the following is true for amphibians but not reptiles?
1. They have slimy skin. 2. The shed their skin. 3. They are cold-blooded. 4. They have scaly skin.
8. Ectothermic means: 1. aquatic. 2. warm-blooded. 3. light-sensitive. 4. cold-blooded.
9. Frog calls serve several functions. Which of the following is not a reason frogs call?
1. distress. 2. attract a mate. 3. announce territories. 4. announce the capture of prey.
10. Why do frogs lay lots of eggs?
11. What
1. Inflate the population. 2. Increase the chance of species survival. 3. Provide companionship for each other when
they hatch. 4. Allow for cross-breeding.
kind 1. 2. 3. 4.
of protective covering do A layer of dried blood. A leathery shell. A layer of skin. A jelly coating.
frog eggs have?
12. Which best describes the life cycle of a frog? 1. larvae to egg to adult 2. egg to larvae to adult 3. egg to adult 4. liveborn to adult
(Go to next page)
13. A snake's 1. 2. 3. 4.
150
tongue helps it sense odors. feel vibrations. catch insects. sense heat.
14. Which herp has all of the following:
A. prehensile tail B. skin that changes colors c. independently moveable eyes D. sticky tongue.
1. poison arrow frog 2. python 3. komodo dragon 4. chameleon
15. Where do sea turtles lay their eggs? 1. in the ocean 2. on beaches 3. in saltwater rivers 4. they do not lay eggs
16. What is the most serious threat to sea turtle populations today.
1. territoriality 2. predation of adults 3. overharvesting 4. the blue-banded sea snake
True or False On the answer sheet, color in the circle containing the number 11 1 11 for each correct or true statement and color in the circle containing the number 11 2 11 for each incorrect or false statement.
17. The bodies of reptiles are covered with scales.
18. All snakes lay eggs.
19. Reptiles and amphibians are invertebrates.
20. Snakes and salamanders rely on vibrations to hear. (Go to next page)
151
21. Snakes have moveable eyelids.
22. Snakes hear vibrations through tiny bones at the back of their jaw.
23. The vertebrae and ribs of a snake are used for support and protection.
24. The skin of lizards and snakes is covered with slime for protection.
25. Picking up a toad will give you warts.
26. Snakes help control rodent populations.
27. All snakes are meat eaters.
28. Most snakes are poisonous.
29. It is against the law to take animals out of National Parks and refuges.
30. Salamanders are a type of lizard.
31. Most herps shed their skin in tiny pieces, a little bit at a time.
32. Alligators are amphibians.
152
Discovering Deserts
General Instructions for completing Answer Sheet:
You are to enter all answers on the on the Answer Sheet according to the Directions given for each set of questions. Circles must be completely filled in using a number 2 pencil. Make your marks dark enough so you cannot read the number or letter inside the circle. Stray pencil marks may be interpreted as answers so mark only the circle containing the desired answer and erase mistakes carefully.
The top portion of the Answer Sheet should be filled in as follows:
NAME - print your name. COURSE - leave blank. DATE - print the date. ID NUMBER - print the number given by your
teacher. Place only 1 digit in each box and blacken the corresponding circle under that box.
FORM - print 11 D11 in the box and blacken the circle containing the letter 11 D11 •
SEAT NO. - print the number given by your teacher and blacken the corresponding circle under that box.
GROUP - print the number 11 5 11 if you are in the fifth grade and the number 11 7 11 if you are in the seventh grade. Blacken the corresponding circle under the box.
The follwing section will provide examples of how to complete each type of question on the test.
Multiple Choice Please select the one answer that best answers the question or completes the statement. On the line of the answer sheet corresponding to the question number, please color in the circle containing the number of the correct answer.
(Go to next page)
153
Example 1. The earth's only natural satelite is
1. the moon. 2. the sun. 3. Jupiter. 4. Mars.
(Since the correct answer is moon, color in the circle containing the number 11 1 11 on line 1 of the coding sheet.)
True or False On the answer sheet, color in the circle containing the number 11 1 11 for each correct or true statement and color in the circle containing the number 11 2 11 for each incorrect or false statement.
Example 2. The earth's only natural satelite is the sun.
(Because this is false, color in the circle containing the number 11 2 11 on line 2 of the coding sheet.)
Matching
On the line of the answer sheet corresponding to Column A, color in the circle containing the number of the best response in Column B. Each answer may be used more than once.
Column A
Example 3. The earth's only natural
satelite.
Column B
1. Jupiter
(Since the correct answer is moon, color in the circle containing the number 11 5 11 on line 3 of the coding sheet.)
2. Neptune
3. Venus
4. Mercury
5. Moon
(Turn to next page to begin test)
154
4. The temperature of the ground is 100° F and water is beginning to evaporate from the ground. What will happen to the ground temperature?
1. decrease 2. increase 3 •. remain the same
5. What desert landform is formed by a river that flows
6.
7.
through the same area for thousands of years? 1. oasis 2. playas 3. canyon 4. mesa
dunes are shaped by Natural 1. 2. 3. 4.
sand particles blown by the wind. the tilt of the earth on it's axis. animals grazing on the plants. trees that die and fall on the dunes.
The final result of irrigation of desert lands 1. reducing wasted space.
is
2. increasing growth of cactus in the area. 3. increasing populations of desert animals
the area.
the
in
4. changing of the area to a non-desert form.
a. Which of the following describes how cactus spines help cactuses survive?
1. Spines protect cactuses from the sun. 2. Spines collect water that cactuses can use. 3. Spines provide support for the cactuses. 4. Spines keep cactuses warm by absorbing heat.
9. The waxy covering of many cactuses helps 1. increase transpiration. 2. reduce water loss. 3. reflect the sun. 4. add color to the plant.
10. Plants that sprout quickly after a desert rain, bloom and produce seeds, then die within a few weeks are called ~~~~-
1. perennial 2. biennial 3. ephemeral 4. annual
(Go to next page)
155
11. The sidewinder rattlesnake's name refers to 1. the snake's unique method of crawling through
the sand. 2. the way in which the snake kills it's prey. 3. the camouflage pattern on the snake's body. 4. the snake's unusual defense tactics.
13. The desert iguana, ground squirrel, jackrabbit, and roadrunner are examples of what type of desert animals?
1. scavengers 2. reptiles 3. predators 4. diurnal
Deserts are known for their dry conditions. Use the graph below to answer questions 14 and 15 concerning rainfall.
5.0
4.5 #
4.0 #
3.5 # # #
Rainfall 3.0 # # in inches # #
2.5 #
2.0 #
1.5
1. 0 * * * * 0.5 * * * *
* * * J F I M I A M I J I J I A I s 0 -I D Month
* = Smithville # = Jonesville
(go to next page)
#
* N
156
14. Which of these cities is in a desert local? 1. Smithville 2. Jonesville
15. In what month does Smithville average the most rainfall?
1. February 2. April 3. August 4. October
16. If you get lost in the desert, the best thing to do is 1. remove your shirt to stay cool. 2. walk toward the sun. 3. walk in larger and larger circles. 4. stay in one place.
17. Most environmentalists dislike all of the following uses of desert lands EXCEPT
1. increasing agricultural production through. irrigation
2. exploring for oil deposits. 3. allowing native plants to grow. 4. crossing the desert in off-road vehicles.
18. What continent contains the largest desert? 1. America 2. Africa 3. Asia 4. Europe
19. The process of changing land from a form that supports life to one that can't support life is called
1. aestivation. 2. desertification. 3. convergent evolution. 4. enhancement.
20. The changing of liquid to vapor is 1. renovation. 2. transpiration. 3. aestivation. 4. evaporation.
(Go to next page)
21.
157
An animal that is active only at night is 1. capricious. 2. nocturnal. 3. colloidal. 4. predacious
True or False On the answer sheet, color in the circle containing the number 11 1 11 for each correct or true statement and color in the circle containing the number 11 2 11 for each incorrect or false statement.
22. As temperatures go up, evaporation rates go down.
23. Desert plants help reduce soil erosion.
24. Vandalism is not a major factor in the decline of many species of cactus.
25. Due to it's limited supply, desert animals compete for water.
26. There are very few species of plants in the desert.
27. The spadefoot toad sleeps through the driest time of the year.
Matching
28.
29.
30.
Animals and Their Food
On the line of the answer sheet corresponding to the Animal, color in the circle containing the number of the correct Food Source. Each answer may be used more than once.
Animal Food Source
roadrunner 1. Insects
coyote 2. desert grasses
peccary 3. small rodents
31.
32.
33.
34.
158
Sidewinder Rattlesnake 4. prickly pear
5. lizards
Desert Tribes
To match each desert area with the people who live there, first find the number on your answer sheet that is the same as the number of the desert area. Color in the circle containing the number of the correct desert peoples. You can use each answer more than once.
Location Tribe
Western Australia 1. Angolans
southwestern United States 2. Hopi Indians
Sahara desert 3. Bushmen
4. Tua regs
5. Aborigines
159
ENDANGERED SPECIES: WILD & RARE
General Instructions for completing Answer Sheet:
You are to enter all answers on the on the Answer Sheet according to the Directions given for each set of questions. Circles must be completely filled in using a number 2 pencil. Make your marks dark enough so you cannot read the number or letter inside the circle. Stray pencil marks may be interpreted as answers so mark only the circle containing the desired answer and erase mistakes carefully.
The top portion of the Answer Sheet should be filled in as follows:
NAME - print your name. COURSE - leave blank. DATE - print the date. ID NUMBER - print the number given by your
teacher. Place only 1 digit in each box and blacken the corresponding circle under that box.
FORM - print "D" in the box and blacken the circle containing the letter "D".
SEAT NO. - print the number given by your teacher and blacken the corresponding circle under that box.
GROUP - print the number 11 5 11 if you are in the fifth grade and the number 11 7 11 if you are in the seventh grade. Blacken the corresponding circle under the box.
The following section will provide examples of how to complete each type of question on the test.
Multiole Choice Please select the one answer that best answers the question or completes the statement. On the line of the answer sheet corresponding to the question number, please color in the circle containing the number of the correct answer.
(Go to next page)
160
Example 1. The earth's only natural satelite is
1. the moon. 2. the sun. 3. Jupiter. 4. Mars.
(Since the correct answer is moon, color in the circle containing the number 11 1 11 on line 1 of the coding sheet.)
True or False
on the answer sheet, color in the circle containing the number 11 1 11 for each correct or true statement and color in the circle containing the number 11 2 11 for each incorrect or false statement.
Example 2. The earth's only natural satelite is the sun.
(Because this is false, color in the circle containing the number 11 2 11 on line 2 of the coding sheet.)
Matching
On the line of the answer sheet corresponding to Column A, color in the circle containing the number of the best response in Column B. Each answer may be used more than once.
Column A
Example 3. The earth's only natural
satelite.
Column B
1. Jupiter
(Since the correct answer is moon, color in the circle containing the number "5" on line 3 of the coding sheet.)
2. Neptune
3. Venus
4. Mercury
5. Moon
161
4. The primary food source for many sea mammals is 1. krill. 2. turtles. 3. sea oats. 4. algae.
5. The term for a species whose population has declined but whose numbers aren't low enough to be in immediate danger of extinction is said to be:
1. Extirpated. 2 . Endangered. 3. Threatened. 4. Stable.
6. Which species below is currently endangered?
7.
1. alligator 2. red wolf 3. snowy egret 4. armadillo
Which of people's
1. 2. 3. 4.
these species did not become extinct due to activities? sabre-tooth cat passenger pigeon Carolina parakeet dodo
8. How has the rate of extinction changed as the human population has grown?
1. It has increased. 2. It has decreased. 3. It has stayed the same. 4. Cannot tell about change.
9. Certain animals have characteristics that make them more likely to become extinct. Which of the following is not one of those characteristics?
1. having special nesting requirements. 2. being migratory. 3. living in a cold climate. 4. interfering with people's activities.
(Go to next page)
162
10. How might a tropical rain forest change when almost all of trees are chopped down?
1. The forest environment will be warmer and drier.
2. Nothing important will change for the animals.
, 3. After a few years, there will be more animal species.
4. The trees remaining will be healthier.
11. ? chokes out native vegetation and was introduced into the United States?
1. Multiflora rose 2. Kudzu 3. Wild orchid. 4. Virginia creeper.
12. Exotic species may be harmful to native species because:
A. The exotics may have no natural enemies to control their populations.
B. The exotics compete with native species for food and nesting areas.
c. The exotics may transmit diseases.
1. A only. 2. A and B only. 3. A and C only. 4. A,B, and c are correct.
13. Worldwide, the most important factor causing animals and plants to become endangered is
1. habitat loss. 2. killing the animals for food, fur, and other
products. 3. taking animals from the wild as pets. 4. pollution.
14. People can help stop the unlawful trade in wildlife by 1. buying wild animals and releasing them. 2. keeping their pets on leash. 3. having only traditional pets. 4. not keeping pets.
(Go to next page)
163
True or False On the answer sheet, color in the circle containing the number 11 1 11 for each correct or true statement and color in the circle containing the number 11 2 11 for each incorrect or false statement.
15. Cultural traditions, economics, and political issues are not important in the management of endangered species.
16. In most natural communities, plants are not as important as animals.
17. As the human population has increased, the number of endangered species has decreased.
18. People compete with plants and wildlife for space.
19. Changes in habitat may cause animal populations to decline.
20. Some zoos breed endangered species and then return them to the wild.
21. There is ample room remaining in the rain forest of South America for most mammals.
22. Starlings are a bird native to the United States.
23. Flying foxes are a type of fruit-eating bird.
24. Non-native species are often a big problem on islands.
25. Whooping crane populations have recently increased due to the efforts of scientists.
26. Species that do not produce many young are less likely to become endangered.
(Go to next page)
164
Matching
On the line of the answer sheet corresponding to Column A, color in the circle containing the number of the best response in Column B. Each answer may be used more than once.
Definitions
Column A
27. A species that lives in a limited area.
28. a species which is
no longer living.
29. a species whose numbers are beginning to decline.
30. A species in immediate danger of extinction.
Endangered Species
Column A
31. Endangered due to loss of habitat.
32. Endangered due to pollution.
33. Endangered primarily due to commercial hunting.
Column B
1. rare
2. introduced
3. native
4. endangered
5. extinct
6. threatened
Column B
1. humpback whale
2. Passenger pigeon
3. Pitcher plant
4. snail darter
5. Bald eagle
165
Wild About Weather
General Instructions for completing Answer Sheet:
You are to enter all answers on the on the Answer Sheet according to the Directions given for each set of questions. Circles must be completely filled in using a number 2 pencil. Make your marks dark enough so you cannot read the number or letter inside the circle. Stray pencil marks may be interpreted as answers so mark only the circle containing the desired answer and erase mistakes carefully.
The top portion of the Answer Sheet should be filled in as follows:
NAME - print your name. COURSE - leave blank. DATE - print the date. ID NUMBER - print the number given by your
teacher. Place only 1 digit in each box and blacken the corresponding circle under that box.
FORM - print "D" in the box and blacken the circle containing the letter "D".
SEAT NO. - print the number given by your teacher and blacken the corresponding circle under that box.
GROUP - print the number "5" if you are in the fifth grade and the number "7" if you are in the seventh grade. Blacken the corresponding circle under the box.
The follwing section will provide examples of how to complete each type of question on the test.
Multiple Choice Please select the one answer that best answers the question or completes the statement. On the line of the answer sheet corresponding to the question number, please color in the circle containing the number of the correct answer.
(Go to next page)
166
Example 1. The earth's only natural satelite is
1. the moon. 2. the sun. 3. Jupiter. 4. Mars.
(Since the correct answer is moon, color in the circle containing the number 11 1 11 on line 1 of the coding sheet.)
True or False
On the answer sheet, color in the circle containing the number 11 1 11 for each correct or true statement and color in the circle containing the number 11 2 11 for each incorrect or false statement.
Example 2. The earth's only natural satelite is the sun.
(Because this is false, color in the circle containing the number 11 2 11 on line 2 of the coding sheet.)
Matching
On the line of the answer sheet corresponding to Column A, color in the circle containing the number of the best response in Column B. Each answer may be used more than once.
Column A
Example 3. The earth's only natural
satelite.
Column B
1. Jupiter
(Since the correct answer is moon, color in the circle containing the number 11 5 11 on line 3 of the coding sheet.)
2. Neptune
3. Venus
4. Mercury
5. Moon
167
Multiple Choice Please select the one answer that best answers the question or completes the statement. on the line of the answer sheet corresponding to the question number, please color in the circle containing the number of the correct answer.
4. The earth's surface is heated unevenly, causing the air to move. Moving air is known as
1. transpiration. 2. hypothermia. 3. wind. 4. kinetics.
5. When the North Pole is tilted toward the sun, Virginia will have what two seasons?
1. Fall & Winter 2. Summer & Fall 3. Winter & Spring 4. Spring & Summer
6. Air always moves from 1. high pressure to low pressure. 2. low pressure to high pressure. 3. place to place for no apparent reason. 4. north to south.
7. Without an atmosphere, the earth would have 1. the same temperatures as it has now. 2. very cold days and very hot nights. 3. very hot days and very cold nights. 4. -constant daylight.
8. Which of the following never happens when a warm air mass and a cold air mass meet?
1. The air from the two masses mixes evenly. 2. The warm air mass moves up over the sides of
the cold air mass. 3. The cold air mass digs under the warm air
mass. 4. Rain, sleet, hail, or snow may fall.
{Go to next page)
168
9. When clouds pile up in the sky, a thunderstorm is probably on the way.
1. cirrocumulus 2. cumulonimbus 3. altostratus 4. stratocumulus
10. The amount of water vapor in the air is called 1. precipitate. 2. saturation density. 3. average rainfall. 4 . humidity.
11. "Fair today, but increasing cloudiness tonight as a low pressure system moves into the area." Why is this forecast reasonable?
12.
1. A low pressure system often indicates precipitation.
2. A fair weather system indicates that rain will soon follow.
3. A low pressure system usually indicates fair weather.
4. Clouds usually accompany fair weather.
Which of 1.
2.
3. 4.
the following best describes how clouds form? Snowflakes pack together in the upper atmosphere. Tiny water droplets condense, forming tiny droplets. Water vapor condenses, forming tiny droplets. Raindrops that have not fallen reflect the sun's light.
13. The trapping of heat from the earth by carbon dioxide in the atmosphere is known as the
1. solar buildup. 2. evaporation phenomenon. 3. greenhouse effect. 4. stellar factor.
True or False On the answer sheet, color in the circle containing the number "1" for each correct or true statement and color in the circle containing the number "2" for each incorrect or false statement.
(Go to next page)
169
14. Air has no weight.
15. Ground water is put back into the atmosphere through vegetation.
16. The earth, sun air, and water act together to influence our weather.
17. Hot air tends to stay close to the ground. (Go to next page)
18. A weather front forms when two air masses with different temperatures and humidities meet.
19. When warm air replaces cold air a cold front is formed.
20. Almost all clouds form at the same height above the earth.
21. It is not safe to be fishing during a lightning storm.
22. For snow to occur all of the following conditions must be present:
a) a cloud containing water droplets at below freezing temperatures;
b) particles of ice for water vapor to condense on;
c) cold air temperatures so the snow doesn't melt as it falls.
23. Cold air holds more water vapor than warm air.
24. Weather does not affect the behavior of plants.
25. Some plants bend toward the sun.
26. Acid rain is a result of air pollution.
Matching
Weather Conditions
On the line of the answer sheet corresponding to the Condition on the left, color in the circle containing the number of the correct Term on the right. Each answer may be used more than once.
(Go to next page)
Condition
27. Constant wind speeds over 74 miles per hour.
28. Drifting, blowing snow.
29. A funnel cloud.
170
Weather Symbols
Term
1. Monsoon
2. Northeaster
3. Blizzard
4. Tornado
5. Hurricane
The map below contains several symbols commonly used in reporting the weather. On the line of the answer sheet corresponding to Column A, color in the circle containing the number of the correct symbol on the map. Each answer may be used more than once.
Column A Column B
30. A 1. Clear skies
31. B 2. Drizzle
32. c 3. Heavy snow
33. D 4. Partly cloudy 5. Cold front
(MAP)
Appendix c NatureScope Objectives As They Appear in the Text
Let's Hear It for Herps
Hands-on Herps Describe some general characteristics of reptiles and amphibians.
Talk about the ways the two groups are alike and different.
Herps on stage Give an example of a characteristic that reptiles and amphibians have in common and one makes them different.
Hot 'n' Cool Herps Define the terms cold-blooded, warm-blooded, ectotherm, and endotherm.
Describe the behaviors that some reptiles and amphibians use to regulate their body temperatures.
Herps Around the World Name some reptiles and amphibians and describe the habitats they live in.
Point out the continents where these herps are found on a world map.
Call of the Wild List several reasons frogs call and explain how they call. Imitate the call of a frog.
Far-out Frogs Name several frogs and explain how they help their young survive.
Watchers at the Pond Explain the life cycle of a frog, toad, or salamander.
Describe the eggs and larvae of an amphibian.
171
172
Let's Hear It for Herps (continued)
Close-Up on Cobras Describe several characteristics of snakes.
Name some of the ways a snake is adapted for survival.
Project Reptile Interpret data from a chart.
Describe and compare several unusual reptiles.
Living Like a Reptile Describe, demonstrate and compare several reptile adaptations.
People and Herps Discuss some of the misconceptions people have about herps. Describe some of the ways people have viewed herps through history.
Hooray for Herps Discuss why some people dislike herps.
Name several positive things about reptiles and amphibians.
Race to the Sea Discuss some of the dangers that threaten sea turtles.
What Would You Do? Discuss some of the problems reptiles and amphibians face.
Give an example of how personal feelings and beliefs can affect situations involving herps.
173
Wild About Weather
The Weather Master Myth Describe how the earth, the sun, water and air affect
weather.
Practice creative writing skills.
Balloons Around the Sun Demonstrate how the earth's tilt affects weather patterns and our seasons.
The Pressure Is On Demonstrate that air has weight and moves from an area of higher pressure to an area of lower pressure.
Water on the Move Observe the weather cycle in a closed system.
Describe the path a water droplet might follow over a period of ten days.
Draw a diagram of the water cycle.
The Weather Game Describe several ways the earth, the sun, air, and water affect our weather.
Crazy Hot Air Balloons Observe that hot air rises.
Follow The Front Explain what a front is.
Define the terms warm front and cold front.
Warm Front Follies Explain what a warm front is.
Name two types of clouds associated with a warm front.
Windblown Poetry Describe haiku and cinquain poetry.
Write a poem about the wind.
A Fold-Out Cloud Book Record and identify different types of clouds.
174
Wild About Weather (continued)
Lightning on the Loose Observe how a static electric charge is formed.
Name some lightning safety rules.
Snow in a Box Observe and describe the conditions that cause snow to form.
Weather Wizards Discuss two weather-related characteristics for each of the following: a sunny day, a tornado, a blizzard, and a hurricane.
Make a Simple Weather Watcher Define humidity and demonstrate how it changes from day to day.
Feathered and Furry Forecasters Discuss how animals have been used in folklore to predict the weather.
Say It with Symbols Draw five weather symbols and explain what they mean.
Use weather symbols to create a story.
Fishy Forecasts Describe a weather forecast.
Identify errors in faulty forecasts.
First with the Forecast Use weather symbols to interpret a simplified weather map.
Weather by the Chart Explain how cloud types, wind direction, and pressure can be used to forecast the weather.
Weather Scavenger Hunt Describe several ways weather affects plants, soil, and people and other animals.
175
Wild About Weather (continued)
What Do You Think? Discuss how people feel about different aspects of the weather.
Define majority and minority.
The Weather Zapper: A Radio Play to Finish Develop an ending to a radio play about weather control.
Discuss some of the pros and cons of controlling the weather.
Weagevia Describe several ways weather affects different parts of the world.
Review world geography.
176
Endangered Species: Wild & Rare
Get the Connection! Discuss why people value some living things more than others.
Explain why all plants and animals are ecologically important.
The Rare Scare Define endangered, threatened, and extinct.
Describe several characteristics that make an animal extremely susceptible to extinction.
A Timely Activity Name several species that have become extinct since the 1600s and discuss why they died out.
Talk about some of the steps people have taken to protect animals, plants, and their habitats.
The People Factor Describe several ways that human population growth affects wildlife.
Relate human population growth to the increased number of extinct birds and mammals.
All Around the World Define endangered and threatened.
Describe the habitats of several endangered or threatened species.
Explain that most species are in trouble due to habitat loss.
Sizing Up Reserves Describe some of the ways rain forest destruction affects plants and animals that live there.
Paradise Lost Describe how introduced species can affect native plants and animals.
Name some introduced species in your area.
177
Endangered Species: Wild & Rare (cont.)
Short Takes Name several endangered species.
Describe some of the reasons animals and plants become endangered.
Wildlife for Sale Name several plants and animals that are in trouble because of excessive and illegal wildlife trading.
Describe some of the ways people can help stop harmful wildlife trade.
People Power Name some of the ways individuals and groups can help endangered species.
Recovery! Describe some of the ways people help endangered species.
Name some plants and animals that were once nearly extinct, but are now doing better.
Meet the Press Name several species whose populations have increased because of help from people.
Describe some of the ways people have helped these species.
178
Discovering Deserts
Evaporation Tricks Define evaporation.
Explain how evaporation affects plants, animals, and landscapes in the desert.
Grit, the Sand Grain Describe how sand can help wind and water create desert landforms.
Draw a picture of a sandy desert scene.
Desert Landforms Come to Life Describe several types of desert landforms and explain how they form.
Desert Graphics Compare average daily temperatures and rainfall of a desert area and a coastal area.
Plot data to make a line graph and a bar graph.
Ice Plants and Elephant Trees Name two desert plants and state one interesting fact about each.
Desert Plants in Trouble Describe three reasons why desert plants are in trouble.
Discuss the importance of desert plants to desert soil and wildlife.
Close-Up Comparisons Describe two ways cactuses are adapted to the desert.
Discuss two differences between cactuses and woodland plants.
Grow'em on Your Own Recognize differences in the growth rates of different desert plants.
Describe the life cycle of a desert ephemeral.
179
Discovering Deserts (cont.)
Desert Relay Race Describe three desert animals and the way they move.
Talk about some of the ways these animals are adapted to the desert.
sunrise, Sunset Define the words diurnal and nocturnal.
Name three desert animals that are diurnal and three that are nocturnal.
Animal Match-Up Describe four desert animals and some of their adaptations to desert life.
A Desert Variety Show Use song, dance, or drama to describe some aspect of life in a desert community.
The Desert Scramble Explain what competition is .
. Discuss limited resources and how they can affect desert communities.
The Desert Daily Times Write a newspaper article or special feature about some aspect of desert life.
Describe how a newspaper is arranged.
Community Squares Describe three plant and animal relationships that exist in the Sonoron desert.
Explain how plants and animals in a community depend on the non-living environment.
Lost in the Desert Name four things you should do if you get lost outside.
Talk about ways to prepare for a hike in the desert.
Desert Culture Day . Describe the lifestyle of one group of desert people.
180
Discovering Deserts (continued)
The Spreading Desert Describe three ways desert lands are overused.
Define desertification.
Desert Quest Research the answers to desert-related questions.
Discuss three unusual desert-related facts.
Appendix D
Intermediate and Advanced Objectives Reorganized based on Environmental Topics
(non-environmental objectives omitted, duplicate objectives should be given greater emphasis on tests, omitted objectives appear at end of Appendix III)
Let's Hear It for Herps
I. Describe general characteristics of reptiles and amphibians.
A. Give examples of characteristics that reptiles and amphibians have in common.
1. Define the words cold-blooded, warm-blooded, ectotherm and endotherm.
2. Describe the behavior that some reptiles and amphibians use to regulate their body temperatures.
B. Give examples of characteristics that make reptiles different from amphibians.
II. Describe the habitats of some specific reptiles and amphibians.
III. Using a world map, point out the continents where specific herps are found.
IV. Explain how frogs call.
V. List several reasons why frogs call.
VI. Explain the life cycle of a frog, toad, or salamander.
A. Name several frogs and explain how they help their young survive.
B. Describe the eggs and larvae of an amphibian.
VII. Describe several characteristics of snakes.
VIII.Describe some of the ways a snake is adapted for survival.
181
182
IX. Compare several unusual reptiles.
X. Compare several reptile adaptations.
XI. Describe some of the ways people have viewed herps through history.
XII. Discuss some of the misconceptions people have about herps.
XIII.Name several positive things about reptiles and amphibians.
XIV. Discuss some of the problems reptiles and amphibians face.
xv. Discuss some of the dangers that threaten sea turtles.
(omitted herp objectives) 1. Imitate the call of a frog. 2. Interpret data from a chart. 3. Discuss why some people dislike herps (affective). 4. Give an example of how personal feelings and beliefs can affect situations involving herps (affective).
183
Wild About Weather
I. Describe how the earth, sun, water, and air affect weather. x2
A. Demonstrate how the earth's tilt affects weather patterns and our seasons.
B. Demonstrate that air has weight and moves from an area of higher pressure to an area of lower pressure.
1. Observe that hot air rises.
II. Observe the weather cycle in a closed system.
III. Draw a diagram of the water cycle.
A. Describe the path a water droplet might follow over a period of ten days.
IV. Describe the terms warm front and cold front. x3
v. Record and identify different types of clouds.
VI. Name two types of clouds associated with a warm front.
VII. Observe how a static electric charge is formed.
VIII.Name some lightning safety rules.
IX. Discuss two weather-related characteristics for each of the following: a sunny day, a tornado, a blizzard, and a hurricane.
x. Observe and describe the conditions that cause snow to form.
XI. Demonstrate how humidity changes from day to day.
184
XII. Explain how cloud types, wind direction, and pressure can be used to predict the weather.
A. Draw five weather symbols and explain what they mean.
B. Use weather symbols to interpret a simplified weather map.
c. Describe a weather forecast.
D. Identify errors in faulty forecasts.
XIII.Describe several ways weather affects plants, soil, and people and other animals.
A. Discuss how people feel about different aspects of the weather.
B. Discuss some of the pros and cons of controlling the weather.
c. Describe several ways weather affects different parts of the world.
(omitted weather objectives) 1. Practice creative writing skills using "The weather myth" 2. Describe haiku and cinquain poetry. 3. Write a poem about the wind. 4. Use weather symbols to create a story. 5. Define majority and minority. 6. Develop an ending to a radio play about weather control. 7. Review world geography.
185
Endangered Species: Wild & Rare
I. Discuss why people value some living things more than others.
II. Explain why all plants and animals are ecologically important.
III. Define endangered, threatened, and extinct. x2
A. Describe several characteristics that make an animal or plant extremely susceptible to extinction. x2
B. Name several species that have become extinct since the 1600's and discuss why they died out.
c. Talk about some of the steps people have taken to protect animals, plants, and their habitats. x3
D. Describe several ways that human population growth affects wildlife.
E. Relate human population growth to the increased number of extinct birds and animals.
IV. Describe the habitats of several endangered or threatened species.
A. Explain that most species are in trouble due to habitat loss.
B. Describe some of the ways rain forest destruction affects plants and animals that live there.
v. Describe how introduced species in your area can affect native plants and animals. x2
·vI. Name several endangered species.
VII. Name several plants and animals that are in trouble because of excessive and illegal wildlife trading.
A. Describe some of the ways people can help stop harmful wildlife trade.
186
VIII.Name some plants and animals that were once nearly extinct, but are now doing better.
IX. Discuss how people have helped several species to increase.
Discovering Deserts
I. Explain how evaporation affects plants, animals, and landscapes in the desert.
A. Define evapor.ation.
II. Describe several types of desert landforms and explain how they form.
III. Explain how desert landforms are shaped. x2
IV. Compare average daily temperatures and rainfall of a desert area and coastal area.
V. Discuss the importance of desert plants to desert soil.
VI. Discuss the importance of desert plants to desert wildlife. x2
VII. Name two desert plants and state one interesting fact about each.
VIII.Describe three reasons why desert plants are in trouble.
IX. Discuss two differences between cactuses and woodland plants.
X. Describe two ways cactuses are adapted to the desert.
XI. Recognize differences in the growth rates of different desert plants.
XII. Describe the lifecycle of a desert ephemeral.
XIII.Describe four desert animals and some of their adaptations to desert life. xJ
187
XIV. Name three desert animals that are nocturnal and three that are diurnal.
A. Define the words diurnal and nocturnal.
XV. Discuss limited resources and how they can affect desert communities.
A. Explain what competition is.
XVI. Explain how plants and animals in a community depend on the non~living environment.
XVII.Talk about ways to prepare for a hike in the desert.
XVIII.Name four things you should do if you get lost outside.
IX. Describe the lifestyle of one group of desert people.
XX. Describe three ways desert lands are overused.
XXI. Define desertification.
XXII.Research the answers to desert-related questions.
A. Discuss three unusual desert-related facts.
(omitted desert objectives)
1. Draw a picture of a sandy desert scene. 2. Plot data to make a line graph and a bar graph. 3. Use song, dance, or drama to describe some aspect of life
in a desert community. 4. Write a newspaper article or special feature about some
aspect of desert life. 5. Describe how a newspaper is arranged.
Appendix E
General Instructions for Completing Answer Sheet:
You are to enter the number which corresponds to the response that most accurately describes your opinion on this statement. Use a number 2 pencil. Make your marks dark enough so you cannot read the number or letter inside the circle. Stray pencil marks may be interpreted as answers so mark only the circle containing the desired answer and erase mistakes carefully.
The top portion of the Answer Sheet should be filled in as follows:
NAME - print your name. COURSE - leave blank. DATE - print the date. ID NUMBER - print the number given by your
teacher. Place only 1 digit in each box and blacken the corresponding circle under that box.
FORM - print 11 E11 in the box and blacken the circle containing the letter 11 E11 • .
SEAT NO. - print the number given by your teacher and blacken the corresponding circle under that box.
GROUP - print the number 11 5 11 if you are in the fifth grade and the number 11 7 11 if you are in the seventh grade. Blacken the corresponding circle under the box.
1. Companies should not have to clean up their
2.
pollution. 1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
Recycling 1. 2. 3. 4.
newspapers costs more Strongly disagree Disagree Agree Strongly agree
188
than it is worth.
189
3. Special habitats should be set aside for endangered species.
1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
4. It is okay to litter on someone else's property. 1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
5. Laws should be passed to prevent more plants and animals from becoming extinct.
1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
6. The future of our planet depends on people. 1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
7. Swamps should be filled in and used for farming.
8.
1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
Stricter passed.
1. 2. 3. 4.
laws against water pollution should be
Strongly disagree Disagree Agree Strongly agree
9. People should try to reduce noise pollution. 1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
10. Animals 1. 2. 3. 4.
190
that eat other animals are mean. Strongly disagree Disagree Agree Strongly agree
11. It is okay to pollute rivers if you clean them up later.
1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
12. Animals that provide meat for people are the most important ones to protect.
1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
13. Water should be pumped into deserts so they can be
14.
used for farming. 1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
Littering 1. 2. 3. 4.
looks bad, but does Strongly disagree Disagree Agree Strongly agree
not hurt anything.
15. When you are on vacation, the hot water heater should be turned off.
1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
16. It is okay to use all the gasoline we want as long as there is enough for everyone.
1. Strongly disagree 2. Disagree 3. Agree 4. Strongly agree
191
Appendix F
Teacher Survey
Thank you for assisting in the NatureScope evaluation. The information gained will help the National Wildlife Federation improve this educational program. Please answer the following questions related to your views on environmental education and NatureScope. Thank you for your cooperation.
Environmental Education
1. Have you taught a unit on the environment to your students in the past 2 years?
yes no
2. Is this the first time you have used any NatureScope materials?
yes no
If no, please indicate which issues you have used (Check all that apply):
Let's Hear It for Herps Amazing Mammals I Incredible Insects Wading into Wetlands Rain Forests
Geology: The Active Earth Digging Into Dinosaurs
Species:Wild & Rare Trees Are Terrific
Birds, Birds, Birds Amazing Mammals II Discovering Deserts Diving into Oceans Astronomy
Adventures Wild About Weather Endangered
192
3. Have you ever attended an environmental education workshop?
yes ___ no
If yes, please briefly describe the nature of that workshop:
4. Place an "X" on the line to indicate your level of environmental awareness.
Low High 1-----1-----1-----1-----1-----1 1 2 3 4 5 6
5. Which of the following materials produced by National Wildlife Federation have you used in teaching?
--- Ranger Rick Wildlife Week --- National Wildlife --- International Wildlife
___ none
The following questions are designed to proyide us with information concerning your opinions NatureScope.
6. Rate NatureScope as to its quality in each of the following:
Low High
1---1---1---1---1---1 1 2 3 4 5 6
Background information given in the activity guide prior to an activity.
193
1---1---1---1---1---1 1 2 3 4 5 6
1---1---1---1---1---1 1 2 3 4 5 6
1---1---1---1---1---1 1 2 3 4 5 6
1---1---1---1---1---1 1 2 3 4 5 6
1---1---1---1---1---1 1 2 3 4 5 6
1---1---1---1---1---1 1 2 3 4 5 6
The match between the activities and the objectives stated for that activity.
The usefulness of illustrations associated with each activity for clarifying activiti-es.
Practicality of crafts.
Benefits of doing crafts.
Usefulness of activities for teaching different subjects.
Usefulness of materials for lesson plans and school objectives.
7. While participating in the NatureScope evaluation, did you use any of the supplemental sources recommended in the bibliography?
yes no
If yes, which two were most useful?~~~~~~~-
194
Background Information
8. What grade(s) do you teach? K 1 2 3 4 5 6 7 8
9. What subject(s) do you teach? Social Studies Mathematics Science Language Arts Art History other
10. What is the highest degree you hold? Bachelors Bachelors plus Masters Masters plus Specialist Doctorate
11. How many years have you been teaching this grade?
Appendix G
NATURES COPE
HANDBOOK AND RECORD BOOK
Wild About Weather
195
196
Introduction What is NatureScope?
The goal of Naturescope as stated by the National Wildlife Federation (NWF) is to develop "a creative education series dedicated to inspiring in children an understanding and appreciation of the natural world while developing the skills they will need to make responsible decisions about the environment".
The list of NatureScope issues currently available address a wide variety of environmental topics ranging from endangered species to weather. Each activity in each issue is designed to be independent of the others, making it possible to use one activity without being forced to do a prerequisite activity.
You have agreed to participate in an evaluation of certain aspects of the NatureScope program. The development and evaluation of environmental education programs is an area of high interest to the NWF.
What is needed from each participant?
Prior to using any of the NatureScope materials in their
classrooms each teacher will administer a pretest booklet containing three tests. The first is for the issue for which that teacher will be using the activity guide. The second is for the activity guide for which that teacher's students will serve as the control group. The third pretest is for the measure of environmental attitudes.
After distributing the test booklet and answer sheets teachers should provide direction in properly completing the answer sheet. Instructions for completing the identification portion of the answer sheet are included on each test. Teachers should also go over the first question of each section as an example of how to complete the coding sheet. At this time teachers may wish to walk around the room to check student understanding of the directions.
Immediately upon completion, these tests should be returned to the field coordinator. The field coordinator will then return the completed tests to Virginia Tech.
197
For one month after giving the pretests teachers in the treatment groups will use the NatureScope issue they have selected. Teachers may use any of the activities designated for intermediate or advanced, but should follow the directions given in the activity. The forms contained in this booklet are specific to the issue of NatureScope you will be using. Please complete the appropriate form for each activity.
At the close of the one month period, posttests will be administered following the same format as the pretests. While the students are completing the posttests, teachers should fill out the survey at the end of this handbook. These tests and surveys should then be returned to the field coordinator. The field coordinator will return them to Virginia Tech.
198
ID NUMBER ~~-
ACTIVITY: The Weather Master Myth
DATE(S) USED NUMBER OF STUDENTS
How long did it take to complete the activity?
Things you liked about the activity
Things you disliked about the activity
Appendix H
How do students exposed to NatureScope differ in
environmental attitudes compared to those not exposed?
Introduction
Analysis of variance was used to determine if students
exposed to one issue of NatureScope differed in
environmental attitudes from students exposed to a different
issue of NatureScope or from the control group. The mean
across the four groups of NatureScope was 50.944 (minimum 16
point, maximum 64 points) on the environmental attitudes
scale.
Considering the large sample size which makes rejection
of the null hypothesis of no difference between groups
easier, these results are statistically significant but not
practically significant.
Analysis and Results
Class means were used as the unit of analysis for
question 2. However, a comparison of the effects of sample
size produced very different results. Statistical analysis
of student scores on the attitude scale produced the means
listed in Table 18. Analysis of variance showed these 199
200
groups to differ significantly at the .oooo level with an F
statistic of 9.7095 (Table 19). However, a difference of
only 3.40 (5.3%) exists between the highest mean (Endangered
Species: Wild & Rare) and the lowest mean (Wild About
Weather).
Table 18. Attitude Means for each unit of analysis
Issue students N Class Avg.
Let's Hear It for Herps 50.3253 292 49.19
Discovering Deserts 51.5146 139 51. 07
Wild About Weather 48.6336 131 50.07
Endangered Species 52.0339 236 51. 02
Pure Control Group 48.9098 122 49.91
Grand Mean 50.4471 1020 50.37
Table 19.
ANOVA for student attitude means by knowledge test
Source
Between
Within
Total
DF
4
1015
1019
MS
293.6910
30.2477
F F prob.
9.7095 .0000*
Groups exposed to Wild About Weather actually had a lower
mean score on the attitude scale than the control group.
N
13
11
5
9
3
41
201
Analysis of the data using class means produced
different results because of the associated degrees of
freedom (see Table 18). The rank order of the means changed
when a different unit of analysis was used. Discovering
Deserts had the largest mean, followed by Endangered
Species: Wild & Rare, Then Wild About Weather. The smallest
group mean was Let's Hear It for Herps.
Students who did activities from Endangered Species:
Wild & Rare had the highest mean score on the scale of
environmental attitudes, followed by Discovering Deserts,
Let's Hear It for Herps, and Wild About Weather (see Table
18). Post hoc analysis using a Scheffe test showed groups
exposed to Endangered Species: Wild & Rare, Discovering
Deserts, and Let's Hear It for Herps differed from the Wild
About Weather groups at the .05 level of significance.
"Endangered Species", "Deserts", and "Herps" also differed
from the control group. These issues are value-sensitive; a
factor that should increase attitude effects. The "Weather"
group only differed from the control group by .28 points
(>1%) •
Summary
The statistical significance of this question varied
greatly depending on the unit of analysis. If effect sizes
202
between the means are considered, a difference between the
extreme means of 3.4% for classes and 5.4% for students is
achieved. From a practical standpoint, none of the four
NatureScope issues had an effect on environmental attitudes.
Appendix I
"Good correlation with geography" "Good math correlation" "[Demonstrations] were quite effective" "helped students expand their imaginations" "creative approach" "very informative activity" "reinforced objectives from previous lessons" "fun way to learn" "pictures were very helpful" "gave students practice in using reference materials" "involved research, cooperative learning, higher level
thinking skills" "very concrete" "good feedback from the students" "Nice for students to work together in groups" "Correlates well with physical education curriculum" "Lots of information given" "correlates with social studies" "good graphing skills" "students engaged in higher levels of thinking" "promotes discussion skills" "Problem-solving skills" "promotes map skills" "hands-on activities" "interesting way of getting ideas across" "very well organized" "Allows for creativity" "Students were curious and eager"
Many teachers felt that if the activities were taken in
order students were prepared for the next activity or had
previous activities reinforced.
Comments concerning things that were disliked about
NatureScope included:
"Recordings of frog calls not available" "pond not available" [needs]"wide range of materials" "lack of references in the school library" ~more realistic pictures"
·"did not copy well" "Copycat pages too crowded."
203
204
"Needs to be more activity-oriented" "sunflower seeds would not stick to the clay" "Information was spread out" "trouble following the directions" "My ability group found it difficult" "not enough time" "Didn't feel the game was very exciting" "more background information" "book could be easier to follow" "high cost for materials"
The most negative comments came from a seventh grade honors
teacher who felt the materials were too elementary for
students at that level. [Students] "knew the outcome and
therefore most of the activities were unnecessary."
205
Appendix J
Results of the Homogeneity of Regression Analysis
Let's Hear It for Herps
R2 Full model = .58989 R2 Partial model = .58960 N = i8 Ki = 3 K2 = 2 F obtained = .0099 F critical for 3 and i4 degrees of freedom= 3.34 Fail to Reject
Discovering Deserts
R2 Full model = .642i3 R2 Partial model = .64046 N = i6 Ki = 3 K2 = 2 F obtained = .056 F critical for 3 and i2 degrees of freedom= 3.49 Fail to Reject
Wild About Weather
R2 Full model = .42i52 R2 Partial model = .ii980 N = ii Ki = 3 K2 = 2 F obtained = 6.259 F critical for 3 and 7 degrees of freedom = 4.35 Reject
Endangered Species: Wild & Rare
R2 Full model = .60635 R2 Partial model = .59662 N = i3 Ki = 3 K2 = 2 F obtained = .222 F critical for 3 and 9 degrees of freedom= 3.86 Fail to Reject
Appendix K
CELL MEANS
5th 7th Pretest Post test Adjusted Pretest Post test Adjusted Mean Mean Mean Mean Mean Mean SD SD SD SD SD SD (N) (N) (N) (N) (N) (N)
He rps Trt 16.38 19.76 17.98 18.01 19.43 13.98
2.11 2.62 2.31 2.72 (10) (10) ( 4) ( 4)
cntr 15.21 14.71 16.18 8.00 13.74 19.50 2.47 4.46 .oo .00
( 3) ( 3) (1) ( 1) Deserts Trt 13.75 14.82 14.75 14.55 14.84 13.84
1. 04 2.88 4.70 5.88 ( 7) (7) (4) ( 4)
Cntr 12.99 10.92 11.77 11. 91 11. 91 12.14 .29 2.77 2.17 2.17 ( 3) ( 3) ( 2) ( 2)
Weather Trt 15.04 19.46 19.61 11. 90 16.22 16.90
4.66 2.60 5.40 3.44 ( 4) ( 4) ( 2) (2)
Cntr 18.89 19.47 18.98 18.09 17.74 17.39 3.05 2.53 4.20 4.40 ( 3) ( 3) ( 2) ( 2)
End. Sp. Trt 14.29 15.06 15.67 15.98 17.17 14.08
1. 73 4.66 1. 00 4.86 (7) ( 7) ( 3) ( 3)
Cntr 13.44 13.69 16.16 15.64 NA NA 1. 01 1. 60 .00
( 3) ( 3) ( 1)
206
He rps
Pretest Mean SD (N)
Cntr 15.82 2.08
6 Deserts Cntr 13.93
1.17 7
Weather Cntrl 17.17
3.07 10
End. Sp. Trt 14.03
1. 60 9
Cntrl 12.81 1. 01
10
Attrition 5th
Posttest Adjusted Mean Mean SD SD (N) (N)
207
Means 7th
Pretest Posttest Adjusted Mean Mean Mean SD SD SD
(N) (N) (N)
14.52 1. 43
7
18.38 3.01
3
The vita has been removed from the scanned document
ABSTRACT
AN EVALUATION MANAGEMENT MODEL
FOR ENVIRONMENTAL EDUCATION PROGRAMS
by
James Barry Armstrong
Committee Co-Chairman: James c. Impara and Daniel E. Vogler Administrative and Educational Services
The study was designed to develop a management model
for the evaluation of environmental education supplements
developed by nonprofit special-interest groups.
Naturescope, an interdisciplinary environmental education
supplement developed by the National Wildlife Federation
(NWF), was used as a vehicle for developing this model. The
first component of the study involved the development of the
model, while the second component measured evaluation
outcomes relative to NatureScope.
Working in conjunction with NWF program staff, four
issues of NatureScope were randomly selected from the
fifteen issues available at the time of the study.
Volunteer fifth and seventh grade teachers from Virginia and
Georgia were used in the study. A separate knowledge test
was developed for each of the four issues. In addition, a
16-item likert scale was developed to measure environmental
attitudes. Qualitative information related to the
usefulness of NatureScope was collected through a teacher
questionnaire.
Evaluator interactions with the client, school
administrators, and teachers influenced the development and
success of the evaluation. The politically-motivated hidden
agendas of the client resulted in unexpected modifications
to the evaluation process. School administrators displayed
a reluctance to permit teachers to participate in the study.
Teachers who volunteered expressed concern over a lack of
familiarity with the topics and the evaluation process.
The evaluation outcomes of the study measured the
effects of NatureScope on environmental knowledge and
attitudes. Students exposed to Let's Hear It for Herps
demonstrated the greatest differences in subject knowledge
when compared to the control group. None of the issues had
a significant effect on environmental attitudes when
compared to the control group. Fifth grade students
systematically scored higher than seventh grade students on
all measures. Teachers responded favorably to NatureScope,
however, the materials were criticized as too simple for
seventh grade. The NatureScope evaluation portion of the
study indicates that the materials may be best suited for
fifth grade students of average ability.