Paper ID #11443

Engineering Summer Programs: A Strategic Model

Dr. Laura Bottomley, North Carolina State University

Dr. Laura Bottomley, ASEE Fellow, is the Director of Women in Engineering and The Engineering Placefor K-20 Outreach and a Teaching Associate Professor in the Colleges of Engineering and Educationat NC State University. She teaches an Introduction to Engineering class for incoming freshmen in theCollege and Children Design, Invent, Create, a course for elementary education students that introducesthem to engineering design and technology as well as various electrical engineering classes.

In 2009 Dr. Bottomley was selected for a Presidential Award for Excellence in Mathematics, Scienceand Engineering Mentoring by the White House Office of Science and Technology Policy and by theEducational Activities Board of the IEEE for an Informal Education Award. She was also inducted intothe YWCA Academy of Women in 2008 for her contributions to eliminating racism and empoweringwomen and was selected as the 2011 Woman of the Year by the RTP chapter of Women in Transportation.In 2013 she was named one of 125 Transformational Women by NC State University.

In her role as director of The Engineering Place at NC State, Dr. Bottomley and her colleagues reach morethan 10,000 students, 2000 teachers and 1500 parents each year. The programs she leads include sum-mer camps for K-12 students; programs that send undergraduates and graduate students into schools towork with elementary and middle school students; training sessions for NC State engineering alumni whowant to be volunteer teachers in their communities; and professional development and classroom supportfor K-12 teachers who want to introduce engineering concepts to their young students. In addition, sheco-authored statewide engineering standards for K-12 and delivers teacher professional development inintegrated STEM. Bottomley also directs NC State’s Women in Engineering program, which works toboost the number of women engineers in academia and industry. The NC State Women in EngineeringProgram was selected as the outstanding program for 2008 by WEPAN, the Women in Engineering Pro-gram Advocates Network for the progress made in recruiting and retaining women students in engineeringat NC State University. In addition to her roles at the University, Dr. Bottomley has taught fifth gradescience as a volunteer consultant, helped schools reinvent themselves as engineering magnet schools andacted as a consultant to the N.C. Dept. of Public Instruction and Wake County Public Schools. She servedon a national team for the National Assessment of Educational Progress developing an assessment forengineering and technological literacy, works with IEEE and the National Academy of Engineering onthe Engineering Equity Extension Project and served as a curriculum consultant on a National ScienceFoundation Gender Equity grant. She also co-authored the Engineering Connections to STEM documentpublished by the North Carolina Department of Public Instruction. She is currently serving on a commit-tee with the National Academy of Engineering, Guiding the Implementation of K-12 Engineering.

Dr. Jerome P. Lavelle, North Carolina State University

Jerome P. Lavelle is Associate Dean of Academic Affairs in the College of Engineering at North CarolinaState University. His teaching and research interests are in the areas of engineering economic analysis,decision analysis, project management, leadership, engineering management and engineering education.

Mrs. Susan Beth D’Amico, NC State University

Susan B. D’Amico Coordinator of Engineering K-12 Outreach Extension The Engineering Place Collegeof Engineering NC State University

Susan earned a B.S in Industrial Engineering from NC State and has worked in the

Telecom and Contract Manufacturing Industries for over 25 years as an Industrial Engineer, Process Engi-neer, Manufacturing Engineer, Project Manager, Business Cost Manager and Program Manager. Inspiredby coursework she developed and presented as an engineer, her professional path made a turn towardseducation by completing coursework for lateral entry teaching.

Susan now works for The Engineering Place, the K-12 outreach arm for NC State University’s College ofEngineering, as a coordinator for Outreach. Her main responsibility is to manage the week long Day and

c©American Society for Engineering Education, 2015

Page 26.644.1

Paper ID #11443

Residential Summer Engineering Camps for rising 3rd through 12th graders in Raleigh and throughoutthe growing number of partner locations throughout the state of North Carolina. Over 1,700 children willbe attending one of her engineering camps during the summer of 2015.

Mr. Landon Drew LaPorte, North Carolina State University

Graduate Research Assistant at the Friday Institute, North Carolina State University

c©American Society for Engineering Education, 2015

Page 26.644.2

Engineering Summer Programs: A Strategic Model (Evaluation)

The Engineering Place is the umbrella program for all engineering K-12 outreach, extension and

engagement activities at North Carolina State University. Operating under the Office of

Academic Affairs this unit last year had over 10,000 touches with K-12 students, parents and

teachers across North Carolina. From the website at www.engr.ncsu.edu/theengineeringplace/:

The Engineering Place is NC State’s K–20 education and resource headquarters for exploring

engineering. Through hands-on summer camps, in-school mentoring, dynamic volunteer

programs, topical workshops and much more, The Engineering Place builds excitement around

engineering for students and teachers.

Engineering summer camps have been offered at NC State University for almost 20 years. Over

time the focus, purpose and strategy associated with planning and executing the camps has

matured to support the current 41 weeks of camp per summer. In the most recent summer these

camps engaged over 1,700 students in grades 2-12 at various locations across the state. Several

design elements of The Engineering Place summer camps are particularly unique, including the

staff mix. In our camps we assemble a combination of engineering educators, K-12 educators,

engineering undergraduate students, and high school students using a tiered mentoring

arrangement. This model was developed as part of our NSF GK-12 grant and has been shown to

have positive and long-term impact on all of the participants. The camps themselves are

designed to be financially self-supporting, including provision for at least five percent

scholarships/aid for those families needing financial assistance. The camp curriculum is linked to

cutting edge research activities in the College, with specific attention to the tenets put forward in

the NAE document, Changing the Conversation5. Attendance at the camps averages 30-40%

female and 35-40% underrepresented ethnic minorities with no specific targeted recruiting.

This paper describes the details of the design of the summer programs and provides assessment

results from more than fifteen years of camps within the College.

Introduction:

The Engineering Place began offering engineering camps almost 20 years ago with a middle

school camp for 30 students and 6 teachers. Since then the program has grown to serve over

1,700 K-12 students per year across the state of North Carolina. This growth has necessitated the

development of a systematic organized approach to planning and implementation, as the team

feels very strongly that the unique flavor of the camps must be maintained. Therefore, the team

has identified distinct measureable goals to which all aspects of the camp are tied: approach,

activities, advertising, application process, assessment and budget.

The mission statement for Engineering Summer Camps is: To provide an enlightening

educational, hands-on experience for elementary, middle and high school students and teachers

that introduces, broadens perspectives and enhances experiences in the disciplines of

engineering and to attract a diverse population to the field of engineering by providing initial or

reinforcing positive experiences to all populations.

Page 26.644.3

The goals are:

Goal 1: Attract a diverse population to summer programs.

Goal 2: Provide an overview of engineering and its many areas of expertise, always highlighting

the true nature of engineering through the Habits of Mind6.

Goal 3: Improve students’ attitudes towards learning in STEM disciplines.

Goal 4: Improve teacher’s attitudes towards teaching in STEM disciplines.

Goal 5: Recruit future engineering students to engineering, with a preference for NC State

College of Engineering.

Goal 2 is particularly important in supporting each of the other goals. Many programs, whether

summer camp in school or out of school, may focus on very narrow aspects of engineering, or

may even be teaching, what the authors would refer to as, technology. To date, therefore, the

camps at The Engineering Place have avoided robotics and Legos to the extent possible to ensure

that participants get a broad view of practical engineering.

This paper is organized to deliberately integrate the mechanics of the camp operation and the

underlying philosophy of those same mechanics. These camps differ from others in the literature

in many ways. The philosophical basis for the camps seems to be completely unique. A

sampling of the literature concerning summer camps yields a variety of publications about

engineering camps for middle and high school students10,11

. Reference 11, in particular, contains

a discussion of the types of camps available and their purposes. Many camps are focused on

robotics. Some are single gender. None of the references discovered mentioned an elementary

engineering camp, and the typical numbers of attendees was under thirty. The camps at The

Engineering Place have some essential differences. The goals are unique. The longevity of the

programs is unique, and the number of attendees, sixty for elementary camps and ninety for 9th

and 10th

grade camps, are significantly different. In addition, there is no other program that

offers a continuum of camps designed on a common platform for students in grades 2-12.

None of the literature discusses staff training, and none of the camps appear to have a tiered

mentoring structure like that found in these camps. For these reasons, the details of these

elements are included in this paper, rather than just assessment results.

The selection process for camp does not give priority to the children whose parents are able to

apply quickly. Giving first-come-first-serve preference was shown to be biased toward more

wealthy parents, which was not intended. The application period runs from the first Monday in

January until April 1st. For the day camps, the selection process is determined by the level of

interest in an applicant’s answers to the questions: “Why do you want to attend camp?” and

“What have you recently learned that excited you?” Upper division high school applicants have a

more rigorous application process for the simple reason that admission to the College of

Engineering is increasingly competitive, and these applicants are within a year of applying. The

goals of the camp have shifted slightly at this level to recruit more directly to the College. The

rubric used for evaluating these applications combines grade point average (40%), course rigor

(30%), personal statement essay (20%), and class rank/standardized test scores (10%).

Page 26.644.4

Camp budgets consist largely of salary expenses for teachers and undergraduate and high school

students. The combined planned budget for the 2015 camps is $482K, comprising 37% for day-

camps and 63% for overnight (residential)-camps. Below are budget percentages:

Residential Camps Budget Breakdown:

Salaries (Full and Part Time Staff) 31%

Meals 18%

Housing 16%

Materials 14% Admin Materials 1%

Misc. Expenses 20%

Day Camps Breakdown:

Salaries (Full and Part Time Staff) 48%

Meals 16%

Materials 6%

Admin Materials 6%

Misc. Expenses 24%

To give a framework for how the camps are organized, sample weekly agendas are shown in

Figure 1. The agenda is designed to be fast-paced, while giving adequate time for each activity

to meet its goals. For example, activities during the beginning of the week need to concentrate

on teaching how to use the design process and on the Engineering Habits of Mind of optimism—

sticking with it until success is achieved—or teamwork. Later in the week, activities may be

more designed to teach a particular scientific concept; however, each activity is multifunctional

and multidisciplinary. Later in the week, the design process becomes a habit in itself, and the

students are functioning better as a team. Activities frequently become more challenging and

more open-ended.

Note that Friday afternoons are dedicated to parent showcases for all students, including student

presentations, and fun design competitions that are assigned Friday morning for younger

students. The design competitions are chosen to incorporate lessons learned during previously

completed activities and may even be a complete redesign of the same activity. Students are not

bored by this repeat; rather they embrace the chance to improve on what they have learned.

An additional important philosophy of the camp is to teach both the attendees AND the

participating teachers about the true and broad nature of engineering. For example, the camps

avoid incorporating robotics. This is not because robots are not a part of engineering, but many

schools use robotics as a substitute for engineering. The fraction of engineers who actually

design robots is terribly small. A more likely subject would be shoe design or food-related

applications of engineering.

Page 26.644.5

Monday Tuesday Wednesday Thursday Friday

9:00 - 9:15Team Assignments/

Badges/Journals/Pre-survey

9:15 - 9:30

9:30 - 9:45

9:45 - 10:00

10:00 - 10:15

10:15 - 10:30

10:30 - 10:45

10:45 - 11:00

11:00 - 11:15

11:15 - 11:30

11:30 - 11:45

11:45 - 12:00

12:00 - 12:15

12:15 - 12:30

12:30 - 12:45

12:45 - 1:00

1:00 - 1:15 Closing Remarks

1:15 - 1:30 Travel to Competition Venue

1:30 - 1:45

1:45 - 2:00

2:00 - 2:15

2:15 - 2:30

2:30 - 2:45

2:45 - 3:00

3:00 - 3:15 Project Ranking

3:15 - 3:30 Dismissal Dismissal Dismissal Dismissal

Gallery WalkGallery Walk of Projects

Awards/ Closing/ Dismissal

Overview of Week

Activity: Ionic Printing of

Hydrogels

Mentor: Daniel Morales

Activity: Medical Systems

Mentor: Julie Ivy, PhD

Activity: Biomymicry -

Mechanical Engineering

Mentor: Leyf Starling

Activity: Airplane Challenge

Mentor: Leyf Starling

Activity: Engineering World

Health

Mentor: Carlos Amaral, PhD

Campers are placed in their

project teams and assigned

new breakout rooms.

Challenges are presented.

Project Dependent Breakout

Time: Campers work on an

engineering project of their

choosing.

Welcome/ Camp Overview/

Presentation to Parents

Lunch Lunch Lunch Lunch Lunch

Activity: Civil Engineering

Project

Mentor: Emily Berglund,

PhD

Prepare for Gallery Walk

Clean-up Breakout Room and

Set Up for Exhibitions

NC STATE COLLEGE OF ENGINEERING K-12 OUTREACH PROGRAM

2014 RALEIGH HIGH SCHOOL SUMMER ENGINEERING DAY CAMP AGENDA

Research/ Project Work

* Finish Projects

* Complete Week Long Activity

* Complete Post Camp Surveys

Welcome/ Intros/ Overview/

Rules and Expectations/ Grand

Challeges

Activity: Double Egg Drop

Figure 1: Sample Camp Agendas

Page 26.644.6

Strategic Camp Design Process

Over the course of many years designing, offering, assessing, re-designing, re-assessing, etc. our

summer camps we have developed a strategic design process approach for existing and new

camps. The simple steps are: Make it Easy, Make it Fun, Make it Work, Work the Mechanics,

and Assessment and Reflection. Each of these is described in more detail below:

Making it Easy

After the camp leadership team has planned the agendas and teachers have vetted them, an

activity journal is created for each camper (see Figure 2). These become the “engineering

notebook” for the week. Space is allowed for each stage of the design process, all activities are

outlined, and a place is provided for student reflection. Not only does this make teaching easier

in the camp, it give the campers a great tool to take home, share with parents and come back to

any activity in the future.

Figure 2: Example student activity journals

While the activity journals describe each design challenge, the undergraduate students are

responsible for working together to develop and implement the testing protocols for each design.

Camp managers must develop a sense of the potential success of a particular test before

implementing at the camp—this is part of the camp protocol so that these undergraduates gain

ownership of that process.

Page 26.644.7

Making it Fun

The camp schedule is designed so that the majority of the day is spent in hands-on work.

Further, the projects are chosen so that 90% of them require creative design (e.g., design a water

squirter that you can use to knock down a paper wall), 8% of them are strictly analytical (e.g.,

learning to do root cause analysis cases), and 2% are procedural (e.g., put together something

from instructions). The focus is not on what the product looks like. In fact, for most of the

projects at camp, it is impossible to predict what the resultant designs will look like, and that is

the point. The projects have carefully constructed constraints and a testing protocol, but the

students generate their own ideas. Examples of outcomes are never shown, so as not to cause the

students to take a particular tack or to limit their creative intent.

Teachers and undergraduates are trained to facilitate the design process to reduce any frustration,

while not overly directing the outcome. Figures 3 and 4 show examples of PowerPoint slides

used in the training sessions for camp staff. Most of the students who attend camps need to be

taught how to engage in the design process and in the engineering habits of mind6, so the weekly

schedule is carefully crafted with this in mind. Shorter, simpler projects with fewer degrees of

freedom are scheduled early in the week with multiple opportunities for iteration. Some of these

shorter projects will be built upon by more complex design opportunities later in the week,

helping to provide scaffolding that increases the probability of success and learning.

Figure 3: Slide for teacher/undergraduate student training P

age 26.644.8

Figure 4: Slide for teacher/undergraduate training

Communications among and within the groups are maintained dynamically. For example, an

“engineering change management” memo can be originated from the undergraduates and passed

to the camp leaders for posting and communicating to the teams. (This most often happens with

new projects that have not been through a camp cycle before.) This might originate because a

participant asked a question that led to a need to make a call to allow an innovation in the design

or to not allow such. The undergraduate students, as a group, make the decision. (For example,

if masking tape is provided, but not duct tape, and a participant team asks for duct tape, will it be

allowed?) Projects are sometimes refined by participants asking unanticipated questions. If a

teacher receives such a question, they communicate with the undergraduates who then determine

whether to make an engineering change or not. An example of such a change would be if a

catapult for marshmallows is being designed that requires both accuracy and distance, and a

participating student group realizes that the rubric will score a catapult that significantly

overshoots the target more than one that just misses it, the teacher may want to make a change in

the way the project is described to disallow this solution…or they may not, depending on what

they want to teach. (This is a real example from camp.)

Making it Work

The camp staffing plan is structured with tiered mentoring and bridging across age groups.

Some team pictures are shown in Figure 5.

Page 26.644.9

Figure 5: Sample team pictures from middle and elementary camp

Using the numbers from an elementary engineering camp, seventy-two elementary students (just

finished grade 2 to just finished grade 4) are divided into six teams of twelve. Each team of

twelve is assigned to a classroom with another team. Each of those teams has a team lead, who

is a teacher with experience teaching the relevant age group, and an undergraduate counselor.

There are also sufficient high school assistants to distribute two to each room. Mentoring takes

place between each of the “levels” of staffing, as well as with each “level” and the student

attendees, as indicated in figure 3. This means that all of the elements are in place to complete a

bridge from K-12 to engineering!

When the camp agendas are designed, function is not a uniting factor, meaning that camps are

not defined as “electrical engineering camp,” or the like. The uniting factor is the engineering

design process, Figure 6. The depiction of the process that is used is based on the five steps

identified by the Museum of Science, Boston4.

Figure 6: The Engineering Design Process used in the NC State University Engineering camps

Page 26.644.10

Because the design process applies to every discipline, every Grand Challenge, etc., it makes an

optimal unifying factor. The camps are also designed to reflect the NAE Grand Challenges for

Engineering7. The camp workbooks, camp tee shirts and the open and closing presentations refer

to the Challenges and connect what goes on in the camp to the broader world of engineering.

Figure 7: Camp tee shirt and workbook content design

New and different camp activities are always included in the agendas as a result of collaboration

with active researchers in the engineering departments. For example, a recent favorite activity

involved designing a hydrogel that can be ionically imprinted from a penny using electricity.

Students learned about the current research being used for hydrogels from the researcher and his

graduate students and were able to create their own. Another favorite activity is a water resource

distribution and model building activity designed by an environmental engineering graduate

student which is based on her research topic.

Working the Mechanics

Each year, the number of camps and partner sites may vary for reasons usually associated with

personnel (e.g., if a departmental contact takes leave and is unable to identify another contact),

but most years the number and variety of camps increase as we grow across the state. Partnering

sites are important because we are able to bring the program and content to areas throughout the

state that don’t have similar local opportunities. The key factor towards a successful partnering

site is to balance providing guidance and direction with encouraging a partner to make it their

own locally flavored camp. This helps integrate relevance into the activities, making the camper

experience more meaningful. Table 1 lists the camps offered in summer 2014.

Page 26.644.11

Table 1: 2014 Engineering Camps by the Numbers: Residential & Day

Day Camps Number of weeks

1,153 Campers

3rd

– 10th

Grade

8 Raleigh

480

4 Charlotte

288

3 Rocky Mount

144

5 Hickory

162

1 Havelock

60

1 Wilson

19

Residential

393

11th/ 12th Grade

6 Raleigh - Week 1

124

6 Raleigh - Week 2

133

6 Raleigh – Week 3

136

Weeks 40 Total Attending 2014 Camps

1,546

Figure 8: Residential & Day Camp Attendance Numbers: 2010 - 2014

Page 26.644.12

As indicated in Figure 8, student attendance at the camps has increased dramatically over the

years since their inception. The demographics of the camps by gender and ethnicity have also

changed. Targeted advertising (through organizations like Girl Scouts and Girls and Boys Club)

is used to attract a diverse group, with some success. The demographics for 2014 are illustrated

in Figures 9 and 10 below.

Figure 9: 2014 Raleigh Day Camp Demographics by Gender

Figure 10: 2014 Raleigh Day Camp Demographics by Ethnicity

Assessment and Reflection: Since their beginnings approximately 15 years ago, the engineering

camps have undergone many changes. One of the most important changes has been the

continuous improvement of the camp assessment system, which in itself leads to changes in the

Page 26.644.13

design and activities of the camps. The assessments have always been chosen to address the

camp goals, as stated in the Introduction, but they have not always been well designed to do so!

It is important to realize that this is not intended to be a research project. The assessments are

designed to match how well the camps meet the camp goals, which can be modified if desired by

the team. Research can be (and has been) overlaid. A paper about a project done by our College

of Education partners has been submitted to this conference. In addition, the camps frequently

encompass the broader impacts portions of grants belonging to fellow researchers by

incorporating research-based activities. The camps make an excellent test bed for research, but

this paper does not incorporate those results.

The first type of assessment done was simply a Likert-style survey of whether the participants

enjoyed various aspects of camp. The assessment plan still includes a survey of this type for

formative assessment. Statistical analysis is not done on this data, as it is deemed uninstructive.

If a certain number of participants do not like an activity, it is modified. Sample results for the

camps are shown in Figure 11 below.

Figure 11: 2014 Elementary Day Camps combined. Number of respondents choosing 1-5 for,” How did you like each of the activities you did at camp”

Sample conclusions from this data would be that students enjoyed the theme park ride design and

the marble wall run design and that the artificial hear/hydraulic arm needs to be revised or

replaced. The activities with the highest ratings are ones that we are likely to select for our

partnering camp locations in the following years. These data are also used to form the process

for creating new activities. For example, the theme park ride is a very open-ended design

project. The attendees clearly preferred this activity to the artificial heart/hydraulic arm activity,

which is very procedural. Future activities will be more open-ended.

The second type of question asks the students to describe their feelings about various aspects of

camps (see Figure 12).

Page 26.644.14

Figure 12: 2014 Middle School Day Camps combined. Number of respondents choosing 1-5 for,

“Select the option that best describes your feelings”

A sample conclusion from this chart would be that students want more time for the activities.

This is a tough concept that is best addressed early in the week so that improvements in time

management are realized as the week progresses. Next year’s staff training sessions and intro

activities will incorporate more of this important concept.

Although ratings for the elementary, middle and early high school (9th

and 10th

grade) day camps

are compiled as above, ratings are separated for each of the late high school (11th

and 12th

grade)

workshops, as they are taught by a different set of staff in the departments. The Camps Director

then meets with each department to go over the analysis and suggest changes to make in

activities, structure, etc.

The second level of assessment asks attendees to self-rate on characteristics that match with the

camp goals (as stated in the introduction). Figure 13 is an example of such a rating from the high

school day camp, which has campers select a research topic to pursue after the first day of camp.

Page 26.644.15

Figure 13: 2014 High School Day Camp Assessment Data, Number of respondents choosing 1-5

From the output above, the camp administrator concluded that the attendees felt comfortable

working in the team setting, which is an important collaborative element to achieve. The

administrator also found evidence for a need to reevaluate the incorporation of the engineering

design process and whether all staff were sufficiently prepared to use it appropriately. A deeper

level of evaluation of the goals of the camp is measured as in Figure 14.

Figure 14: 2014 High School Day Camp Assessment Data, Number of respondents choosing 1-5

Campers felt unsure about coming up with ideas and developing creative solutions for their

group project. The training for the next year camp will incorporate more mentoring techniques in

staff training sessions to promote creativity and resulted in the administration designing an

opening activity that introduces creativity.

Starting in 2006, camp assessments also included collection of data using a survey standardized

by the Burroughs Wellcome Fund. Sample results are given in Table 2. Several problems

existed with using this type of assessment, including confusion between how engineering and

science relate. This effect could have contributed to a lack of certainly of how to interpret the

data. In addition, one cannot interpret an answer that the program will not encourage the student

Page 26.644.16

to take more science classes as negative, if it only confirmed their decision to do so.

Nonetheless, the funder required that we use this assessment, to which we added our own

formative assessment questions. Table 2 shows sample results from this assessment.

Table 2: 2006 Summer Camp Attitude Assessment Strongly

disagree Disagree Uncertain Agree Half

way (4.5)

Strongly agree

No response

1a. This program helped me understand science better.

2.0% 4.0% 16.0% 50.0% 0.0% 26.0% 2.0%

1b. Because of this program, I feel better about being able to learn science.

2.0% 8.0% 18.0% 46.0% 0.0% 24.0% 2.0%

1c. I learned some things in this program that I can use in science class at school.

2.0% 6.0% 10.0% 46.0% 0.0% 34.0% 2.0%

1d. Because of this program, I think I am more aware of the importance of science in everyday living.

2.0% 2.0% 14.0% 54.0% 0.0% 26.0% 2.0%

1e. I tell my family or friends about the things we do in this program.

2.0% 6.0% 10.0% 34.0% 0.0% 44.0% 2.0%

1f. Because of this program, I am more excited about science.

4.0% 10.0% 16.0% 38.0% 2.0% 24.0% 6.0%

1g. Because of this program, I think I have a better understanding of what scientists do.

4.0% 6.0% 6.0% 40.0% 2.0% 40.0% 2.0%

In 2011 The Engineering Place was offered the opportunity to participate in a deeper level of

assessment by the MISO (Maximizing the Impact of STEM Outreach) project, an NSF funded I3

project3. The MISO project was created to unify the evaluation of STEM outreach projects at our

university and to track participants longitudinally. The survey collects attitudinal data about

math attitudes, science attitudes, engineering and technology attitudes and 21st century learning,

using a five point Likert scale. The attitude tests were devised and validated by the MISO staff9.

A snapshot of the MISO attitudes report from the 2014 camps is shown in Figure 15.

Page 26.644.17

Figure 15: MISO Attitude Survey Sample Results from 2014 Camps

The attitude surveys are administered in the morning on the first day of camp and again at the

end of camp. This leads to the ability to do paired sample analysis. Here are findings for

engineering camps in the summer of 2014. Constructs are formed from multiple questions in

Page 26.644.18

four areas of interest, math, science, engineering and technology and 21st Century Skills. All

attitudes tests are paired sample t-tests from pre and post measures of individuals.

Elementary

The S-STEM assessment created by the MISO project3 has four sections, math, science,

Engineering/Tech, 21st century skills. Students are asked questions designed to elicit their

attitudes in these areas. A construct composite is made by averaging responses to questions in

each of the four sections. Here answers are averaged for each participant and then pre scores

and compared to post scores.

For elementary cohort A, students showed significant attitudinal gains in all areas. For

elementary cohort B, students showed significant attitudinal gains in Math, Science, and 21st

century learning showing significant, as indicated in Table 3. Engineering attitudes did not show

significant improvement, although the pre-mean rating was already 4.2 out of 5 on the Likert

Scale.

Table 3: Elementary Engineering Camp Post vs. Pre Attitude Analysis

Paired Samples Test Difference in Means,

Post-Pre-test

constructs,

Elementary

Paired Differences Sig. (2-tailed) Mean Std Dev N 95% Confidence

Interval of the Difference

Lower Upper

Math, cohort A .0777 .3519 113 .0121 .1433 .021

Science, cohort A .2286 .4283 112 .1485 .3089 .000

21st Century Skills, cohort

A .1315 .3355

109 .0678 .1952 .000

Engr/Tech, cohortA .0958 .4061 109 .0187 .1729 .015

Math, cohort B .1136 .3638 110 .0449 .1824 .001 Science, cohort B .2253 .4461 108 .1402 .3104 .000 21

st Century Skills, cohort

B .1633 .3570 103 .0935 .2330 .000

Engr/Tech, cohort B .0279 .4598 108 -.0598 .1156 .530

Middle and High School Camps, Grouped

All attitude scores improved significantly from Pre to Post for both cohorts C and D, as indicated

in Tables 4 and 5.

Page 26.644.19

Table 4: Cohort C Middle and High School Engineering Camp Post vs. Pre Attitude

Analysis Paired Samples Test

Difference in Means,

Post-Pre-test constructs,

Middle and High

Paired Differences Sig. (2-

tailed) Mean Std.

Deviatio

n

N 95% Confidence

Interval of the

Difference

Lower Upper

Math .0661 .3001 475 .0391 .0932 .000

Science .1073 .3709 472 .0738 .1409 .000

Engr/Tech .0642 .3446 469 .0329 .0955 .000

21st Century Skills .0798 .3604 468 .0471 .1126 .000

Table 5: Cohort D Middle and High School Engineering Camp Post vs. Pre Attitude

Analysis Paired Samples Test

Difference in Means,

Post-Pre-test constructs,

Middle and High

Paired Differences Sig. (2-

tailed) Mean Std.

Deviatio

n

N 95% Confidence

Interval of the

Difference

Lower Upper

Math .0517 .28058 580 .0288 .0746 .000

Science .0993 .3583 578 .0700 .1286 .000

Engr/Tech .0299 .3849 576 -.0016 .0614 .063

21st Century Skills .0482 .3782 573 .0172 .0793 .002

Some of the most interesting data are coming from the longitudinal analysis that is beginning to

come out of the MISO project utilizing data from the National Student Clearinghouse8.

Engineering Place camp participants are just beginning to be old enough to have been surveyed

by MISO and graduate. The numbers are small, as yet, but they will continue to increase. Some

sample results are outlined in Table 6.

Table 6: Sample Longitudinal Tracking Results from Independent Samples t-tests

Persistence (Fall 2012-Fall 2013)

105 students matriculated in Fall 2012 and 94.29% continued in Fall 2013

This was significantly higher than the National Average of 68.7% persistence (t = 11.24, df =

104, p < .01)

Extended Persistence (Entered College Before Fall 2013)

Of the 317 students that matriculated to college from the Engineering Place, 95.90% persisted by

enrolling in consecutive Fall semesters

This was significantly higher than the National Average of 68.7% (t = 4.38, df = 316, p < .01)

On-track in College

Engineering Place average of 89% of students remained on-track during college (students did not

miss consecutive Fall or Spring semester enrollment, did not withdraw from enrollment during

the semester, and did not enroll less than full-time)

Page 26.644.20

This was significantly higher than the National Average of 84.7% (t = 2.59, df = 346, p = 0.01)

Time to Graduate from Earliest College Enrollment

Engineering Place average time to graduate was 3.73 years for 27 students that did graduate from

4-year institutions

This was largely significant lower than the National Average of 4.41 years (t = -8.05, df = 26, p <

.01) -

Two-Year Graduates

Engineering Place had 27 four-year college graduates and 10 two-year college graduates

Note that the metrics, although they are statistically significant where indicated, do not take into

account any self-selection bias for students who choose to come to an engineering camp.

Additionally, numbers are still too low to do gender and ethnicity analysis, but these results will

be interesting, when available. The sample sizes for time to graduate and two-year graduate

data are much smaller than other samples, but the findings remain valid. The MISO data and the

longitudinal analysis of it will enable the staff to monitor the long-term effects on camp

attendees. These will, in-turn, allow the camps to be changed, if necessary. Initial results show

very positive long-term impact and do not suggest changes.

Conclusions

After almost 20 years of offering summer camp, The Engineering Place has accumulated a great

deal of information about what works and doesn’t work. One of the most important lessons

learned has been to be sure that everyone on the planning and implementation teams understands

and commits to the overall mission and goals for the camps. This dedication has allowed The

Engineering Place to post very effective outcomes for the camp participants. One of the

hallmarks of work originating with The Engineering Place is to share results, materials, lesson

plans and stories with any interested party, with the only requirement being for

acknowledgement. Others are encouraged to contact the authors for more information.

This paper outlines the camp mechanics and some assessment results focused on students.

Additional outcomes are sought for the influence the camps have on participating teachers.

Assessment of these effects will be the subject of future work, although anechdotal evidence is

very encouraging.

Acknowledgements

Portions of this material are based upon work supported by the National Science Foundation

under Grant No. DUE-1038154 – any opinions, findings, and conclusions or recommendations

expressed in this material are those of the author(s) and do not necessarily reflect the views of

the National Science Foundation.

Page 26.644.21

References

1. Bottomley, Laura and Elizabeth Parry, (2002). K 12 Redux: Sending College Students Back (In)To Schools.

ASEE Annual Conference Proceedings, 2002.

2. LaPorte, L. & Wiebe, E. (April, 2015). Operationalizing Metrics of Persistence and On-Track in the STEM

Pipeline Using the National Clearinghouse Database. Accepted for presentation at the NARST Annual

Meeting, Chicago, IL.

3. MISO, Maximizing the Impact of STEM Outreach. (2014). MISO Project Homepage. Accessed January,

2015 at http://miso.ncsu.edu.

4. Museum of Science, Boston, (2014) EIE Homepage, Accessed January, 2015 at http://www.eie.org/.

5. National Academy of Engineering. (2008) Changing the Conversation: Messages for Improving Public

Understanding of Engineering. Washington, DC: The National Academies Press, 2008.

6. National Academy of Engineering and the National Research Council , (2009) Engineering in K–12

Education: Understanding the Status and Improving the Prospects Washington, DC: The National

Academies Press, 2009.

7. National Academy of Engineering, (2015) Grand Challenges for Engineering Homepage, Accessed

January, 2015 at http://www.engineeringchallenges.org/.

8. NSC, National Student Clearinghouse (2015). Homepage, Accessed January, 2015 at http://

http://www.studentclearinghouse.org/

9. Unfried, A., Faber, M., Stanhope, D. & Wiebe, E. (in press, 2014). The development and validation of a

measure of student attitudes toward science, technology, mathematics, and engineering. Journal of

Psychoeducational Assessment.

10. Barger Marilyn, Richard Gilbert, and Marie A. Boyette, (2011) Best Practices for Student Robotic Camps,

ASEE Annual Conference Proceedings, 2011.

11. Yilmaz, M.; Jianhong Ren; Custer, S.; Coleman, J., "Hands-On Summer Camp to Attract K–12 Students to

Engineering Fields," Education, IEEE Transactions on , vol.53, no.1, pp.144,151, Feb. 2010

Page 26.644.22