the judge factor: the effect of an adjudicator’s teaching
Post on 14-May-2022
3 Views
Preview:
TRANSCRIPT
The Judge Factor: The Effect of an Adjudicator’s Teaching Level
on Assigned UIL Ratings
By
Cody Newman, B.M. A Thesis
In Music Education
Submitted to the Graduate Faculty of Texas Tech University in
Partial Fulfillment of the Requirements for
the Degree of
Master of Music Education
Approved
Dr. Janice Killian Chair of Advisory Committee
Dr. Keith Dye
Dr. Jacqueline C. Henninger
Dr. Mark A. Sheridan Vice Provost for Graduate and Postdoctoral Affairs
Dean of the Graduate School
December, 2014
© 2014, Cody Newman
Texas Tech University, Cody Newman, December, 2014
ii
ACKNOWLEDGMENTS
I would like to say thank you to the many people who have guided me on my
journey through my years at Texas Tech and into the world of music education. The
constant support and inspiration I received from the Texas Tech faculty, my beautiful
wife Candice, and my peers in music education have lifted me to this point. I would like
to specifically thank Dr. Killian for the years of training and teaching she invested in me
as well as her boundless patience through this journey.
Texas Tech University, Cody Newman, December, 2014
iii
TABLE OF CONTENTS
ACKNOWLEDGMENTS ii
ABSTRACT v
LIST OF TABLES vi
LIST OF FIGURES vii
CHAPTER
I. INTRODUCTION 1
Vignette 1
Justification 2
II. REVIEW OF RELATED LITERATURE 4
Director Controlled Influences 4
Non-Director Controlled Influences 5
Judge Reliability 6
Purpose 7
III. METHODOLOGY 8
Terminology 8
Participants 11
Procedures and Materials 14
IV. RESULTS 15
Comprehensive Results by Director 15
Comprehensive Results by Grade Level Experience 22
Comprehensive Results Final Comparison 24
Texas Tech University, Cody Newman, December, 2014
iv
VI. DISCUSSION 26
Implications for Further Research 27
Implication for Educators 29
REFERENCES 30
APPENDIX 33
Texas Tech University, Cody Newman, December, 2014
v
ABSTRACT
Previous research established the reliability of adjudicators and their rankings
and ratings when judging in their field of expertise. This study examined the extent to
which the grade level of judges’ prior teaching experience affects the ratings they assign
during band adjudication (N = 11, 7 high school judges, 4 middle school judges).
Adjudication data were collected from a 14 year period, 2001 – 2014 and included varsity
band ratings from the UIL Concert and Sightreading Contest in the state of Texas.
At each contest studied, judging panels consisted of three Texas Music
Adjudicator Association approved judges. Each of the three judges individually assigned
a rating, and the three ratings were combined for a composite rating. Results were
determined by evaluating the statistical significance between the rating assigned by each
of the judges in this study as compared to the composite rating of the panel for that
contest. These results were then compared based upon the adjudicator’s previous grade
level of teaching experience in relation to the grade level of ensembles adjudicated. The
emphasis of this study was on the relationship between the ratings assigned to ensembles
in which the adjudicator had the most experience teaching in that grade level compared to
the ensembles the adjudicator had little or no experience teaching in that grade level.
Results indicated that there was no statistical significance found in any of the judge’s
ratings when compared to the adjudicators’ previous grade level of teaching and the
ensembles in which they adjudicate.
Texas Tech University, Cody Newman, December, 2014
vi
LIST OF TABLES
1. UIL Rating Table p. 10
2. Participant Data p. 13
3. Unpaired T Test Results p. 25
Texas Tech University, Cody Newman, December, 2014
vii
LIST OF FIGURES
1. High School Judge A Analyzed Results p. 16
2. High School Judge B Analyzed Results p. 16
3. High School Judge C Analyzed Results p. 17
4. High School Judge D Analyzed Results p. 17
5. High School Judge E Analyzed Results p. 18
6. High School Judge F Analyzed Results p. 18
7. High School Judge G Analyzed Results p. 19
8. Middle School Judge AA Analyzed Results p. 19
9. Middle School Judge BB Analyzed Results p. 20
10. Middle School Judge CC Analyzed Results p. 20
11. Middle School Judge DD Analyzed Results p. 21
12. All High School Analyzed Results p. 22
13. All Middle School Analyzed Results p. 23
Texas Tech University, Cody Newman, December, 2014
1
CHAPTER I
INTRODUCTION
Vignette
To sit in the audience and listen to a high school or middle school band is
something many people have done, whether as a parent or friend of a performer or just
a supportive community member. It is a common occurrence, but every year in the
state of Texas, thousands of bands go to the University Interscholastic League,
henceforth referred to as UIL, Concert and Sightreading Contest to test their abilities
against a standard, each seeking the coveted sweepstakes award signifying the group
as a “superior” performing ensemble. Having taken my own band to this contest
numerous times, I know the incredible importance and expectation placed on this one
day of the year and the pressure associated with it. Then I had the opportunity to
experience the other side of the auditorium, the job of the judge. The weight of my
decision along with the rest of the judging panel, could positively shape another
director’s career or, frankly, dissuade them from the profession. It could make for a
celebration by the students celebrating their hard work and the journey they took
together, or, unfortunately, something else altogether. As brutal as it seems, this is the
reality, but there are always deeper layers to every scenario and situation. Each
adjudicator, like the ensembles he or she judges, brings a vast array of experiences in
his or her field. Some have a treasure trove of experience in one level, and next to
nothing in another, yet they often judge across grade levels. Is this just a technicality
that really does not matter because “band is band” regardless of the grade level, or
does the previous grade level of teaching have an impact in the ratings assigned?
Texas Tech University, Cody Newman, December, 2014
2
Justification
In Texas, performance ratings are an everyday part of a music educator’s life,
and much work and focus is based on the performance at the spring contest. The
University Interscholastic League (UIL) is the governing body of all public school
academic, athletic and music contests in the state of Texas (University Interscholastic
League, 2014 a). The Concert and Sightreading Contest occurs each spring and is
attended by middle school and high school bands, choirs, and orchestras (2014 a). This
contest is not a competition between groups or schools but an assessment on an
established standard. That standard is specifically described by judging rubrics created
by the Texas Music Adjudicators Association (2014). This association not only
provides the rubric by which the performing group will be compared, but also
identifies high-quality judges in each field. The TMAA member judges have applied
for and been accepted to the Texas Music Adjudicator’s Association (TMAA),
attended specific training seminars, and displayed a proven track record of excellence
at the school in which they teach (Texas Music Adjudicators Association, 2014).
These judges are then charged with comparing a group’s performance to the rubric and
assigning a rating. A rating of Division 1, or “Superior”, is the goal of each group and
the highest rating possible. A rating of a Division 2 is considered “Excellent”, a
Division 3 is “Average”, a Division 4 is “Below Average” and the lowest rating of
Division 5 is considered “Poor” (University Interscholastic League, 2014 b). Many
factors influence an adjudicator’s assigned rating (Geringer & Johnson, 2007), and
with this being an important piece of a student’s educational experience each year, it
could be beneficial for music educators to understand the impact a judge’s teaching
Texas Tech University, Cody Newman, December, 2014
3
career may have had on his or her final ratings at UIL Concert and Sightreading
Contest.
Beyond the rubric that is used to judge these bands, however, is a human being
who has been charged with using his or her own knowledge and experiences to
accurately rate the group’s performance. All judges are selected on the basis of their
successful teaching experiences (Texas Music Adjudicators Association, 2014), but
each judge has different backgrounds and teaching experience, which raises the
question: How does a judge’s previous experience, particularly the grade level on
which he or she taught, affect the rating assigned to not only the group they have the
most experience teaching, but also the grade levels he or she has less, little, or no
experience teaching? Therefore, the purpose of this study was to determine the extent
to which the grade level of a judge’s prior teaching experience affects the assigned
rating of the bands he or she adjudicates.
Texas Tech University, Cody Newman, December, 2014
4
CHAPTER II
REVIEW OF RELATED RESEARCH
There have been several in depth studies related to the adjudication of music
contests (Bergee, 2003, 2007; Brakel, 2006; Cavitt, 1997; Conrad, 2003; Fiske, 1975,
1978; King & Burnsed, 2009; Smith, 1999), as well as studies and publications
regarding the numerous factors that music educators must address while preparing
their groups for competition that might ultimately affect their contest rating (Baker,
2004; Geringer & Johnson, 2007; Hewitt, 2007; Killian, 1998, 1999, 2000). However,
a study identifying an adjudicator’s previous grade level of teaching in relation to the
ratings he or she gave has not been discovered. Although there are numerous studies
related to music adjudication, the spectrum of information relevant to the proposed
study included studies closely related to the following concepts:
1. Director controlled influences over the rating received
2. Non-director controlled influences over the rating received
3. Previous research related to judge reliability
Director Controlled Influences
Each year, the director of any organization makes many choices that can
positively or negatively affect the year-end outcome and overall success of that
organization. The same is true for music organizations in the state of Texas (May,
1989).
Among the many influences a music director has over his or her ensemble is
the selection of repertoire. Much emphasis has been placed upon the director to make
Texas Tech University, Cody Newman, December, 2014
5
wise repertoire choices for his or her ensemble’s contest performances as described by
Killian (1998, 1999, 2000) in regards to choirs in Texas. Moving from repertoire
choices made outside of the classroom, Smith (1999) examined the actual rehearsal
procedures and practices that music educators, specifically marching band directors,
utilize to influence the eventual contest ratings of their groups. These two factors,
repertoire selection and rehearsal behaviors, are both decisions controlled primarily by
the music director. Further, Price (2006) and Morrison, Price, Geiger and Cornacchio
(2009) explored the director’s physical conducting itself and the many impacts the
level of preparation and conducting style have on the overall outcome of the ensemble
being conducted.
Non-Director Controlled Influences
Although the director of a music program has the ability to positively or
negatively affect the outcome of the group’s performance through decisions made
previous to and during the year, there are factors that can affect the rating and a
judge’s perception during a contest. Previous research would indicate that in solo and
ensemble competitions, judges’ ratings were influenced positively by an afternoon
performance time, belonging to a large, high expenditure school, and being a solo
vocalist (Bergee 2006; Bergee & McWhirter 2005; Bergee & Piatt, 2003). This
example of non-director controlled influence on a judge’s rating was contrary to the
findings of Brakel (2007) who probed ratings of solo musical performances across the
variables of occasion, sequence, and rater, finding lack of bias and general consistency
within the performances evaluated.
Texas Tech University, Cody Newman, December, 2014
6
In 2008, Rickels examined multiple variables that influenced the results of
marching band festivals in the state of Arizona in 2004. Rickels found that elements
such as band budget, number of directors and assistants, marching band enrollment,
the number of festivals attended, school enrollment, and the concurrence of a concert
band program to have the greatest relationship to final scores, but that the director’s
tenure and total years of teaching experience had no significant statistical relationship
to the outcome of the contest.
Judge Reliability
The question of judge reliability is one that has been studied quite extensively
by Brakel (2006), King and Burnsed (2009) and Bergee (2007) who each examined
large contests at the state level for judge reliability.
In 2006, Brakel examined the reliability of judge’s ratings in the state of
Indiana; the results indicated a high level of reliability between judges. The groups
that performed at a higher performance quality yielded the closest agreement between
judges, but as the performance quality of the group declined, so did the agreement
between judges’ ratings, but only on a miniscule level. Brakel gave much credit to the
Indiana State School Music Association Instrumental Festival organization for their
comprehensive examination of judging results and the adjudication process as well as
the implementation of a training session for judges.
Similarly, King and Burnsed (2009) examined the 2005 Virginia Band and
Orchestra Directors Association State Marching Band Festivals and the judge’s ratings
assigned to the 124 competing bands. Just as in Brakel’s study, the reliability of the
Texas Tech University, Cody Newman, December, 2014
7
judge’s ratings was very high. Additionally, King and Burnsed explored further,
studying the judge’s overall ratings in comparison to detailed captions such as: Quality
of Sound, Technique, and Musicianship. The results indicated the inter-correlations of
the caption ratings were very high as well. On the contrary, Bergee (2007) examined
the relationship between the adjudicator, performer, occasion and sequence of the
performance. The findings suggested virtually no measurable error when considering
occasion and sequence, but did find a substantive measurement error amongst the
adjudicators themselves.
Purpose
Given the data indicating the influence of a variety of factors on ratings, but the
lack of data regarding grade level of adjudicators’ prior teaching experience, the
purpose of this study was to determine the extent to which the grade level of a judges’
prior teaching experiences affects the assigned rating of the bands he or she
adjudicates.
Texas Tech University, Cody Newman, December, 2014
8
CHAPTER III
METHODOLOGY
This study was designed to examine the extent to which the grade level of a
judge’s prior teaching experience affects the assigned rating of the bands he or she
adjudicates.
The data collected for this study consisted of ratings from UIL Concert and
Sightreading Contests from across the state of Texas over a period of fourteen years,
2001-2014. The ratings were gathered and sorted into two groups, an “assigned rating”
group (n = 3,493) and a “composite rating” group (n = 3,493), both of which are
defined below.
Terminology
University Interscholastic League Concert and Sightreading Contest
The University Interscholastic League (UIL) was created by the University of
Texas at Austin in 1910 to govern Texas public school academics, athletics and music
contests. It is the largest inter-school organization of its kind in the world
(University Interscholastic League, 2014 a). Within the UIL is the division of music,
which governs the various music specific events that occur annually throughout the
state (University Interscholastic League, 2014 b). Through the supervision of the UIL
Music Division and under the Constitution and Contest Rules (2014 b, 2014 c), the
Concert and Sightreading Contest is conducted each spring. These contests consist of
musical performances of a concert component and a sightreading component. The
concert component includes three prepared pieces by the ensemble adjudicated by a
Texas Tech University, Cody Newman, December, 2014
9
three member-judging panel. The sightreading component consists of a reading of
literature composed specifically for the sightreading contest. As is on the concert
component, a three-judge panel adjudicates the sightreading performance.
Varsity Ensemble
In the state of Texas, the highest quality group at a given school is designated
the varsity group and performs a more difficult level of literature and sightreading than
the non-varsity groups during the UIL Concert and Sightreading contest (University
Interscholastic League, 2014 b). Second ensembles are designated as non-varsity and
lower ensembles are designated as sub non-varsity. Only varsity ratings in both
concert and sightreading were considered in this study to eliminate any discrepancies
caused by each school’s own guidelines for determining the makeup of a non-varsity
group.
Assigned Ratings
An assigned rating is the rating an individual adjudicator gave to a varsity
ensemble at a UIL Concert and Sightreading Contest within the time period of this
study. An assigned rating is specific to each judge and can vary from judge to judge.
The term “assigned rating” was applied because the adjudicator, through the use of a
judging rubric provided by the Texas Music Adjudicators Association (Texas Music
Adjudicators Association, 2014), assigned a rating of one through five to that
ensemble’s performance on that day.
Texas Tech University, Cody Newman, December, 2014
10
Composite Ratings
A composite rating is the combined rating the ensemble received from the
judging panel that day. There were three judges on each panel (University
Interscholastic League, 2014 c) and each judge gave an assigned rating. This
composite rating takes into account each judge’s rating, but is not precisely an average
among the three-judge panel. Using the three assigned ratings applied to the University
Interscholastic League Rating Table (2014 c), Table 1, the composite rating is
determined for the group. The composite rating is in bold, and the assigned ratings of
the three judges and all various combinations are below.
Table 1 University Interscholastic League Rating Table
Rating I Rating II Rating III Rating IV Rating V
1-1-1 1-2-2 1-3-3 1-4-4 1-5-5
1-1-2 1-2-3 1-3-4 1-4-5 2-5-5
1-1-3 1-2-4 1-3-5 2-4-4 3-5-5
1-1-4 1-2-5 2-3-3 2-4-5 4-5-5
1-1-5 2-2-2 2-3-4 3-4-4 5-5-5
2-2-3 2-3-5 3-4-5
2-2-4 3-3-3 4-4-4
2-2-5 3-3-4 4-4-5
3-3-5
(University Interscholastic League, 2014 c)
Texas Tech University, Cody Newman, December, 2014
11
Participants
Each participant was selected by identifying individual judges who had both a
high overall frequency of judging UIL contests, and less than nine years experience as
a head director at one grade level with the remainder of said career at the opposite
grade level. The Texas Music Adjudicators Association has, as an organization, been
challenged with creating a set of standards, not only for the ensembles they adjudicate,
but also for the adjudicator. The requirements to be a University Interscholastic
League judge necessitate a person to:
1. Display successful teaching
2. Maintain employment in music education
3. Attend applicable TMAA approved workshops once every four years
4. Have a record of superior performance
5. Be active as an adjudicator
(Texas Music Adjudicators Association, 2014)
Each judging panel according to the UIL Constitution and Contest Rules (2014
b) must be comprised of at least two TMAA qualified adjudicators.
First, to gather a pool of possible directors, I examined scores publicly
available on UILforms.com website (UILforms, 2014), Brynn Park Productions
(2014), and on the Texas Music Adjudicators Association (2014) website. This process
eliminated many of the possible candidates, as they did not meet the constraint of nine
years or less at one grade level, middle school or high school, with the remainder of
the career spent at the opposing grade level. Next, further research was done into the
frequency with which the participant actually was contracted by the UIL to adjudicate
Texas Tech University, Cody Newman, December, 2014
12
Concert and Sightreading Contests over the time period 2001-2014. These data are
publically available at Brynn Park Productions (2014). There was not necessarily a
minimum number of times hired by UIL constraint because each UIL Concert and
Sightreading Contest can be very different. One contest may have only varsity
ensembles, another may have a mix of varsity and non-varsity, and yet another contest
may contain only non-varsity ensembles. The important data for this study were the
number of varsity ratings assigned by the adjudicator over the time period. In addition
to being hired to judge varsity ensembles at a UIL contest, each participating judge
had to have no fewer than 20 ratings assigned in each grade level, middle school and
high school, and no fewer than 80 ratings assigned overall. After these constraints
were met, eleven (N = 11) participants emerged. The average number of assigned
ratings for the high school judge group was 148 and the average for the middle school
judge group was 464. Details regarding the number of judging experiences per each
selected adjudicator appear in Table 3.
Texas Tech University, Cody Newman, December, 2014
13
Table 3
Participant Data _____________________________________________________________________
Starting in the year 2001 and ending in the year 2014, all ratings assigned to
varsity bands in the state of Texas by these eleven adjudicators (N = 11) were
compiled amassing 3,493 (n = 3,493) assigned ratings and the same number of
corresponding composite ratings (n = 3,493).
Level of Teaching
Experience
Relative to Study
Total High School
Ensemble Ratings
Assigned
Total Middle
School Ensemble
Ratings Assigned
Total Varsity
Ratings
Assigned
Judge A 26 HS / 0 MS 53 35 88
Judge B 15 HS / 8 MS 135 183 318
Judge C 40 HS / 0 MS 108 183 196
Judge D 25 HS / 0 MS 75 22 97
Judge E 30 HS / 3 MS 295 68 363
Judge F 29 HS / 6 MS 163 28 191
Judge G 26 HS / 8 MS 157 226 383
Judge AA 2 HS / 25 MS 65 423 488
Judge BB 9 HS / 21 MS 145 616 761
Judge CC 3 HS / 27 MS 102 212 314
Judge DD 0 HS / 32 MS 26 268 294
Texas Tech University, Cody Newman, December, 2014
14
Procedure and Materials
For each judge, data were compiled and analyzed into four mean numbers:
1. The judge’s assigned rating mean for high school ensembles
2. The corresponding composite rating mean for high school ensembles
3. The judge’s assigned rating mean for middle school ensembles
4. The corresponding composite rating mean for middle school ensembles
After these calculations were completed for each judge, the judges and the
calculated means above were grouped into four different scenarios for comparison.
Each scenario would then yield a mean assigned rating and mean composite rating
based upon the judge’s level of experience, middle school or high school, and the type
of ensemble adjudicated, middle school or high school.
1. High school experienced judges adjudicating high school ensembles
2. High school experienced judges adjudicating middle school ensembles
3. Middle School experienced judges adjudicating high school ensembles
4. Middle school experienced judges adjudicating middle school ensembles
These final mean numbers were then compared using t-tests for all combinations,
testing for statistical significance.
Texas Tech University, Cody Newman, December, 2014
15
CHAPTER IV
RESULTS
The data consisted of information collected regarding the assigned ratings of
specific, experienced, judges (N = 11) and was compared then to the composite rating
for each varsity ensemble judged. The data gathered were used to determine to what
extent the judge’s previous grade level of teaching experience affected the assigned
rating in comparison to the composite rating of the judging panel.
Comprehensive Results by Director
These ratings were then compiled for each adjudicator and calculated into a
judge’s mean rating assigned and a corresponding composite rating assigned. This
allows for a comparison over a 14-year period of each adjudicator’s mean assigned
rating to the overall composite rating the ensemble received. See Figures 1 through 7
for each high school experienced judge’s analyzed results. See Figures 8 though 11 for
each middle school experienced judge’s analyzed results. The judges have been
assigned letter names, single letters for high school experienced adjudicators and
double letters for middle school experienced adjudicators.
When reading each figure, please note that the number atop each column
represents the mean of all ratings applicable to that column and can be compared
directly to the analyzed means of the other columns. Also note that a lower number
indicates a more favorable rating because ratings could range from 1 (high) to 5 (low).
Texas Tech University, Cody Newman, December, 2014
16
Figure 1
High School Judge A Analyzed Results
_____________________________________________________________________
Figure 2
High School Judge B Analyzed Results
Texas Tech University, Cody Newman, December, 2014
17
Figure 3
High School Judge C Analyzed Results
_____________________________________________________________________
Figure 4
High School Judge D Analyzed Results
Texas Tech University, Cody Newman, December, 2014
18
Figure 5
High School Judge E Analyzed Results
_____________________________________________________________________
Figure 6
High School Judge F Analyzed Results
Texas Tech University, Cody Newman, December, 2014
19
Figure 7
High School Judge G Analyzed Results
_____________________________________________________________________
Figure 8
Middle School Judge AA Analyzed Results
Texas Tech University, Cody Newman, December, 2014
20
Figure 9
Middle School Judge BB Analyzed Results
_____________________________________________________________________
Figure 10
Middle School Judge CC Analyzed Results
Texas Tech University, Cody Newman, December, 2014
21
Figure 11
Middle School Judge DD Analyzed Results
_____________________________________________________________________
Texas Tech University, Cody Newman, December, 2014
22
Comprehensive Results by Grade Level Experience
After reviewing the results for each individual judge and the their data over the
14 year period, all high school data were then compiled for a broader look at the
judging outcomes. See Figure 13 for the calculated mean of all high school directors,
and the comparison between the assigned and composite ratings.
_____________________________________________________________________
Figure 13
All High School Judges Combined Analyzed Results
_____________________________________________________________________
Texas Tech University, Cody Newman, December, 2014
23
Similarly, all data collected for each of the middle school experienced directors
were also compiled and analyzed as can be seen in Figure 14.
_____________________________________________________________________
Figure 14
All Middle School Judges Combined Analyzed Results
_____________________________________________________________________
Texas Tech University, Cody Newman, December, 2014
24
Comprehensive Results Final Comparison
After these calculations were completed for each judge, the judges were grouped
into four different scenarios for comparison:
1. High school experienced judges adjudicating high school ensembles
2. High school experienced judges adjudicating middle school ensembles
3. Middle School experienced judges adjudicating high school ensembles
4. Middle school experienced judges adjudicating middle school ensembles
Results were analyzed using an unpaired t-test in each of the four groups. Through
each test, the results returned to reveal a lack of statistical significance in any
comparison.
Texas Tech University, Cody Newman, December, 2014
25
Table 3
Unpaired T Test Results
Mean Assigned Rating
Mean Composite Rating
Two-Tailed P Value
High School Judge
Adjudicating
High School Ensembles
1.40304343 1.38604 0.7211
High School Judge
Adjudicating
Middle School Ensembles
1.426025 1.35671283 0.5285
Middle School Judge
Adjudicating
High School Ensembles
1.354239 1.321053275 0.8628
Middle School Judge
Adjudicating
Middle School Ensembles
1.48094175 1.4723975 0.8192
Texas Tech University, Cody Newman, December, 2014
26
CHAPTER V
DISCUSSION
This study arose from personal experiences with UIL Concert and Sightreading
Contest in my early years of teaching as a high school band director. The results of
that contest did not go the way I wanted, but through that situation, this study arose.
The purpose of this study was to determine to the extent to which the grade level of a
judge’s prior teaching experience affects the assigned rating of the bands he or she
adjudicates.
Looking at the final results of the study, the consistency of the eleven (N = 11)
adjudicators was notable. The lack of statistical significance in the judges’ ratings in
relation to their previous grade level of teaching experience closely aligns with the
studies of King and Burnsed (2009) and Brakel (2006) who both found a high level of
reliability among judges’ scores. When looking at each judge’s results (See Figures 1 -
11), note each individual judge’s analysis with emphasis on how close all the judges
were in their assigned rating mean and composite rating mean. Considering the total
number of assigned ratings (n = 3,493), once again the consistency of the adjudicators
to accurately rate an ensemble’s performance is evident. Results should be generalized
with caution due to the fact that these data came from a single state and a single
adjudication training procedure.
Texas Tech University, Cody Newman, December, 2014
27
Implications for Further Research
A few interesting scenarios developed while collecting data that could be cause
for further study, one of which was apparent pockets of ratings, sometimes lasting just
a few ensembles, and, more rarely, lasting several days of adjudication where a judge
displayed a higher level of “missed” ratings, meaning their assigned rating was not the
same as the composite for an extended period of ratings. Most of these strings of
misses were on the high side, meaning the adjudicator during this string of ratings was
judging more strictly, or more strictly than the other judges on the panel. Keep in
mind, in this contest, the ensemble’s goal is to attain a lower number rating, i.e. “1”,
which indicates a higher level of performance. An interesting study would be to isolate
these strings and conduct interviews with actual adjudicators to try and capture their
mindset during this period of judging. Could it have been the style of teaching in a
geographical region that influenced the judge, musical choices made by the directors
and students, or something that has affected the adjudicator on a more personal level?
Based on these findings, additional research could be conducted involving the
non-varsity ensemble. The non-varsity ensemble creates many different and interesting
scenarios for the adjudicator who knows that these are not necessarily the top players
at the school and has to apply the standards from the Texas Music Adjudicators
Association equally to them as he or she does to any other ensemble. Although this
study was limited to the examination of varsity ensembles, the researcher had the
opportunity to peruse the non-varsity ratings while collecting the varsity ensemble
data. After reviewing these non-varsity ratings assigned by the adjudicators that were
included in this study, there seems to be much more discrepancy in ratings. To
Texas Tech University, Cody Newman, December, 2014
28
speculate, these discrepancies in ratings could be due to the very nature of the non-
varsity ensemble, which is generally comprised of less musically developed and less
technically advanced students. Those factors carry different weight with each
adjudicator, which may lead to instability in the accuracy and agreement between
judges. To study this might reveal how an adjudicator who has a stellar record with
varsity groups transfers that level of adjudication to the non-varsity ensemble.
Finally, I believe the most interesting result from this study would center
around how most of the adjudicators included in the study had higher mean assigned
ratings than composite rating assigned by corresponding judging panel. Keep in mind
Table 1, which illustrates how the composite rating system works. The composite
rating is not the mean rating of the three judges; in fact, it is not necessarily a
representation of agreement by the three judges on a rating. The only adjudicator to
have an assigned mean lower in both grade levels, middle school and high school, was
Middle School Judge CC, and only two adjudicators had one mean assigned higher
than the composite in one grade level, High School Judge B and Middle School Judge
BB. All three of these situations were ones in which mean average although higher on
the composite side, was still no more than 0.02 rating points in total difference. All
other judges in this study, in both grade levels, displayed a higher mean assigned
rating than composite rating, meaning that these changes on average resulted in a
higher rating to an ensemble than the judges on the panel sitting next to them. I would
speculate that this is because of the volume of ratings each of these judges has given
over their careers along with the confidence to assign a rating based upon that
experience.
Texas Tech University, Cody Newman, December, 2014
29
Implication for Educators
After many months of collecting and analyzing data, the final results are what
we all as educators and adjudicators would hope for. No statistical significance in this
situation is an outstanding result that should be applauded by all parties involved.
Through the results of this study, it is clear that the intentions of the University
Interscholastic League and Texas Music Adjudicators Association are successfully
producing individuals who are not being swayed, positively or negatively, from a
ratings standpoint, by their previous grade level of teaching.
Texas Tech University, Cody Newman, December, 2014
30
REFERENCES
Baker, V. (2004). The effect of repertoire selection on university interscholastic league Choral concert ratings. Texas Music Education Research. Retrieved from http://www.tmea.org/assets/pdf/research/Bak2004.pdf Bergee, M. J. (2003). Faculty interjudge reliability of music performance evaluation. Journal of Research in Music Education, 51, 137-150. doi:10.2307/3345847 Bergee, M. (2006). Validation of a model of extramusical influences on solo and
small-ensemble festival ratings. Journal of Research in Music Education, 54, 244-256.
Bergee, M. (2007). Performer, rater, occasion, and sequence as sources of variability
in music performance assessment. Journal of Research in Music Education, 55, 344-358.
Bergee, M. & McWhirter, J. (2005) Selected influences on solo and small-ensemble
festival ratings: replication and extension. Journal of Research in Music Education, 53, 177-190.
Bergee, M. J., & Piatt, M. C. (2003). Influence of selected variables on solo and small-
Ensemble festival ratings. Journal of Research in Music Education, 51, 342- 353. doi:10.2307/3345660
Bergee, M. J., & Westfall, С. R. (2005). Stability of a model explaining selected
extramusical influences on solo and small-ensemble festival ratings. Journal of Research in Music Education, 53, 358-374. doi:10.1177/002242940505300407
Brakel, T. (2006). Inter-judge reliability of the Indiana State School Music
Association High School Instrumental Festival. Journal of Band Research, 42, 59-69.
Brynn Park Productions (2014). Complete UIL Results. Retrieved from
http://www.brynnpark.com/#!archives/c1syx Burnsed, V., Hinkle, D., & King, S. (1985). Performance evaluation reliability at selected concert festivals. Journal of Band Research, 2/(1), 22-29. Cavitt, M. E. (1997). Effects of expectations on evaluators' judgments of music performance. Texas Music Education Research. Retrieved from http://www.tmea.org/assets/pdf/research/Cavl997.pdf
Texas Tech University, Cody Newman, December, 2014
31
Conrad, D. (2003). Judging the judges: Improving rater reliability at music contests. NFHS Music Association Journal, 20(2), 27-31 Fiske, H. E. (1975). Judge-group differences in the rating of secondary school trumpet
performances. Journal of Research in Music Education, 23, 186-189. doi:10.2307/3344643
Fiske, H. E. (1978). The effect of a training procedure in music performance
evaluation on judge reliability. Ontario Educational Research Council Report, Canada.
Geringer, J. & Johnson, C. (2007). Effects of excerpt duration, tempo, and
performance level on musicians' ratings of wind band performances. Journal of Research in Music Education, 55, 289-301.
Hewitt, M. (2007) Influence of primary performance instrument and education level
on music performance evaluation. Journal of Research in Music Education, 55, 18-30.
Killian J. N. (1998). Characteristics of successful choirs in a contest setting. Texas
Music Education Research, 39-43. Retrieved from http://www.tmea.org/assets/pdf/research/Kill998.pdf
Killian J. (1999). Music selection of successful choirs at UIL and non-UIL contests.
Texas Music Education Research, 51-56. Retrieved from http://www.tmea.org/assets/pdfresearch/Kill999.pdf
Killian, J. (2000). Effect of music selection on contest ratings: Year three of a
continuing study. Texas Music Education Research. Retrieved from http://www.tmea.org/assets/pdf/research/Kil2000.pdf
King, S. & Burnsed, V. (2009). A study of the reliability of adjudicators ratings at the
2005 Virginia Band and Orchestra Directors Association State Marching Band Festivals. Journal of Band Research, 45, 27-32.
May, W. (1989, October). On Competition. Southwestern Musician, 6, 8. Morrison, S., Price, H., Geiger, C. & Cornacchio, R. (2009). The effect of conductor
expressivity on ensemble performance evaluation. Journal of Research in Music Education, 57, 37-49.
Price, H. (2006). Relationships among conducting quality, ensemble performance
quality, and state festival ratings. Journal of Research in Music Education, 54, 203-214.
Texas Tech University, Cody Newman, December, 2014
32
Rickels, D. (2008). A comparison of variables in Arizona marching band festival results. Journal of Research in Music Education. 44, 25-39.
Smith, J.W. (1999). Correlation of discreet and continuous contest ratings and
marching band director rehearsal behaviors. (Doctoral dissertation, University of Kansas, 1999). Dissertation Abstracts International, 60 (09A), 3303.
Texas Music Adjudicators Association. (2014). Membership Requirements. Retrieved
from http://www.txmaa.org/tmaameminfo.php UIL Forms (2014). Results. Retrieved from http://www.uilforms.com/results.asp University Interscholastic League. (2014 a). About the UIL. Retrieved from
http://www.uiltexas.org/about University Interscholastic League. (2014 b). Constitution and contest rules: Section
1109-1110. Retrieved from http://www.uiltexas.org/files/constitution/uil-ccr-section-1109-1110.pdf
University Interscholastic League. (2014 c). Constitution and contest rules: Section
1111-1115. Retrieved from http://www.uiltexas.org/files/constitution/uil-ccr-section-1111-1115.pdf
Texas Tech University, Cody Newman, December, 2014
33
APPENDIX
Texas Tech University, Cody Newman, December, 2014
High School Judge MEAN
TOTAL SUM OF RATINGS
TOTALNUMBER OF RATINGS
AHS 53 HSassigned 1.377358491 73 assigned 1 1 1 1 2 1 1 1 1 1 2 2 2 1 2 1 2 2 1 1 1 1 2 3 1composite 1.320754717 70 composite 1 1 1 1 2 1 1 1 1 1 2 1 2 1 2 1 1 2 1 1 1 1 2 3 1MS 35 MSassigned 1.314285714 46 assigned 1 1 2 2 1 1 1 1 1 2 2 2 1 1 1 1 1 1 1 2 1 4 1 1 1composite 1.228571429 43 composite 1 1 2 2 1 1 1 1 1 2 2 2 1 1 1 1 1 1 1 3 1 2 1 1 1
HSassigned 2 1 2 2 1 1 1 1 1 1 1 1 2 1 1 2 2 1 1 3 1 1 1 1 2composite 2 1 2 2 1 1 1 1 1 1 1 1 1 1 1 2 2 1 1 3 1 1 1 1 2MSassigned 1 1 1 1 1 1 1 2 2 1composite 1 1 1 1 1 1 1 1 1 1
HSassigned 1 2composite 1 2MSassignedcomposite
34
Texas Tech University, Cody Newman, December, 2014
High School Judge MEAN
TOTAL SUM OF RATINGS
TOTALNUMBER OF RATINGS
BHS 135 HSassigned 1.348148148 182 assigned 1 1 2 1 2 1 1 1 1 1 1 1 1 1 2 1 2 3 1 3 3 1 1 1 3composite 1.355555556 183 composite 1 1 2 1 2 1 1 1 1 1 1 1 1 1 2 1 1 3 1 3 3 1 1 1 3MS 183 MSassigned 1.513661202 277 assigned 2 1 1 3 3 1 4 1 1 2 1 3 2 3 1 2 1 2 1 3 4 1 2 1 1composite 1.49726776 274 composite 2 1 1 3 3 1 4 1 1 2 1 2 2 3 1 2 1 2 1 3 3 1 2 1 1
HSassigned 3 1 1 1 1 2 1 4 1 1 1 2 2 1 1 1 3 2 2 1 3 1 1 1 1composite 3 1 1 1 1 2 1 4 1 1 1 2 3 1 1 1 3 2 2 1 3 1 1 1 1MSassigned 1 1 1 1 3 1 1 1 1 1 4 1 2 2 1 2 1 3 2 1 2 1 2 1 2composite 1 1 2 1 2 1 1 1 1 1 4 1 2 1 1 2 1 3 2 1 2 1 2 1 1
HSassigned 1 2 3 2 1 2 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 2 1 1 1composite 1 2 3 2 1 2 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 2 2 1 1MSassigned 1 1 1 2 1 4 1 1 1 1 2 3 2 1 1 1 2 2 1 2 1 1 2 3 1composite 1 1 1 2 1 4 1 1 1 1 1 3 2 2 1 1 3 2 2 2 1 2 2 2 1
35
Texas Tech University, Cody Newman, December, 2014
B (2 of 2)
HSassigned 2 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 4 1 1 2 2 1 1 1 1 2 1 1 1 1 1 1 3 2 1 3 1 1 1 1 1composite 2 1 1 1 1 1 1 1 2 1 1 3 1 1 1 1 1 1 1 1 1 1 1 1 1 1 3 1 1 2 2 1 1 1 1 2 1 1 1 1 1 1 3 2 1 3 1 1 1 1 1MSassigned 3 2 1 1 1 1 2 1 1 2 1 3 2 1 1 1 1 1 2 1 1 1 1 2 1 2 3 2 1 1 3 3 1 1 1 2 2 1 2 1 3 1 1 2 1 1 1 2 1 1 1composite 3 2 1 1 1 2 2 1 1 2 1 3 2 1 1 1 1 1 2 1 1 2 1 2 1 2 3 2 1 1 2 2 1 1 1 2 2 1 2 1 3 1 1 2 1 1 1 3 1 1 1
HSassigned 1 2 1 1 1 1 1 1 1composite 1 1 1 1 1 1 1 1 1MSassigned 2 2 3 1 1 1 1 3 1 1 1 1 1 1 1 2 1 2 1 1 1 3 2 1 1 1 1 1 1 1 1 1 1 1 1 2 3 1 2 1 1 1 1 1 1 1 1 1 2 1 1composite 2 1 3 1 1 1 1 2 2 2 1 1 1 1 1 2 1 2 1 1 1 2 2 1 1 1 1 1 1 1 1 1 1 1 2 1 3 1 2 1 1 1 1 1 1 1 1 1 1 1 1
HSassignedcompositeMSassigned 1 2 1 1 3 1composite 1 2 1 2 2 1
36
Texas Tech University, Cody Newman, December, 2014
High School Judge MEAN
TOTAL SUM OF RATINGS
TOTAL NUMBER OF RATINGS
CHS 108 HSassigned 1.305555556 141 assigned 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 2 2 1 1 3 1 1 1 1 1composite 1.305555556 141 composite 2 1 1 1 1 1 2 1 1 1 1 1 1 1 1 2 2 1 1 3 1 1 1 1 1MS 88 MSassigned 1.613636364 142 assigned 2 1 3 1 3 2 2 1 1 1 2 1 2 1 2 1 1 3 1 1 1 1 1 1 1composite 1.568181818 138 composite 2 1 3 1 3 2 2 1 1 1 1 1 2 1 2 1 1 3 2 1 1 1 1 1 1
HSassigned 3 1 2 3 2 1 4 2 3 1 4 1 1 1 1 1 1 1 1 1 1 1 1 1 1composite 3 1 2 2 2 1 4 2 3 1 4 1 1 1 1 1 1 1 1 1 1 1 1 1 1MSassigned 1 1 1 1 2 2 1 3 1 2 1 2 1 4 2 1 1 1 1 1 1 1 2 1 1composite 1 1 1 1 2 2 1 2 1 2 2 2 1 4 2 1 1 1 1 1 1 1 2 1 1
HSassigned 2 1 1 1 1 3 1 1 1 4 1 1 2 4 1 2 1 1 1 1 1 1 1 1 1composite 2 1 1 1 1 3 1 1 1 4 1 1 2 3 1 2 1 1 1 1 1 1 1 1 1MSassigned 1 2 1 3 1 4 1 1 1 3 3 2 1 2 1 4 2 1 2 1 1 1 1 2 2composite 1 2 1 3 1 4 1 1 1 3 3 2 1 2 1 2 2 1 2 1 1 1 1 3 2
37
Texas Tech University, Cody Newman, December, 2014
C (2 of 2)
HSassigned 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 1 1 1 1 1 1 1 1 1 1 1composite 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 1 1 1 1 1 1 1 1 1 1 1MSassigned 1 2 1 2 2 3 4 3 3 1 1 1 1composite 1 1 1 2 2 2 3 2 3 1 1 1 2
HSassignedcompositeMSassignedcomposite
HSassignedcompositeMSassignedcomposite
38
Texas Tech University, Cody Newman, December, 2014
High School Judge MEAN
TOTAL SUM OF RATINGS
TOTALNUMBER OF RATINGS
DHS 75 HSassigned 1.546666667 116 assigned 3 4 2 1 1 1 1 1 2 1 3 1 1 5 3 1 1 1 3 2 1 1 2 1 1composite 1.533333333 115 composite 4 4 2 1 1 1 1 1 1 1 3 1 1 5 3 1 1 1 3 2 1 1 2 1 1MS 22 MSassigned 1.318181818 29 assigned 2 1 1 1 1 2 2 1 2 1 1 1 3 1 1 1 1 2 1 1 1 1composite 1.272727273 28 composite 2 1 1 1 1 2 1 1 1 1 2 1 3 1 1 1 1 2 1 1 1 1
HSassigned 1 1 1 3 3 1 3 3 3 1 1 3 1 1 1 1 1 1 1 2 1 1 1 1 3composite 1 1 1 3 3 1 2 3 3 1 1 3 1 1 1 1 1 1 1 2 1 1 1 1 3MSassignedcomposite
HSassigned 2 1 1 1 2 1 3 1 1 1 1 1 1 1 1 1 1 2 2 2 1 1 1 1 1composite 2 2 1 1 2 1 2 1 1 1 1 1 1 1 1 1 1 2 2 2 1 1 1 1 1MSassignedcomposite
39
Texas Tech University, Cody Newman, December, 2014
High School Judge MEAN
TOTAL SUM OF RATINGS
TOTALNUMBER OF RATINGS
EHS 295 HSassigned 1.376271186 406 assigned 1 2 1 1 1 1 2 1 1 1 1 1 1 1 1 2 1 1 4 1 3 2 3 1 1composite 1.359322034 401 composite 1 2 1 1 1 1 2 1 1 1 1 2 1 1 2 2 1 1 4 1 3 2 3 1 1MS 68 MSassigned 1.102941176 75 assigned 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1composite 1.073529412 73 composite 2 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
HSassigned 1 1 1 2 1 2 1 1 1 1 1 3 2 2 2 1 3 3 2 1 1 1 2 2 2composite 1 1 1 2 1 2 1 1 1 1 2 3 2 2 1 1 3 3 2 1 1 1 2 2 2MSassigned 2 2 1 1 3 1 1 2 1 1 1 1 1 3 1 2 2 1 1 1 1 1 1 3 1composite 2 2 1 1 3 1 1 1 1 1 1 1 1 2 1 2 2 1 1 1 1 1 1 3 1
HSassigned 1 1 2 1 4 4 2 1 1 2 1 2 1 1 1 1 2 1 1 1 1 1 1 2 2composite 1 1 2 1 5 3 2 1 1 2 1 1 1 1 1 1 2 1 1 1 1 1 1 2 2MSassigned 1 1 3 3 1 1 2 1composite 1 1 2 3 1 1 2 1
40
Texas Tech University, Cody Newman, December, 2014
E (2 of 3)
HSassigned 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 3 2 1 1 2 1 1 1 1 1 1 2 1 1 1 1 1 3 1 2 1 2 1 3 1 1 1 1 2 1composite 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 1 1 2 1 1 1 1 1 1 2 1 1 1 1 1 2 1 2 1 1 1 3 1 1 1 1 1 1MSassignedcomposite
HSassigned 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 2 2 1 2 3 1 4 2 3 3 2 1 4 2 1 1 3 1 1 1 1composite 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 2 1 1 1 1 1 2 1 1 1 2 2 2 1 2 3 1 4 2 3 3 2 1 4 2 1 1 3 1 1 1 1MSassignedcomposite
HSassigned 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 2 1 1 1 1 1 2composite 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 2 1 1 1 1 1 1MSassignedcomposite
41
Texas Tech University, Cody Newman, December, 2014
E (3 of 3)
HSassigned 1 1 1 4 1 1 1 2 1 1 3 1 1 1 3 2 1 2 1 1 3 3 2 1 1 3 1 1 1 2 3 1 1 1 2 1 3 1 1 2 1 1 1 1 3 1 1 3 2 1 1composite 1 1 1 3 2 1 1 1 1 1 3 1 1 1 3 2 1 1 1 1 3 3 1 1 1 3 1 1 1 2 3 2 1 1 1 1 3 1 1 2 1 1 1 1 4 1 1 3 2 1 1MSassignedcomposite
HSassigned 1 1 1 3 1 1 1 1 1 1 1 1 1 1 1 1composite 1 1 1 3 1 1 1 1 1 1 1 1 1 1 1 1MSassignedcomposite
42
Texas Tech University, Cody Newman, December, 2014
High School Judge MEAN
TOTAL SUM OF RATINGS
TOTALNUMBER OF RATINGS
FHS 163 HSassigned 1.472392638 240 assigned 1 2 3 2 1 4 1 1 2 1 1 3 1 1 2 2 1 3 1 1 1 1 3 1 2composite 1.441717791 235 composite 1 2 3 2 1 4 1 1 2 1 1 2 1 1 2 1 1 3 1 1 1 1 3 1 2MS 28 MSassigned 1.5 42 assigned 2 2 1 1 1 1 2 1 1 1 1 2 2 1 2 2 4 2 3 1 2 1 1 1 1composite 1.5 42 composite 3 2 2 1 1 1 2 1 1 1 1 2 1 1 2 2 3 2 3 1 2 1 1 1 1
HSassigned 1 1 2 1 1 2 1 1 1 1 1 1 1 1 1 1 1 2 1 1 2 1 1 1 1composite 1 1 2 1 1 2 1 1 1 1 1 1 1 1 1 1 1 2 1 1 2 1 2 1 1MSassigned 1 1 1composite 1 1 1
HSassigned 1 2 1 1 1 1 1 2 2 3 1 2 1 1 1 1 2 1 1 1 1 1 1 1 1composite 1 2 1 1 1 1 1 2 2 2 1 2 1 1 1 1 2 1 1 1 1 1 1 1 1MSassignedcomposite
43
Texas Tech University, Cody Newman, December, 2014
F (2 of 2)
HSassigned 2 1 1 1 1 1 1 1 1 1 2 2 1 1 2 1 3 2 3 2 1 2 3 1 3 2 1 2 1 1 2 1 1 1 1 1 3 2 3 3 2 3 3 2 1 3 2 1 3 1 2composite 2 2 1 1 1 1 1 1 1 1 2 1 1 1 2 1 3 2 3 2 1 2 3 1 3 1 1 2 1 1 2 1 1 1 1 1 3 2 3 4 2 2 3 2 1 2 2 1 3 1 2MSassignedcomposite
HSassigned 2 1 1 1 1 1 1 1 2 1 1 3 1 3 1 1 2 2 1 2 1 1 1 2 1 1 1 1 2 1 1 1 1 1 1 1 2composite 2 1 1 1 1 1 1 1 2 1 2 2 1 3 2 1 2 1 1 2 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 2MSassignedcomposite
44
Texas Tech University, Cody Newman, December, 2014
High School Judge MEAN
TOTAL SUM OF RATINGS
TOTALNUMBER OF RATINGS G (1 of 2)
GHS 157 HSassigned 1.394904459 219 assigned 1 1 1 2 1 1 3 1 1 1 1 2 1 1 1 1 1 1 1 2 2 1 1 1 1composite 1.388535032 218 composite 1 1 1 2 1 1 3 1 1 1 1 2 1 1 2 1 1 1 1 2 2 1 1 1 1MS 226 MSassigned 1.619469027 366 assigned 4 3 2 4 1 2 5 1 2 2 1 1 1 1 2 1 1 1 1 1 1 1 2 1 1composite 1.553097345 351 composite 3 3 2 4 1 2 5 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
HSassigned 1 1 1 1 1 2 2 1 1 1 2 1 1 1 2 1 1 1 1 1 1 2 1 1 1composite 1 1 1 1 1 3 2 2 1 1 3 1 1 1 2 1 1 1 1 1 1 3 1 1 1MSassigned 1 1 3 1 2 2 1 3 2 3 3 1 1 1 2 1 1 2 1 2 2 1 1 1 1composite 1 1 3 1 2 2 1 2 2 3 3 1 1 1 2 1 1 2 1 2 2 1 1 1 1
HSassigned 1 2 2 2 1 2 3 2 1 1 2 2 1 1 1 1 1 2 2 1 1 1 1 1 1composite 1 2 3 2 1 2 1 1 1 1 3 2 1 1 1 1 1 2 2 1 1 1 1 1 1MSassigned 1 3 1 1 1 2 1 1 2 1 1 2 1 1 1 1 1 1 1 1 1 1 2 1 2composite 1 3 1 1 1 2 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 2
45
Texas Tech University, Cody Newman, December, 2014
G (2 of 2)
HSassigned 1 1 2 3 3 1 2 1 1 1 1 2 3 1 2 3 1 2 1 3 1 1 1 1 1 1 1 1 1 1 2 2 2 2 1 1 1 1 2 1 1 2 2 1 1 1 2 1 1 1 1composite 1 1 2 3 2 1 2 1 1 1 1 2 2 2 3 2 1 2 1 3 1 1 1 1 1 1 1 1 1 1 1 2 2 2 1 2 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1MSassigned 1 2 1 1 3 1 2 1 1 2 2 2 2 2 1 1 2 1 3 2 2 2 1 3 2 3 1 1 2 5 1 2 1 1 2 2 1 1 3 1 1 2 2 2 1 1 2 1 1 1 1composite 1 2 1 1 3 1 2 1 1 2 2 2 2 1 1 1 2 1 3 3 2 2 1 2 2 3 1 1 3 5 1 2 1 1 2 2 1 1 3 1 1 2 2 1 1 1 2 1 1 1 1
HSassigned 1 1 1 2 3 1 1 1 2 1 1 2 1 1 1 1 1 2 1 3 1 1 1 2 1 2 4 3 2 1 2composite 1 1 1 2 3 1 1 1 1 1 2 2 1 1 1 1 1 2 1 2 1 1 1 2 1 2 4 3 2 1 2MSassigned 2 1 1 1 1 3 1 2 1 2 3 2 1 1 1 1 1 1 2 2 1 2 2 2 2 1 1 2 1 1 1 1 2 1 2 2 2 3 2 1 2 2 1 2 1 1 1 3 1 1 1composite 2 1 2 1 1 3 1 2 1 1 2 2 1 1 1 1 1 1 1 1 1 2 2 2 2 1 1 2 1 1 1 1 2 1 2 2 2 3 1 1 2 2 1 1 1 1 1 3 1 1 1
HSassignedcompositeMSassigned 1 2 2 2 1 3 2 2 1 3 3 1 2 2 3 3 1 1 1 1 1 1 1 1 2 1 2 1 1 2 3 3 1 1 1 3 2 1 3 2 1 2 1 2 2 3 3 2 2composite 1 2 2 2 1 3 2 2 1 3 3 1 1 1 3 3 1 1 1 1 1 1 1 1 2 1 2 1 2 2 3 3 1 1 1 3 2 1 3 2 1 2 1 2 1 3 3 2 2
46
Texas Tech University, Cody Newman, December, 2014
Middle School Judge MEAN
TOTAL SUM OF RATINGS
TOTALNUMBER OF RATINGS
AAHS 65 HSassigned 1.569230769 102 assigned 1 2 2 1 1 2 1 1 1 1 1 1 1 1 2 1 1 1 1 2 1 3 1 2 1composite 1.584615385 103 composite 1 2 2 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 3 1 2 1MS 423 MSassigned 1.517730496 642 assigned 3 3 2 3 1 1 4 1 1 1 1 1 1 1 1 1 2 1 1 1 2 2 3 3 2composite 1.48463357 628 composite 3 3 2 3 1 2 5 1 1 1 1 1 1 1 1 1 2 1 1 2 3 2 3 3 1
HSassigned 3 2 2 1 2 4 1 3 2 1 1 1 2 2 1 1 1 1 3 1 2 1 3 2 2composite 3 2 2 1 2 4 2 3 2 1 1 1 2 2 1 1 1 1 3 1 2 1 3 2 2MSassigned 1 1 1 2 1 1 1 1 2 1 2 1 1 2 1 2 1 1 1 3 2 1 2 1 2composite 1 1 1 1 1 1 1 1 2 1 2 1 1 1 1 2 1 1 1 3 1 1 1 1 2
HSassigned 1 2 4 1 3 2 1 1 1 2 2 1 1 1 1composite 1 2 4 2 3 2 1 1 1 2 2 1 1 1 1MSassigned 1 1 1 1 1 3 1 1 2 2 1 2 1 1 2 1 1 1 4 1 2 1 2 3 1composite 1 1 1 1 1 3 1 1 2 2 1 2 1 1 2 1 1 1 4 1 2 1 2 2 1
47
Texas Tech University, Cody Newman, December, 2014
AA (2 of 4)
HSassignedcompositeMSassigned 2 1 1 1 1 1 1 1 2 1 3 3 2 1 3 2 1 3 1 1 1 1 2 2 1 1 2 1 1 1 2 3 1 2 2 1 2 2 2 1 1 1 1 2 1 1 1 1 3 1 4composite 1 1 1 1 1 1 1 1 2 1 2 3 2 1 3 2 2 4 1 1 1 1 2 2 1 1 2 1 1 1 2 3 2 3 3 1 2 2 2 1 1 1 1 2 1 1 1 1 3 1 3
HSassignedcompositeMSassigned 3 1 1 1 1 1 1 1 1 1 2 2 3 1 2 3 5 1 1 1 1 2 1 2 1 2 1 1 2 1 1 1 1 1 1 2 1 2 4 4 1 1 1 2 2 2 3 2 1 4 2composite 3 1 1 1 1 1 1 1 1 1 2 2 3 1 1 3 5 1 2 1 1 2 2 1 1 3 1 1 2 1 1 1 1 1 1 2 2 1 4 4 1 1 1 2 2 2 3 3 1 4 2
HSassignedcompositeMSassigned 2 1 1 2 2 1 1 1 1 1 1 2 1 2 1 2 1 2 2 1 1 1 1 2 1 2 1 1 1 1 1 2 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 2 3 3composite 3 1 1 1 2 1 1 1 1 1 1 2 1 1 1 2 1 2 2 2 1 1 1 2 1 2 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 3 2
48
Texas Tech University, Cody Newman, December, 2014
AA (3 of 4)
HSassignedcompositeMSassigned 1 2 1 2 2 1 1 2 2 1 3 1 2 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 3 2 1 1 3 1 3 1 3 2 1 2 1 2composite 2 2 1 2 2 1 1 2 2 1 3 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 3 2 1 1 2 1 3 1 3 1 1 2 1 2
HSassignedcompositeMSassigned 2 2 2 3 2 1 2 2 2 1 3 2 2 2 1 2 2 1 1 1 1 1 1 2 1 2 1 1 1 2 1 2 1 1 2 1 1 1 1 1 1 2 2 1 1 1 1 3 1 2 1composite 3 2 2 3 2 1 2 2 1 1 3 2 2 2 1 2 2 1 1 1 2 1 1 1 1 2 1 1 1 2 1 2 1 1 1 1 1 1 1 1 1 2 2 1 1 1 1 3 1 2 1
HSassignedcompositeMSassigned 2 2 2 2 2 2 1 2 1 2 1 2 2 1 1 1 1 1 1 2 1 1 3 2 2 1 1 1 1 2 2 1 1 1 1 1 1 3 2 1 2 2 3 1 2 3 3 2 2 1 2composite 2 1 2 2 3 1 1 2 1 2 1 2 2 1 1 1 1 1 1 2 1 1 3 2 2 1 1 1 1 2 2 1 1 1 1 1 1 3 2 1 2 2 2 1 1 3 3 1 2 1 1
49
Texas Tech University, Cody Newman, December, 2014
AA (4 of 4)
HSassignedcompositeMSassigned 1 2 2 1 1 1 1 2 2 2 1 1 1 1 2 2 1 3 1 1 1 2 1 1 2 2 1 3 2 2 1 1 1 1 1 2 1 1 1 1 1 1composite 1 2 2 1 1 1 1 1 2 1 1 1 1 1 2 2 1 2 1 1 1 1 1 1 2 2 1 3 2 2 1 1 1 1 1 2 1 1 1 1 1 1
HSassignedcompositeMSassignedcomposite
HSassignedcompositeMSassignedcomposite
50
Texas Tech University, Cody Newman, December, 2014
Middle School Judge MEAN
TOTAL SUM OF RATINGS
TOTALNUMBER OF RATINGS
BBHS 145 HSassigned 1.289655172 187 assigned 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 3 2 1composite 1.317241379 191 composite 1 1 1 1 1 1 1 1 1 2 1 1 2 1 1 1 1 1 1 1 1 2 3 2 1MS 616 MSassigned 1.50974026 930 assigned 3 3 3 2 1 2 5 1 1 1 1 1 1 1 1 2 1 2 3 1 1 2 1 1 2composite 1.5 924 composite 3 3 2 3 1 2 5 1 1 1 1 1 1 1 1 2 1 2 3 1 2 2 1 2 2
HSassigned 1 2 2 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1composite 1 2 2 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1MSassigned 1 2 1 1 1 1 1 2 1 1 2 1 3 2 1 1 2 2 1 2 2 1 1 1 1composite 1 2 1 1 1 2 1 3 1 1 2 1 2 2 1 1 1 2 1 2 2 1 1 1 1
HSassigned 1 2 3 2 2 1 1 1 1 1 1 1 1 1 1 1 1 3 3 1 1 1 2 1 1composite 1 2 3 2 2 1 1 1 1 1 1 1 1 1 1 1 1 3 3 1 1 1 1 1 2MSassigned 1 3 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1composite 1 3 1 1 1 2 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 2
51
Texas Tech University, Cody Newman, December, 2014
BB (2 of 6)
HSassigned 1 1 2 1 2 1 1 1 3 1 1 1 3 1 1 2 3 1 2 1 1 1 1 2 1 1 2 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 2 1 1composite 1 1 2 1 2 1 1 2 3 1 1 1 3 1 1 2 2 1 2 1 1 1 1 2 1 1 2 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 2 1 1MSassigned 1 1 1 1 2 2 4 3 3 1 1 1 2 2 1 1 1 1 4 1 2 2 1 1 1 1 1 1 1 2 1 1 1 1 1 2 1 1 1 2 1 1 2 1 1 1 1 2 1 1 2composite 1 1 1 1 2 2 4 3 3 1 1 1 2 2 1 1 1 1 4 1 2 2 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 2 1 1 2 1 1 1 1 2 1 1 2
HSassigned 1 1 1 1 1 1 1 2 1 2 1 2 2 1 3 3 1 2 2composite 1 1 1 2 1 1 1 2 1 2 1 2 2 1 3 3 1 2 2MSassigned 2 3 1 1 1 1 1 2 2 1 1 1 1 1 2 1 1 2 1 1 1 1 1 1 1 2 1 4 3 1 1 1 1 3 2 2 1 2 1 2 1 5 4 1 2 1 3 2 1 1 1composite 2 3 1 1 1 1 1 2 2 1 1 1 1 1 3 1 1 1 1 1 1 1 1 1 1 1 1 3 3 1 1 1 1 3 2 1 1 2 1 2 1 5 4 1 2 1 3 2 1 1 1
HSassignedcompositeMSassigned 2 1 1 1 2 1 1 1 2 2 1 1 1 1 2 1 2 1 3 1 1 4 2 1 3 1 1 1 1 2 4 3 1 1 1 3 2 1 1 1 2 1 2 1 1 3 1 1 1 1 1composite 2 1 1 1 2 1 1 1 2 1 1 1 1 1 2 1 2 1 3 1 1 4 2 1 2 1 1 1 1 2 4 3 1 1 1 3 2 1 1 1 2 1 2 1 1 3 1 1 1 1 1
52
Texas Tech University, Cody Newman, December, 2014
BB (3 of 6)
HSassignedcompositeMSassigned 2 1 3 2 3 2 1 1 1 2 1 1 1 2 2 1 1 1 1 2 1 2 1 3 1 1 4 2 1 3 1 1 1 1 2 4 3 1 1 1 3 2 1 1 1 2 1 2 1 1 3composite 1 1 3 1 2 2 1 1 1 2 1 1 1 2 1 1 1 1 1 2 1 2 1 3 1 1 4 2 1 2 1 1 1 1 2 4 3 1 1 1 3 2 1 1 1 2 1 2 1 1 3
HSassignedcompositeMSassigned 1 1 1 1 1 2 1 3 2 3 2 1 2 1 1 1 1 2 2 1 1 1 3 1 2 3 1 2 2 2 1 2 2 2 3 3 1 1 2 1 1 1 1 1 1 1 2 2 1 1 1composite 1 1 1 1 1 1 1 3 1 2 2 1 2 1 1 2 1 2 1 1 1 1 3 1 2 3 1 2 2 2 1 1 2 2 3 3 1 1 2 1 1 2 1 1 1 2 2 2 1 1 1
HSassignedcompositeMSassigned 2 1 3 1 2 2 1 1 2 1 1 1 2 1 1 1 1 2 1 1 1 1 1 1 1 2 3 1 2 2 2 2 1 1 2 2 2 1 1 2 1 1 3 2 1 2 1 2 1 1 1composite 2 1 3 1 2 1 1 1 2 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 3 3 2 1 1 1 2 2 1 1 1 1 1 1 3 2 1 2 1 2 1 1 2
53
Texas Tech University, Cody Newman, December, 2014
BB (4 of 6)
HSassignedcompositeMSassigned 2 3 1 4 2 1 2 1 1 1 1 3 1 5 1 1 2 2 1 2 2 1 2 1 1 2 2 1 2 3 1 1 1 1 1 1 1 1 1 2 2 1 1 1 1 2 2 1 1 3 1composite 2 2 1 4 2 1 2 1 1 1 2 3 1 5 1 1 2 1 1 1 3 2 2 1 1 2 2 2 2 3 1 1 1 1 1 1 1 1 1 2 2 1 1 1 1 2 2 1 1 3 1
HSassignedcompositeMSassigned 2 1 2 1 1 1 1 2 1 2 2 1 3 1 4 2 1 2 1 1 1 1 3 1 5 1 1 2 2 1 2 2 1 2 1 1 2 2 1 2 3 1 1 1 1 1 1 1 1 1 2composite 1 1 1 1 1 1 1 2 1 2 2 1 2 1 4 2 1 2 1 1 1 2 3 1 5 1 1 2 1 1 1 3 2 2 1 1 2 2 2 2 3 1 1 1 1 1 1 1 1 1 2
HSassignedcompositeMSassigned 2 1 1 1 1 2 2 1 1 3 1 2 1 2 1 1 1 1 2 1 2 2 1 1 2 2 1 2 2 1 2 3 1 1 1 1 3 1 2 2 2 1 1 1 2 1 1 1 1 2 1composite 2 1 1 1 1 2 2 1 1 3 1 1 1 1 1 1 1 1 2 1 2 2 1 1 2 2 1 2 2 1 2 2 1 1 2 1 3 1 2 2 2 1 1 1 1 1 1 2 1 2 1
54
Texas Tech University, Cody Newman, December, 2014
BB (5 of 6)
HSassignedcompositeMSassigned 2 1 1 2 1 3 3 2 1 2 2 1 4 1 1 1 2 3 1 1 1 2 2 1 2 4 2 1 2 2 1 2 2 3 1 4 3 1 1 1 1 1 1 2 1 1 1 2 2 1 2composite 2 1 1 2 1 3 3 2 1 2 2 1 4 1 1 1 1 3 1 1 1 3 2 1 2 4 2 1 2 2 1 2 2 2 1 4 3 1 1 1 1 1 1 2 1 1 1 2 2 1 2
HSassignedcompositeMSassigned 1 1 1 1 2 1 2 2 1 1 1 1 2 1 1 1 1 1 2 1 1 1 2 1 2 2 1 1 1 2 1 3 2 2 2 1 1 3 1 3 3 1 2 2 1 1 1 2 2 3 2composite 1 1 1 1 1 1 3 2 1 1 1 1 2 1 1 1 1 1 2 2 1 1 2 1 2 1 1 2 1 2 1 3 1 2 2 1 1 3 1 3 2 1 2 2 1 1 1 2 1 3 2
HSassignedcompositeMSassigned 3 4 2 1 1 1 1 2 3 1 2 2 1 1 1 3 1 1 1 1 1 1 1 2 2 1 2 1 1 2 2 2 2 1 1 1 1 1 1 1 1 1 2 1 1 2 1 1 1 2 3composite 3 3 3 1 1 1 1 2 3 1 2 2 1 1 1 3 1 1 1 1 1 1 1 2 2 1 2 1 1 2 2 2 2 1 1 1 1 1 1 1 1 1 2 1 1 2 2 1 1 2 2
55
Texas Tech University, Cody Newman, December, 2014
BB (6 of 6)
HSassignedcompositeMSassigned 1 1 1 1 2 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 1 1 1 2 2 1 1 1 1 1 3 1 2 1 1composite 1 1 1 1 2 1 1 1 2 1 1 1 1 1 1 1 1 1 2 2 1 1 1 1 2 2 2 2 2 2 2 1 1 1 2 2 1 1 2 2 1 3 1 2 1 1
56
Texas Tech University, Cody Newman, December, 2014
Middle School Judge MEAN
TOTAL SUM OF RATINGS
TOTALNUMBER OF RATINGS
CCHS 102 HSassigned 1.519607843 155 assigned 3 3 1 2 2 1 1 3 2 1 1 1 1 1 1 1 1 1 1 3 2 2 1 1 1composite 1.529411765 156 composite 4 4 1 1 2 1 1 2 2 1 1 1 1 1 1 1 1 1 1 3 2 3 1 1 1MS 212 MSassigned 1.485849057 315 assigned 5 1 1 1 1 1 3 2 1 2 1 1 3 1 3 1 2 3 3 4 4 1 2 2 2composite 1.509433962 320 composite 5 1 1 2 1 1 3 2 1 2 1 1 2 2 3 1 2 4 3 4 3 2 2 3 2
HSassigned 1 2 2 1 1 1 1 3 1 1 2 2 1 3 1 1 1 1 1 1 1 2 3 2 2composite 1 3 3 1 1 1 1 3 1 1 2 3 2 3 1 1 1 1 1 1 1 2 3 1 1MSassigned 3 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 1 1 1composite 3 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 1 1 1
HSassigned 1 1 3 3 2 1 2 1 1 4 1 1 1 1 1 1 1 1 1 2 1 3 1 2 3composite 1 1 2 3 1 1 2 1 1 3 1 1 1 1 1 1 1 1 1 2 1 3 2 1 2MSassigned 1 3 1 1 1 1 1 1 2 2 2 3 2 3 2 1 1 1 1 2 1 1 2 2 2composite 1 3 1 1 1 2 1 1 2 1 2 3 2 3 2 1 1 1 1 2 1 1 2 3 2
57
Texas Tech University, Cody Newman, December, 2014
CC (2 of 2)
HSassigned 1 3 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 2 1 2 2 1 3 3 1 2 2composite 1 3 1 1 2 1 1 1 1 1 1 1 1 1 2 2 1 2 1 2 2 1 3 3 1 2 2MSassigned 3 1 1 1 1 1 1 1 2 2 1 1 1 1 1 1 1 2 1 1 1 1 1 3 1 1 3 2 1 1 1 2 1 1 3 2 1 1 1 2 2 1 2 4 3 1 2 4 1 1 1composite 3 1 1 1 1 1 1 1 2 2 1 1 1 1 1 1 1 2 1 1 1 1 1 2 1 1 3 2 1 1 1 2 1 1 3 2 1 1 1 2 2 1 3 4 3 1 2 4 1 1 1
HSassignedcompositeMSassigned 1 1 1 1 2 1 1 1 1 1 2 1 1 1 2 1 1 1 1 1 1 2 2 2 1 5 1 1 3 4 1 2 2 1 2 3 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2composite 1 1 1 1 2 1 1 1 1 1 2 1 1 1 2 1 1 1 1 1 1 2 2 2 2 5 1 1 3 4 1 1 2 1 2 3 1 1 1 1 1 1 1 1 1 2 1 1 2 1 1
HSassignedcompositeMSassigned 1 2 1 1 1 1 2 2 1 1 1 2 1 1 2 1 1 1 2 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 2composite 1 2 1 1 1 1 2 3 1 1 1 2 3 1 2 1 1 1 2 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1
58
Texas Tech University, Cody Newman, December, 2014
Middle School Judge MEAN
TOTAL SUM OF RATINGS
TOTALNUMBER OF RATINGS
DDHS 26 HSassigned 1.038461538 27 assigned 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1composite 1.038461538 27 composite 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1MS 268 MSassigned 1.410447761 378 assigned 2 1 1 2 1 2 1 1 1 2 2 1 2 1 2 1 1 2 2 1 1 1 2 2 1composite 1.395522388 374 composite 1 1 1 1 1 2 1 1 1 1 2 2 2 1 2 1 1 2 2 1 1 1 1 2 1
HSassigned 1composite 1MSassigned 1 1 1 2 3 1 1 1 1 2 2 1 1 1 1 1 2 1 1 1 1 2 1 1 1composite 1 1 1 2 3 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1
HSassignedcompositeMSassigned 1 1 1 1 1 1 1 1 2 3 1 1 1 1 2 1 1 1 2 1 1 2 1 2 1composite 1 1 1 1 1 2 1 1 2 3 1 1 1 1 1 1 1 1 2 1 1 2 1 2 1
59
Texas Tech University, Cody Newman, December, 2014
DD (2 of 3)
HSassignedcompositeMSassigned 1 2 1 1 2 3 2 2 1 1 2 1 1 1 1 2 2 2 1 1 3 3 2 1 1 1 1 1 2 1 2 2 1 3 1 2 1 1 1 1 1 1 2 2 1 1 2 2 1 2 1composite 1 2 1 1 2 3 2 2 1 1 2 1 1 1 1 2 2 2 1 1 3 2 2 1 1 2 1 1 2 1 2 2 1 2 2 2 1 1 1 1 1 1 2 3 1 1 1 2 1 2 1
HSassignedcompositeMSassigned 2 2 1 1 3 3 2 2 2 1 1 2 1 3 1 2 1 2 1 1 1 2 1 1 1 2 2 3 3 1 1 1 1 1 1 2 2 1 2 1 1 2 2 2 1 1 2 2 2 1 1composite 2 2 1 1 3 3 2 2 2 1 1 2 1 3 1 2 1 3 1 1 2 2 1 1 1 2 1 3 3 1 1 1 1 1 1 2 2 1 2 1 1 2 2 2 2 1 1 2 1 1 1
HSassignedcompositeMSassigned 1 1 1 1 1 2 1 1 1 2 2 2 1 1 2 2 1 1 1 1 1 1 1 1 1 1 1 1 2 3 2 2 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 2 1 1 1composite 1 1 1 1 1 1 2 1 1 3 2 2 1 1 2 2 1 1 1 1 1 1 1 1 1 1 1 1 2 2 1 2 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1
60
Texas Tech University, Cody Newman, December, 2014
DD (3 of 3)
HSassignedcompositeMSassigned 1 1 3 1 2 1 2 3 2 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 2 1 1 1 2 1 2 1 2 1 3 2 1 1 1 2composite 2 1 3 1 2 1 2 3 3 1 1 1 2 1 1 1 1 1 1 2 1 1 1 1 2 1 1 1 2 1 2 1 2 1 3 2 1 1 1 2
HSassignedcompositeMSassignedcomposite
HSassignedcompositeMSassignedcomposite
61
top related