Research in Developmental Disabilities 33 (2012) 2067–2071

Contents lists available at SciVerse ScienceDirect

Research in Developmental Disabilities

Factor structure of the Matson Evaluation of Social Skills withYoungsters-II (MESSY-II)

Johnny L. Matson *, Daniene Neal, Julie A. Worley, Alison M. Kozlowski, Jill C. Fodstad

Department of Psychology, Louisiana State University, Baton Rouge, LA 70803, United States

A R T I C L E I N F O

Article history:

Received 17 May 2010

Accepted 27 September 2010

Available online 1 July 2012

Keywords:

Social skills

Assessment

MESSY

Rating scale

Factor structure

A B S T R A C T

The importance of social skills in development is a well studied area of research, and

deficits in these skills can have implications long into adulthood. Therefore, assessment

tools must be able to aid clinicians in identifying areas of weaknesses to target in

treatment. The purpose of the current paper was to investigate the factor structure of a

well researched measure of social skills, the Matson Evaluation of Social Skills with

Youngsters (MESSY), which has recently been re-normed to update its psychometric

properties. As such, this measure has now been dubbed the MESSY-II. This new norm

sample was utilized in the current study to determine whether the original two factor

structure for the MESSY would remain for the MESSY-II. Based on factor analysis, a three

factor model was found to be ideal. Implications of these findings are discussed.

� 2012 Elsevier Ltd. All rights reserved.

The development of social skills is an important process in young childhood and adolescence. Deficits present inchildhood that are left undetected and/or untreated can lead to increased problems into adulthood (Greene et al., 1999). Inaddition, impairments in social skills may be related to larger problems such as developmental disability, attention deficit/hyperactivity disorder (ADHD), depression, anxiety, antisocial behavior, and other mental health problems (Davis et al.,2011; Lugnegard, Hallenback, & Gillberg, 2011; Mahan & Matson, 2011; Matson & Wilkins, 2009; Worley & Matson, 2011).Social skills deficits may occur as a result of these disorders or as part of the disorders themselves. As a result, identification ofsocial weaknesses is essential for providing treatment and improving prognosis and quality of life. Identifying socialstrengths is also important for treatment and can guide clinicians to use assets that the child already possesses to helpimprove the areas of deficit.

The most common method of assessing social skills is the use of rating scales (Matson & Wilkins, 2009). While there aremany broad band measures that include a social skills subscale, there are few well researched scales assessing socialstrengths and weaknesses exclusively. The Matson Evaluation of Social Skills with Youngsters (MESSY; Matson, 1988) is onesuch rating scale that is intended to measure solely social behaviors, both positive (e.g., eye contact, smiling at others,helping others) and negative (e.g., bragging, getting into fights, lying). The MESSY was originally designed and used to assessthe social skills of typically developing children and adolescents, but it has also been researched with a number of varyingpopulations (e.g., developmental disabilities, hearing and vision impaired, intellectual disability, mental health disorders)and translated into several different languages. It has been found to have sound psychometric properties across studiesconducted over many years. However, in order for a measure to remain useful and current, the norms on which it is basedshould be updated and new studies regarding psychometrics and standardization need to be conducted. As such, the MESSY

* Corresponding author.

E-mail address: johnmatson@aol.com (J.L. Matson).

0891-4222/$ – see front matter � 2012 Elsevier Ltd. All rights reserved.

http://dx.doi.org/10.1016/j.ridd.2010.09.026

J.L. Matson et al. / Research in Developmental Disabilities 33 (2012) 2067–20712068

was recently renormed using a large sample of typically developing children (i.e., no prior psychiatric condition, medical/genetic disorder, physical disability, intellectual disability, or developmental delay) as determined via parent/legal guardianreport and/or record review (Matson et al., 2010). The revised scale will from this point on be referred to as the MESSY-II inorder to reflect the changes that have occurred since the MESSY’s inception. Only the parent form of the scale was consideredduring this renorming process. Although both parent and teacher forms of the original MESSY exist, the original MESSY hadbeen documented to be used largely in community and clinic settings while other assessments which focus on schoolbehaviors are used to assess social skills within the educational environment. Children included in the norming sample were2–16 years of age, thus expanding the lower limit target age range of the original MESSY scale which had originally beenaimed at assessing social skills in children 4–18 years of age. This revision was done in order to enhance the utility of thisscale as the assessment of social skills in children of younger and younger ages has been deemed most important. For thepurposes of the reliability and validity studies, the children included were broken down into age cohorts: 2–5 year olds, 6–9year olds, and 10–16 year olds. Initial studies have shown strong psychometric properties across age cohorts includinginternal consistency, and convergent and divergent validity.

The purpose of the current study was to examine the factor structure of the MESSY-II given the updated norm sample.When it was originally developed, a factor analysis of the MESSY yielded a two factor-structure: Appropriate Social Skills andInappropriate Assertiveness (Matson, Rotatori, & Helsel, 1983). Since the MESSY has recently been renormed and named theMESSY-II, however, there has not been updated research to examine whether this factor structure holds. An updated factorstructure based on the updated norms would likely assist in diagnosis by reflecting changes to the latent variable of socialskills since the MESSY’s inception nearly three decades ago. The social repertoire of children and adolescents since that timeis likely to have changed, thereby resulting in the previously found factor structure to be outdated. The goal of this paperthen, was to determine the new factor structure of the renormed MESSY-II.

1. Method

1.1. Participants

A sample of 886 children was selected from a larger group of 1065 children whose parents or caregivers completed theMESSY-II. For the purposes of this study, only children who were determined to be typically developing (i.e., no priorpsychiatric condition, medical/genetic disorder, physical disability, intellectual disability, or developmental delay) viarecord review and/or parent report were retained. Therefore, the 179 children who did not meet this inclusion criterion wereeliminated from further analyses. The ages of the children included in the study ranged from 2 through 16 (M = 7.90;SD = 3.71), reflecting the age range utility of the MESSY-II. Breakdown of gender within the sample was 44% male and 56%female. The ethnic representation of the sample consisted of children of Caucasian (83%), African American (8%), Hispanic(3%), and Other (6%) descent.

1.2. Measures

Matson Evaluation of Social Skills with Youngsters-II (MESSY-II; Matson et al., 2010). The MESSY-II is a social skills measurefor a broad range of children, ages 2–16, based on observations of both appropriate and inappropriate social behaviors. Thismeasure is a renormed version of the original MESSY, which was designed to assess social skills in children ages 4–18 (Matson,Rotatori, et al., 1983). The original scale had two different forms: a self-report form and parent/teacher report form consisting of62 and 64 items, respectively. At present, the MESSY-II only has one form, which is a parent/caregiver report form. During therenorming process it was decided that social skills would best be examined through parent/caregiver report as opposed to self-report due to difficulties with poor insight in the populations frequently administered the MESSY. Also, since the measure’sutility has largely been clinic and community focused, there is a decreased need for a teacher report form. The MESSY-II has 64items identical to the original MESSY parent/teacher report form, which are each rated on a Likert-type rating scale from 1 (‘‘notat all’’) to 5 (‘‘very much’’). Recent studies indicate that the scale has strong psychometric properties including internalconsistency, and convergent and divergent validity (Matson et al., 2010). Although the original MESSY parent/teacher reportform yielded a two factor structure (i.e., Inappropriate Assertiveness/Impulsiveness scale and Appropriate Social Skills scale),the factor structure of the MESSY-II has yet to be established.

1.3. Procedures

The participants for this investigation were recruited throughout the United States from a variety of clinic, community,and school settings via self or professional referral. Flyers describing the study and procedures for participation weredisseminated at the aforementioned settings. Participants came from sites in California, Connecticut, Florida, Georgia,Kentucky, Louisiana, Maryland, Massachusetts, Michigan, and Pennsylvania. The project was described as a study on thedevelopment of measures to assess the social and emotional behaviors of children and adolescents. As such, parents or legalguardians completed a battery of assessments which included, but was not limited to, a demographic questionnaire and anassessment of social skills in children and adolescents (i.e., MESSY-II). This investigation was approved by the Louisiana StateUniversity Institutional Review Board and informed consent was obtained from parents or legal guardians.

J.L. Matson et al. / Research in Developmental Disabilities 33 (2012) 2067–2071 2069

1.4. Data analysis

In order to determine the factor structure of the MESSY-II, an exploratory factor analysis with Principle axis factoring wasused on the 64 items of the MESSY-II. Given the likelihood of high correlations among the underlying constructs of thefactors, an oblique promax rotation was run (Fabrigar, Wegener, MacCallum, & Strahan, 1999). The optimal factor structurewas determined via examination of the scree plot, comprehensibility of factors, and Kaiser Criterion (Costello & Osborne,2005). Item correlation coefficients greater than .30 were retained for each factor (Field, 2005). Internal consistency of thefactors was examined using Chronbach’s alpha. Alpha coefficients of .80 and higher were considered indicative of strongsupport for the respective factor (Clark & Watson, 1995).

2. Results

Exploratory factor analysis yielded a three-factor solution for the MESSY-II. The total variance accounted for by the three-factor model was 32.06%. Factor 1 (Hostile) accounted for 17.54% of the variance. Factor 2 (Adaptive/Appropriate) accountedfor 8.80% percent of the variance. Factor 3 (Inappropriately Assertive/Overconfident) accounted for 5.73% of the variance. Atotal of seven items (i.e., item 1: Makes others laugh; item 9: Slaps or hits when angry; item 18: Walks up and initiatesconversation; item 20: Is afraid to speak to people; item 31: Shows feelings; item 35: Makes sounds that bother others; anditem 58: Explains things more than necessary) did not meet the criteria of .30, and were removed from the measure. Table 1lists the factors and corresponding items.

Next, internal consistency was examined for the three factors of the MESSY-II using Chrobach’s alpha. Factor 1 (Hostile)had an internal consistency of .92 (M = 42.60; SD = 12.89). Factor 2 (Adaptive/Appropriate) had an internal consistency of .88(M = 72.31; SD = 10.87). Factor 3 (Inappropriately Assertive/Overconfident) had an internal consistency of .86 (M = 58.70;SD = 7.66). Item-total correlations were considered for each of the retained factors to determine if the removal of additionalitems was warranted due to coefficients below .30 (Field, 2005). However, no item on any of the scales had a correlation ofless than .30 and, therefore, all items were retained following the seven items that had been removed during the exploratoryfactor analysis.

3. Discussion

Deficits in broad and specific social skills can impede development in other areas of functioning. Thus, an assessment toolmeasuring social skills (adaptive and maladaptive skills) is necessary to determine if a child or adolescent demonstratesappropriate social skills for their developmental age. A social skills measure is useful to identify strengths and weaknesses insocial skills and can be utilized repeatedly over the course of treatment (e.g., social skills treatment) to monitor progress andtreatment gains.

The purpose of this paper then, was to determine the factor solution of a measure used to assess social skills, the MESSY-II(Matson et al., 2010). The original MESSY was initially developed nearly three decades ago, but recently renormed (seeMatson et al., 2010). Exploratory factor analysis of the MESSY-II yielded a three factor solution. Two of the factors wereconsistent with inappropriate social skills while the other consisted of items relating to appropriate and adaptive socialskills. More specifically, factor 1 consisted of items representing anger, teasing, and jealousy and was therefore named‘‘Hostile.’’ Factor 2 included items representing prosocial skills relating to helping others, manners, and being respectful andwas labeled ‘‘Adaptive/Appropriate.’’ Factor 3 was comprised of items that related to being domineering, inflexible, andstubborn and was labeled ‘‘Inappropriately Assertive/Overconfident.’’ The factor structure of the MESSY-II appears to berobust as it measures important dimensions of social skills.

Although the initial two factor solution for the original MESSY (Matson, Rotatori, et al., 1983) did not remain, whatremained consistent was that both inappropriate and appropriate social skills loaded onto separate factors. It is suspectedthat the single inappropriate social skills factor from the original MESSY was broken down into two separate inappropriatesocial skills factors targeting specific dimensions of inappropriate social skills due to differences in patterns of inappropriatebehavior that may have arisen in the past three decades. Furthermore, this revised factor structure will greatly assist in socialskills interventions by aiding clinicians in identifying the exact domain of maladaptive social skills with which individualsare exhibiting excesses. Such information may prove beneficial in guiding specific treatment interventions. Secondly, whatalso did not remain consistent was the number of items retained in the final measure. That is, six items were removed due tolow factor loadings. The low factor loadings of these items indicated less importance of the items to the specific factors, andtherefore, the removal of them made the scale more parsimonious.

Although this study further highlights the usefulness of the MESSY-II in the assessment of social skills, the current study isnot without limitations. First, the collection sites used to recruit participants varied (i.e., clinic, community, and schoolsettings). While not ideal, potential variance due to this limitation was somewhat negated by the use of only parents or legalguardians serving as respondents. This prevented the use of teachers, clinicians, or other professionals working with thechildren as informants. However, the use of both parent and legal guardians was another possible limitation of the currentstudy. Where possible, parent reports were utilized; however, in extenuating circumstances the legal guardians were neededto complete the measures. Although this may be viewed as a potential limitation, in these situations, the legal guardian wasconsidered the most appropriate individual to complete the assessment measures for the child. Furthermore, this reflects

Table 1

Factor structure of the MESSY-II.

Item Factor 1: Hostile Factor 2:

Adaptive/Appropriate

Factor 3: Inappropriately

Assertive/Overconfident

53 Is jealous of other people .758

23 Makes fun of others .738

64 Wants to get even with someone who hurts him/her .646

13 Picks out other Children’s faults/mistakes .644

43 Always thinks something bad is going to happen .638

12 Feels angry or jealous when someone else does well .616

36 Brags too much when he/she wins .603

52 Gets in fights a lot .597

21 Hurts others’ feelings on purpose .587

63 Hurts others’ feelings when teasing them .579

32 Thinks people are picking on him/her when they are not .572

44 Tries to be better than everyone else .541

30 Acts as if he/she better than others .531

46 Feels lonely .515

5 Gripes or complains often .503

61 Talks a lot about problems or worries .500

3 Becomes angry easily .493

15 Breaks promises .480

16 Lies to get what he/she wants .468

11 Gives other children dirty looks .465

24 Blames others for own problems .465

17 Picks on people to make them angry .432

8 Brags about self .429

60 Hurts others to get what he/she wants .397

62 Thinks that winning is everything .391

2 Threatens people or acts like a bully .379

22 Is a sore loser .343

54 Does nice things for others who are nice to him/her .723

40 Asks if he/she can be of help .704

41 Feels good if he/she helps others .668

56 Asks others how they are, what they have been doing, etc. .645

25 Sticks up for friends .623

34 Works well on a team .609

10 Helps a friend who is hurt .597

19 Says ‘‘thank you’’ and is happy when someone does

something for him/her

.551

59 Is friendly to new people he/she meets .541

50 Joins in games with other children .539

33 Thinks good things are going to happen .534

47 Feels sorry when he/she hurts others .523

28 Smiles at people he/she knows .495

51 Plays by the rules of a game .471a �.425

37 Takes care of others’ property as if it were his/her own .471a �.476

26 Looks at people when they are speaking .385

45 Asks questions when talking with others .322a .316

39 Calls people by their names .311

42 Defends self .305

14 Always wants to be first .684

4 Is bossy (tells people what to do instead of asking) .647

6 Speaks (breaks in) when someone else is speaking .646

55 Tries to get others to do what he/she wants .598

38 Speaks too loudly .596

7 Takes or uses things that are not his/hers

without permission

.595

27 Thinks he/she knows it all .545

49 Likes to be the leader .357 .522a

29 Is stubborn .479

48 Gets upset when he/she has to wait for things .472

57 Stays with others too long (wears out welcome) .375a Item assignment for items that cross-loaded.

J.L. Matson et al. / Research in Developmental Disabilities 33 (2012) 2067–20712070

situations that would be encountered in clinical settings using the MESSY-II. Lastly, the total variance explained by the threefactor solution was less than 50%. However, in order to explain the variance with as few factors as possible, a three factorsolution provided the best trade-off between total variance explained and straightforwardness of the scale.

Reliable assessments of social skills remain important. Just as important for clinicians is a measure that assists indetermining children and adolescents who have more profound difficulties/deficits in social skills. Thus, it seems imperativefor future studies to examine potential cutoff scores of the MESSY-II. In addition, previous studies have been conducted

J.L. Matson et al. / Research in Developmental Disabilities 33 (2012) 2067–2071 2071

examining the usefulness of the original MESSY with specific populations aside from the typically developing population:autism (Matson, Stabinsky-Compton, & Sevin, 1991), visually impaired (Matson, Heinze, Helsel, Kapperman, & Rotatori,1986), and hearing impaired (Matson, Macklin, & Helsel, 1985). Therefore, the renormed MESSY-II should again beinvestigated for these populations to determine its continued applicability outside of the typically developing population.

References

Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309–319.Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical

Assessment, Research, & Evaluation, 10, 1–9.Davis, T. E., III, Hess, J. A., Moree, B. N., Fodstad, J. C., Dempsey, T., Jenkins, W. S., et al. (2011). Anxiety symptoms across the lifespan in people diagnosed with

autism disorder. Research in Autism Spectrum Disorders, 5, 112–118.Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological

Methods, 4, 272–299.Field, A. (2005). Discovering statistics using SPSS. London: Sage Publications Inc.Greene, R. W., Biederman, J., Farone, S. U., Wilens, T. E., Mick, E., & Blier, H. K. (1999). Further validation of social impairment as predictor of substance use

disorders. Findings from a sample of siblings of boys with and without ADHD. Journal of Clinical Child Psychology, 28, 349–354.Lugnegard, T., Hallerback, M. U., & Gillberg, C. (2011). Psychiatric comorbidity in young adults with a clinical diagnosis of Asperger’s syndrome. Research in

Developmental Disabilities, 32, 1910–1917.Mahan, S., & Matson, J. L. (2011). Children and adolescents with autism spectrum disorders copared to typically developing controls on the Behavioral Assessment

system for children, Second Edition (BASC-2). Research in Autism Spectrum Disorders, 5, 230–236.Matson, J. L. (1988). The Matson Evaluation of Social Skills with Youngsters (MESSY). Worthington, OH: International Diagnostic Systems.Matson, J. L., Heinze, A., Helsel, W. J., Kapperman, G., & Rotatori, A. (1986). Assessing social behaviors in the visually handicapped: The Matson Evaluation of Social

Skills with Youngsters (MESSY). Journal of Clinical Child Psychology, 15, 78–87.Matson, J. L., Macklin, G. F., & Helsel, W. J. (1985). Psychometric properties of the Matson Evaluation of Social Skills with Youngsters (MESSY) with emotional

problems and self-concept in deaf children. Journal of Behavior Therapy and Experimental Psychiatry, 16, 117–123.Matson, J. L., Neal, D., Hess, J. A., Fodstad, J. C., Mahan, S., & Rivet, T. T. (2010). Reliability and Validity of the Matson Evaluation of Social Skills with Youngsters

(MESSY). Behavior Modification, 34, 539–558.Matson, J. L., Rotatori, A. F., & Helsel, W. J. (1983). Development of a rating scale to measure social skills in children: The Matson Evaluation of Social Skills with

Youngsters (MESSY). Behaviour Research Therapy, 21, 335–340.Matson, J. L., Stabinsky-Compton, L., & Sevin, J. A. (1991). Comparison and item analysis of the MESSY for autistic and normal children. Research in Developmental

Disabilities, 12, 361–369.Matson, J. L., & Wilkins, J. (2009). Psychometric testing methods for children’s social skills. Research in Developmental Disabilities, 30, 249–274.Worley, J. A., & Matson, J. L. (2011). Psychiatric symptoms in children diagnosed with an autism spectrum disorder: An examination of gender differences. Research

in Autism Spectrum Disorders, 3, 1086–1091.