Article
Limited Nomination Reliability Using Single- and Multiple-item Measures
Ben Babcock,
Peter E. L. Marks, Nicki R. Crick, Antonius H. N. Cillessen,
Corresponding Author
Ben Babcock
The American Registry of Radiologic Technologists
Correspondence should be addressed to Ben Babcock, ARRT, 1255 Northland Drive, St. Paul, MN 55120, USA. Email: [email protected]Search for more papers by this authorBen Babcock,
Peter E. L. Marks, Nicki R. Crick, Antonius H. N. Cillessen,
Corresponding Author
Ben Babcock
The American Registry of Radiologic Technologists
Correspondence should be addressed to Ben Babcock, ARRT, 1255 Northland Drive, St. Paul, MN 55120, USA. Email: [email protected]Search for more papers by this authorAbstract
This article examines a variety of reliability issues as related to limited nomination sociometric measures. Peer nomination data were collected from 77 sixth grade classrooms. Results showed that, although some single-item peer nomination measures were relatively reliable, many single-item peer nomination measures using limited nominations were quite unreliable. Overt aggression nomination items were the only set of single-item measures where mean classroom reliability estimates were .75 or greater. Combining multiple items led to substantially better reliability, as combining the two least reliable items for a category into a single measure made the composite more reliable than the most reliable single measure. Having more nominators in the sample also increased reliability. The limited nomination items overall tended to be less reliable than similar unlimited nomination items from other studies. The authors end with recommendations for obtaining the most reliable peer nomination data possible from a study.
References
-
Babad, E. (2001). On the conception and measurement of popularity: More facts and some straight conclusions. Social Psychology of Education, 5, 3–29. doi: 10.1023/A:1012780232587.
10.1023/A:1012780232587 Google Scholar
- Becker, B. E., & Luthar, S. S. (2007). Peer-perceived admiration and social preference: Contextual correlates of positive peer regard among suburban and urban adolescents. Journal of Research on Adolescence, 17, 117–144. doi: 10.1111/j.1532-7795.2007.00514.x.
-
Bronfenbrenner, U. (1944). A constant frame of reference for sociometric research: Part II: Experiment and inference. Sociometry, 7, 40–75. doi: 10.2307/2785536.
10.2307/2785536 Google Scholar
- Brown, W. (1910). Some experimental results in the correlation of mental abilities. British Journal of Psychology, 3, 296–322. doi: 10.1111/j.2044-8295.1910.tb00207.x.
- Cillessen, A. H. N., & Marks, P. E. L. (2011). Conceptualizing and measuring popularity. In A. H. N. Cillessen, D. Schwartz, & L. Mayeux (Eds.), Popularity in the peer system (pp. 25–56). New York: Guildord Press.
-
Cohen, J. (1968). Multiple regression as a general data-analytic system. Psychological Bulletin, 6, 426–443. doi: 10.1037/h0026714.
10.1037/h0026714 Google Scholar
- Coie, J. D., Dodge, K. A., & Coppotelli, H. (1982). Dimensions and types of status: A cross-age perspective. Developmental Psychology, 18, 557–570. doi: 10.1037/0012-1649.18.4.557.
- Crick, N. R. (1997). Engagement in gender normative versus nonnormative forms of aggression: Links to social–psychological adjustment. Developmental Psychology, 33, 610–617. doi: 10.1037/0012-1649.33.4.610.
- Crick, N., & Grotpeter, J. (1995). Relational aggression, gender, and social-psychological adjustment. Child Development, 66, 710–722. doi: 10.1111/j.1467-8624.1995.tb00900.x.
- Crocker, L., & Algina, J. (1986). Introduction to classical & modern test theory. Belmont, CA: Wadsworth Group.
- Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. doi: 10.1007/BF02310555.
- Eng, E., & French, R. L. (1948). The determination of sociometric status. Sociometry, 11, 368–371. doi: 10.2307/2785197.
- Furr, R. M., & Bacharach, V. R. (2008). Psychometrics: An introduction. Thousand Oaks, CA: Sage.
-
Hartshorne, H., & May, M. A. (1929). Studies in the nature of character II: Studies in service and self-control. New York: Macmillan. doi: 10.1037/11334-000.
10.1037/11334‐000 Google Scholar
- Jarecky, R. K. (1959). Identification of the socially gifted. Exceptional Children, 25, 415–419.
- LaFontana, K. M., & Cillessen, A. H. N. (2002). Children's perceptions of popular and unpopular peers: A multimethod assessment. Developmental Psychology, 38, 635–647. doi: 10.1037/0012-1649.38.5.635.
- Lattin, J., Carroll, J. D., & Green, P. E. (2003). Analyzing multivariate data. Pacific Grove, CA: Brooks/Cole-Thompson Learning.
- Liu, Y., & Salvendy, G. (2009). Effects of measurement errors on psychometric measurements in ergonomics studies: Implications for correlations, ANOVA, linear regression, factor analysis, and linear discriminant analysis. Ergonomics, 52, 499–511. doi: 10.1080/00140130802392999.
- Malcolm, K. T., Jensen-Campbell, L. A., Rex-Lear, M., & Waldrip, A. M. (2006). Divided we fall: Children's friendships and peer victimization. Journal of Social and Personal Relationships, 23, 721–740. doi: 10.1177/0265407506068260.
- Marks, P. E., Babcock, B., Cillessen, A. H. N., & Crick, N. R. (2013). The effects of participation rate on the internal reliability of peer nomination measures. Social Development. doi: 10.1111/j.1467-9507.2012.00661.
- McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
-
Moreno, J. L. (1934). Who shall survive? A new approach to the problem of human relations. Washington, DC: Nervous and Mental Disease Publishing Company. doi: 10.7326/0003-4819-8-1-104_2.
10.7326/0003‐4819‐8‐1‐104_2 Google Scholar
- Parkhurst, J. T., & Asher, S. R. (1992). Peer rejection in middle school: Subgroup differences in behavior, loneliness, and interpersonal concerns. Developmental Psychology, 28, 231–241. doi: 10.1037//0012-1649.28.2.231.
- Peery, J. C. (1979). Popular, amiable, isolated, rejected: A reconceptualization of sociometric status in preschool children. Child Development, 50, 1231–1234. doi: 10.2307/1129356.
- Rose, A. J., Swenson, L. P., & Waller, E. M. (2004). Overt and relational aggression and perceived popularity: Developmental differences in concurrent and prospective relations. Developmental Psychology, 40, 378–387. doi: 10.1037/0012-1649.40.3.378.
- Schafer, J. S., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 147–177. doi: 10.1037/1082-989X.7.2.147.
-
Spearman, C. (1907). Demonstration of formulæ for true measurement of correlation. American Journal of Psychology, 18, 161–169. doi: 10.2307/1412408.
10.2307/1412408 Google Scholar
- Spearman, C. C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 3, 271–295. doi: 10.1111/j.2044-8295.1910.tb00206.x.
-
Terry, R. (2000). Recent advances in measurement theory and the use of sociometric techniques. In A. H. N. Cillessen & W. M. Bukowski (Eds.), Recent advances in the measurement of acceptence and rejection in the peer system. New directions for child and adolescent development (Vol. 88, pp. 27–53). San Francisco, CA: Jossey-Bass. doi: 10.1002/cd.23220008805.
10.1002/cd.23220008805 Google Scholar
- Waldrip, A. M., Malcolm, K. T., & Jensen-Campbell, L. A. (2008). With a little help from your friends: The importance of high-quality friendships on early adolescent adjustment. Social Development, 17, 832–852. doi: 10.1111/j.1467-9507.2008.00476.x.
- Wang, S. S., Houshyar, S., & Prinstein, M. J. (2006). Adolescent girls' and boys' weight-related health behaviors and cognitions: Associations with reputation- and preference-based peer status. Health Psychology, 25, 658–663. doi: 10.1037/0278-6133.25.5.658.
-
Zeleny, L. D. (1940). Measurement of social status. American Journal of Sociology, 45, 576–582. doi: 10.1086/218376.
10.1086/218376 Google Scholar
