PREDICTING LONG-TERM FIREFIGHTER PERFORMANCE FROM COGNITIVE AND PHYSICAL ABILITY MEASURES
and requests for reprints should be addressed to Norman D. Henderson, Department of Psychology, Severance Laboratory, Oberlin College, Oberlin, OH 44074, USA; [email protected].
Abstract
Firefighters from 1 academy training class were observed for 23 years, beginning with their selection test consisting of a g-saturated written exam (GCA) and firefighting simulations loaded on a strength/endurance (SE) factor. Operational validity coefficients for both GCA and SE were high for training success and remained consistently high for job performance ratings throughout the study. The operational validity for combined GCA and SE predictors was .86 for a composite job rating measure covering 21 years of service. A structural model produced similar results for more broadly defined GCA and SE latent variables. Both analyses suggested approximately equal weighting for GCA and SE for a fire service selection test. Results indicate considerable latitude in choosing cognitive and physical predictors for firefighter screening if the predictors are highly loaded on GCA and SE.
REFERENCES
- Arnold JD, Rauschenberger JM, Soubel WG, Guion RM. (1982). Validation and utility of a strength test for steelworkers. Journal of Applied Psychology, 67, 588–604.
- Arvey RD, Nutting SM, Landon TE. (1992). Validation strategies for physical ability testing in police and fire settings. Public Personnel Management, 21, 301–312.
- Ästrand P, Rodhal K. (1986). Textbook of work physiology. New York : McGraw-Hill.
- Barrett GV, Polomsky MD, McDaniel MA. (1999). Selection tests for firefighters: A comprehensive review and meta-analysis. Journal of Business and Psychology, 13, 507–514.
- Beaton RU, Murphy S, Salazar M, Johnson LC. (2002). Neck, back, and shoulder pain complaints in urban firefighters: The benefits of aerobic exercise. Journal of Musculoskeletal Pain, 10, 57–67.
- Berger RA. (1963). Classification of students on the basis of strength. Research Quarterly, 34, 514–515.
- Berk RA. (1996). Standard setting: The next generation (where few psychometricians have gone before!). Applied Measurement in Education, 9, 215–235.
- Berkman v. City of New York, 536 F. Supp. 177 (E. D. N. Y. 1982).
- Biddle DA, Nooren PM. (2006). Validity generalization vs. Title VII: Can employers successfully defend tests without conducting local validation studies? Labor Law Journal, 57, 216–237.
- Blakley BR, Quinones MA, Crawford MS, Jago IA. (1994). The validity of isometric strength tests. Personnel Psychology, 47, 247–274.
- Bobko P. (1983). An analysis of correlations corrected for attenuation and range restriction. Journal of Applied Psychology, 68, 584–589.
- Bownas DA, Heckman RW. (1976). Job analysis of the entry level firefighter position. (Report No. TS 77–8). Washington , DC : U. S. Office of Personnel Management.
- Bradley et al. 443 F. Supp. 2d 145 (D. Mass. 2006).
- Brown LE, Weir JP. (2001). ASEP procedures recommendation I: Accurate assessment of muscular strength and power. Journal of Exercise Physiology online, 4, 1–21.
- Browne MW, Cudeck R. (1993). Alternative ways of assessing model fit. In KA Bollen, JS Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park , CA : Sage.
- Cady LD, Bischoff DP, O’Connell ER, Thomas BA, Allan JH. (1979). Strength and fitness and subsequent back injuries in firefighters. Journal of Occupational Medicine, 21, 269–272.
-
Campbell JP. (1982). Some remarks from an outgoing editor.
Journal of Applied Psychology, 67, 691–700.
10.1037/h0077946 Google Scholar
-
Carroll JB. (1993). Human cognitive abilities.
Cambridge
: Cambridge University Press.
10.1017/CBO9780511571312 Google Scholar
- Clark W. (1973). Firefighter principles and practices. New York : Dunn-Donnelley.
- Cleary TA, Humphreys LG, Kendrick SA, Wesman A. (1975). Educational uses of tests with disadvantaged students. American Psychologist, 30, 15–41.
- Cortez L. (2001). Fire company staffing requirements: An analytical approach. Fire Technology, 37, 199–218.
- Cotton DJ. (1971). A modified step test for group cardiovascular testing. Research Quarterly, 42, 91–95.
- Coward WM, Sackett PR. (1990). Linearity of ability-performance relationships: A reconfirmation. Journal of Applied Psychology, 75, 297–300.
- Cronbach LJ. (1975). Five decades of public controversy over mental testing. American Psychologist, 30, 1–14.
- Cucina JM, Vasilopoulos NL. (2005). Non-linear personality-performance relationships and the spurious moderating effects of traitedness. Journal of Personality, 71, 227–259.
- Davis JA, Wilmore JH. (1990). Validation of a step test for cardiorespiratory fitness classification of emergency service personnel. Journal of Occupational Medicine, 21, 671–673.
- Davis PO, Dotson CO, Santa Maria DL. (1982). Relationship between simulated fire fighting tasks and physical performance measures. Medicine and Science in Sports and Exercise, 14, 65–71.
- Dawes RM, Corrigan B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106.
- Doolittle TL, Kaiyala K. (1986). Strength and musclo-skeletal injuries of firefighters. Proceedings of the Annual Conference of the Human Factors Association of Canada, 49–52. Vancouver , BC .
- Draper N, Smith H. (1981). Applied regression analysis. ( Revised ed.). New York : Wiley.
- EEOC v. Atlas Paper, 868 F. 2d 1487 (6th Cir. 1989), cert. denied, 493, U.S. 814.
- Fiske DW. (1971). Measuring the concepts of personality. Chicago : Aldine-Atherton.
- Grofman B, Merrill S. (2004). Anticipating likely consequences of lottery-based affirmative action. Social Science Quarterly, 85, 1447–1468.
- Gutman A. (2009). Major EEO issues relating to personnel selection decisions. Human Resource Management Review, 19, 232–250.
- Harbin G, Olson J. (2005). Post-offer, pre-placement testing in industry. American Journal of Industrial Medicine, 47, 296–307.
- Henderson ND. (1985). Validity and utility of a cognitive-physical screening test for firefighters. (Technical report). Oberlin , OH: Henderson & Associates .
- Henderson ND, Berry MW, Matic T. (2007). Field measures of strength and fitness predict firefighter performance on physically demanding tasks. Personnel Psychology, 60, 431–473.
- Hoffman CC. (1999). Generalizing physical ability test validity: A case study using test transportability, validity generalization, and construct related validation evidence. Personnel Psychology, 52, 1019–1042.
- Hogan J. (1991a). Physical abilities. In MD Dunnette, LM Hough (Eds.), Handbook of industrial-organizational psychology ( 2nd ed., pp. 753–831). Palo Alto , CA : Consulting Psychologists Press.
- Hogan J. (1991b). The structure of physical performance in occupational tasks. Journal of Applied Psychology, 76, 495–507.
- Hunter JE, Schmidt FL. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Beverly Hills , CA : Sage.
- Hunter JE, Schmidt FL. (1996). Intelligence and job performance: Economic and social implications. Psychology, Public Policy and Law, 2, 447–472.
-
Hunter JE,
Schmidt FL. (2004). Methods of meta-analysis: Correcting error and bias in research findings.
Thousand Oaks
,
CA
: Sage.
10.4135/9781412985031 Google Scholar
- Hunter JE, Schmidt FL, Le H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91, 594–612.
- International Public Management Association for Human Resources. (1996). B3R and B4R Technical Report. Alexandria , VA : Author.
- International Public Management Association for Human Resources. (2009). B-5/B-5a and 300 Series Technical Report. Alexandria , VA : Author.
- Jackson AS. (1994). Preemployment physical evaluation. Exercise and sport sciences reviews, 22, 53–90.
- Jöreskog KG, Sörbom D. (1996). Structural equation modeling. Workshop presented for the NORC Social Science Research Professional Development Training Sessions. Chicago , IL .
- Klein GA. (1996). The effect of acute stressors in decision making. In JE Driskell, E Salas. (Eds.), Stress and human performance (pp. 49–88). Mahwah , NJ : LEA
- Landy, Jacobs, Associates. (1987). Report on the criterion-related validity of the physical capabilities test (PCT) for entry level firefighter positions in Columbus, Ohio. State College , PA : Author.
- Lanning v. Southeastern Pa. Transp. Auth. (Lanning I), 181 F .3d 478 (3d Cir. 1999).
- Lanning v. Southeastern Pa. Transp. Auth. (Lanning II), 308 F .3d 286, 290 (3d Cir. 2002).
- Larsen GE, George JD, Alexander JL, Fellingham GW, Aldana SG, Parcell AC. (2002). Prediction of maximum oxygen consumption from walking, jogging, or running. Research Quarterly for Exercise and Sport, 73, 66–72.
- Le HA. (2003). Correcting for indirect range restriction in meta-analysis: Testing a new meta-analytic method. Unpublished doctoral dissertation, University of Iowa .
- Lubinski D. (2004). Introduction to the special section on cognitive abilities: 100 years after Spearman's (1904) “general intelligence,” objectively determined and measured. Journal of Personality and Social Psychology, 86, 96–111.
- Luke v. City of Cleveland, 2006 WL 43759 (N.D. Ohio).
- MacCallum RC, Austin JT. (2000). Applications of structural equation modeling in psychological research. In ST Fiske, DL Schacter, C Zahn-Waxler (Eds.), Annual review of psychology (Vol. 51, pp. 201–226). Palo Alto , CA : Annual Reviews, Inc.
- MacCallum RC, Widaman KF, Zhang S, Hong S. (1999). Sample size in factor analysis. Psychological Methods, 4, 84–99.
- Marsh HW. (1993). The multidimensional structure of physical fitness invariance over gender and age. Research Quarterly for Exercise and Sport, 64, 256–273.
- Mead TP, Legg DL. (1994). Exploratory versus confirmatory factor analysis of collegiate physical fitness. Education Resources Information Center, (ED 379336), 1–10.
- Medbo JI, Tabata I. (1989). Relative importance of aerobic and anaerobic energy release during short-lasting exhausting bicycle exercise. Journal of Applied Physiology, 67, 1881–1886.
- Mendoza JL, Bard DE, Mumford MD, Ang SC. (2004). Criterion-related validity in multiple hurdle designs: Estimation and bias. Organizational Research Methods, 7, 418–441.
- Mendoza JL, Mumford M. (1987). Correction for attenuation and range restriction on the predictor. Journal of Educational Statistics, 12, 282–293.
-
Meyers DC,
Gebhardt DL,
Crump CE,
Fleishman EA. (1993). The dimensions of human physical performance: Factor analyses of strength, stamina, flexibility and body composition measures.
Human Performance, 6, 309–344.
10.1207/s15327043hup0604_2 Google Scholar
- Morgeson FP, Campion MA, Dipboye RL, Hollenbeck JR, Murphy K, Schmitt N. (2007a). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology, 60, 683–729.
- Morgeson FP, Campion MA, Dipboye RL, Hollenbeck JR, Murphy K, Schmitt N. (2007b). Are we getting fooled again? Coming to terms with limitations in the use of personality tests for personnel selection. Personnel Psychology, 60, 1029–1049.
- Murphy KR, Dzieweczynski JL. (2005). Why don't measures of broad dimensions of personality perform better as predictors of job performance? Human Performance, 18, 343–357.
- Orasanu J. (1997). Stress and naturalistic decision making: Strengthening the weak links. In R Flin, E Salas, M Strub, L Martin (Eds.), Decision making under stress: Emerging themes and applications (pp. 43–66). Ashgate : Aldershot.
- Potosky D, Bobko P, Roth PL. (2005). Forming composites of cognitive ability and alternative measures to predict job performance and reduce adverse impact: Corrected estimates and realistic expectations. International Journal of Selection and Assessment, 13, 304–315.
- Rhea MR, Alvar BA, Gray R. (2004). Physical fitness and job performance of firefighters. Journal of Strength and Conditioning Research, 18, 348–352.
- Rosenfeld M, Thornton RF. (1976). The development and validation of a firefighter physical selection test for the City of Philadelphia. Princeton , NJ : Educational Testing Service, Center for Occupational and Professional Assessment.
- Rosse JG, Stecher MD, Miller JL, Levin RA. (1998). The impact of response distortion on preemployment personality testing and hiring decisions. Journal of Applied Psychology, 83, 634–644.
- Ryan AM, Greguras GJ, Ployhart RE. (1996). Perceived job relatedness of physical ability testing for firefighters: Exploring variations in reactions. Human Performance, 9, 219–240.
- Ryan AM, Ployhart RE, Friedel LA. (1998). Using personality testing to reduce adverse impact: A cautionary note. Journal of Applied Psychology, 83, 298–307.
- Sackett PR, Lievens F, Berry CM, Landers RN. (2007). A cautionary note on the effects of range restriction on predictor intercorrelations. Journal of Applied Psychology, 92, 538–544.
- Sackett PR, Yang H. (2000). Correction for range restriction: An expanded typology. Journal of Applied Psychology, 85, 112–118.
- Sarno MR. (2003). Issues in the third circuit: Employers who implement pre-employment tests to screen their applicants, beware (or not?): An analysis of Lanning v. Southeastern Pennsylvania Transportation Authority and the business necessity defense as applied in Third Circuit employment discrimination cases. Villanova Law Review, 48, 1403–1428.
- Schmidt FL, Hunter J. (2004). General mental ability in the world of work: Occupational attainment and job performance. Journal of Personality and Social Psychology, 86, 162–173.
- Schmidt FL, Hunter J, Outerbridge AN, Goff S. (1988). Joint relation of experience and ability with job performance: Test of three hypotheses. Journal of Applied Psychology, 73, 46–57.
- Schmidt FL, Shaffer JA, Oh IS. (2008). Increased accuracy for range restriction corrections: Implications for the role of personality and general mental ability in job and training performance. Personnel Psychology, 61, 827–868.
- Schmitt N, Rogers W, Chan D, Sheppard L, Jennings D. (1997). Adverse impact and predictive efficiency of various predictor combinations. Journal of Applied Psychology, 82, 719–730.
- Sharkey BJ, Davis PO. (2008). Hard work: Defining physical work performance requirements. Champaign , IL : Human Kinetics.
- Shaver LG. (1971). Maximum dynamic strength, relative dynamic endurance, and their relationships. Research Quarterly, 42, 460–465.
- Sothmann MS, Gebhardt DL, Baker TA, Kastello GM, Sheppard VA. (2004). Performance requirements of physically strenuous occupations: Validating minimum standards for muscular strength and endurance. Ergonomics, 47, 864–875.
- Stauffer JM, Mendoza JL. (2001). The proper sequence for correcting correlation coefficients for range restriction and unreliability. Psychometrika, 66, 63–68.
- Steiger JH, Lind JM. (1980). Statistically based tests for the number of common factors. Paper presented at the annual meeting of the Psychometric Society, Iowa City , Iowa .
- Stevenson JM, Weber CL, Smith JT, Dumas GA, Albert WJ. (2001). A longitudinal study of the development of low back pain in an industrial population. Spine, 26, 1370–1377.
-
Terman LM. (1917). A trial of mental and pedagogical tests in a civil service examination for policemen and fireman.
Journal of Applied Psychology, 1, 17–29.
10.1037/h0073841 Google Scholar
- United States v. City of New York, 637 F. Supp. 2d 77 (E.D.N.Y. 2009).
-
Wilcox RR. (2001). Fundamentals of modern statistical methods: Substantially improving power and accuracy.
New York
: Springer-Verlag.
10.1007/978-1-4757-3522-2 Google Scholar
- Winer BJ, Brown DR, Michels KM. (1991). Statistical principles in experimental design. New York : McGraw-Hill.
- Winkelspecht C, Lewis P, Thomas A. (2006). Potential effects of faking on the NEO-PI-R: Willingness and ability to fake changes who gets hired in simulated selection decisions. Journal of Business and Psychology, 21, 243–259.
- Wolff WM, North, AJ. (1951). Selection of municipal firemen. Journal of Applied Psychology, 35, 25–29.
