Development and Validation of the Cebeci Test of Creativity: A Computerized Test of Figural Creativity
Sukru Murat Cebeci
Renzulli Learning, New Haven, Connecticut, USA
Search for more papers by this authorCorresponding Author
Selcuk Acar
Department of Educational Psychology, University of North Texas, Denton, Texas, USA
Correspondence:
Selcuk Acar ([email protected])
Search for more papers by this authorSukru Murat Cebeci
Renzulli Learning, New Haven, Connecticut, USA
Search for more papers by this authorCorresponding Author
Selcuk Acar
Department of Educational Psychology, University of North Texas, Denton, Texas, USA
Correspondence:
Selcuk Acar ([email protected])
Search for more papers by this authorABSTRACT
This study presents the Cebeci Test of Creativity (CTC), a novel computerized assessment tool designed to address the limitations of traditional open-ended paper-and-pencil creativity tests. The CTC is designed to overcome the challenges associated with the administration and manual scoring of traditional paper and pencil creativity tests. In this study, we present the first validation of CTC, demonstrating strong internal and external validity across two studies with a large sample size of over 14,000 students in grades 1–8. The results provide support for the proposed unidimensional factor structure of CTC, with robust reliability (ω = 0.833 and 0.872). Analyses of measurement invariance showed that the unidimensional factor structure of CTC holds consistently across all grade levels, with factor loadings exhibiting notable similarity. Additionally, the item intercepts demonstrate considerable uniformity across grades 3–5. The composite CTC scores were positively correlated with creative self-efficacy but not with Standard Progressive Matrices. The outcomes of our study indicate that CTC is a valuable and efficient tool for assessing creativity in educational settings. Its scalability and comprehensive evaluation of four key dimensions of creative ideation (i.e., fluency, flexibility, originality, and elaboration) make it particularly advantageous for educators seeking to assess students' creative potential.
Conflicts of Interest
The first author is also the developer of the Cebeci Test of Creativity, and the second author served as a consultant for this validation study.
Open Research
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
- Acar, S. 2023. “Does the Task Structure Impact the Fluency Confound in Divergent Thinking? An Investigation With TTCT-Figural.” Creativity Research Journal 35, no. 1: 1–14. https://doi.org/10.1080/10400419.2022.2044656.
- Acar, S., A. M. A. Alabbasi, M. A. Runco, and K. Beketayev. 2019. “Latency as a Predictor of Originality in Divergent Thinking.” Thinking Skills and Creativity 33: 100574. https://doi.org/10.1016/j.tsc.2019.100574.
10.1016/j.tsc.2019.100574 Google Scholar
- Acar, S., K. Berthiaume, K. Grajzel, D. Dumas, C. T. Flemister, and P. Organisciak. 2023. “Applying Automated Originality Scoring to the Verbal Form of Torrance Tests of Creative Thinking.” Gifted Child Quarterly 67, no. 1: 3–17. https://doi.org/10.1177/00169862211061874.
- Acar, S., D. Dumas, P. Organisciak, and K. Berthiaume. 2024. “Measuring Original Thinking in Elementary School: Development and Validation of a Computational Psychometric Approach.” Journal of Education & Psychology 116: 953–981. https://doi.org/10.1037/edu0000844.
- Acar, S., L. E. Lee, and R. Scherer. 2024. “A Reliability Generalization of the Torrance Tests of Creative Thinking-Figural.” European Journal of Psychological Assessment 40, no. 5: 396–411. https://doi.org/10.1027/1015-5759/a000819.
- Acar, S., U. Ogurlu, and A. Zorychta. 2023. “Exploration of Discriminant Validity in Divergent Thinking Tasks: A Meta-Analysis.” Psychology of Aesthetics, Creativity, and the Arts 17, no. 6: 705–724. https://doi.org/10.1037/aca0000469.
- Acar, S., P. Organisciak, and D. Dumas. 2024. “Automated Scoring of Figural Tests of Creativity With Computer Vision.” Journal of Creative Behavior 59, no. 1: e677. https://doi.org/10.1002/jocb.677.
- Acar, S., and M. A. Runco. 2014. “Assessing Associative Distance Among Ideas Elicited by Tests of Divergent Thinking.” Creativity Research Journal 26, no. 2: 229–238. https://doi.org/10.1080/10400419.2014.901095.
- Acar, S., M. A. Runco, and H. Park. 2020. “What Should People Be Told When They Take a Divergent Thinking Test? A Meta-Analytic Review of Explicit Instructions for Divergent Thinking.” Psychology of Aesthetics, Creativity, and the Arts 14, no. 1: 39–49. https://doi.org/10.1037/aca0000256.
- Agresti, A. 2002. Categorical Data Analysis. Wiley.
10.1002/0471249688 Google Scholar
- An, D., Y. Song, and M. Carr. 2016. “A Comparison of Two Models of Creativity: Divergent Thinking and Creative Expert Performance.” Personality and Individual Differences 90: 78–84. https://doi.org/10.1016/j.paid.2015.10.040.
- Anderson, L. W., D. R. Krathwohl, P. W. Airasian, et al. 2001. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives (Complete Edition). Longman.
- Backes, B., and J. Cowan. 2019. “Is the Pen Mightier Than the Keyboard? The Effect of Online Testing on Measured Student Achievement.” Economics of Education Review 68: 89–103. https://doi.org/10.1016/j.econedurev.2018.12.007.
- Baghdarnia, M., R. F. Soreh, and R. Gorji. 2014. “The Comparison of Two Methods of Maximum Likelihood (ML) and Diagonally Weighted Least Squares (DWLS) in Testing Construct Validity of Achievement Goals.” Journal of Educational and Management Studies 4, no. 1: 22–38.
- Barbot, B. 2018. “The Dynamics of Creative Ideation: Introducing a New Assessment Paradigm.” Frontiers in Psychology 9: 2529. https://doi.org/10.3389/fpsyg.2018.02529.
- Beaty, R. E., D. R. Johnson, D. C. Zeitlen, and B. Forthmann. 2022. “Semantic Distance and the Alternate Uses Task: Recommendations for Reliable Automated Assessment of Originality.” Creativity Research Journal 34, no. 3: 1–16. https://doi.org/10.1080/10400419.2022.2025720.
- Browne, M. W., and R. Cudeck. 1993. “ Alternative Ways of Assessing Model Fit.” In Testing Structural Equation Models, edited by K. A. Bollen and J. S. Long, 136–162. Sage.
- Bump, W. M. 1994. “A Comparative Analysis of Spelling Skills Utilizing a Computer-Based Spelling Assessment Instrument.” Unpublished doctoral dissertation, Texas A&M University.
- Burke, H. R. 1972. “Raven's Progressive Matrices: Validity, Reliability, and Norms.” Journal of Psychology 82, no. 2: 253–257. https://doi.org/10.1080/00223980.1972.9923815.
10.1080/00223980.1972.9923815 Google Scholar
- Burke, H. R., and W. C. Bingham. 1969. “Raven's Progressive Matrices: More on Construct Validity.” Journal of Psychology 72, no. 2: 247–251.
10.1080/00223980.1969.10543505 Google Scholar
- Chen, F. F. 2007. “Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance.” Structural Equation Modeling: A Multidisciplinary Journal 14, no. 3: 464–504. https://doi.org/10.1080/10705510701301834.
- Christensen, P. R., J. P. Guilford, and R. C. Wilson. 1957. “Relations of Creative Responses to Working Time and Instructions.” Journal of Experimental Psychology 53, no. 2: 82–88. https://doi.org/10.1037/h0045461.
- Clark, L. A., and D. Watson. 1995. “Constructing Validity: Basic Issues in Objective Scale Development.” Psychological Assessment 7, no. 3: 309–319. https://doi.org/10.1037/1040-3590.7.3.309.
- Clark, P. M., and H. L. Mirels. 1970. “Fluency as a Pervasive Element in the Measurement of Creativity.” Journal of Educational Measurement 7, no. 2: 83–86. https://doi.org/10.1111/j.1745-3984.1970.tb00699.x.
- Costa, P. T., Jr., and R. R. McCrae. 1992. NEO-PI–R Professional Manual. Psychological Assessment Resources.
- Cropley, D. H., and R. L. Marrone. 2025. “Automated Scoring of Figural Creativity Using a Convolutional Neural Network.” Psychology of Aesthetics, Creativity, and the Arts 19, no. 1: 77–86. https://doi.org/10.1037/aca0000510.
- Cruz-Cázares, C., C. Bayona-Sáez, and T. García-Marco. 2013. “You Can't Manage Right What You Can't Measure Well: Technological Innovation Efficiency.” Research Policy 42, no. 6–7: 1239–1250. https://doi.org/10.1016/j.respol.2013.03.012.
- Davis, G. A. 1989. “Testing for Creative Potential.” Contemporary Educational Psychology 14, no. 3: 257–274. https://doi.org/10.1016/0361-476X(89)90014-3.
- Dumas, D., P. Organisciak, and M. Doherty. 2021. “Measuring Divergent Thinking Originality With Human Raters and Text-Mining Models: A Psychometric Comparison of Methods.” Psychology of Aesthetics, Creativity, and the Arts 15, no. 4: 645–663. https://doi.org/10.1037/aca0000319.
- Erwin, J. O., and F. C. Worrell. 2012. “Assessment Practices and the Underrepresentation of Minority Students in Gifted and Talented Education.” Journal of Psychoeducational Assessment 30, no. 1: 74–87. https://doi.org/10.1177/0734282911428197.
- Forthmann, B., C. Szardenings, and H. Holling. 2020. “Understanding the Confounding Effect of Fluency in Divergent Thinking Scores: Revisiting Average Scores to Quantify Artifactual Correlation.” Psychology of Aesthetics, Creativity, and the Arts 14, no. 1: 94–112. https://doi.org/10.1037/aca0000196.
- Forthmann, B., S. Weiss, and B. Goecke. 2025. “A Cognitive Interpretation Is Not at Odds With Equal Odds: A Latent Variable Investigation of Creative Thinking.” Imagination, Cognition and Personality 44, no. 4: 362–386. https://doi.org/10.1177/02762366241311561.
10.1177/02762366241311561 Google Scholar
- Gold, A. H., A. Malhotra, and A. H. Segars. 2001. “Knowledge Management: An Organizational Capabilities Perspective.” Journal of Management Information Systems 18, no. 1: 185–214. https://doi.org/10.1080/07421222.2001.11045669.
- Grajzel, K., S. Acar, and G. Singer. 2023. “The Big Five and Divergent Thinking: A Meta-Analysis.” Personality and Individual Differences 214: 112338. https://doi.org/10.1016/j.paid.2023.112338.
- Guilford, J. P. 1950. “Creativity.” American Psychologist 5, no. 9: 444–454.
- Guilford, J. P. 1968. Intelligence, Creativity, and Their Educational Implications. Robert R. Knapp.
- Guilford, J. P. 1973. Characteristics of Creativity. Illinois State Office of the Superintendent of Public Instruction, Gifted Children Section.
- Guilford, J. P., P. R. Christensen, P. R. Merrifield, and R. C. Wilson. 1960. Alternative Uses Manual. Sheridan Supply Co.
- Guo, J. 2019. “Web-Based Creativity Assessment System That Collects Both Verbal and Figural Responses: Its Problems and Potentials.” International Journal of Information and Education Technology 9, no. 1: 27–34.
10.18178/ijiet.2019.9.1.1168 Google Scholar
- Harrington, D. M. 1975. “Effects of Explicit Instructions to “Be Creative” on the Psychological Meaning of Divergent Thinking Test Scores.” Journal of Personality 43, no. 3: 434–454. https://doi.org/10.1111/j.1467-6494.1975.tb00715.x.
- Hocevar, D. 1979. “Ideational Fluency as a Confounding Factor in the Measurement of Originality.” Journal of Educational Psychology 71, no. 2: 191–196. https://doi.org/10.1037/0022-0663.71.2.191.
- Horkay, N., R. E. Bennett, N. Allen, B. Kaplan, and F. Yan. 2006. “Does It Matter if I Take My Writing Test on Computer? An Empirical Study of Mode Effects in NAEP.” Journal of Technology, Learning, and Assessment 5, no. 2: n2. https://doi.org/10.1002/j.2162-6057.1998.tb00809.x.
10.1002/j.2162?6057.1998.tb00809.x Google Scholar
- Jäger, A. O., H. Holling, F. Preckel, et al. 2005. “Berliner Intelligenzstruktur-Test Für Jugendliche: Begabungs- Und Hochbegabungsdiagnostik (BIS-HB) [Berlin Structure-of-Intelligence Test for Youth: Diagnosis of Talents and Giftedness].” Hogrefe.
- Jöreskog, K. 2001. “Analysis of Ordinal Variables 2: Cross-Sectional Data.” Unpublished Manuscript. http://www.ssicentral.com/lisrel/ordinal.htm.
- Kettner, N. W., J. P. Guilford, and P. R. Christensen. 1959. “A Factor-Analytic Study Across the Domains of Reasoning, Creativity, and Evaluation.” Psychological Monographs: General and Applied 73, no. 9: 1–31. https://doi.org/10.1037/h0093745.
10.1037/h0093745 Google Scholar
- Kim, K. H. 2017. “The Torrance Tests of Creative Thinking-Figural or Verbal: Which One Should We Use?” Creativity. Theories–Research-Applications 4, no. 2: 302–321. https://doi.org/10.1515/ctra-2017-0015.
10.1515/ctra-2017-0015 Google Scholar
- Kline, R. B. 2011. Principles and Practice of Structural Equation Modeling. Guilford Press.
- Kwon, M., E. T. Goetz, and R. D. Zellner. 1998. “Developing a Computer-Based TTCT: Promises and Problems.” Journal of Creative Behavior 32, no. 2: 96–106.
- Kwon, M. C. 1996. An Exploratory Study of a Computerized Creativity Test: Comparing Paper-Pencil and Computer-Based Versions of the Torrance Tests of Creative Thinking. Texas A&M University.
- Lau, S., and P. C. Cheung. 2010. “Creativity Assessment: Comparability of the Electronic and Paper-And-Pencil Versions of the Wallach–Kogan Creativity Tests.” Thinking Skills and Creativity 5, no. 3: 101–107. https://doi.org/10.1016/j.tsc.2010.09.004.
- Lubart, T. I., M. Besançon, and B. Barbot. 2011. Evaluation Du Potential Créatif (EPoC). Editions Hogrefe France.
- Luria, S. R., R. L. O'Brien, and J. C. Kaufman. 2016. “Creativity in Gifted Identification: Increasing Accuracy and Diversity.” Annals of the New York Academy of Sciences 1377, no. 1: 44–52. https://doi.org/10.1111/nyas.13136.
- Mazzeo, J., and A. L. Harvey. 1988. “The Equivalence of Scores From Automated and Conventional Educational and Psychological Tests: A Review of the Literature.” ETS Research Report Series 1988, no. 1: i–27.
10.1002/j.2330-8516.1988.tb00277.x Google Scholar
- McBee, M. T., S. J. Peters, and C. Waterman. 2014. “Combining Scores in Multiple-Criteria Assessment Systems: The Impact of Combination Rule.” Gifted Child Quarterly 58, no. 1: 69–89. https://doi.org/10.1177/0016986213513794.
- McCrae, R. R. 1987. “Creativity, Divergent Thinking, and Openness to Experience.” Journal of Personality and Social Psychology 52, no. 6: 1258–1265. https://doi.org/10.1037/0022-3514.52.6.1258.
- McDonald, R. P. 1999. Test Theory: A Unified Treatment. Erlbaum.
- Mindrila, D. 2010. “Maximum Likelihood (ML) and Diagonally Weighted Least Squares (DWLS) Estimation Procedures: A Comparison of Estimation Bias With Ordinal and Multivariate Non-Normal Data.” International Journal of Digital Society 1, no. 1: 60–66.
10.20533/ijds.2040.2570.2010.0010 Google Scholar
- OECD. 2023. “PISA 2022 Assessment and Analytical Framework.” OECD Publishing. https://doi.org/10.1787/dfe0bf9c-en.
10.1787/dfe0bf9c?en Google Scholar
- Organisciak, P., S. Acar, D. Dumas, and K. Berthiaume. 2023. “Beyond Semantic Distance: Automated Scoring of Divergent Thinking Greatly Improves With Large Language Models.” Thinking Skills and Creativity 49: 101356. https://doi.org/10.1016/j.tsc.2023.101356.
- Osborn, A. F. 1963. Applied Imagination: Principles and Procedures of Creative Problem-Solving. Scribner.
- Palaniappan, A. K. 2012. “Web-Based Creativity Assessment System.” International Journal of Information and Education Technology 2, no. 3: 255–258. https://doi.org/10.7763/IJIET.2012.V2.123.
10.7763/IJIET.2012.V2.123 Google Scholar
- Parnes, S. J. 1961. “Effects of Extended Effort in Creative Problem Solving.” Journal of Educational Psychology 52, no. 3: 117–122. https://doi.org/10.1037/h0044650.
- Partnership for 21st Century Skills. 2008. “21st Century Skills, Education & Competitiveness: A Resource and Policy Guide.” https://files.eric.ed.gov/fulltext/ED519337.pdf.
- Patterson, J. D., B. Barbot, J. Lloyd-Cox, and R. Beaty. 2022. “AuDrA: An Automated Drawing Assessment Platform for Evaluating Creativity.” https://psyarxiv.com/t63dm.
- Paulus, D. H., and J. S. Renzuli. 1968. “Scoring Creativity Tests by Computer.” Gifted Child Quarterly 12, no. 2: 79–83. https://doi.org/10.1177/001698626801200202.
10.1177/001698626801200202 Google Scholar
- Pender, W. C. 2020. “The Impact of Computer Based Versus Paper Pencil on PARCC Test Results for Illinois Public Elementary Schools (Doctoral Dissertation, University of St. Francis).”
- Peters, S. J. 2022. “The Challenges of Achieving Equity Within Public School Gifted and Talented Programs.” Gifted Child Quarterly 66, no. 2: 82–94. https://doi.org/10.1177/00169862211002535.
- Puccio, G. J. 2017. “From the Dawn of Humanity to the 21st Century: Creativity as an Enduring Survival Skill.” Journal of Creative Behavior 51, no. 4: 330–334. https://doi.org/10.1002/jocb.203.
- Raven, J. 2000. “The Raven's Progressive Matrices: Change and Stability Over Culture and Time.” Cognitive Psychology 41, no. 1: 1–48. https://doi.org/10.1006/cogp.1999.0735.
- Raykov, T. 1997. “Estimation of Composite Reliability for Congeneric Measures.” Applied Psychological Measurement 21, no. 2: 173–184.
- Reiter-Palmon, R., B. Forthmann, and B. Barbot. 2019. “Scoring Divergent Thinking Tests: A Review and Systematic Framework.” Psychology of Aesthetics, Creativity, and the Arts 13, no. 2: 144–152. https://doi.org/10.1037/aca0000227.
- Richardson, A. G. 1986. “Two Factors of Creativity.” Perceptual and Motor Skills 63, no. 2: 379–384. https://doi.org/10.2466/pms.1986.63.2.379.
- Runco, M. A. 1986. “Flexibility and Originality in Children's Divergent Thinking.” Journal of Psychology 120, no. 4: 345–352. https://doi.org/10.1080/00223980.1986.9712632.
10.1080/00223980.1986.9712632 Google Scholar
- Runco, M. A. 1992. “Children's Divergent Thinking and Creative Ideation.” Developmental Review 12, no. 3: 233–264. https://doi.org/10.1016/0273-2297(92)90010-Y.
10.1016/0273-2297(92)90010-Y Google Scholar
- Runco, M. A., and S. Acar. 2012. “Divergent Thinking as an Indicator of Creative Potential.” Creativity Research Journal 24, no. 1: 66–75. https://doi.org/10.1080/10400419.2012.652929.
- Runco, M. A., and R. S. Albert. 1985. “The Reliability and Validity of Ideational Originality in the Divergent Thinking of Academically Gifted and Nongifted Children.” Educational and Psychological Measurement 45, no. 3: 483–501. https://doi.org/10.1177/001316448504500306.
- Said-Metwaly, S., B. Fernández-Castilla, E. Kyndt, W. Van den Noortgate, and B. Barbot. 2021. “Does the Fourth-Grade Slump in Creativity Actually Exist? A Meta-Analysis of the Development of Divergent Thinking in School-Age Children and Adolescents.” Educational Psychology Review 33: 275–298. https://doi.org/10.1007/s10648-020-09547-9.
- Schroeders, U., and O. Wilhelm. 2010. “Testing Reasoning Ability With Handheld Computers, Notebooks, and Paper and Pencil.” European Journal of Psychological Assessment 26, no. 4: 284–292. https://doi.org/10.1027/1015-5759/a000038.
- Shoemaker, A. L., and D. M. Bolt. 1992. “Computer Measurement of the Autokinetic Effect.” Perceptual and Motor Skills 75, no. 3: 771–777. https://doi.org/10.2466/pms.1992.75.3.771.
- Silvia, P. J., E. C. Nusbaum, C. Berg, et al. 2009. “Openness to Experience, Plasticity, and Creativity: Exploring Lower-Order, High-Order, and Interactive Effects.” Journal of Research in Personality 43, no. 6: 1087–1090.
- Simonton, D. K. 2004. Creativity in Science: Chance, Logic, Genius, and Zeitgeist. Cambridge University Press. https://doi.org/10.1017/CBO9781139165358.
10.1017/CBO9781139165358 Google Scholar
- Sung, Y.-T., H.-H. Cheng, H.-C. Tseng, et al. 2024. “Construction and Validation of a Computerized Creativity Assessment Tool With Automated Scoring Based on Deep-Learning Techniques.” Psychology of Aesthetics, Creativity, and the Arts 18, no. 4: 493–509. https://doi.org/10.1037/aca0000450.
- Tierney, P., and S. M. Farmer. 2002. “Creative Self-Efficacy: Its Potential Antecedents and Relationship to Creative Performance.” Academy of Management Journal 45, no. 6: 1137–1148. https://doi.org/10.5465/3069429.
- Torrance, E. P. 1966. The Torrance Tests of Creative Thinking-Norms-Technical Manual Research Edition-Verbal Tests, Forms A and B – Figural Tests, Forms A and B. Personnel Press.
- Torrance, E. P. 1968. “A Longitudinal Examination of the Fourth Grade Slump in Creativity.” Gifted Child Quarterly 12, no. 4: 195–199.
10.1177/001698626801200401 Google Scholar
- Torrance, E. P. 1998. Torrance Test of Creative Thinking: Manual for Scoring and Interpreting Results, Verbal Forms A&B. Scholastic Testing Service.
- Torrance, E. P., and O. E. Ball. 1984. The Torrance Tests of Creative Thinking: Streamlined Scoring Guide Figural A and B. Scholastic Testing Service.
- Urban, K. K., and H. G. Jellen. 1996. Test for Creative Thinking - Drawing Production (TCT-DP). Swets and Zeitlinger.
- Van der Ven, A. H. G. S., and J. L. Ellis. 2000. “A Rasch Analysis of Raven's Standard Progressive Matrices.” Personality and Individual Differences 29, no. 1: 45–64. https://doi.org/10.1016/S0191-8869(99)00177-4.
- Wallach, M. A., and N. Kogan. 1965. Modes of Thinking in Young Children: A Study of the Creativity-Intelligence Distinction. Holt, Rinehart & Winston.
- Ward, T. B., and Y. Kolomyts. 2019. “ Creative Cognition.” In The Cambridge Handbook of Creativity, edited by J. C. Kaufman and R. J. Sternberg, 175–199. Cambridge University Press. https://doi.org/10.1017/9781316979839.011.
10.1017/9781316979839.011 Google Scholar
- Wilson, R. C., J. P. Guilford, P. R. Christensen, and D. J. Lewis. 1954. “A Factor-Analytic Study of Creative-Thinking Abilities.” Psychometrika 19: 297–311. https://doi.org/10.1007/BF02289230.
- Wollscheid, S., J. Sjaastad, C. Tømte, and N. Løver. 2016. “The Effect of Pen and Paper or Tablet Computer on Early Writing–A Pilot Study.” Computers & Education 98: 70–80.
- Zabramski, S. 2014. “Creating Digital Traces of Ideas: Evaluation of Computer Input Methods in Creative and Non-Creative Drawing (Doctoral Dissertation, Acta Universitatis Upsaliensis).”
- Zarnegar, Z., D. Hocevar, and W. B. Michael. 1988. “Components of Original Thinking in Gifted Children.” Educational and Psychological Measurement 48, no. 1: 5–16. https://doi.org/10.1177/001316448804800103.