Assessing Integrated Skills
Abstract
A recent trend in language assessment has been to assess examinees’ abilities to integrate source reading or listening material into their writing and speaking performance in ways that simulate the cognitive, communication, and literacy demands of real-life academic or vocational tasks. This chapter describes the justifications for this innovation in interactionist theories of communication, cognition, and assessment; through appeals to authenticity and relevance; and as a means to counter the effects of testing methods. The chapter explains how and why integrated skills assessment differs from the convention of assessing writing, speaking, reading, and listening as separate skills but also produces unique complications for measurement, test design, and examinee performance. Future directions are proposed to refine the constructs and tasks guiding integrated skills assessment, to increase their value for diagnostic purposes, and to verify their proposed benefits for teaching, learning, and high stakes decision making such as university admissions or employment.
References
- Anderson, J. R. (1995). Learning and memory. New York, NY: John Wiley.
- Artemeva, N., & Fox, J. (2010). Awareness vs. production: Probing students’ antecedent genre knowledge. Journal of Written and Business Communication, 24, 476–515.
- Bachman, L. (1990). Fundamental considerations in language testing. Oxford, England: Oxford University Press.
- Bereiter, C. (2002). Education and mind in the knowledge age. Mahwah, NJ: Lawrence Erlbaum.
- Byrnes, H. (2008). Assessing content and language. In E. Shohamy & N. H. Hornberger (Eds.), Encyclopedia of language and education. Volume 7: Language testing and assessment ( 2nd ed., pp. 37–52). New York, NY: Springer.
- Byrnes, H., Maxim, H., & Norris, J. (2010). Realizing advanced foreign language writing development in collegiate education: Curricular design, pedagogy, assessment (Monograph). Modern Language Journal, 94, Suppl. 1.
- Carroll, J. B. (1975). The teaching of French as a foreign language in eight countries. New York, NY: John Wiley and Sons.
-
Chalhoub-Deville, M.
(2003).
Second language interaction: Current perspectives and future trends.
Language Testing, 20, 369–83.
10.1191/0265532203lt264oa Google Scholar
-
Charge, N., &
Taylor, L.
(1997).
Recent developments in IELTS.
ELT Journal, 51, 374–80.
10.1093/elt/51.4.374 Google Scholar
-
Colpin, M., &
Gysen, S.
(2006).
Developing and introducing task-based language tests. In van den Branden, K. (2006),
Task-based language education (pp. 151–74).
New York, NY:
Cambridge University Press.
10.1017/CBO9780511667282.008 Google Scholar
- B. Cope, & M. Kalantazis (Eds.) (2000). Multiliteracies: Literacy learning and the design of social futures. London, England: Routledge.
- Cumming, A. (2013). Assessing integrated writing tasks for academic purposes: Promises and perils. Language Assessment Quarterly, 10, 1–18.
-
Cumming, A.,
Grant, L.,
Mulcahy-Ernt, P., &
Powers, D.
(2004).
A teacher-verification study of speaking and writing prototype tasks for a new TOEFL.
Language Testing, 21, 159–97. (See also TOEFL Monograph Report 26 at http://www.ets.org/research/policy_research_reports/rm-04-05_toefl-ms-26)
10.1191/0265532204lt278oa Google Scholar
-
Cumming, A.,
Kantor, R.,
Baba, K.,
Erdosy, U.,
Eouanzoui, K., &
James, M.
(2005).
Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL.
Assessing Writing, 10, 5–43.
10.1016/j.asw.2005.02.001 Google Scholar
-
Cumming, A.,
Rebuffot, J., &
Ledwell, M.
(1989).
Reading and summarizing challenging texts in first and second languages.
Reading and Writing: An Interdisciplinary Journal, 2, 201–19.
10.1007/BF00377643 Google Scholar
- Davies, A. (2008). Assessing academic English: Testing English proficiency 1950–2005, the IELTS solution. Cambridge, England: Cambridge University Press.
- Esmaeili, H. (2002). Integrated reading and writing tasks and ESL students’ reading and writing performance in an English language test. Canadian Modern Language Review, 58, 599–622.
- Flowerdew, J., & Li, Y. (2007). Language re-use among Chinese apprentice scientists writing for publication. Applied Linguistics, 28, 440–65.
- Frost, K., Elder, C., & Wigglesworth, G. (2012). Investigating the validity of an integrated listening–speaking task: A discourse-based analysis of test takers’ oral performances. Language Testing, 29, 345–69.
-
B. Harley,
P. Allen,
J. Cummins, &
M. Swain (Eds.).
(1990).
The development of second language proficiency.
New York, NY:
Cambridge University Press.
10.1017/CBO9781139524568 Google Scholar
- Harwood, N., & Petric, B. (2012). Performance in the citing behavior of two student writers. Written Communication, 29, 55–103.
- Hawkey, R. (2004). A modular approach to testing English language skills: The development of the Certificates in English Language Skills (CELS) examination. Cambridge, England: Cambridge University Press.
- Hidi, S., & Anderson, V. (1986). Producing written summaries: Task demands, cognitive operations, and implications for instruction. Review of Educational Research, 56, 473–93.
- Hillocks, G., Jr. (2002). The testing trap: How state assessments of writing control learning. New York, NY: Teachers College Press.
- Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY: Cambridge University Press.
- Knoch, U. (2009). Diagnostic writing assessment: The development and validation of a rating scale. Frankfurt, Germany: Peter Lang.
-
Koda, K.
(2007).
Reading and language learning: Crosslinguistic constraints on second language reading development.
Language Learning, 57, Suppl. 1, 1–44.
10.1111/0023-8333.101997010-i1 Google Scholar
- Lado, R. (1961). Language testing: The construction and use of foreign language tests. London, England: Longman.
- Leki, I. (2007). Undergraduates in a second language: Challenges and complexities of academic literacy development. New York, NY: Erlbaum.
- Leki, I., & Carson, J. (1997). “ Completely different worlds”: EAP and the writing experiences of ESL students in university courses. TESOL Quarterly, 31, 39–69.
-
Lewkowicz, J. A.
(2000).
Authenticity in language testing: Some outstanding questions.
Language Testing, 17, 43–64.
10.1177/026553220001700102 Google Scholar
- Mislevy, R., & Yin, C. (2009). If language is a complex adaptive system, what is language assessment? In N. Ellis & D. Larsen-Freeman (Eds.), Language as a complex adaptive system. Supplement to Language Learning, 59, 249–67.
- Morrow, K. (1977). Techniques of evaluation for a notional syllabus. London, England: Royal Society of Arts.
-
Norris, J.
(2002).
Interpretations, intended uses and designs in task-based language assessment.
Language Testing, 19, 337–46.
10.1191/0265532202lt234ed Google Scholar
- Peirce, B. (1992). Demystifying the TOEFL reading test. TESOL Quarterly, 26, 665–89.
-
Plakans, L.
(2008).
Comparing composing processes in writing-only and reading-to-write test tasks.
Assessing Writing, 13, 111–29.
10.1016/j.asw.2008.07.001 Google Scholar
- Plakans, L. (2009). Discourse synthesis in integrated second language writing assessment. Language Testing, 26, 561–87.
-
Plakans, L., &
Gebril, A.
(2012).
A close investigation into source use in integrated second language writing tasks.
Assessing Writing, 17, 18–34.
10.1016/j.asw.2011.09.002 Google Scholar
- Raimes, A. (1990). The TOEFL test of written English: Causes for concern. TESOL Quarterly, 24, 427–42.
- Rosenfeld, M., Leung, S., & Oltman, P. (2001). The reading, writing, speaking, and listening tasks important for academic success at the undergraduate and graduate levels (TOEFL Monograph Report 21). Princeton, NJ: Educational Testing Service.
- Sawaki, Y., Quinlin, T., & Lee, Y. (2013). Understanding learner strengths and weaknesses: Assessing performance on an integrated writing task. Language Assessment Quarterly, 10, 73–95.
- Shi, L. (2004). Textual borrowing in second-language writing. Written Communication, 21, 171–200.
- Shi, L. (2010). Textual appropriation and citing behaviors of university undergraduates. Applied Linguistics, 31, 1–24.
- Sternglass, M. (1997). Time to know them: A longitudinal study of writing and learning at the college level. Mahwah, NJ: Erlbaum.
- van Lier, L. (1989). Reeling, writhing, drawling, stretching, and fainting in coils: Oral interviews as conversation. TESOL Quarterly, 23, 489–508.
-
Wesche, M.
(1987).
Second language performance testing: The Ontario Test of ESL as an example.
Language Testing, 4, 28–47.
10.1177/026553228700400103 Google Scholar
- Yang, H., & Plakans, L. (2012). Second language writers’ strategy use and performance on an integrated reading-listening-writing task. TESOL Quarterly, 46, 80–103.
-
Yu, G.
(2009).
The shifting sands in the effects of source text summarizability on summary writing.
Assessing Writing, 14, 116–37.
10.1016/j.asw.2009.04.002 Google Scholar
- Yu, G. (2013). The use of summarization tasks: Some conceptual and lexical analyses. Language Assessment Quarterly, 10, 96–109.
- Xi, X., Higgins, D., Zechner, K., & Williamson, D. (2008). Automated scoring of spontaneous speech using SpeechRater V 1.0 (ETS Research Report 08-62). Princeton, NJ: Educational Testing Service.
Suggested Readings
- C. Chapelle, M. Enright, & J. Jamieson (Eds.). (2008). Building a validity argument for the Test of English as a Foreign Language (TOEFL). London, England: Routledge.
-
Cumming, A.
(2007).
New directions in testing English language proficiency for university entrance. In
J. Cummins &
C. Davison (Eds.),
International handbook of English language teaching (Vol. 1, pp. 473–86).
New York, NY:
Springer.
10.1007/978-0-387-46301-8_34 Google Scholar
- Shaw, S., & Weir, C. (2007). Examining writing: Research and practice in assessing second language writing. New York, NY: Cambridge University Press.
- G. Yu (Ed.). (2013). Use of integrated writing tasks in language assessment (Special issue). Language Assessment Quarterly, 10.
Online Resources
- Brown, A., Iwashita, N., & McNamara, T. (2005). An examination of rater orientations and test-taker performance on English-for-Academic-Purposes speaking tasks (TOEFL Monograph Report 29). Princeton, NJ: Educational Testing Service. Retrieved December 7, 2012 from https://www.ets.org/Media/Research/pdf/RR-05-05.pdf
- Cambridge ESOL. (2012). First Certificate in English. Retrieved July 26, 2012 from http://www.cambridgeesol.org/exams/fce/index.html#wr
- Carleton University. (n.d.). Carleton Academic English Language (CAEL) Assessment Practice Test. Topic: Rainforest. Ottawa: Carleton University. Retrieved May 14, 2012 from http://www.cael.ca/taker/Rainforest.shtml
- Certificaat Nederlands als Vreemde Taal. (2012). Centre for Language and Education, Catholic University of Leuven. Retrieved July 26, 2012 from http://www.cnavt.org/main.asp
- Cumming, A., Kantor, R., Baba, K., Eouanzoui, K., Erdosy, U., & James, M. (2006). Analysis of discourse features and verification of scoring levels for independent and integrated prototype tasks for the new TOEFL (TOEFL Monograph Report 30). Princeton, NJ: Educational Testing Service. Retrieved December 7, 2012 from http://www.ets.org/Media/Research/pdf/RR-05-13.pdf
-
Deane, P.
(2011).
Writing assessment and cognition.
Princeton, NJ:
Educational Testing Service. Retrieved May 14, 2012 from http://www.ets.org/Media/Research/pdf/RR-11-14.pdf
10.1002/j.2333-8504.2011.tb02250.x Google Scholar
- Educational Testing Service. (2005). TOEFL iBT Writing Sample Responses. Princeton, NJ: Educational Testing Service. Retrieved May 14, 2012 from http://www.ets.org/Media/Tests/TOEFL/pdf/ibt_writing_sample_responses.pdf
- Fraser, W. (2002). The role of reflection in the Canadian Academic English Language (CAEL) Assessment. Ottawa, Canada: Carleton University. Retrieved May 14, 2012 from http://www.cael.ca/pdf/wendypaper.pdf
- New York State Education Department. (2012). Regents Examination. Albany, NY: Office of Assessment Policy, Development and Administration. Retrieved July 26, 2012 from http://www.nyregents.org/ComprehensiveEnglish/
- Powers, D. E. (2010). The case for a comprehensive, four-skills assessment of English language proficiency. (TOEIC Compendium Report 12). Princeton, NJ: Educational Testing Service. Retrieved May 14, 2012 from http://www.ets.org/Media/Research/pdf/TC-10-12.pdf
- Swain, M., Huang, L., Barkaoui, K., Brooks, L., & Lapkin, S. (2009). The speaking section of the TOEFL iBT (SSTiBT): Test-takers’ reported strategic behaviors (TOEFL iBT Research Report 10). Princeton, NJ: Educational Testing Service. Retrieved May 14, 2012 from http://www.ets.org/Media/Research/pdf/RR-09-30.pdf
- University of Auckland. (n.d.). Diagnostic English Language Needs Assessment (DELNA): Handbook for candidates at the University of Auckland. Auckland, New Zealand: University of Auckland. Retrieved May 14, 2012 from http://www.delna.auckland.ac.nz/webdav/site/delna/shared/delna/documents/delna-handbook.pdf
- Wall, D., & Horak, T. (2008). The impact of changes in the TOEFL examination on teaching and learning in central and eastern Europe: Phase 2, coping with change (TOEFL-iBT Research Report 5). Princeton, NJ: Educational Testing Service. Retrieved May 14, 2012 from http://www.ets.org/Media/Research/pdf/RR-08-37.pdf