Free Access

Assessing English in Europe

Volume IV. Assessment Around the World
Part 13. Current Practices in Assessing English
Gad S. Lim

Gad S. Lim

University of Cambridge, ESOL Examinations, England

Search for more papers by this author
First published: 11 November 2013

Abstract

One of the results of integration in the European area has been the development of a unified language policy promoting “plurilingualism,” providing some measure of coherence to the learning, teaching, and assessment of language ability. In practice, English has been the primary additional language learned, and it is learned in and out of school systems from ever earlier ages by ever more people, for purposes of personal development, migration, survival, education, and employment.

A long tradition of formal English language assessment exists in parts of Europe, often for high stakes purposes, though informal and formative assessments have also been promoted in such projects as DIALANG and the European Language Portfolio. In recent years, language assessment has been discussed with reference to the Common European Framework of Reference for Languages (CEFR). Examinations claim “alignment” to the CEFR, and test users ask for results in terms of the six levels into which the CEFR divides language ability.

There is debate, however, as to whether using the CEFR for such alignment is possible, defensible, or even desirable. The meaning and role of the framework remains to be agreed. The direction of this debate, the language-related policies of users, and the changing demographic of English language learning will influence future ESL assessment practice in Europe and beyond.

Introduction

A number of factors have converged to make Europe a locus of developments in language teaching and assessment in our time. The continent is a compact conglomeration of states, most of which have decided upon union for political and economic reasons, but whose citizens speak many different languages, creating the need for a lingua franca. This reality required new approaches to language learning, teaching, and assessment whose goal was not formal knowledge but communication. This reality also meant that political institutions were in place to promote required developments. Finally, the reality of our historic moment made it inevitable that, desired or not, and the designation of other official languages notwithstanding, the European lingua franca would be and is English.

Teaching–Learning Contexts

A major influence on the learning, teaching, and assessment of languages in the continent at present is the Common European Framework of Reference for Languages (CEFR), produced by the Council of Europe (2001) to provide “a common basis for the elaboration of language syllabuses, curriculum guidelines, examinations, textbooks, etc. across Europe” (p. 1). The framework divides language ability into six levels—A1 and A2 for basic users, B1 and B2 for independent users, and C1 and C2 for proficient users of a language. It espouses a communicative approach to language teaching—the earliest work on which was done in Europe (e.g., Wilkins, 1976; Widdowson, 1978)—and is illustrated by descriptors of language ability that are phrased as can-do statements. While learning expectations, curricula, and textbooks across Europe are written referencing the CEFR, the extent to which these are faithful to its principles is much debated. Also quite unfortunately, many people have reduced the CEFR to the illustrative descriptors, forgetting other aspects and features of the framework, such as approach to language learning (Jones & Saville, 2009).

Reflecting European policy, the CEFR encourages “plurilingualism”—that is, multilingualism as an individual rather than a societal phenomenon (Beacco & Byram, 2003). European school systems generally make provision for the learning of multiple foreign languages in compulsory education, and from an increasingly early age. The exceptions to compulsory foreign language learning are, perhaps unsurprisingly, countries where English is the primary language—the United Kingdom and Northern Ireland. In practice, English has been by far the dominant foreign language learned in almost every European country. The number of English language learners has been growing the fastest in eastern and southern Europe. In upper secondary school, approximately 90% of students learn English whether or not it is compulsory in their country (European Commission, 2006; Eurydice, 2008). In some contexts, the perceived inadequacy of English language instruction within regular school contexts has resulted in the growth of language tutors and schools offering supplementary private tuition. These include large international organizations such as the British Council, Education First, and Eurocentres, as well as smaller independent schools.

In higher education, in the interest of promoting greater exchange, the Bologna Process has created the European Higher Education area, standardizing programs largely according to the model followed in the United Kingdom. This development has also led to a rapid rise in the number of programs taught in English; one study showed that there are approximately 2,500 such programs (Wächter & Maiworm, 2008).

A strong curricular trend in Europe at all levels of education is content and language integrated learning (CLIL) (Fortanet-Gomez & Raisanen, 2008; Marsh, Mehisto, Wolff, & Martin, 2011). In a sense, CLIL is an extension of, or at least bears a family resemblance to, the communicative approach, embedding language teaching in the very disciplines that students are studying. However, it seems that CLIL is being adopted as much out of practical necessity as for pedagogical/theoretical reasons. Given the limited amount of time available in a crowded curriculum, CLIL appears to be a convenient way of claiming that both content and language have been covered. It also bears mentioning that relatively few teachers are trained in the approach, though steps are apparently being taken to address this shortcoming (Marsh et al., 2011).

In the United Kingdom, and on a smaller scale in the Republic of Ireland, the English-teaching industry is a major source of income, with people from all over the world enrolling in English language courses for varying lengths of time. Degree-level international students also often require English for academic purposes support. In addition, knowledge of the English language is also required for migration, asylum, and citizenship purposes (Shohamy & McNamara, 2009; Strik, Bocker, Luiten, & van Oers, 2010). Provision of adult ESOL programs for these people is mandatory by law, but support for them has generally been less than adequate. These learners tend to be quite diverse and quite different from traditional English as a foreign language (EFL) learners—needing English for survival purposes, for example, but often with low literacy skills even in their first languages—but learning resources do not usually account for these differences (Cooke & Simpson, 2008; Little & Simpson, 2009).

Assessment Practices

Most European countries do not have a tradition of external examinations. Where they did exist, there was no emphasis on the use of psychometric procedures to ensure valid, reliable, and comparable outcomes. Instead, there is a tradition of localized assessment, with the individual teacher being regarded as the expert, and with validity and reliability presumed (Spolsky, 1995). This carried into the context of external assessment. For example, the Certificate of Proficiency in English offered by the University of Cambridge in 1913 was an offshoot of academic tests offered by the university. It was a combined proficiency and teaching test that included sections on translation (into German or French, and vice versa), English literature, and essay writing, among others. The test was marked by a professor in much the same way as teachers marked classroom assignments (Weir & Milanovic, 2003).

Present-day realities dictate examination outcomes that are more demonstrably valid and reliable, and the lack of a tradition that enables the production of such can be seen as one factor that led to the development of the CEFR. Many eastern and southern European countries have been reforming their school-leaving examinations (called matura or maturita), including those for English, to reflect best practices in educational assessment. Desired performance outcomes on these examinations are expressed in terms of CEFR levels; the most common level expected at the end of secondary school is B2 (Council of Europe, n.d.). However, cultural and political considerations have resulted in many of these reforms being thwarted or watered down (Pizorn & Nagy, 2009). Testing of the productive skills, especially speaking, is often on the verge of being excluded. Where the productive skills are tested, testing tends to be under nonstandardized conditions, or marked by the students' own teachers, making outcomes less than meaningful and trustworthy. Cut scores on tests are adjusted so low that virtually everyone passes.

Other test providers have filled the space created by state examinations' inability to provide reliable information about people's language abilities. Among these are examination bodies associated with academic institutions, for instance University of Cambridge ESOL Examinations and the University of Michigan, as well as other nonprofit and for-profit organizations, such as telc and Pearson. These providers typically have a range of English language proficiency exams for use in various contexts and at a number of different ability levels, and operate in multiple countries. Examinations are generally structured around and usually cover the four skills—reading, writing, listening, and speaking.

No longer in the “traditional” (Spolsky, 1995) stage of language testing, these international test providers pay equal attention to psychometrics. Test items are routinely pretested or trialed, analyzed and calibrated, and then collected in item banks. Test performances are scored and equated using classical and item response theory. Where examiners are involved, for instance in writing and speaking tests, they are generally trained, certificated, and monitored on a regular basis. Reliability information for exams and their components is generally calculated and reported. Computers have also been used in the delivery and in the scoring of tests.

Perhaps owing to their academic origins, a number of these level-based examinations are distinctive in that test takers follow a course of study in preparation for taking the tests—as opposed to cram schools that focus on test-taking strategy, or exams for which test takers simply show up on the day. Thus, these examinations are more closely tied to language teaching than others, and are to a certain extent not just proficiency measures but also achievement measures. In some countries such as Greece, preparing for and taking these exams are a rite of passage for teenagers, and certificates obtained are displayed prominently in homes (Tsagari, 2009).

On the other hand, a problem created by these exams is that their popularity and their ubiquity have made them and their associated materials very easy to use—even in contexts where they are not the most suitable or appropriate. For example, in the Republic of Ireland it was found that traditional coursebooks for EFL exams were being used with adult refugees, even though these materials did not cover the everyday, survival language these learners needed (Little & Simpson, 2009).

Fortunately, more appropriate assessments have also since been developed for nontraditional learners with different needs. One example is DIALANG, an online self-assessment available in 14 languages, including English, which learners can use to determine where they are generally on the CEFR levels (Alderson, 2005). Another example is the European Language Portfolio (ELP), of which there are multiple versions officially approved by the Council of Europe (Little & Perclova, 2001). The portfolio contains learners' language biographies, examples of their work in a language, and results of self-, teacher, and formal assessments. This has the dual function of allowing learners to report their progress in a language, and to guide further learning in the language. It is hoped that, through the ELP, learners can take more responsibility for planning their language learning according to their needs, rather than simply following some externally defined requirement.

More formal assessments for adults in the United Kingdom include officially supported Skills for Life exams, which yoke the assessment of language skills with literacy and numeracy skills. Migrants who seek naturalization as British citizens also need to take an ESOL course or a “Life in the UK” test, a test which combines English language skills with knowledge of British life and culture.

Challenges and Future Directions

Interest in and use of English language tests has been growing in Europe. This can be seen in the formation of organizations such as the Association of Language Testers in Europe (ALTE) and the European Association for Language Testing and Assessment (EALTA), as well as the existence of sections devoted to testing, evaluation, and assessment in associations of applied linguists and language teachers (e.g., BAAL, BALEAP, and IATEFL). These groups have been at the forefront of advancing language-testing practice in the continent. For example, ALTE has been closely involved in the development of the CEFR, has devised frameworks, codes, and guidelines, and audits the practices and processes of its members. Similarly, EALTA has developed a code of testing practice that is available in 35 European languages.

However, keeping in mind that such large-scale, high stakes testing is a relatively new concept for most Europeans, it is perhaps inevitable that testing—and, along with it, comparisons and rankings, access or nonaccess to society's goods—would be controversial and contentious. For example, requiring English language proficiency as attested to by a language test has been a convenient way for the British government to reduce migration from certain demographics (Shohamy & McNamara, 2009). Commercial interests were also affected when the number of recognized test providers was reduced.

At the center of many debates, not surprisingly, is the CEFR. Among other things, the framework has been criticized for being atheoretical (e.g., Fulcher, 2004), though some critics appear to be thinking of the CEFR's illustrative scales rather than the CEFR itself. More broadly, objections to the CEFR seem to stem from its being used (or misused) as an instrument of policy. The CEFR was developed to be a reference, as its name makes clear. That is to say, language syllabi and tests would reflect certain standards, and when necessary (e.g., a student moving to another country) the CEFR could be used as a reference (e.g., to select a suitable course to take next). However, the transitivity has been reversed in many contexts, with the CEFR becoming the standard, and with language syllabi and tests having to conform to it instead (Jones & Saville, 2009). Its being used out of context is also reflected in the framework being adopted in non-European contexts as far afield as Japan, Taiwan, the Middle East, and Latin America, regions whose linguistic realities and requirements may be quite different from those in Europe (e.g., learners may be lower-level and need a framework with finer gradations of those levels).

One major area of contention has been the matter of demonstrating “alignment” to the CEFR. The Council of Europe (2009) has put together a manual with suggested procedures for this purpose. On the other hand, there are those who think the CEFR is underspecified for any sort of alignment (Weir, 2005). Many test providers claim a relationship to the CEFR. However, because the bases for these claims are not published, or, when they are, they do not seem to match with one another, some have been led to doubt the veracity of these claims (Lim, Geranpayeh, Khalifa, & Buckendahl, 2013). In this the CEFR has, perhaps accidentally, proved a positive development for measurement theory. The consensus in standard-setting theory is that divergent standard-setting outcomes are acceptable, though theory has apparently developed in that direction partly as a result of the general absence of criterion measures. The current use of the CEFR has resulted in a situation where there are multiple criterion measures claiming to measure the same thing for the same contexts of use, and under such conditions it is but right that their outcomes should match.

In any event, it appears that the CEFR is here to stay. In policy settings, it generally makes sense for imperfect instruments to be improved upon instead of thrown away. Thus, regarding the CEFR being underspecified, the levels are being fleshed out for a number of European languages; for English, this is being done by English Profile (e.g., Hawkins & Filipovic, 2012). These developments should help users to focus not just on the vertical dimension of the framework (levels) but on the horizontal dimension as well (nature of each level), both of which are in fact called for by the CEFR (Council of Europe, 2001). The CEFR is also not sufficiently defined for specific demographics (e.g., young learners) and contexts (e.g., CLIL) at this time, and will require further elaboration for those purposes.

Uses of the CEFR for its original intended purpose—comparisons across European contexts and languages—also continue apace. The European Commission, for example, has sponsored the European Survey of Language Competences, which assessed a sample of 1,500 students in each participating country on reading, writing, and listening, providing information about foreign language learning in those countries, and the extent to which they are reaching their goals of citizens becoming plurilingual (European Commission, 2012).

As those citizens become more proficient in English from ever earlier ages, the day will come (or perhaps already has come) when the number of English language learners will fall (Graddol, 2006) and, along with it, the number of English language examinations taken. On the other hand, language being the vital thing that it is, new varieties and uses of it might be found, perhaps leading to ever more specified English for specific purposes testing. If there are new uses for English in the world, it should be no surprise if Europe is once again at the vanguard, influencing English language assessment practice in the continent and beyond.

SEE ALSO: Chapter Language Testing for Immigration to Europe, Language Tests for Immigration to Europe; Chapter Portfolio Assessment in the Classroom, Portfolio Assessment in the Classroom; Chapter Using Standards and Guidelines, Using Standards and Guidelines; Chapter Standard Setting in Language Testing, Standard Setting in Language Testing; Chapter English as a Lingua Franca, English as a Lingua Franca

    The full text of this article hosted at iucr.org is unavailable due to technical difficulties.