“The standard you walk past is the standard you accept.” – David Morrison
Since I became a teacher some thirteen years ago, I have been listening to teachers bemoan the competition in English organized by the Croatian Education and Teacher Training Agency (AZOO), whether at elementary or high school level. I know a number of colleagues who have sent letters of complaint throughout these years, and I know a few who were at some point members of the National Committee for the Competition and who tried to change the system from within. Obviously and unfortunately, the letters and the attempts were in vain.
What are the major complaints? That the tests are too difficult, that some tasks are inappropriate, that there are answers that should be accepted, but aren’t, that elementary school students only get to compete in grade 8, and – perhaps most important – that a great number of fantastic students don’t feel they “know” English well upon taking the test.
My discontent with the competition was dormant for a couple of years as I no longer worked in elementary school, but it culminated in 2020 when I realized that nothing has changed in more than a decade. That is when I thoroughly analyzed the test for the school level of the competition in grade 8 in 2020, as well as the regulations regarding the work of the Committee. The whole investigation is available on my site, so I will not go into all the details here. Rather, I’ll focus on the key findings in form of a list that shows all the ways the competition fails students and teachers.
1) There is no record of the goal of the competition. Is it to test grammar, to identify talented students, to measure communicative competence? How can a student work toward a goal if they don’t know what it is?
2) Similar to the previous point, there is no catalog of what students need to know or be able to do in order to do well in the competition. This means that students and their teachers know neither what the goal is nor the means to achieve that goal; in other words, they don’t know what to do to prepare for the competition.
3) Tests in the competition include tasks that are several levels above the students’ target level. Some may say this is fine because the competition is for the best students who we presume are above the target level, but that level should be defined, along with the goal and catalog of knowledge and skills.
4) Many tasks on the tests in the competition are inspired by those found in Cambridge ESOL exams (FCE, CAE, CPE). Each of the Cambridge exams has a very specific format, and a number of tasks found in those exams is usually not used in regular English classes in Croatia. Learners who want to take a Cambridge exam know well that they have to prepare for it – not just by learning the language but by learning the format of the exam as this has a major impact on how well they will do on the exam. Hence, the tasks in the competition are not very familiar to our students who are not preparing for a Cambridge exam.
5) We don’t know the goal of the competition so it is difficult to ruminate on which tasks would be appropriate for it, but I’m sure they should be aimed at testing communicative competence and not isolated skills, such as grammatical competence. Students are taught in line with the communicative approach, but that is not how they are tested in the competition.
6) The instructions for the competition state that culture is an essential part of the test in the competition. We all know that culture is intertwined with language – there’s no question about it. Thus, intercultural competence is considered a part of overall communicative competence and should be developed and assessed. However, the culture task in the competition doesn’t assess intercultural competence but rather tests students’ knowledge of historical and geographical facts. In 2020, it tested students’ familiarity with London, exclusively. After eight years of learning English, will facts about London help the student get their message across in English? (Consider this question especially in light of the fact that there are three times more non-native than native speakers.)
7) The work of the Committee is not transparent. The Agency has confirmed that the Committee doesn’t keep minutes of their meetings and that the reports on their work are – lost. Without these, it is impossible to know which issues they faced and whether and how they addressed them. Furthermore, there are around twenty members of the Committee, and it is not clear who does what; for example, I know that one person deigns the test, and one more takes a look at it, but this isn’t public information. Some members have been on the Committee for ages, which would only make sense if they were doing a good job. Perhaps if more people were rotated annually or every couple of years, the competition would benefit from new perspectives and ideas.
8) Looking just at the test at the school level of the competition in grade 8 in 2020, it is obvious that the person who made it did a poor job. The test-makers should be versed in test development, and more members of the Committee should be involved in providing feedback to the test-maker before the test is released. In fact, all the members of the Committee should be trained in assessment matters.
9) In the midst of the 2020 debacle, three exceptional and very qualified professionals (a top Croatian scholar and ELT methodology professor, a textbook author and pre-service mentor, and a curriculum developer, to name just a few roles) left the Committee because their expert opinion, which differed from that of the Committee President, was ignored and suppressed. A committee in which a person’s expert opinion isn’t acknowledged simply because it is different from the centrally enforced one is a committee that stifles critical thinking. We have already established that there is no external evaluation of the committee (see n°7), and it is clear that any attempt at change coming from within is quickly thwarted.
10) It would be easy to live with the fact we have a subpar competition because it takes place only once a year so you sort of get it over with and get over it, but the problem is that such a competition has a negative washback effect and thus could influence the way students are taught and assessed. Do we want students to think that knowing English is mostly about knowing how to transform sentences and words and being more informed about London than the Queen herself? Or do we want them to understand that English is about getting your message across to two billion people who speak it?
For all of the reasons above, I find the system of competition in English ill-conceived – it fails bright students and hard-working teachers. We have let it fester for too long.
This text was published in HUPEzine in May 2020.