Last in a series about the challenges facing the country’s public-education system
Felix Barrantes, 58, is a ball of energy, eager to provide a high-speed tour of the small world he commands in downtown San José and to provide colorful explanations of the work done there – but he doesn’t want his picture taken, or that of his staff.
The reason? He and his crew are Public Enemy Number One in the eyes of many a high-school student, because his is not just any administrative office. It’s the Public Education Ministry’s Quality Control Division, which oversees the nation’s standardized tests, including what’s arguably the biggest controversy facing the ministry: the high-stakes secondary graduation exams that in years past have brought teachers, students and parents to the streets in San José. The many students who failed one or more of the exámenes de bachillerato – 11,000 in 2004 and 10,000 in 2005, though the numbers were later reduced by thousands after the ministry discarded questioned test items – saw their chances at university or good jobs endangered or dashed altogether by the test results.
Teachers’ unions want to get rid of the tests, which cover Spanish, civics, social studies, science (students choose physics, biology or chemistry) and a foreign language (French or English). That’s an unlikely prospect. Education Minister Leonardo Garnier maintains the tests are essential to monitor schools’ quality, and a recent study by investigators from the University of Costa Rica (UCR) upheld the quality of the much-aligned math exam, strengthening that position.
However, change is clearly in order. Garnier says the exams and their results are not being used as they should to influence teacher training and affect ministry policy.
According to Barrantes, who eventually relented on the photography, school principals and instructors don’t do enough with the information they’re given. And Miguel Gutiérrez, who directed the first annual State of Education report this year says the ministry should expose its students to different and more thorough types of tests to get a better picture of why students are falling short.
On Again, Off Again
Costa Rican high-stakes testing has gone through two very different phases, according to Barrantes. Tests were first implemented in 1954, but were discontinued in 1974 because teachers’ unions and other critics made some of the same arguments they make today: the exams proved a barrier for students and served no useful purpose.
However, during the test-free period when high school graduation rates were entirely in the hands of secondary teachers and principals, those rates dropped to 50%, Barrantes said. The UCR study explained that the ministry conducted some diagnostics in 1986 that showed student performance had dropped, prompting the government to reinstate testing in 1988.
According to Garnier, this was a positive step.
“When they implemented tests again in the 1980s, during the first administration of Oscar (Arias), we discovered that quality had fallen,” he told The Tico Times. “If one doesn’t have a test of quality, it drops.”
Barrantes said the new exam system implemented that year, when he started working at Quality Control, featured several key changes. Before 1974, students had to pass high school, then pass the test to get their degree. Since 1988, the exams are worth only 60% of the score that determines graduation, with the average grades from the last two years of high school making up the remaining 40%. Students who excel in their classes have less pressure going into the tests.
Additionally, students are given guidelines about what kind of content will be on the test, which wasn’t done before, he added. The ministry also changed the way the tests are put together. Before, one teacher would create a test and the ministry would use it on a national level. Now, a complex process with several levels of approval takes place. The ministry buys 70% of its new questions for ¢1,000 apiece (about $2) from active and retired K-12 and university educators, and 30% are written by specialists on the Quality Control staff – people with degrees in the subject area for which they’re writing questions.
Once received, the questions are tested and reviewed by five other teachers. If, despite this screening process, a question gets through that’s too difficult, the division automatically eliminates questions answered correctly by less than 20% of students. The Education Ministry often disqualifies additional questions as the result of appeals.
Gradually, the ministry is building up its supply of “item bank” questions – those that proved successful in past exams and are recommended for use again. Countries with advanced testing systems, like France, take 100% of their questions from item banks to avoid having to eliminate questions, but Costa Rica’s 18 years of experience have yielded only enough for 10% of this year’s questions to come from the bank. Barrantes said the Ombudsman’s Office has told the ministry it must release copies of the exam to students each year once test-taking is done, instead of keeping them sealed as many countries do. The result is that Quality Control must wait years before reusing a question.
Criticism
For Gonzalo Ortiz, treasurer of the Association of Secondary School Educators (ANDE) and, like Barrantes, a former high school teacher, none of these assurances are enough. The tests must go.
“The bachillerato exams have not improved quality. They’ve been an exclusion mechanism,” he said. “Many kids, for example, have failed one subject and have taken the exam four or five times, and the ministry doesn’t have any mechanism to (support) that kid. There are students who haven’t been able to study in university, who haven’t been able to have a good job, who haven’t been able to make it professionally because they failed one subject of the bachillerato.”
Hundreds of parents, students and teachers stormed the Education Ministry in San José in 2004, demanding that year’s exam, which only half of the 22,000 test-takers passed in the initial count, be re-graded on a curve (TT, Dec. 17, 2004).
Barrantes says it’s up to educators and ministry leaders to change this situation.
“They’re not using it, neither the institutions, nor the Education Ministry’s central offices, nor the regional offices,” he said, pushing across the table copies of the school-specific and national reports Quality Control provides to schools each year. “It’s as if you went to the doctor and he told you that you have high cholesterol… and you don’t pay attention.”
Garnier echoed this point, emphasizing that the original purpose of national exams is not only to evaluate students, but also to influence national and local decisions about teacher training and curriculum.
“Almost nothing has been done with respect (to these goals),” he said. “The exams have served only to see if students pass or don’t pass. They’ve had practically no impact on the education system, the ministry, the regional leadership, the teachers, nothing. They’re not even used to define what additional training teachers need.”
He’s established improving teacher training and nationwide use of exam results as a priority, and added that earlier tests are also necessary to provide teachers and administrators with feedback throughout students’ careers, not just an “autopsy” at the end.
However, according to analyst Gutiérrez, the nature of the tests themselves deserves more scrutiny, and the ministry should subject its school system to international tests as well to get a more diverse picture of how Costa Rican students are performing.
“We’re not subjecting ourselves to international tests, even though we have a clear idea that we have pretty severe problems in terms of educational quality,” Gutiérrez said, adding that the bachillerato exams “are characterized by a lot of content, not so much evaluation of skills and abilities.”
Barrantes said these are points he’d love to address at Quality Control, given more funds. For example, he’s argued in favor of establishing written responses (as opposed to multiple choice, the only question style for all exams except Spanish) or even oral portions for the language exams, but the division’s ¢400 million annual budget ($779,727) won’t allow for it.
Gutiérrez said Quality Control’s analysis of Costa Rica’s exam results also fail to take into account economic factors or other elements of students’ lives, which would help the ministry detect patterns. Providing more thorough analysis is one of the goals for the State of Education, he said.
“These tests (could) give an idea of how certain elements of quality are distributed in the country…For the second report (next year) we hope to have a qualitative study that allows us to isolate factors that have a big influence on quality,” he said. “What things are at play in national (test) results? It’s the kind of thing that… should be being done, to identify these factors and use them when it’s time to plan.”
