Cross-national Research on Education
Helena Laaksonen
The Finnish Institute for Educational Research (FIER) has long experience in producing high-quality research on education. Based at the University of Jyväskylä, the institute specializes in comparative assessment of learning outcomes, educational cultures and systems, the most well-know example being the PISA study.
Researchers working at the institute are not too pleased about the enthusiasm produced by
the comparative assessment results.
– Country rankings are a visible part of the results but we are talking about just a
few measurements, conducted from a certain
viewpoint. The differences between countries are small. Country rankings tend to lose
information. Education is not competitive sports, says a staff member, professor Pekka
Kupari.
– There is a risk that countries which do well on the rankings become satisfied with
the prevailing situation, thinking there is no need for change. The goal of
international comparative assessments is to look for important factors influencing learning
results both on the national level and by country groups, and to try to figure out what can
be learned from them. Consequently, the studies collect a lot of background information on
pupils, school environment and teaching.
Forty years of comparative assess
The Finnish Institute for Educational Research was established in 1968. It is responsible for the Finnish part of the PISA study (Programme for International Student Assessment), as well as for other OECD-funded study programmes such as the Program for the International Assessment of Adult Competencies (PIACC) and the Assessment of Higher Education Learning Outcomes (AHELO).
The institute has been a member of the International Association for the Evaluation of Educational Achievement (IEA) ever since its establishment. Being a IEA member has led to the institute conducting also the International Civic and Citizenship Education Study (ICCS), Progress in International Reading Literacy Study (PIRLS) and the Trends in International Mathematics and Science Study (TIMSS) in Finland.
Increase in evaluation studies
Pekka Kupari is the most practiced researcher working with cross-national comparative assessments, having been involved with such studies ever since the 1970s. He is currently involved in all cross-national assessments conducted by the institute. Kupari says that the number of evaluation studies has increased rapidly over the past ten years.
This has led to a situation where the institute is preparing, carrying out or reporting
several assessment studies at the same time. In 2010, for instance, the institute reported on
the 2009 PISA findings, prepared the 2012 PISA study, and pretested measurements for both
TIMS and PIRLS. All this work was executed by a small group of ten researchers. Senior Researcher Sari
Sulkunen emphasizes that this would be impossible without very experienced staff. With such
expertise, there is always the risk that someone leaves, taking their know-how with them.
– There is no-one we could hire to as a quick replacement of a person leaving.
Cross-national assessment process
Cross-national assessments are all collected in waves, generally every 3 - 5 years. If the
cycle is frequent, there may be problems with non-response and refusals to participate. To
ensure comparability, all research projects have an international consortium which lays down the rules regarding the schedules, instructions and quality criteria which the national level operators must abide by. – For example, the PIACC guidelines are 300 pages long, says Senior Researcher Antero Malin.
All cross-national studies proceed in a similar manner. First the assignments and measurements are planned and pre-tested. At this stage the national operators have a say in methods and measurements.
– Once the actual data collection stage is reached, there is no longer any room
for changes, Kupari explains.
National cultures influence results
The researchers working at the Finnish Institute for Educational Research stress that
measurements used in cross-national assessment studies are always compromises. This should
be taken into account when considering their validity. Cultural environment always affects the results.
– Still, the main goal is to achieve as high comparability as possible.
Researchers know how to take problematic factors into consideration, Malinen points
out.
Even with strict guidelines, schedules and quality requirements, researchers working at the
institute find international cooperation rewarding. Both Kupari and Sulkunen feel it is educational to write joint papers.
–One achieves knowledge and skills it would be hard to get otherwise, Sulkunen
says.
More information: Finnish Institute for
Educational Research
|