Wpływ testu na środowisko edukacyjne

Numer JOwS: 
str. 56

Dzięki współpracy redakcji JOwS z Cambridge English Language Assessment mamy przyjemność udostępnić Państwu na łamach naszego czasopisma artykuł Nicka Saville’a - jednego z czołowych badaczy, zajmujących się problematyką testowania biegłości językowej. Tekst opisuje model umożliwiający badanie, w jaki sposób ocenianie umiejętności językowych kształtuje konteksty edukacyjne. Tekst w języku angielskim jest poprzedzony krótką syntezą w języku polskim, opracowaną przez dr Agnieszkę Dryjańską, redaktor JOwS.

Research methods and roles of researchers

Contemporary theories of knowledge and learning have played a prominent role in developing Cambridge English model of impact and the search for a ‘paradigm worldview’ (epistemology and ontology) which provides an effective conceptualisation and has drawn on relevant theories in the social sciences. A ‘realist’ stance now underpins Cambridge English approach, drawing on ‘critical realism’ (e.g. Sayer 1984, 2000) and contemporary views on pragmatism.

Constructivism is also important for the re-conceptualisation of impact for two reasons: first because contemporary approaches to teaching and learning in formal contexts now appeal to constructivist theories; secondly, because it is most appropriate to finding out ‘what goes on’ in contexts of test use. From the learner’s perspective, affective factors are vital for motivation and feedback that highlights strengths positively tends to lead to better learning (i.e. learning oriented assessment). These considerations are relevant in designing language assessment systems which have learning oriented objectives and a concern in impact research is whether these objectives have been met effectively.

The current model of impact looks to ‘real world’ research paradigms to provide tools which can shed light on what happens in testing contexts, including mixed methods and quasi-experimental designs. Case studies are especially useful for investigating impact at the micro level and for understanding the complexities of interaction between macro level policies and implementation in local settings. Without such methods it is difficult to find out about and understand how the interaction of differing beliefs and attitudes can lead to consensus or to divergence and diversity.

Mixed method research designs are becoming increasingly relevant to addressing impact research questions. Creswell and Plano Clark (2011:69) discuss six prototypical versions of mixed method research designs which seek to integrate qualitative and quantitative data in parallel and sequential ways and these are becoming central to the Cambridge English approach, as illustrated by the studies reported in this issue.

The Cambridge English ‘impact toolkit’ of methods and approaches is now being used to carry out analyses of both large-scale aggregated data, as well as micro analyses of views, attitudes and behaviours in local settings (as in the earlier case of the Progetto Lingue 2000 impact study reported by Hawkey 2006). Quantitative analysis of macro level group data allows us to capture overall patterns and trends, while the qualitative analysis of multiple single cases enables the research team to monitor variability in local settings and to work with the ‘ecological’ features of context. It is the integration of both analyses to provide the insights and interpretations which is particularly important.

Finally it is important to highlight the make-up of the impact research teams; where possible, the team should comprise both Cambridge-based staff with appropriate skills in research design and analysis, as well as local researchers who may be ‘participants’ in the teaching/learning context itself and who bring a deeper understanding of the educational and cultural context which is under investigation. Again this is illustrated in the studies reported in this issue, including Gu and Saville working jointly with other participants in the Chinese context.

Conclusion

The ability to change in order to improve educational outcomes or mitigate negative consequences associated with the examinations is ultimately the most important dimension of the impact by design model. Anticipating impacts and finding out what happens in practice are not enough if improvements do not occur as a result. Being prepared to manage change is therefore critical to a theory of action. In working closely with the stakeholders in their own contexts, this approach is now providing us with the necessary tools to determine what needs to be done and when/how to do it.

References

  • Bachman, L., Palmer, A. (1996) Language Testing in Practice. Cambridge: Cambridge University Press.
  • Bailey, K.M. (1996) Working for Washback: A Review of the Washback Concept in Language Testing. W: Language Testing, nr 13 (3), 257-279.
  • Cambridge ESOL (2011) Principles of Good Practice: Quality Management and Validation in Language Assessment [online] [dostęp 8.04.2015].
  • Cheng, L. (1997) The Washback Effect of Public Examination Change on Classroom Teaching: An Impact Study of the 1996 Hong Kong Certificate of Education in English on the Classroom Teaching of English in Hong Kong Secondary Schools [niepublikowana praca doktorska]. University of Hong Kong [niepublikowana praca doktorska].
  • Cheng, L. (2005) Changing Language Teaching through Language Testing: A Washback Study. W: Studies in Language Testing, nr 21. Cambridge: UCLES/Cambridge University Press.
  • Cheng, L., Watanabe, Y, Curtis, A. (red.) (2004) Washback in Language Testing: Research Contexts and Methods, Mahwah. Nowy Jork: Lawrence Erlbaum Associates.
  • Creswell, J.W., Plano Clark, V.L. (2011) Designing and Conducting Mixed Methods Research. Thousand Oaks. California: Sage.
  • Fullan, M. (1991) The New Meaning of Educational Change. London: Cassell.
  • Fullan, M. (1993) Change Forces: Probing the Depths of Educational Reform. London: The Falmer Press.
  • Fullan, M. (1999) Change Forces: The Sequel. London: The Falmer Press.
  • Green, A. (2003) Test Impact and EAP: A Comparative Sudy in Backwash Between IELTS Preparation and University Pre-sessional Courses [niepublikowana praca doktorska]. University of Surrey.
  • Green, A. (2007) IELTS Washback in Context: Preparation for Academic Writing in Higher Education. W: Studies in Language Testing, nr 25, Cambridge: UCLES/Cambridge University Press.
  • Hawkey, R. (2006) Impact Theory and Practice: Studies of the IELTS Test and Progetto Lingue 2000. W: Studies in Language Testing, nr 24. Cambridge: UCLES/Cambridge University Press.
  • Hughes, A. (1989) Testing for Language Teachers. Cambridge: Cambridge University Press.
  • Messick, S. (1996) Validity and washback in language testing. W: Language Testing, nr 13 (3), 241-256.
  • Milanovic, M., Saville, N. (1996) Considering the Impact of Cambridge EFL Examinations [raport wewnętrzny]. Cambridge: Cambridge ESOL.
  • Saville, N. (2003) The Process of Test Development and Revision within UCLES EFL. W: C. Weir, M. Milanovic (red.) Continuity and Innovation: Revising the Cambridge Proficiency in English Examination 1913-2002. Studies in Language Testing, nr 15. Cambridge: UCLES/Cambridge University Press, 57-120.
  • Saville, N. (2009) Developing a Model for Investigating the Impact of Language Assessment within Educational Contexts by a Public Examination Provider [niepublikowana praca doktorska]. University of Bedfordshire
  • Saville, N. (2010) Developing a Model for Investigating the Impact of Language Assessment. W: Research Notes, nr 42, 2-8.
  • Sayer, A. (1984) Method in Social Science: A Realist Approach. London: Routledge.
  • Sayer, A. (2000) Realism and Social Science. London: Sage.
  • Thelen, E., Smith, L.B. (1994) A Dynamic Systems Approach to the Development of Cognition and Action. Cambridge, Massachusetts: The MIT Press.
  • Van Geert, P. (2007) Dynamic Systems in Second Language Learning: Some General Methodological Reflections. W: Bilingualism: Language and Cognition, nr 10, 47-49.
  • Wall, D. (1999) The Impact of High-stakes Examinations on Classroom Teaching: A Case Study Using Insights From Testing and Innovation Theory [niepublikowana praca doktorska]. Lancaster University.
  • Wall, D. (2005) The Impact of High-Stakes Examinations on Classroom Teaching: A Case Study Using Insights from Testing and Innovation Theory. W: Studies in Language Testing, nr 22. Cambridge: UCLES/ Cambridge University Press.
  • Weir, C.J. (2005) Language Testing and Validation: An Evidence-based Approach. Basingstoke: Palgrave Macmillan.

* Artykuł przedrukowany z Reseach Notes Issue nr 50.


[1] Termin impact może być tłumaczony także jako oddziaływanie testu, jak proponuje Angielsko-polsko-słoweński glosariusz terminów z zakresu testowania biegłości językowej pod redakcją W. Martyniuka, konsultacja naukowa prof. B. Niemierko, prof. H. Komorowska, prof. W. Miodunka, dr J. Magiera, Kraków, 2004. Jednakże w obecnej polskiej literaturze przedmiotu częstsze jest użycie terminu wpływ. Należy jednak podkreślić, że definicja pojęcia impact prezentowana i dyskutowana przez Cambridge English Language Assessment w niniejszym artykule wykracza poza tradycyjne rozumienie tego pojęcia.

[2] Diagram (Fig. 1.) dostępny w tekście oryginalnym artykułu.

[3] Ibidem.