APPLICATION OF META-ANALYSIS METHOD IN HUMANITARIAN AND PEDAGOGICAL RESEARCH
DOI: 10.23951/1609-624X-2021-5-78-89
Introduction. The implementation of the modern model of education requires teaching staff to possess certain skills in research work and to use scientifically grounded forms and methods in their professional activities. It seems that reliance on science should make it possible to achieve the required quality of education and thereby ensure the fulfillment of the social and state order for the education system. In particular, it is necessary to fix the fact that in empirical research used in science, there are, today, serious problems regarding the reproducibility of the process and the results of pedagogical research. In this regard, in fulfilling the requirement to achieve the quality of education, which is verified, first of all, by empirical research, teachers are faced with a serious problem, namely, with a crisis of reproducibility. The reproducibility crisis is a serious methodological crisis, the essence of which is the impossibility of repeating the results of studies that were previously published or the impossibility of repeating the study itself (for technical and other reasons). Most often, this is due to the absence or insufficient representation of the following elements of pedagogical research: the presence of an experiment, a clear description of the planning and course of the experiment, verifiable data – when it is not clear what, how and in what conditions to reproduce. The goal is to scientifically substantiate the feasibility of extrapolating the meta-analysis method from evidencebased medicine to pedagogical research as a tool to increase the reproducibility of both experiments and the results obtained in pedagogical research. Material and methods. The research methodology is based on the following methods: study and analysis of literature, analytical generalization, forecasting, abstraction, analogy, measurement, data synthesis, generalization, modeling, analogy, statistical processing of the information received. After the main reasons causing the crisis of reproducibility in science were considered, the use of the meta-analysis method in pedagogical research was substantiated as a means of minimizing the negative impact of the crisis of reproducibility. Results and discussion. The current literature on the research problem is analyzed from different positions. The essential features and causes of the crisis of reproducibility in pedagogical research have been studied and generalized. It was found that the crisis of reproducibility is interdisciplinary in nature. The problem of finding means to minimize the negative impact of the crisis of reproducibility on the development of pedagogical research is actualized. As such a tool, a meta-analysis is considered and an assumption is made about the advisability of extrapolating the method of meta-analysis from medicine to pedagogical research. The theoretical and practical aspects of the application of the meta-analysis method are considered. The definitions of the concepts of meta-analysis and a systematic review are derived. It was found that meta-analysis intersects and partially includes elements of such methods and technologies as content analysis, applied scientometrics, blockchain. The organizational and substantive features of the meta-analysis are considered, which make it possible to apply this method to assess the quality of pedagogical research. As a practical justification for the applicability of the method, a contour plan for the meta-analysis of scientific publications reflecting the results of pedagogical research is proposed. A number of controversial provisions that could become the basis for further research is put forward. A way is outlined for a comprehensive study of tools to minimize the negative impact of the crisis of reproducibility in pedagogical sciences. Conclusion. The has been substantiated the use of meta-analysis as an effective tool for: 1) assessing the quality of pedagogical research; 2) planning and organizing the experiment; 3) presentation of the results of empirical research in pedagogy.
Keywords: meta-analysis, blockchain, systematic review, reproducibility crisis
References:
1. Devyatko I. F. Novyye dannyye, novaya statistika: ot krizisa voproizvodimosti k novym trebovaniyam k analizu i predstavleniyu dannykh v sotsial’nykh naukakh [New data, new statistics: from the crisis of reproducibility to new requirements for the analysis and presentation of data in the social sciences]. Sotsiologicheskiye issledovaniya “K XX Kharchevskim chteniyam” [Sociological research “In honour of XX Kharchev’s readings”]. Moscow, Russian Academy of Sciences Publ., 2018, no. 12, pp. 30–38 (in Russian).
2. Kuznetsov D. Rezul’taty boleye poloviny psikhologicheskikh issledovaniy okazalis’ nevosproizvodimy [The results of more than half of psychological studies were irreproducible]. N + 1 (in Russian). URL: https://nplus1.ru/news/2015/08/28/psychology-studies-fail-reproducibility (accessed 24 May 2021).
3. Mazilov V. A. Krizis psikhologii: novoye ponimaniye i traktovka [The crisis of psychology: a new understanding and interpretation]. Yaroslavskiy pedagogicheskiy vestnik – Yaroslavl Pedagogical Bulletin, 2019, no. 3 (108), pp. 90–100 (in Russian).
4. Raven J. Nekotoryye prestupnyye (poka yeshche ne kriminalizovannyye) zloupotrebleniya “naukoy”, logikoy i vlast’yu na primere shkol’nogo obrazovaniya (na primere nachal’nogo obrazovaniya) [Some criminal (not yet criminalized) abuse of “science”, logic and power on the example of school education (on the example of primary education)] Azimut nauchnykh issledovaniy: pedagogika i psikhologiya – Azimuth of Scientific Research: Pedagogy and Psychology, 2019, vol. 8, no. 3 (28), pp. 218–227 (in Russian).
5. GOST R ISO 5725-1-2002. Gosudarstvennyy standart Rossiyskoy Federatsii. Tochnost’ (pravil’nost’ i pretsizionnost’) metodov i rezul’tatov izmereniy. Chast’ 1. Osnovnyye polozheniya i opredeleniya (prinyat i vveden v deystviye Postanovleniyem Gosstandarta Rossii ot 23.04.2002 N 161-st) [GOST R ISO 5725-1-2002. State standard of the Russian Federation. Accuracy (correctness and precision) of measurement methods and results. Part 1. Basic provisions and definitions (adopted and put into effect by the Resolution of the Gosstandart of Russia of 23.04.2002 N 161-st)] (in Russian). URL: http://www.consultant.ru/cons/cgi/online.cgi?req=doc&base=OTN&n=27181#012174768938398195 (accessed 24 May 2021).
6. Monya Baker. 1,500 scientists lift the lid on reproducibility. Nature, 2016, no. 533, pp. 452–454. URL: https://www.nature.com/articles/533452a (accessed 5 July 2021).
7. Zakoldayev D. A., Yamshchikov R. V., Yamshchikova N. V. Tekhnologiya blokcheyna v Rossii: dostizheniya i problemy [Blockchain technology in Russia: achievements and problems]. Vestnik Moskovskogo gosudarstvennogo oblastnogo universiteta – Bulletin of the Moscow Region State University, 2018, no. 2, p. 94 (in Russian). URL: https://www.elibrary.ru/download/elibrary_35395217_74405648.pdf (accessed 5 July 2021).
8. Kosmarsky A. A. Blokcheyn dlya nauki: revolyutsionnyye vozmozhnosti, perspektivy vnedreniya, potentsial’nyye problemy [Blockchain for science: revolutionary opportunities, implementation prospects, potential problems]. Monitoring obshchestvennogo mneniya: Ekonomicheskiye i sotsial’nyye peremeny – Monitoring of Public Opinion: Economic and Social Changes Journal, 2019, no. 2 (150), pp. 388–409 (in Russian). DOI: 10.14515/monitoring.2019.2.16
9. Ferstrate M. Vozmozhnosti i ogranicheniya metaanaliza [Possibilities and limitations of meta-analysis]. Mezhdunarodnyy zhurnal interventsionnoy kardioangiologii – International Journal of Interventional Cardioangiology, 2004, no. 5, pp. 11–15 (in Russian).
10. Omel’yanovskiy V. V., Avksent’yeva M. V., Sura M. V. et al. Metodicheskiye rekomendatsii po provedeniyu meta-analiza [Methodological recommendations for conducting a meta-analysis]. Moscow, FGBU “TSEKKMP” Minzdrava Rossii Publ., 2017. 28 p. (in Russian).
11. Dzhalalov S. Ch., Dzhalalova D. Kh., Khoch D. S. Rol’ sistematicheskogo obzora i metaanaliza pri otsenke tekhnologiy v zdravookhranenii [The role of systematic review and meta-analysis in the assessment of technologies in health care]. Meditsinskiye tekhnologii. Otsenka i vybor – Medical technologies. Assessment and choice, 2012, no. 3, pp. 10–16 (in Russian).
12. Lukyanova A. L. Otdacha ot obrazovaniya: chto pokazyvayet metaanaliz: Preprint WP3/2010/03 [Return on Education: What Meta-Analysis Shows: Preprint WP3/2010/03]. Moscow, State University – Higher School of Economics Publ., 60 p. (in Russian).
13. Otsenka meditsinskikh tekhnologiy. Obshchiye polozheniya. GOST R 56044-2014 [Assessment of medical technologies. General Provisions. GOST R 56044-2014]. Ed. O. A. Stoyanovskaya. Moscow, Standartinform. Publ., 2015. 50 p. (in Russian).
14. Nicholas A. Bowman. College diversity experiences and cognitive development: A meta-analysis [College diversity experiences and cognitive development: A meta-analysis]. Review of Educational Research, March 2010, vol. 80, no. 1, pp. 4–33 (in Russian).
15. Gordeyeva N. O. Ispol’zovaniye mobil’nykh tekhnologiy v obrazovanii: metaanaliz rossiyskikh issledovaniy [The use of mobile technologies in education: a meta-analysis of Russian research]. Sovremennyye problemy nauki i obrazovaniya – Modern problems of science and education, 2018, no. 3,147 p. (in Russian).
( 845.82 kB ) 
( 706.32 kB )
Issue: 5, 2021
Series of issue: Issue 5
Rubric: TEACHER EDUCATION
Pages: 78 — 89
Downloads: 739