Effects of AI-generated feedback on l2 writing: a PRISMA 2020 systematic review of performance, revision, and engagement outcomes

  • Kristian Burhan 
    Universitas Negeri Padang
    Indonesia

Abstract

Generative artificial intelligence (GenAI) tools can provide immediate feedback during second-language (L2) writing, but evidence on their effects across writing outcomes remains fragmented. This PRISMA 2020 systematic review synthesized peer-reviewed empirical studies published from 2023 to 2025 that examined GenAI-generated feedback for English writing in EFL/ESL contexts. Searches of Scopus, PubMed, and ScienceDirect identified 1,106 records; after 12 duplicates were removed, 1,094 records were screened, 71 full texts were assessed, and 20 studies met the eligibility criteria. Eligible studies were experimental, quasi-experimental, mixed-methods, qualitative, or design-based investigations reporting writing performance, revision, feedback engagement/literacy, or AI-feedback accuracy. Because study designs, writing tasks, outcome measures, and reported statistics were heterogeneous, statistical meta-analysis and pooled effect-size estimation were not conducted; findings were synthesized thematically and by effect direction. The evidence indicates that GenAI feedback most consistently supports local accuracy, lexical development, revision activity, and learner engagement, whereas effects on syntactic complexity, higher-order composing, and long-term retention are mixed or conditional. Comparative evidence generally supports a complementary model in which AI provides rapid, high-volume local feedback and teachers address global structure, disciplinary expectations, contextual appropriateness, and affective support. Key limitations of the evidence base include short intervention periods, small samples, heterogeneous measurement, limited longitudinal evidence, and uneven geographic representation. Future research should preregister screening procedures, use independent double screening with agreement statistics, report extractable effect sizes and delayed post-tests, and examine equity, transfer, and hybrid AI-human feedback models.

Keywords
  • Generative artificial intelligence, ChatGPT, Automated writing evaluation, EFL/ESL writing, Written corrective feedback
Lisensing Information
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

References
  1. Abdi Tabari, M., Kushki, A., & Wang, Y. (2025). Comparing the effects of teacher- and AI-mediated corrective feedback on accuracy, complexity, and quality in L2 written narratives. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2025.2561608
  2. Abduljawad, S. A. (2024). Investigating the Impact of ChatGPT as an AI Tool on ESL Writing: Prospects and Challenges in Saudi Arabian Higher Education. International Journal of Computer-Assisted Language Learning and Teaching. https://doi.org/10.4018/IJCALLT.367276
  3. Albadarin, Y., Saqr, M., Pope, N., & Tukiainen, M. (2024). A systematic literature review of empirical research on ChatGPT in education. Discover Education, 3, 60. https://doi.org/10.1007/s44217-024-00138-2
  4. Alhamam, A. A. (2025). Investigating the Role of AI Tool Interactions in Enhancing English Language Acquisition Among Saudi College Students. Arab World English Journal. https://doi.org/10.24093/awej/vol16no3.26
  5. Aljohani, N. J. (2025). Enhancing Writing Accuracy and Empowering Students: The Transformative Influence of ChatGPT’s Informative Feedback on Students’ Writing. Forum for Linguistic Studies. https://doi.org/10.30564/fls.v7i7.9849
  6. Alsaweed, W., & Aljebreen, S. (2024). Investigating the Accuracy of ChatGPT as a Writing Error Correction Tool. International Journal of Computer-Assisted Language Learning and Teaching. https://doi.org/10.4018/IJCALLT.364847
  7. Annamalai, N., Rashid, R. A., Munir Hashmi, U., Mohamed, M., Harb Alqaryouti, M., & Eddin Sadeq, A. (2023). Using chatbots for English language learning in higher education. Computers and Education: Artificial Intelligence, 5, 100153. https://doi.org/10.1016/j.caeai.2023.100153
  8. Balcı, Ö. (2024). The role of ChatGPT in English as a Foreign Language (EFL) learning and teaching: A systematic review. International Journal of Current Educational Studies, 3(1), 66–82. https://doi.org/10.46328/ijces.107
  9. Baz, M. A., & Hasirci Aksoy, S. (2025). The Effect of Feedback on Informative Text Writing: AI or Teacher?. Open Praxis. https://doi.org/10.55982/openpraxis.17.3.871
  10. Bhutoria, A. (2022). Personalized education and Artificial Intelligence in the United States, China, and India: A systematic review using a Human-In-The-Loop model. Computers and Education: Artificial Intelligence, 3, Article 100068. https://doi.org/10.1016/j.caeai.2022.100068
  11. Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: the state of the field. International Journal of Educational Technology in Higher Education, 20(1), Article 22. https://doi.org/10.1186/s41239-023-00392-8
  12. Crompton, H., Edmett, A., Ichaporia, N., & Burke, D. (2024). AI and English language teaching: Affordances and challenges. British Journal of Educational Technology, 55(6), 2503–2529. https://doi.org/10.1111/bjet.13460
  13. Dang, T. V. A., Haworth, P., & Ashton, K. (2024). Pre-service teachers’ belief changes in an English for specific purposes teacher education context. English for Specific Purposes, 73, 110–123. https://doi.org/10.1016/j.esp.2023.10.009
  14. Derakhshan, A., & Ghiasvand, F. (2024). Is ChatGPT an evil or an angel for second language education and research? A phenomenographic study of research-active EFL teachers’ perceptions. International Journal of Applied Linguistics (United Kingdom), 34(4), 1246–1264. https://doi.org/10.1111/ijal.12561
  15. Deygers, B., Buelens, L., Chan, D., Schildt, L., Van Parys, A., & Vanbuel, M. (2025). The impact of automated writing evaluation on writing gains. ELT Journal. https://doi.org/10.1093/elt/ccaf020
  16. Escalante, J., Pack, A., & Barrett, A. (2023). AI-generated feedback on writing: insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-023-00425-2
  17. Fazackerley, L. A., Perrin, D., & Minett, G. M. (2025). Harnessing generative AI in exercise and sports science education: enhancing real-world learning and overcoming traditional barriers in data analysis. Advances in physiology education, 49(2), 496-502. https://doi.org/10.1152/advan.00249.2024
  18. Feng, W., Lai, X., Zhang, X., Fan, X., & Du, Y. (2025). Research on the construction and application of intelligent tutoring system for english teaching based on generative pre-training model. Systems and Soft Computing, 7, 200232. https://doi.org/10.1016/j.sasc.2025.200232
  19. Fudiyartanto, F. A. (2024). English language teaching in a globalised world: A sociological investigation of Indonesian higher education. International Journal of Educational Research, 127, 102395. https://doi.org/10.1016/j.ijer.2024.102395
  20. Gozali, I., Wijaya, A. R. T., Lie, A., Cahyono, B. Y., & Suryati, N. (2024). ChatGPT as an automated writing evaluation (AWE) tool: feedback literacy development and AWE tools’ integration framework. JALT CALL Journal. https://doi.org/10.29140/jaltcall.v20n1.1200
  21. Guo, K., Pan, M., Li, Y., & Lai, C. (2024). Effects of an AI-supported approach to peer feedback on university EFL students' feedback quality and writing ability. Internet and Higher Education. https://doi.org/10.1016/j.iheduc.2024.100962
  22. Guo, Y., & Wang, Y. (2025). Exploring the Effects of Artificial Intelligence Application on EFL Students' Academic Engagement and Emotional Experiences: A Mixed-Methods Study. European Journal of Education, 60(1), Article e12812. https://doi.org/10.1111/ejed.12812
  23. Guo, K., Zhang, E. D., Li, D., & Yu, S. (2025). Using AI-supported peer review to enhance feedback literacy: An investigation of students' revision of feedback on peers' essays. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13540
  24. Haddaway, N. R., Page, M. J., Pritchard, C. C., & McGuinness, L. A. (2022). PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams. Campbell Systematic Reviews, 18(2), e1230. https://doi.org/10.1002/cl2.1230
  25. Holland, A. M., Lorenz, W. R., Cavanagh, J. C., Smart, N. J., Ayuso, S. A., Scarola, G. T., Kercher, K. W., Jorgensen, L. N., Janis, J. E., Fischer, J. P., & Heniford, B. T. (2024). Comparison of Medical Research Abstracts Written by Surgical Trainees and Senior Surgeons or Generated by Large Language Models. JAMA network open, 7(8), e2425373. https://doi.org/10.1001/jamanetworkopen.2024.25373
  26. Hong, Q. N., Gonzalez-Reyes, A., & Pluye, P. (2018). Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). Journal of Evaluation in Clinical Practice, 24(3), 459–467. https://doi.org/10.1111/jep.12884
  27. Huang, X., Zou, D., Cheng, G., Chen, X., & Xie, H. (2023). Trends, Research Issues and Applications of Artificial Intelligence in Language Education. Educational Technology and Society, 26(1), 112–131. https://doi.org/10.30191/ETS.202301_26(1).0009
  28. Jaashan, H. M. S., & Alashabi, A. A. (2025). Using AI Large Language Model (LLM-ChatGPT) to Mitigate Spelling Errors of EFL Learners. Forum for Linguistic Studies. https://doi.org/10.30564/fls.v7i3.8438
  29. Jeon, J. (2024). Exploring AI chatbot affordances in the EFL classroom: young learners’ experiences and perspectives. Computer Assisted Language Learning, 37(1-2), 1–26. https://doi.org/10.1080/09588221.2021.2021241
  30. Jeon, J., Lee, S., & Choe, H. (2023). Beyond ChatGPT: A conceptual framework and systematic review of speech-recognition chatbots for language learning. Computers and Education, 206, Article 104898. https://doi.org/10.1016/j.compedu.2023.104898
  31. Karataş, F., Abedi, F. Y., Ozek Gunyel, F., Karadeniz, D., & Kuzgun, Y. (2024). Incorporating AI in foreign language education: An investigation into ChatGPT’s effect on foreign language learners. Education and Information Technologies, 29(15), 19343–19366. https://doi.org/10.1007/s10639-024-12574-6
  32. Khoso, A. K., Honggang, W., & Darazi, M. A. (2025). Empowering creativity and engagement: The impact of generative artificial intelligence usage on Chines EFL students' language learning experience. Computers in Human Behavior Reports, 18, 100627. https://doi.org/10.1016/j.chbr.2025.100627
  33. Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering (EBSE Technical Report EBSE-2007-01). Keele University and University of Durham.
  34. Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). Exploring generative artificial intelligence preparedness among university language instructors: A case study. Computers and Education: Artificial Intelligence, 5, Article 100156. https://doi.org/10.1016/j.caeai.2023.100156
  35. Kurt, G., & Kurt, Y. (2024). ENHANCING L2 WRITING SKILLS: CHATGPT AS AN AUTOMATED FEEDBACK TOOL. Journal of Information Technology Education: Research. https://doi.org/10.28945/5370
  36. Lai, W. Y. W., & Lee, J. S. (2024). A systematic review of conversational AI tools in ELT: Publication trends, tools, research methods, learning outcomes, and antecedents. Computers and Education: Artificial Intelligence, 7, 100291. https://doi.org/10.1016/j.caeai.2024.100291
  37. Law, L. H. L. (2024). Application of generative artificial intelligence (GenAI) in language teaching and learning: A scoping literature review. Computers and Education Open, 6, 100174. https://doi.org/10.1016/j.caeo.2024.100174
  38. Li, B., Lowell, V. L., Wang, C., & Li, X. (2024). A systematic review of the first year of publications on ChatGPT and language education: Examining research on ChatGPT’s use in language learning and teaching. Computers and Education: Artificial Intelligence, 7, 100266. https://doi.org/10.1016/j.caeai.2024.100266
  39. Liebenow, L. W., Schmidt, F. T. C., Meyer, J., & Fleckenstein, J. (2025). Self-assessment accuracy in the age of artificial Intelligence: Differential effects of LLM-generated feedback. Computers and Education. https://doi.org/10.1016/j.compedu.2025.105385
  40. Liu, M., Zhang, L. J., & Biebricher, C. (2024). Investigating students’ cognitive processes in generative AI-assisted digital multimodal composing and traditional writing. Computers and Education, 211, Article 104977. https://doi.org/10.1016/j.compedu.2023.104977
  41. Lo, C. K., Hew, K. F., & Jong, M. S.-Y. (2024). The influence of ChatGPT on student engagement: A systematic review and future research agenda. Computers & Education, 219, 105100. https://doi.org/10.1016/j.compedu.2024.105100
  42. Lo, C. K., Yu, P. L. H., Xu, S., Ng, D. T. K., & Jong, M. S.-Y. (2024). Exploring the application of ChatGPT in ESL/EFL education and related research issues: A systematic review of empirical studies. Smart Learning Environments, 11, 50. https://doi.org/10.1186/s40561-024-00342-5
  43. Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
  44. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71
  45. Perales, W., & Bedoya Ulla, M. (2025). Technology in EFL contextual teaching and learning: Teachers’ practices and perspectives in a Thai university. Heliyon, 11, e44169. https://doi.org/10.1016/j.heliyon.2025.e44169
  46. Perkins, M. (2023). Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond. Journal of University Teaching and Learning Practice, 20(2), Article 7. https://doi.org/10.53761/1.20.02.07
  47. Puteri, C. G., Cahyono, B. Y., Widiati, U., & Suryati, N. (2025). Using ChatGPT as a writing assistant: A study on essay quality development among Indonesian EFL University Students. Indonesian Journal of Applied Linguistics. https://doi.org/10.17509/3ebmy158
  48. Seo, J. Y. (2024). Exploring the Educational Potential of ChatGPT: AI-Assisted Narrative Writing for EFL College Students. Language Teaching Research Quarterly. https://doi.org/10.32038/ltrq.2024.43.01
  49. Sharadgah, T. A., & Sa’di, R. A. (2022). A SYSTEMATIC REVIEW OF RESEARCH ON THE USE OF ARTIFICIAL INTELLIGENCE IN ENGLISH LANGUAGE TEACHING AND LEARNING (2015-2021): WHAT ARE THE CURRENT EFFECTS?. Journal of Information Technology Education: Research, 21, 337–377. https://doi.org/10.28945/4999
  50. Teng, M. F. (2025). A Systematic Review of ChatGPT for English as a Foreign Language Writing: Opportunities, Challenges, and Recommendations. International Journal of TESOL Studies, 6(3), 36–57. https://doi.org/10.58304/ijts.20240304
  51. Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8, 45. https://doi.org/10.1186/1471-2288-8-45
  52. Tran, T. T. T. (2025). Enhancing EFL Writing Revision Practices: The Impact of AI- and Teacher-Generated Feedback and Their Sequences. Education Sciences. https://doi.org/10.3390/educsci15020232
  53. Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222. https://doi.org/10.1111/1467-8551.00375
  54. Tsai, C. Y., Lin, Y. T., & Brown, I. K. (2024). Impacts of ChatGPT-assisted writing for EFL English majors: Feasibility and challenges. Education and Information Technologies. https://doi.org/10.1007/s10639-024-12722-y
  55. Wang, P., Liu, T., Yang, Y., & Xiang, X. (2025). Optimizing self-regulated learning: A mixed-methods study on GAI's impact on undergraduate task strategies and metacognition. British Journal of Educational Technology. https://doi.org/10.1111/bjet.70018
  56. Weng, X., & Chiu, T. K. F. (2023). Instructional design and learning outcomes of intelligent computer assisted language learning: Systematic review in the field. Computers and Education: Artificial Intelligence, 4, 100117. https://doi.org/10.1016/j.caeai.2022.100117
  57. Xiao, Y., & Zhi, Y. (2023). An Exploratory Study of EFL Learners’ Use of ChatGPT for Language Learning Tasks: Experience and Perceptions. Languages, 8(3), Article 212. https://doi.org/10.3390/languages8030212
  58. Yang, H., & Kyun, S. (2022). The current research trend of artificial intelligence in language learning: A systematic empirical literature review from an activity theory perspective. Australasian Journal of Educational Technology, 38(5), 180–210. https://doi.org/10.14742/ajet.7492
  59. Zhai, C., & Wibowo, S. (2023). A systematic review on artificial intelligence dialogue systems for enhancing English as foreign language students’ interactional competence in the university. Computers and Education: Artificial Intelligence, 4, Article 100134. https://doi.org/10.1016/j.caeai.2023.100134
  60. Zhai, C., Wibowo, S., & Li, L. D. (2024). Evaluating the AI dialogue System's intercultural, humorous, and empathetic dimensions in English language learning: A case study. Computers and Education: Artificial Intelligence, 7, 100262. https://doi.org/10.1016/j.caeai.2024.100262
  61. Zhang, S., Shan, C., Lee, J. S. Y., Che, S. P., & Kim, J. H. (2023). Effect of chatbot-assisted language learning: A meta-analysis. Education and Information Technologies, 28(11), 15223–15243. https://doi.org/10.1007/s10639-023-11805-6
  62. Zhu, M., & Wang, C. (2025). A systematic review of research on AI in language education: Current status and future implications. Language Learning and Technology, 29(1), 1–29.
  63. Zhuang, Y., Zhao, R., Xie, Z., & Yu, P. L. H. (2025). Enhancing language learning through generative AI feedback on picture-cued writing tasks. Computers and Education: Artificial Intelligence. https://doi.org/10.1016/j.caeai.2025.100450

Downloads

PDF

Statistic

Abstract Views
: 2 times
PDF
: 0 times