The Impact of ChatGPT and Instructor Feedback on ESL Email Writing: A Mixed-Methods Study

AILEEN FARIDA MOHD ADAM, WAN NAJMIYYAH WAN MD ADNAN, MOHD SUHAIMI SULAIMAN

Abstract


This study explores the impact of ChatGPT-generated feedback and hybrid feedback (ChatGPT + instructor) on ESL students’ writing development within a process writing framework. Using a mixed-methods intervention design, 33 Malaysian pre-diploma students completed three email writing tasks: a pre-test (baseline writing), post-test 1 (revised using ChatGPT), and post-test 2 (further revised following instructor feedback). All email responses were assessed using a CEFR-aligned rubric, and student perceptions were collected via a questionnaire containing both close- and open-ended items. Results showed moderate gains in writing performance following ChatGPT feedback, particularly in grammar and vocabulary accuracy. However, the most substantial and consistent improvements were observed after hybrid feedback, especially in coherence, content organisation, and task fulfilment. Questionnaire responses revealed students generally appreciated ChatGPT for its immediacy and clarity but still valued instructor feedback for its personalised and contextualised nature. The findings suggest that combining AI and instructor feedback supports more meaningful revision and writing improvement. This study highlights the potential of integrating generative AI into scaffolded writing instruction and calls for future research to examine hybrid feedback models across broader contexts and proficiency levels.


Full Text:

PDF

References


Bok, E., & Cho, Y. (2023). Examining Korean EFL college students’ experiences and perceptions of using ChatGPT as a writing revision tool. Journal of English Teaching through Movies and Media, 24(4). https://doi.org/10.16875/stem.2023.24.4.15

Creswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). Sage Publications.

Ellis, R. (2009). Corrective feedback and teacher development. L2 Journal, 1(1), 3–18.

Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32(4), 365–387. https://doi.org/10.2307/356600

Gopalan, M., Rosinger, K., & Ahn, J. B. (2020). Use of quasi-experimental research designs in education research: Growth, promise, and challenges. Review of Research in Education, 44(1), 218–243. http://dx.doi.org/10.3102/0091732X20903302

Hyland, F., & Hyland, K. (2001). Sugaring the pill: Praise and criticism in written feedback. Journal of Second Language Writing, 10(3), 185–212. https://doi.org/10.1016/S1060-3743(01)00038-8

Mahapatra, S. (2024). Impact of ChatGPT on ESL students’ academic writing skills: A mixed methods intervention study. Smart Learning Environments, 11(9). https://doi.org/10.1186/s40561-024-00295-9

Mathur, M., & Mahapatra, S. (2022). Impact of ePortfolio assessment as an instructional strategy on students’ academic speaking skills: An experimental study. CALL-EJ, 23(3), 1–23. https://www.researchgate.net/publication/362394646_Impact_of_ePortfolio_Assessment_as_an_Instructional_Strategy_on_Students'_Academic_Speaking_Skills_An_Experimental_Study

McCarthy, K. S., Roscoe, R. D., Allen, L. K., Likens, A. D., & McNamara, D. S. (2022). Automated writing evaluation: Does spelling and grammar feedback support high-quality writing and revision? Assessing Writing, 52, 100608. https://doi.org/10.1016/j.asw.2022.100608

Miao, J., Chang, J., & Ma, L. (2023). Research trends of written corrective feedback in L2 writing: A bibliometric analysis. Sage Open, 13(1). https://doi.org/10.1177/21582440221135172

Molin, F., Haelermans, C., Cabus, S., & Groot, W. (2020). The effect of feedback on metacognition - A randomized experiment using polling technology. Computers & Education, 149, 103885. https://doi.org/10.1016/j.compedu.2020.103885

Najlaa, N. M. R., Aida, M., & Aileen, F. M. A. (2024). Evaluating automated grammar corrective feedback tools: A comparative study of Grammarly and QuillBot in ESL expository essays. Akademika, 94(2), 474–491. https://doi.org/10.17576/akad-2024-9402-27

Nur Ashfaraliana, A. H., Mohamad, F., Md Johar, E., & Abdul Kadir, Z. (2024). Exploring the acceptance of ChatGPT as an assisting tool in academic writing among ESL undergraduate students. International Journal of Research and Innovation in Social Science, 8(10), 2886–2901. https://doi.org/10.47772/IJRISS.2024.8100242

Raheem, B. R., Anjum, F., & Ghafar, Z. N. (2023). Exploring the profound impact of artificial intelligence applications (Quillbot, Grammarly, and ChatGPT) on English academic writing: A systematic review. International Journal of Integrative Research, 1(10), 599–622. http://dx.doi.org/10.59890/ijir.v1i10.366

Shadiev, R., & Feng, Y. (2023). Using automated corrective feedback tools in language learning: A review study. Interactive Learning Environments. https://doi.org/10.1080/10494820.2022.2153145

Shi, H., & Aryadoust, V. (2024). A systematic review of AI-based automated written feedback research. ReCALL, 36(1), 1–23. https://doi.org/10.1017/S0958344023000265

Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on the quality of writing. Assessing Writing, 19(1), 51–65. https://doi.org/10.1016/j.asw.2013.11.007

Storch, N., & Wigglesworth, G. (2010). Learners' processing, uptake, and retention of corrective feedback on writing: Case studies. Studies in Second Language Acquisition, 32(2), 303–334. https://doi.org/10.1017/S0272263109990532

Taskiran, A., & Goksel, N. (2022). Automated feedback and teacher feedback: Writing achievement in learning English as a foreign language at a distance. Turkish Online Journal of Distance Education, 23(2), 108–123. https://doi.org/10.17718/tojde.1096260

Tran, T. T. T. (2025). Enhancing EFL Writing Revision Practices: The Impact of AI- and Teacher-Generated Feedback and Their Sequences. Education Sciences, 15(2), 232. https://doi.org/10.3390/educsci15020232

Turner, E. (2023). Dialogic feedback and literary disciplinary knowledge in L2 writing instruction: How attitude to feedback influences academic achievement. Research Papers in Education, 38(1), 27–46. https://doi.org/10.1080/02671522.2021.1941216

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes (M. Cole, V. John-Steiner, S. Scribner, & E. Souberman, Eds. & Trans.). Harvard University Press. https://doi.org/10.2307/j.ctvjf9vz4

Wang, L., Chen, X., Wang, C., Xu, L., Shadiev, R., & Li, Y. (2024). ChatGPT’s capabilities in providing feedback on undergraduate students’ argumentation: A case study. Thinking Skills and Creativity, 51, 101440. https://doi.org/10.1016/j.tsc.2023.101440

Weigle, S. C. (2002). Assessing writing. Cambridge University Press. https://doi.org/10.1017/CBO9780511732997

Widad, M. A., Azman, C. M., Ghazali Yusri, A. R., & Rozaimi, J. (2024). Exploring the interconnections: Digital competence, teacher training and remote teaching. Akademika, 94(3), 108–123. https://doi.org/10.17576/akad-2024-9403-07

Yao, Y., Yu, S., Zhu, X., Zhu, S., & Pang, W. (2024). Examining the mindsets toward students’ writing competence and feedback-giving practices of Chinese university English writing teachers. Studies in Educational Evaluation, 80, 101318. https://doi.org/10.1016/j.stueduc.2023.101318

Zhang, Z., & Hyland, K. (2018). Student engagement with teacher and automated feedback on L2 writing. Assessing Writing, 36, 90–102. https://doi.org/10.1016/j.asw.2018.02.004

Zhang, Z., Aubrey, S., Huang, X., & Chiu, T. K. F. (2025). The role of generative AI and hybrid feedback in improving L2 writing skills: A comparative study. Innovation in Language Learning and Teaching. https://doi.org/10.1080/17501229.2025.2503890

Zimmerman, B. J., & Labuhn, A. S. (2012). Self-regulation of learning: Process approaches to personal development. In K. R. Harris, S. Graham, T. Urdan, C. B. McCormick, G. M. Sinatra, & J. Sweller (Eds.), APA educational psychology handbook, Vol. 1: Theories, constructs, and critical issues (pp. 399–425). American Psychological Association. https://doi.org/10.1037/13273-014

Zumbrunn, S., Marrs, S., & Mewborn, C. (2016). Toward a better understanding of student perceptions of writing feedback: A mixed methods study. Reading and Writing, 29(2), 349–370. https://doi.org/10.1007/s11145-015-9599-3


Refbacks

  • There are currently no refbacks.