Generative AI use in college writing classes: An analysis of student chat logs and writing projects
DOI:
https://doi.org/10.17239/jowr-2026.17.03.05Keywords:
generative AI, writing pedagogy, higher education, English as a foreign languageAbstract
This study contributes to the emerging research on generative AI and writing pedagogy by exploring how college writing students make use of GAI when offered instruction in a range of responsible uses and latitude to integrate it into their writing process as they see fit. We analyzed chat log data and papers from participants recruited from six sections in which students were guided in experimenting with ChatGPT Plus and permitted to use it to produce up to 50% of submitted work. Through a combination of AI and human thematic content analysis of student chat logs, we found that in 18.6% of prompts, students asked ChatGPT to write for them. The rest of the prompts involved work leading up to or in support of the writing process. Human thematic content analysis of papers showed that students used ChatGPT to generate 8.2% of the writing they submitted. The most common rhetorical purpose of the AI-generated text they included was discussion/analysis/synthesis. English as a foreign language students (EFLs) in the sample prompted ChatGPT to clarify understanding less often than non-EFLs and integrated less AI-generated text into their papers, with a particularly notable difference in their use of AI-generated summaries. This unexpected finding merits further research, but it suggests that EFLs may use GAI for somewhat different purposes than non-EFL peers.
References
Almumen, H. ., & Jouhar, M. (2025). The Impact of ChatGPT as a Brainstorming Tool on Gifted Students’ Persuasive Writing. Journal of Writing Research, 17(2), 181-207. https://doi.org/10.17239/jowr-2025.17.02.01
Boillos, M.M., & Idoiaga, N. (2025) Student perspectives on the use of AI-based language tools in academic writing. Journal of Writing Research.https://doi.org/10.17239/jowr-2025.17.01.06
Chew, R., Bollenbacher, J., Wenger, M., Speer, J., & Kim, A. (2023). LLM-assisted content analysis: Using large language models to support deductive coding. arXiv preprint arXiv:2306.14924. https://doi.org/10.48550/arXiv.2306.14924.
Črček, N., & Patekar J. (2023). Writing with AI: University Students’ Use of ChatGPT. Journal of Language and Education, 9(4), 128-138. https://doi.org/10.17323/jle.2023.17379
Fu, Y. (2024). A Research of the Impact of ChatGPT on Education. Applied and Computational Engineering, 35, 26-31. https://doi.org/10.54254/2755-2721/35/20230354
Gegg‑Harrison, W., & Shapiro, S. (2025). From policing to empowerment: Promoting student
agency in the context of AI text‑generators and AI‑detection tools. In C. Wang & Z. Tian (Eds.), Rethinking writing education in the age of generative AI (pp. 26–41). Routledge. https://doi.org/10.4324/9781003426936-4
Ghosh, S. (2024, June), ChatGPT as a Tool for Equitable Education in Engineering Classes. Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://doi.org/10.18260/1-2--48458
Harvard Business Publishing. (2024). How students are actually using generative AI. https://hbsp.harvard.edu/inspiring-minds/how-students-are-actually-using-generative-ai
Helm, G., & Hesse, F. (2025). Training programmes on writing with AI – but for whom? Identifying students’ writer profiles through two-step cluster analysis. Journal of Writing Research, 17(1), 1-21. https://doi.org/10.17239/jowr-2025.17.01.01
Hutson, J., Plate, D. ., & Berry, K. (2024). Embracing AI in English Composition: Insights and Innovations in Hybrid Pedagogical Practices. International Journal of Changes in Education, 1(1), 19-31.
Joakar, Ajit. (2024, October 5) A scaffolding based pedagogy for AI assisted education. https://www.linkedin.com/pulse/scaffolding-based-pedagogy-ai-assisted-education-ajit-jaokar-yfnle/
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7). https://doi.org/10.1016/j.patter.2023.100779
Nguyen, A., Hong, Y., Dang, B., & Huang, X. (2024). Human-AI collaboration patterns in AI-assisted academic writing. Studies in Higher Education, 49(5), 847–864. https://doi.org/10.1080/03075079.2024.2323593
Nikolopoulou, K. (2024). Generative Artificial Intelligence in Higher Education: Exploring Ways of Harnessing Pedagogical Practices with the Assistance of ChatGPT. International Journal of Changes in Education, 1(2), 103-111. https://doi.org/10.47852/bonviewIJCE42022489
Pattyn, F. (2024). The value of generative AI for qualitative research: A pilot study. Journal of Data Science and Intelligent Systems. https://doi.org/10.47852/bonviewJDSIS4202964
Prescott, M. R., Yeager, S., Ham, L., Rivera Saldana, C. D., Serrano, V., Narez, J., ... & Montoya, J. (2024). Comparing the efficacy and efficiency of human and generative AI: Qualitative thematic analyses. JMIR AI, 3, e54482. https://doi.org/10.2196/54482
Sano-Franchini, J., McIntyre, M., & Fernandes, M. (2024, November). Refusing GenAI in writing studies: A quickstart guide. Refusing Generative AI in Writing Studies. https://refusinggenai.wordpress.com
Selim, A. S. M. (2024). The Transformative Impact of AI-Powered Tools on Academic Writing: Perspectives of EFL University Students. International Journal of English Linguistics, 14(1), 14-29. https://doi.org/10.5539/ijel.v14n1p14
Usher, M., & Amzalag, M. (2025). From Prompt to Polished: Exploring Student–Chatbot Interactions for Academic Writing Assistance. Education Sciences, 15(3), 329. https://doi.org/10.3390/educsci15030329
Wang, L., & Ren, B. (2024). Enhancing academic writing in a linguistics course with generative AI: An empirical study in a higher education institution in Hong Kong. Education Sciences, 14(12), Article 1329. https://doi.org/10.3390/educsci14121329
Wang, C., Aguilar, S. J., Bankard, J. S., Bui, E., & Nye, B. (2024). Writing with AI: What College Students Learned from Utilizing ChatGPT for a Writing Assignment. Education Sciences, 14(9), 976. https://doi.org/10.3390/educsci14090976
Warschauer, M., Tseng, W., Yim, S., Webster, T., Jacob, S., Du, Q., & Tate, T. (2023). The
affordances and contradictions of AI-generated text for writers of English as a second or foreign language. Journal of Second Language Writing, 62, 101071. https://doi.org/10.1016/j.jslw.2023.101071
Yang, K., Raković, M., Liang, Z., Yan, L., Zeng, Z., Fan, Y., Gašević, D., & Chen, G. (2024). Modifying AI, enhancing essays: How active engagement with generative AI boosts writing quality. arXiv Preprint. https://doi.org/10.48550/arXiv.2412.07200
Yang, M., Jiang, S., Li, B., Herman, K., Luo, T., Moots, S. C., & Lovett, N. (2025). Analysing nontraditional students' ChatGPT interaction, engagement, self-efficacy and performance: A mixed-methods approach. British Journal of Educational Technology, 00, 1–28. https://doi.org/10.1111/bjet.13588
Xiao, Z., Yuan, X., Liao, Q. V., Abdelghani, R., & Oudeyer, P. Y. (2023, March). Supporting qualitative analysis with large language models: Combining codebook with GPT-3 for deductive coding. In Companion proceedings of the 28th international conference on intelligent user interfaces (pp. 75-78). https://doi.org/10.1145/3581754.3584136
Xu, T. S., Zhang, L. J., & Gaffney, J. S. (2022). Examining the Relative Effects of Task Complexity and Cognitive Demands on Students’ Writing in a Second Language. Studies in Second Language Acquisition, 44(2), 483–506. https://doi.org/10.1017/S0272263121000310
Zhai, C., Wibowo, S., & Li, L. D. (2024). The effects of over-reliance on AI dialogue systems on students' cognitive abilities: A systematic review. Smart Learning Environments, 11(28). https://doi.org/10.1186/s40561-024-00316-7
Published
Issue
Section
License
Copyright (c) 2026 Sarah Madsen Hardy, Pary Fassihi, Shuang Geng, Christopher McVey, Matt Parfitt
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 Unported License.
