Making sense of L2 written argumentation with keystroke logging


  • Yu Tian Georgia State University
  • Minkyung Kim Korea National University of Education | South Korea
  • Scott Crossley Vanderbilt University



L2 written argumentation, Keystroke logging, Cognitive activities


This study examines associations between writing behaviors manifested by keystroke analytics and the formulation of argument elements in L2 undergraduate writers' writing processes. Ninety-nine persuasive essays written by L2 undergraduate writers were human annotated for Toulmin argument elements. The corresponding keystroke logs were segmented and analyzed to characterize the dynamics of writing processes for different categories of the elements. A multinomial mixed-effects logistic regression model was built to predict argument categories using the keystroke analytics. The study reported that L2 undergraduate writers' text production for final claims and primary claims featured P-bursts (execution processes delimited by pauses exceeding 2 seconds) of longer spans but lower production fluency compared to that for data. In addition, fewer revisions were observed when L2 writers were constructing final claims than when they were formulating data. These findings shed light on the varying cognitive loads and activities L2 undergraduate writers may experience when building different argument elements in written argumentation.


Andriessen, J., Baker, M., & Suthers, D. (2003). Argumentation, computer support, and the educational context of confronting cognitions. In J. Andriessen, M. Baker, & D. Suthers (Eds.), Arguing to learn: Confronting cognitions in computer-supported collaborative learning environments (pp. 1–25). Boston: Kluwer.

Alvès, R. A., Castro, S. L., & Olive, T. (2008). Execution and pauses in writing narratives: Processing time, cognitive effort and typing skill. International Journal of Psychology, 43, 969-979.

Barkaoui, K. (2019). What can L2 writers’ pausing behavior tell us about their L2 writing processes?. Studies in Second Language Acquisition, 41(3), 529-554.

Bayerl, P. S., Lüngen, H., Gut, U., & Paul, K. I. (2003, October). Methodology for reliable schema development and evaluation of manual annotations. In Proceedings of the Workshop on Knowledge Markup and Semantic Annotation at the Second International Conference on Knowledge Capture (K-CAP 2003).

Bell, P., & Linn, M. C. (2000). Scientific arguments as learning artifacts: Designing for learning from the web with KIE. International Journal of Science Education, 22(8), 797–818.

Breuer, E. O. (2019). Fluency in L1 and FL writing: An analysis of planning, essay writing and final revision. In E. Lindgren & K. P. H. Sullivan (Eds.), Observing Writing (pp. 190-211). Brill.

Brooks, S. (1998). Markov chain Monte Carlo method and its application. Journal of the royal statistical society: series D (the Statistician), 47(1), 69-100.

Chan, S. (2017). Using keystroke logging to understand writers’ processes on a reading-into-writing test. Language Testing in Asia, 7(1), 1-27.

Chandrasegaran, A. (2008). NNS students’ arguments in English: Observations in formal and informal contexts. Journal of Second Language Writing, 17(4), 237–254.

Chenoweth, N. A., & Hayes, J. R. (2001). Fluency in writing: Generating text in L1 and L2. Written Communication, 18, 80-98.

Chukharev-Khudilaynen, E. (2014). Pauses in spontaneous written communication: A keystroke logging study. Journal of Writing Research, 6(1), 61.

Chukharev-Hudilainen, E. (2019). Empowering automated writing evaluation with keystroke logging. In E. Lindgren & K. P. H. Sullivan (Eds.), Observing writing (pp.125-142). Brill.

Chukharev-Hudilainen, E., Saricaoglu, A., Torrance, M., & Feng, H. H. (2019). Combined deployable keystroke logging and eyetracking for investigating L2 writing fluency. Studies in Second Language Acquisition, 41(3), 583-604.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1), 37-46.

Coirier, P., Andriessen, J., & Chanquoy, L. (1999). From planning to translating: The specificity of argumentative writing. Foundations of argumentative text processing, 1-28.

Crammond, J. G. (1998). The uses and complexity of argument structures in expert and student persuasive writing. Written Communication, 15(2), 230-268.

Crossley, S., Tian, Y., & Wan, Q. (2022). Argumentation Features and Essay Quality: Exploring Relationships and Incidence Counts. Journal of Writing Research.

Dich, N., & Pedersen, B. (2013). Native Language Effects on Spelling in English as a Foreign Language: A Time-Course Analysis. Canadian Journal of Applied Linguistics/Revue canadienne de linguistique appliquee, 16(1), 51-68.

Ellis, R. (2003). Task-based language learning and teaching. Oxford: Oxford University Press.

Favart, M., & Coirier, P. (2006). Acquisition of the linearization process in text composition in third to ninth graders: effects of textual superstructure and macrostructural organization. Journal of psycholinguistic research, 35(4), 305.

Ferretti, R. P., & Fan, Y. (2016). Argumentative writing. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (2nd ed., pp. 301–315). New York: Guilford Press.

Ferretti, R. P., MacArthur, C. A., & Dowdy, N. S. (2000). The effects of an elaborated goal on the persuasive writing of students with learning disabilities and their normally achieving peers. Journal of Educational Psychology, 92(4), 694.

Ferretti, R. P., Lewis, W. E., & Andrews-Weckerly, S. (2009). Do goals affect the structure of students’ argumentative writing strategies?. Journal of Educational Psychology, 101(3), 577.

Fisher, A. (2011). Critical thinking: An introduction (2nd ed.). Cambridge: Cambridge University Press.

Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College composition and communication, 32(4), 365-387.

Galbraith, D., & Baaijen, V. M. (2019). Aligning keystrokes with cognitive processes in writing. In E. Lindgren & K. P. H. Sullivan (Eds.), Observing Writing (pp. 306-325). Brill.

Godo, A. (2008). Cross-cultural aspects of academic writing: A study of Hungarian and North American college students LI argumentative essays. International Journal of English Studies, 8, 2, pp. 65-111.

Hadfield, J. D. (2010). MCMC methods for multi-response generalized linear mixed models: the MCMCglmm R package. Journal of statistical software, 33, 1-22.

Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing. In M. Levy & S. Ransdell (Eds.), The science of writing (pp. 1-27). Mahwah, NJ: Erlbaum.

Hayes, J. R. (2012). Modeling and remodeling writing. Written Communication, 29(3), 369–388.

Hillocks, G., Jr. (2011). Teaching argument writing, grades 6-12. Portsmouth, NH: Heinemann.

Hirvela, A. (2017). Argumentation & second language writing: Are we missing the boat? Journal of Second Language Writing, 36, 69–74.

Hoang, H. (2019). Metaphorical language in second language learners' texts: Additional baggage of the writing journey? In E. Lindgren & K. P. H. Sullivan (Eds.), Observing Writing (pp. 236-257). Brill.

Jimenez-Aleixandre, M., Rodriguez, M., & Duschl, R. A. (2000). “Doing the lesson” or “doing science”: Argument in high school genetics. Science Education, 84(6), 757–792.<757::aid-sce5>;2-f

Jonassen, D. H., & Kim, B. (2010). Arguing to learn and learning to argue: Design justifications and guidelines. Educational Technology Research and Development, 58(4), 439-457.

Kaufer, D. S., Hayes, J. R., & Flower, L. (1986). Composing written sentences. Research in the Teaching of English, 20, 121-140.

Kellogg, R. T. (1994). The psychology of writing. Oxford: Oxford University Press.

Kim, M., Tian, Y., & Crossley, S. A. (2021). Exploring the relationships among cognitive and linguistic resources, writing processes, and written products in second language writing. Journal of Second Language Writing, 53, 100824.

Knudson, R. E. (1992). Analysis of argumentative writing at two grade levels. The Journal of Educational Research, 85(3), 169-179.

Kubota, R. (2010). Cross-cultural perspectives on writing. In N. H. Hornberger & S. Lee Mckay (Eds.), Sociolinguistics and language education (pp. 265-289) UK: Multilingual Matters.

Kuhn, M., & Johnson, K. (2013). Applied predictive modeling, New York, NY: Springer.

Kunnan, A. J. (2010). Test fairness and Toulmin’s argument structure. Language Testing, 27, 183–189.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. biometrics, 159-174.

Lee, S. H. (2014). Argument structure as an interactive resource by undergraduate students. Linguistics & the Human Sciences, 9(3).

Leijten, M., & Van Waes, L. (2013). Keystroke logging in writing research: Using Inputlog to analyze and visualize writing processes. Written Communication, 30(3), 358-392.

Leijten, M., Van Waes, L., & Ransdell, S. (2010). Correcting text production errors: Isolating the effects of writing mode from error span, input mode, and lexicality. Written Communication, 27(2), 189-227.

Levishina, N. (2015). Bayesian logistic models with MCMCglmm: A brief tutorial. Retrieved from

Limpo, T., & Alvès, R. A. (2017). Tailoring Multicomponent Writing Interventions: Effects of Coupling Self-Regulation and Transcription Training. Journal of Learning Disabilities, 51(4), 381-398.

Lindgren, E., Sullivan, K. P. H., & Stevenson, M. (2008). Supporting the reflective language learner with computer keystroke logging. In B. Barber & F. Zhang (Eds.), Handbook of research on computer enhanced language acquisition and learning (pp. 189–204). Hershey, NY: Information Science Reference, IGI Global.

Lindgren, E. & Sullivan K. P. H. (2006). Analyzing on-line revision. In G. Rijlaarsdam (Series Ed.) and K. P. H. Sullivan, & E. Lindgren. (Vol. Eds.), Studies in Writing, Vol.18, Computer Keystroke Logging: Methods and Applications, (157–188). Oxford: Elsevier.

Lindgren, E., & Sullivan, K. P. H. (Eds.). (2019). Observing writing: Insights from keystroke logging and handwriting. Brill.

Linnemann, M. (2019). Application of audience during writing. In E. Lindgren & K. P. H. Sullivan (Eds.), Observing Writing (pp. 326-345). Brill.

Liu, F., & Stapleton, P. (2014). Counterargumentation and the cultivation of critical thinking in argumentative writing: Investigating washback from a high-stakes test. System, 45, 117–128.

MacArthur C. A., & Graham, S. (2016). Writing research from a cognitive perspective. In C. A. MacArthur, S. Graham, S., & J. Fitzgerald, J. (Eds.), Handbook of writing research (pp. 24–40). NY: Guilford Press.

MacArthur, C. A., Graham, S., & Fitzgerald, J. (Eds.). (2008). Handbook of writing research. Guilford Press.

McCutchen, D. (1996). A capacity theory of writing: Working memory in composition. Educational psychology review, 8(3), 299-325.

McElreath, R. (2020). Statistical rethinking: A Bayesian course with examples in R and Stan. Chapman and Hall/CRC.

Medimorec, S., & Risko, E. F. (2016). Effects of disfluency in writing. British Journal of Psychology, 107(4), 625-650.

Newell, G. E., Beach, R., Smith, J., VanDerHeide, J., Kuhn, D., & Andriessen, J. (2011). Teaching and Learning Argumentative Reading and Writing: A Review of Research. Reading Research Quarterly, 46(3), 273–304.

Nippold, M. A., & Ward-Lonergan, J. M. (2010). Argumentative writing in pre-adolescents: The role of verbal reasoning. Child Language Teaching and Therapy, 26, 238–248.

Nussbaum, E.M. (2008).Using argumentation vee diagrams (AVDs) for promoting argument-counterargument integration in reflective writing. Journal of Educational Psychology, 100, 3, 549–565.

Nussbaum, E. M., & Kardash, C. M. (2005). The Effects of Goal Instructions and Text on the Generation of Counterarguments During Writing. Journal of Educational Psychology, 97(2), 157–169.

Nussbaum, E. M., & Schraw, G. (2007). Promoting argument-counterargument integration in students' writing. The Journal of Experimental Education, 76(1), 59-92.

Olive, T., Alvès, R. A., & Castro, S. L. (2009). Cognitive processes in writing during pause and execution periods. European Journal of Cognitive Psychology, 21(5), 758-785.

Qin, J., & Karabacak, E. (2010). The analysis of Toulmin elements in Chinese EFL university argumentative writing. System, 38(3), 444-456.

Révész, A., Kourtali, N. E., & Mazgutova, D. (2017). Effects of task complexity on L2 writing behaviors and linguistic complexity. Language Learning, 67(1), 208-241.

Sampson, V., & Clark, D. B. (2008). Assessment of the ways students generate arguments in science education: Current perspectives and recommendations for future directions. Science Education, 92(3), 447–472.

Schilperoord, J. (1996). It’s about time. Temporal aspects of cognitive processes in text production. Amsterdam, the Netherlands: Rodopi.

Schwarz, B. B., & Asterhan, C. S. (2010). Argumentation and reasoning. International handbook of psychology in education, 137-176.

Shehab, H. M., & Nussbaum, E. M. (2015). Cognitive load of critical thinking strategies. Learning and Instruction, 35, 51-61.

Simon, D., & Holyoak, K. J. (2002). Structural dynamics of cognition: From consistency theories to constraint satisfaction. Personality and social psychology review, 6(4), 283-294.

Spelman Miller, K. S. (2000). Academic writers on-line: Investigating pausing in the production of text. Language Teaching Research, 4, 123-148.

Spelman Miller, K. S., Lindgren, E., & Sullivan, K. P. (2008). The psycholinguistic dimension in second language writing: Opportunities for research and pedagogy using computer keystroke logging. TESOL Quarterly, 42(3), 433-454.

Stapleton, P., & Wu, Y. A. (2015). Assessing the quality of arguments in students’ persuasive writing: A case study analyzing the relationship between surface structure and substance. Journal of English for Academic Purposes, 17, 12–23.

Stevenson, M., Schoonen, R., & De Glopper, K. (2006). Revising in two languages: A multi-dimensional comparison of online writing revisions in L1 and FL. Journal of Second Language Writing, 15(3), 201-233.

Sullivan, K., & Lindgren, E. (2006). Computer Key-Stroke Logging and Writing: Methods and Applications (Studies in Writing). Elsevier Science Inc..

Tanimoto, T. T. (1958). An elementary mathematical theory of classification and prediction. Internal Rport IBM Corp.

Thorson, H. (2000). Using the computer to compare foreign and native language writing processes: A statistical and case study approach. Modern Language Journal, 84(ii), 155–170.

Torrance, M. (2016) Understanding planning in text production. In C. A. MacArthur, S. Graham, S., & J. Fitzgerald, J. (Eds.), Handbook of writing research (pp. 72–87). NY: Guilford Press.

Toulmin, S. (1958). The uses of argument. Cambridge: Cambridge University Press.

Toulmin, S. E. (2003). The uses of argument. Cambridge: Cambridge university press

Van Waes, L. V., & Leijten, M. (2015). Fluency in Writing: A Multidimensional Perspective on Writing Fluency Applied to L1 and L2. Computers and Composition, 38, 79-95.

Van Waes, L., Leijten, M., Pauwaert, T., & Van Horenbeeck, E. (2019). A multilingual copy task: Measuring typing and motor skills in writing with Inputlog. Journal of open research software.-2013, currens, 7(30), 1-8.

Van Waes, L. & Schellens, P. J. (2003). Writing Profiles: The Effect of the Writing Mode on Pausing and Revision Patterns of Experienced Writers. Journal of Pragmatics, 35:6, 829- 853.

Van Waes, L., Van Weijen, D., & Leijten, M. (2014). Learning to write in an online writing center: The effect of learning styles on the writing process. Computers & Education, 73, 60-71.

Varghese, S. A., & Abraham, S. A. (1998). Undergraduates arguing a case. Journal of Second Language Writing, 7, 287-306.

Voss, J. F., & Means, M. L. (1991). Learning to reason via instruction in argumentation. Learning and instruction, 1(4), 337-350.

Wengelin, Å. (2006). Examining pauses in writing: Theories, methods and empirical data. In K.P.H. Sullivan & E. Lindgren (Eds.), Computer Key-Stroke Logging and Writing: Methods and Applications (pp. 107-130). Amsterdam, the Netherlands: Elsevier.

Wingate, U. (2012). ‘Argument’ helping students understand what essay writing is about. Journal of English for Academic Purposes, 11, 145-154.

Zhang, S. (1987). Cognitive complexity and written production in English as a second language. Language Learning, 37(4), 469-481.



How to Cite

Tian, Y., Kim, M., & Crossley, S. (2023). Making sense of L2 written argumentation with keystroke logging. Journal of Writing Research, 15(3), 435–461.