Argumentation features and essay quality: Exploring relationships and incidence counts


  • Scott Crossley Georgia State University | USA
  • Yu Tian Georgia State University | USA
  • Qian Wan Georgia State University | USA



argumentation, rhetorical structure theory, writing assessment


This study examines links between human ratings of writing quality and the incidence of argumentative features (e.g., claims, data) in persuasive essays along with relationships among these features and their distance from one another within an essay. The goal is to better understand how argumentation elements in persuasive essays combine to model human ratings of essay quality. The study finds that, in most cases, it is not the presence of argumentation features that is predictive of writing quality but rather the relationships between superordinate and subordinate features, parallel features, and the distances between features. This finding has not only theoretical value but also practical value in terms of pedagogical approaches and automated writing feedback.


Applebee, A. N.(1984). Writing and reasoning. Review of Educational Research Winter, 54(4), 577-596.

Azar, M. (1999). Argumentative text as rhetorical structure: An application of rhetorical structure theory. Argumentation, 13(1), 97-114.

Brassart, D. G. (1996). Didactique de l'argumentation écrite: Approches psycho-cognitives. Argumentation, 10(1), 69-87.

Brassart, D. G. (1996). Does a prototypical argumentative schema exist? Text recall in 8 to 13 years olds. Argumentation, 10(2), 163-174.

Berrill, D. P. (1996). Reframing argument from the metaphor of war. In P. Berrill (Ed.), Perspectives on written argument (pp. 171–187). Creskill, NJ: Hampton Press.

Breiman, L. (2001). Decision-tree forests. Machine Learning, 45(1), 5-32.

Brown, H. D., & Abeywickrama, P. (2010). Language assessment: Principles and classroom practices (Vol. 10). White Plains, NY: Pearson Education.

Cohen J. (1960). A coefficient of agreement for nominal scales. Educational Psychology Measurement, 20, 37–46.

Cooper, C.R., Cherry, R., Copley, B., Fleischer, S., Pollard, R., Sartisky, M. (1984). Studying the writing abilities of a university freshman class: strategies from a case study. In Beach, R.,Bridwell, L.S. (Eds.), New Directions in Composition Research. The Guilford Press, New York, pp. 19-52.

Crammond, J. (1998). The uses and complexity of argument structures in expert and student persuasive writing. Written Communication, 15, 230-268.

Crasnich, S., & Lumbelli, L. (2005). The reflection-response in enhancing argumentation ability. L1-educational Studies in Language and Literature, 5(2), 147-174.

Currie, P. (1996). Fullness and sound reasoning: Argument and evaluation in a university content course. In P. Berrill (Ed.), Perspectives on written argument (pp. 121-137). Creskill, NJ: Hampton Press.

Ferretti, R. P., Lewis, W. E., & Andrews-Weckerly, S. (2009). Do goals affect the structure of students’ argumentative writing strategies?. Journal of Educational Psychology, 101(3), 577.

Ferretti, R. P., MacArthur, C. A., & Dowdy, N. S. (2000). The effects of an elaborated goal on the persuasive writing of students with learning disabilities and their normally achieving peers. Journal of Educational Psychology, 92(4), 694.

Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical methods for rates and proportions Wiley-Interscience. Hoboken, NJ.

Gárate, M., & Melero, A. (2005). Teaching How to Write Argumentative Texts at Primary School. Studies In Writing Effective Learning and Teaching of Writing, 323–337.

Green, N. L. (2010). Representation of argumentation in text with rhetorical structure theory. Argumentation, 24(2), 181-196.

Hillocks, G. (2010). " EJ" in Focus: Teaching Argument for Critical Thinking and Writing: An Introduction. The English Journal, 99(6), 24-32.

Hillocks, G. J. (2011). Teaching argumentative writing, Grades 6-12: Supporting claims with relevant evidence and clear reasoning. Portsmouth: Heinemann.

Hirvela, A. (2017). Argumentation & second language writing: Are we missing the boat? Journal of Second Language Writing, 36, 69–74.

Hyland, K. (1997). A genre description of the argumentative essays. RELC Journal 21 (1): 66–78.

Jonassen, D. H., & Kim, B. (2010). Arguing to learn and learning to argue: Design justifications and guidelines. Educational Technology Research and Development, 58(4), 439-457.

Kuhn, M., Wing, J., Weston, S., Williams, A., Keefer, C., Engelhardt, A., ... & Kenkel, B. the R Core Team, Benesty M, Lescarbeau R, Ziem A, Scrucca L, Tang Y, Candan C (2016) Caret: classification and regression training. R package version, 6-0.

Kunnan, A. J. (2010). Test fairness and Toulmin’s argument structure. Language Testing, 27, 183–189.

Lee, J. J., & Deakin, L. (2016). Interactions in L1 and L2 undergraduate student writing: Interactional metadiscourse in successful and less-successful argumentative essays. Journal of Second Language Writing, 33, 21–34.

Liu, F., & Stapleton, P. (2014). Counterargumentation and the cultivation of critical thinking in argumentative writing: Investigating washback from a high-stakes test. System, 45, 117-128.

Mann, W. C., & Matthiessen, C. M. (1991). Functions of language in two frameworks. Word, 42(3), 231-249.

Mann, W. C., & Thompson, S. A. (1986). Relational propositions in discourse. Discourse processes, 9(1), 57-90.

Mann, W. C., & Thompson, S. A. (1987). Rhetorical structure theory: A theory of text organization (pp. 87-190). Los Angeles: University of Southern California, Information Sciences Institute.

Mann, W. C., & Thompson, S. A. (1988). Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3), 243-281.

McCann, T. M. (1989). Student argumentative writing knowledge and ability at three grade levels. Research in the Teaching of English, 62-76.

Nemeth, N., & Kormos, J. (2001). Pragmatic aspects of task-performance: the case of argumentation. Language Teaching Research, 5, 213-240.

Newell, G. E., Beach, R., Smith, J., & VanDerHeide, J. (2011). Teaching and learning argumentative reading and writing: A review of research. Reading Research Quarterly, 46(3), 273-304.

Nippold, M. A., & Ward-Lonergan, J. M. (2010). Argumentative writing in pre-adolescents: The role of verbal reasoning. Child Language Teaching and Therapy, 26, 238–248.

Nussbaum, E. M., & Kardash, C. M. (2005). The Effects of Goal Instructions and Text on the Generation of Counterarguments During Writing. Journal of Educational Psychology, 97(2), 157–169.

Nussbaum, E. M., & Schraw, G. (2007). Promoting argument-counterargument integration in students' writing. The Journal of Experimental Education, 76(1), 59-92.

Pessoa, S., Mitchell, T. D., & Miller, R. T. (2017). Emergent arguments: A functional approach to analyzing student challenges with the argument genre. Journal of Second Language Writing, 38, 42–55.

Qin, J., & Karabacak, E. (2010). The analysis of Toulmin elements in Chinese EFL university argumentative writing. System, 38(3), 444–456.

R Core Team (2017). R: A language and environment for statistical computing. R Found. Stat. Comput. Vienna, Austria.

Ramage, J. D., & Bean, J. C. (1989). Writing arguments: A rhetoric with readings. New York: Macmillan.

Sampson, V., & Clark, D. B. (2008). Assessment of the ways students generate arguments in science education: Current perspectives and recommendations for future directions. Science Education, 92(3), 447–472.

Shehab, H.M. & Nussbaum, E.M. (2015). Cognitive load of critical thinking strategies. Learning and Instruction 35, 51-61.

Simon, S. (2008). Using Toulmin's Argument Pattern in the evaluation of argumentation in school science. International Journal of Research & Method in Education, 31, 277-289.

Song, Y., Deane, P., & Beigman Klebanov, B. (2017). Toward the automated scoring of written arguments: Developing an innovative approach for annotation. ETS Research Report Series, 2017(1), 1-15.

Stapleton, P. (2001). Assessing critical thinking in the writing of Japanese university students: Insights about assumptions and content familiarity. Written communication, 18(4), 506-548.

Stapleton, P., & Wu, Y. A. (2015). Assessing the quality of arguments in students’ persuasive writing: A case study analyzing the relationship between surface structure and substance. Journal of English for Academic Purposes, 17, 12–23.

Strobl, C., Ailhaud, E., Benetos, K., Devitt, A., Kruse, O., Proske, A., & Rapp, C. (2019). Digital support for academic writing: A review of technologies and pedagogies. Computers & Education, 131, 33-48.

Toulmin, S. E. (1958). The uses of argument. Cambridge: Cambridge University Press.

Toulmin, S. E. (2003). The uses of argument. Cambridge: Cambridge university press.

Townsend, M. A., Hicks, L., Thompson, J. D., Wilton, K. M., Tuck, B. F., & Moore, D. W. (1993). Effects of introductions and conclusions in assessment of student essays. Journal of Educational Psychology, 85(4), 670.

Van Daal, T., Lesterhuis, M., Coertjens, L., Donche, V., & De Maeyer, S. (2016). Validity of comparative judgement to assess academic writing: Examining implications of its holistic character and building on a shared consensus. Assessment in Education: Principles, Policy & Practice.

Varghese, S. A., & Abraham, S. A. (1998). Undergraduates arguing a case. Journal of Second Language Writing, 7, 287-306.

Walton, D., Reed, C., and Macagno, F. (2008), Argumentation Schemes, Cambridge: Cambridge University Press.

Wolcott, W., & Legg, S. M. (1998). An Overview of Writing Assessment: Theory, Research, and Practice. Urbana, IL: National Council of Teachers of English.



How to Cite

Crossley, S., Tian, Y., & Wan, Q. (2022). Argumentation features and essay quality: Exploring relationships and incidence counts. Journal of Writing Research, 14(1), 1–34.




Most read articles by the same author(s)