Argumentation features and essay quality: Exploring relationships and incidence counts

Authors

  • Scott Crossley Georgia State University | USA
  • Yu Tian Georgia State University | USA
  • Qian Wan Georgia State University | USA

DOI:

https://doi.org/10.17239/jowr-2022.14.01.01

Keywords:

argumentation, rhetorical structure theory, writing assessment

Abstract

This study examines links between human ratings of writing quality and the incidence of argumentative features (e.g., claims, data) in persuasive essays along with relationships among these features and their distance from one another within an essay. The goal is to better understand how argumentation elements in persuasive essays combine to model human ratings of essay quality. The study finds that, in most cases, it is not the presence of argumentation features that is predictive of writing quality but rather the relationships between superordinate and subordinate features, parallel features, and the distances between features. This finding has not only theoretical value but also practical value in terms of pedagogical approaches and automated writing feedback.

References

Applebee, A. N.(1984). Writing and reasoning. Review of Educational Research Winter, 54(4), 577-596. https://doi.org/10.3102/00346543054004577

Azar, M. (1999). Argumentative text as rhetorical structure: An application of rhetorical structure theory. Argumentation, 13(1), 97-114.

Brassart, D. G. (1996). Didactique de l'argumentation écrite: Approches psycho-cognitives. Argumentation, 10(1), 69-87. https://doi.org/10.1007/bf00126160

Brassart, D. G. (1996). Does a prototypical argumentative schema exist? Text recall in 8 to 13 years olds. Argumentation, 10(2), 163-174. https://doi.org/10.1007/bf00180723

Berrill, D. P. (1996). Reframing argument from the metaphor of war. In P. Berrill (Ed.), Perspectives on written argument (pp. 171–187). Creskill, NJ: Hampton Press.

Breiman, L. (2001). Decision-tree forests. Machine Learning, 45(1), 5-32.

Brown, H. D., & Abeywickrama, P. (2010). Language assessment: Principles and classroom practices (Vol. 10). White Plains, NY: Pearson Education.

Cohen J. (1960). A coefficient of agreement for nominal scales. Educational Psychology Measurement, 20, 37–46. https://doi.org/10.1177/001316446002000104

Cooper, C.R., Cherry, R., Copley, B., Fleischer, S., Pollard, R., Sartisky, M. (1984). Studying the writing abilities of a university freshman class: strategies from a case study. In Beach, R.,Bridwell, L.S. (Eds.), New Directions in Composition Research. The Guilford Press, New York, pp. 19-52. https://doi.org/10.1017/s0142716400006263

Crammond, J. (1998). The uses and complexity of argument structures in expert and student persuasive writing. Written Communication, 15, 230-268.

https://doi.org/10.1177/0741088398015002004

Crasnich, S., & Lumbelli, L. (2005). The reflection-response in enhancing argumentation ability. L1-educational Studies in Language and Literature, 5(2), 147-174.

https://doi.org/10.1007/s10674-005-0918-5

Currie, P. (1996). Fullness and sound reasoning: Argument and evaluation in a university content course. In P. Berrill (Ed.), Perspectives on written argument (pp. 121-137). Creskill, NJ: Hampton Press.

Ferretti, R. P., Lewis, W. E., & Andrews-Weckerly, S. (2009). Do goals affect the structure of students’ argumentative writing strategies?. Journal of Educational Psychology, 101(3), 577.

https://doi.org/10.1037/a0014702

Ferretti, R. P., MacArthur, C. A., & Dowdy, N. S. (2000). The effects of an elaborated goal on the persuasive writing of students with learning disabilities and their normally achieving peers. Journal of Educational Psychology, 92(4), 694.

https://doi.org/10.1037/0022-0663.92.4.694

Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical methods for rates and proportions Wiley-Interscience. Hoboken, NJ. https://doi.org/10.1002/0471445428

Gárate, M., & Melero, A. (2005). Teaching How to Write Argumentative Texts at Primary School. Studies In Writing Effective Learning and Teaching of Writing, 323–337.

https://doi.org/10.1007/978-1-4020-2739-0_22

Green, N. L. (2010). Representation of argumentation in text with rhetorical structure theory. Argumentation, 24(2), 181-196. https://doi.org/10.1007/s10503-009-9169-4

Hillocks, G. (2010). " EJ" in Focus: Teaching Argument for Critical Thinking and Writing: An Introduction. The English Journal, 99(6), 24-32.

Hillocks, G. J. (2011). Teaching argumentative writing, Grades 6-12: Supporting claims with relevant evidence and clear reasoning. Portsmouth: Heinemann.

Hirvela, A. (2017). Argumentation & second language writing: Are we missing the boat? Journal of Second Language Writing, 36, 69–74. https://doi.org/10.1016/j.jslw.2017.05.002

Hyland, K. (1997). A genre description of the argumentative essays. RELC Journal 21 (1): 66–78. http://dx.doi.org/10.1177/003368829002100105

Jonassen, D. H., & Kim, B. (2010). Arguing to learn and learning to argue: Design justifications and guidelines. Educational Technology Research and Development, 58(4), 439-457.

https://doi.org/10.1007/s11423-009-9143-8

Kuhn, M., Wing, J., Weston, S., Williams, A., Keefer, C., Engelhardt, A., ... & Kenkel, B. the R Core Team, Benesty M, Lescarbeau R, Ziem A, Scrucca L, Tang Y, Candan C (2016) Caret: classification and regression training. R package version, 6-0.

Kunnan, A. J. (2010). Test fairness and Toulmin’s argument structure. Language Testing, 27, 183–189. https://doi.org/10.1177/0265532209349468

Lee, J. J., & Deakin, L. (2016). Interactions in L1 and L2 undergraduate student writing: Interactional metadiscourse in successful and less-successful argumentative essays. Journal of Second Language Writing, 33, 21–34. https://doi.org/10.1016/j.jslw.2016.06.004

Liu, F., & Stapleton, P. (2014). Counterargumentation and the cultivation of critical thinking in argumentative writing: Investigating washback from a high-stakes test. System, 45, 117-128.

https://doi.org/10.1016/j.system.2014.05.005

Mann, W. C., & Matthiessen, C. M. (1991). Functions of language in two frameworks. Word, 42(3), 231-249. https://doi.org/10.1080/00437956.1991.11435839

Mann, W. C., & Thompson, S. A. (1986). Relational propositions in discourse. Discourse processes, 9(1), 57-90. https://doi.org/10.1080/01638538609544632

Mann, W. C., & Thompson, S. A. (1987). Rhetorical structure theory: A theory of text organization (pp. 87-190). Los Angeles: University of Southern California, Information Sciences Institute.

Mann, W. C., & Thompson, S. A. (1988). Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3), 243-281. https://doi.org/10.1515/text.1.1988.8.3.243

McCann, T. M. (1989). Student argumentative writing knowledge and ability at three grade levels. Research in the Teaching of English, 62-76.

Nemeth, N., & Kormos, J. (2001). Pragmatic aspects of task-performance: the case of argumentation. Language Teaching Research, 5, 213-240. https://doi.org/10.1177/136216880100500303

Newell, G. E., Beach, R., Smith, J., & VanDerHeide, J. (2011). Teaching and learning argumentative reading and writing: A review of research. Reading Research Quarterly, 46(3), 273-304.

Nippold, M. A., & Ward-Lonergan, J. M. (2010). Argumentative writing in pre-adolescents: The role of verbal reasoning. Child Language Teaching and Therapy, 26, 238–248.

https://doi.org/10.1177/0265659009349979

Nussbaum, E. M., & Kardash, C. M. (2005). The Effects of Goal Instructions and Text on the Generation of Counterarguments During Writing. Journal of Educational Psychology, 97(2), 157–169.

https://doi.org/10.1037/0022-0663.97.2.157

Nussbaum, E. M., & Schraw, G. (2007). Promoting argument-counterargument integration in students' writing. The Journal of Experimental Education, 76(1), 59-92.

https://doi.org/10.3200/jexe.76.1.59-92

Pessoa, S., Mitchell, T. D., & Miller, R. T. (2017). Emergent arguments: A functional approach to analyzing student challenges with the argument genre. Journal of Second Language Writing, 38, 42–55.

https://doi.org/10.1016/j.jslw.2017.10.013

Qin, J., & Karabacak, E. (2010). The analysis of Toulmin elements in Chinese EFL university argumentative writing. System, 38(3), 444–456. https://doi.org/10.1016/j.system.2010.06.012

R Core Team (2017). R: A language and environment for statistical computing. R Found. Stat. Comput. Vienna, Austria.

Ramage, J. D., & Bean, J. C. (1989). Writing arguments: A rhetoric with readings. New York: Macmillan.

Sampson, V., & Clark, D. B. (2008). Assessment of the ways students generate arguments in science education: Current perspectives and recommendations for future directions. Science Education, 92(3), 447–472. https://doi.org/10.1002/sce.20276

Shehab, H.M. & Nussbaum, E.M. (2015). Cognitive load of critical thinking strategies. Learning and Instruction 35, 51-61. https://doi.org/10.1016/j.learninstruc.2014.09.004

Simon, S. (2008). Using Toulmin's Argument Pattern in the evaluation of argumentation in school science. International Journal of Research & Method in Education, 31, 277-289.

https://doi.org/10.1080/17437270802417176

Song, Y., Deane, P., & Beigman Klebanov, B. (2017). Toward the automated scoring of written arguments: Developing an innovative approach for annotation. ETS Research Report Series, 2017(1), 1-15.

https://doi.org/10.1002/ets2.12138

Stapleton, P. (2001). Assessing critical thinking in the writing of Japanese university students: Insights about assumptions and content familiarity. Written communication, 18(4), 506-548. https://doi.org/10.1177/0741088301018004004

Stapleton, P., & Wu, Y. A. (2015). Assessing the quality of arguments in students’ persuasive writing: A case study analyzing the relationship between surface structure and substance. Journal of English for Academic Purposes, 17, 12–23.

https://doi.org/10.1016/j.jeap.2014.11.006

Strobl, C., Ailhaud, E., Benetos, K., Devitt, A., Kruse, O., Proske, A., & Rapp, C. (2019). Digital support for academic writing: A review of technologies and pedagogies. Computers & Education, 131, 33-48.

https://doi.org/10.1016/j.compedu.2018.12.005

Toulmin, S. E. (1958). The uses of argument. Cambridge: Cambridge University Press.

Toulmin, S. E. (2003). The uses of argument. Cambridge: Cambridge university press.

Townsend, M. A., Hicks, L., Thompson, J. D., Wilton, K. M., Tuck, B. F., & Moore, D. W. (1993). Effects of introductions and conclusions in assessment of student essays. Journal of Educational Psychology, 85(4), 670. https://doi.org/10.1037/0022-0663.85.4.670

Van Daal, T., Lesterhuis, M., Coertjens, L., Donche, V., & De Maeyer, S. (2016). Validity of comparative judgement to assess academic writing: Examining implications of its holistic character and building on a shared consensus. Assessment in Education: Principles, Policy & Practice.

https://doi.org/10.1080/0969594x.2016.1253542

Varghese, S. A., & Abraham, S. A. (1998). Undergraduates arguing a case. Journal of Second Language Writing, 7, 287-306. https://doi.org/10.1016/s1060-3743(98)90018-2

Walton, D., Reed, C., and Macagno, F. (2008), Argumentation Schemes, Cambridge: Cambridge University Press.

Wolcott, W., & Legg, S. M. (1998). An Overview of Writing Assessment: Theory, Research, and Practice. Urbana, IL: National Council of Teachers of English.

Published

2022-03-18

How to Cite

Crossley, S., Tian, Y., & Wan, Q. (2022). Argumentation features and essay quality: Exploring relationships and incidence counts. Journal of Writing Research, 14(1), 1–34. https://doi.org/10.17239/jowr-2022.14.01.01

Issue

Section

Articles

Most read articles by the same author(s)