ReaderBench – Automated Essay Grading

Automated evaluation of textual complexity represents a key focus for the linguistic research field as it emphasizes the evolution of technology’s facilitator role in educational processes. From a tutor perspective, the task of identifying accessible materials plays a crucial role in the learning process since inadequate texts, either too simple or too difficult, can cause learners to quickly lose interest. Therefore, we have introduced a multi-dimensional analysis of textual complexity, covering a multitude of factors integrating classic readability formulas, surface metrics derived from automatic essay grading techniques, syntax indices, as well as semantics and discourse.

Essay analysis and grading plays a crucial role in the learning process because it is an indicator for a tutor about the progress of the student and for the latter because it provides a valuable feedback. One of the main features of a well written essay is to have a complexity tailored to a specific destination. Therefore, we have introduced a multi-dimensional analysis of textual complexity, covering a multitude of factors integrating classic readability formulas, surface metrics derived from automatic essay grading techniques, syntax indices, as well as semantics and discourse that are afterwards combined through the use of specific supervised classifiers (e.g., Support Vector Machines, Discriminant Function Analysis). In the end, each factor can be correlated to learners’ comprehension traces, therefore creating a clearer perspective in terms of measurements impacting the perceived difficulty of a given text.

In a nutshell, the proposed evaluation model combines statistical factors and traditional readability metrics with information theory, specific information retrieval techniques, probabilistic parsers, Latent Semantic Analysis and Latent Dirichlet Allocation semantic models for best-matching all components of the analysis. This facilitates a wide range of educational scenarios covering: automated evaluation of summaries with regards to comprehension prediction, assessment of cover letters in terms of language adequacy, as well as potential recommendations for improving one’s writing style.

essay-asset

Current status

  • Integration of new textual complexity indices to predict comprehension

Links