Events

Dr. Florian Weber
Postdoctoral Researcher
Department of Information Systems and Systems Development
University of Kassel

Argumentation is a key competency in higher education and professional contexts, however teaching argumentative writing remains challenging due to limited opportunities for individualized feedback. This challenge is evident across educational and corporate contexts, as for example reflected in the current unequal educator-to-student ratios across educational settings, especially in public schools and universities. As a result, many learners lack the personalized feedback and support needed to further train their argumentation skills. To address the lack of structured support for argumentation in writing exercises, technology-mediated learning (TML) systems have been designed and evaluated. However, most existing systems provide feedback for isolated writing stages rather than supporting the human cognitive writing process. Extending support across cognitive writing stages, such as planning, translating (actual writing), and reviewing, could deepen insights into argumentation skill development and open up new support opportunities beyond feedback-only approaches by foregrounding the human writing process. Therefore, we investigate designing and evaluating the effects of a TML system based on artificial intelligence (AI) with a cognitive process theory perspective to help learners develop argumentative writing skills. To develop a TML system, we followed an echelon design science research (eDSR) approach comprising 14 iterative echelons and derived three novel design principles (DPs). To operationalize and evaluate these principles, we formulated three knowledge claims: a criterion claim, a causal claim, and a context claim. Our knowledge claims were validated through three empirical experiments involving a total of n = 393 university students. In a first proof-of-concept experiment, we show that our system is significantly more effective in supporting argumentative writing than state-of-the-art argumentation systems. Furthermore, we conducted a field experiment to investigate the impact of intelligent feedback across different stages of the cognitive writing process, demonstrating that our system outperforms traditional approaches to teaching argumentative writing. To strengthen our understanding of the system’s causal efficacy validity, we conducted a second field experiment employing a combined evaluation design of intelligent feedback. Alongside all experiments, we show that the system’s efficacy holds across varying educational settings and domains. Based on the validated knowledge claims and our three DPs, we derive three design propositions for developing TML systems for argumentative writing, which not only advance IS design science research but also contribute to theorizing towards intelligent feedback in human cognitive processes.