From Handwritten Student Responses to Individualised Feedback: An Educator-In-The-Loop Framework For AI-Supported Assessment
Keywords:
Assessment for learning, human–AI collaboration, handwritten assessment, educator judgement, individualised feedbackAbstract
Handwritten assessments provide evidence of students’ reasoning, yet converting this evidence into timely, individualised feedback can place substantial demands on educators. Grounded in assessment for learning and human–AI collaboration, this design-and-development paper examines how recognition uncertainty and conflicting artificial intelligence (AI) assessment recommendations can be managed to support feedback while preserving educator judgement.
The approach combines framework development, implementation in a pilot-ready platform and formative educator use. Multimodal optical character recognition interprets handwritten text and mathematical notation. The central mechanism, a Reflective Consensus Loop, compares multiple rubric-grounded AI assessment recommendations that consider students’ methods and reasoning. Disagreements trigger reassessment against the same educator-approved criteria; unresolved cases are escalated for educator review. The framework distinguishes uncertainty about interpreting student work from disagreement about assessment judgements. Educators can inspect original responses, revise proposed marks and rationales, and approve outcomes. Feedback is linked to the approved judgement and designed to highlight potential misconceptions and suggest next steps for teaching and learning.
Systematic empirical evaluation is ongoing, examining recognition quality, agreement with educator judgements, correction requirements, feedback usefulness, transparency and assessment time. Expected results include evidence of disagreement patterns, educator correction needs, and implications for feedback practice and workload.
The paper contributes a theoretically informed, educator-in-the-loop framework linking authentic handwritten evidence, reflective AI reassessment, reviewable and contestable recommendations, and judgement-grounded feedback. It provides a basis for empirically examining the reliability,transparency and educational usefulness of AI-supported assessment while preserving educator professional judgement and decision authority.