Direct multimodal grading of image-based mathematics and computing responses: rubric-constrained decisions and cross-model selective review

Information from the abstract

Introduction Multimodal language models can inspect submission images directly, potentially bypassing a separate optical-character-recognition pipeline, but end-to-end grading still requires explicit decision rules and a defensible mechanism for human review. Methods This retrospective study evaluates a two-stage workflow in authentic undergraduate mathematics and computing courses. In the first experiment, a model generated 922 analytic rubric criteria for 143 problems; instructors rated their immediate usability and completeness. In the second, GPT-4.1, Claude Sonnet 4.6, and Gemini 2.5 Pro graded anonymized handwritten or image-based responses using the same instructor-approved binary rubrics and JSON-constrained prompt. The analysis comprised 3,346 rubric decisions from 31 question units and 57 rubric items, stratified by question type, rubric type, course learning outcome (CLO), and course. Results Individual-model agreement with instructor decisions ranged from 76.5% to 87.6%. Full-coverage majority agreement was 83.7%, below Gemini's 87.6%. With all unanswered outputs routed to review, complete unanimity selected 65.6% of decisions at 91.9% agreement (+8.2 percentage points relative to full-coverage majority), while retrospective majority agreement on referred decisions was 68.0% (-15.7 percentage points). All criteria were acceptance-eligible in 60.9% of submission-question responses. For reasoning-justification items, which occurred in a single six-part group assignment in Discrete Mathematics graded by one instructor, coverage was 84.1%, with award discrepancies in 14.1% of accepted decisions. Discussion These findings support a supervised decision aid: referrals combine binary disagreements and incomplete outputs, while unanimity does not guarantee agreement with the instructor. Rubric decomposition makes decisions auditable; because the study lacks an undecomposed-prompt control, its causal effect on reliability remains unestablished.

Why this record is monitored

This record has an Impact Signal of 85/100 based on recency, source, collaboration, and bibliographic signals. It prioritizes monitoring and is not a judgment of research quality.

Related topics: Artificial Intelligence in Education · Educational Technology and Assessment · Student Assessment and Feedback