Automated grading system

Overview

When I was offered the position of Instructional Student Assistant for introductory CS courses at Cal Poly, I found that the existing autograding system was a fragile jumble of giant, monolithic Bash scripts. The scripts were still working, but just barely, and adding any of the desired new functionality would have been asking for trouble.

To remedy this, I designed a modular grading system, based on the premise that any grading process can be broken down into a sequence of discrete steps. For example, the process for grading a Java assignment might be as follows:

1. Copy student's source files to the grading directory.
2. Copy the test driver to the grading directory.
3. Compile the source files with javac.
4. Run the test driver, and make sure that it passes.
5. Deduct points for overly long lines.
6. Deduct points for tab characters.
7. Deduct points for incomplete documentation.
8. Email scores to students.

Most of these steps are not specific to any one assignment, or even any one programming language. By writing each of these steps independently, it becomes trivially easy to create assignment specifications that can be automatically executed.

Features

• Support for multiple grading periods, and penalties for late submissions or repeated submissions (partial credit). This includes a tool to extract each student’s best score and export the results as CSV for easy input into a gradebook.
• Integration with cron via a configurable schedule file that lists upcoming assignments, grading periods, and parameters.
• Language bindings for steps written in Bash and Python, with the option of easy expansion to other languages.
• Wide variety of built-in steps, including:
  • simple file manipulation scripts (e.g., “copy files from the student directory to the testing directory”);
  • a custom Javadoc doclet to automatically check and score documentation;
  • style checking tools for long lines, tab characters, etc.;
  • a runner for Java test drivers; and
  • utility to mail scores and deduction explanations to students.
• And, of course, comprehensive documentation for instructors and new student graders.

Deployment

My system is in use by Computer Science 102 classes at Cal Poly.

Unfortunately, Cal Poly requires that the source code for this project remain private. Sorry.