AI is rapidly reshaping the way we interact with apps and experiences, and it can be used to make education more efficient, consistent, and fair. Within Gradescope, a paper-to-digital assessment platform, instructors can use AI-assisted grading tools to grade faster, give clearer feedback to students, and get insights into student understanding.
AI-assisted grading with Gradescope enables instructors to first sort student answers into groups, and then grade whole groups at once. For some question types, Gradescope can automatically sort student answers into groups, saving instructors even more time.
We needed to design a user interface that would allow instructors to verify that the answer groups were fully correct, easily fix mistakes if they were not, and communicate the result of a complicated process to the instructor and make them feel comfortable and effective.
This feature is a result of close collaboration between our AI, Design, and Web Development teams. The following three principles of AI Product Design guided us in achieving this mission.
Principle 1: Speak the user’s language
In the early versions of the interface, we used the term “cluster,” which refers to techniques for automatically forming distinct groups of items. We quickly realized that it did not have the same meaning for our users as it did for us. Instead, we decided to use the word “group,” which is just as accurate, but more relevant to the user.
Another example of not speaking the user’s language is the word “autograde” in an early Gradescope prototype. Our team was careful to remove this word from the final versions of the interface because Gradescope AI does not autograde. It only assists the grader in forming answer groups, and requires the grader to sign off on the groups before grading.
Being precise with our language lets the instructor know exactly what our mission is: to assist them, not replace them.
Principle 2: Details matter
The goal of a successful user interface is to make a complex feature simple to use. This can’t be solved with design work alone, you must watch actual people use the interface, notice where they struggle, and improve instruction.
As soon as the AI-assisted grading interface was somewhat usable, we started inviting Gradescope users to alpha-test it. Our office was located close to UC Berkeley, so over a dozen of teaching assistants and instructors found it easy enough to come by on their lunch break.
We would sit next to a user, and silently observe them try to figure out the novel interface. We would watch with dismay as they skipped right past a pop-up with instructions. We would squirm as they struggled to find a clearly visible button. We would notice them try to use keyboard shortcuts, to no effect.
Every single session led to important insights about how things should work and we implemented countless improvements. Individually, they are all small features, and no single one is crucial. But in combination, they make a user interface so intuitive, polished, and delightful, that the user feels safe. They can tell that we care and our product is built with them in mind.
Principle 3: Interactions between the user and AI should benefit both parties
When the user reviews answer groups formed by Gradescope AI assistance, they are interacting with AI and the interaction should be beneficial. This is why we do not launch AI-powered features until the AI engine is good enough to make a major difference in the user experience.
Still, occasionally, our AI makes a mistake. This is why we painstakingly designed the interface to allow the user to quickly and effectively correct mistakes. And when the user corrects a mistake that the AI made, it’s beneficial to the AI.
Most AI applications today learn using large sets of examples (for instance, images of handwritten words, and their corresponding text representations). The more such examples can be provided, the better the AI will get.
Of course, this flywheel effect doesn’t just happen on its own. It requires a lot of work from Design, Web Development, and AI teams. User interactions must be designed in such a way that useful data is generated, then stored in the right place, and finally used for AI development. As we have with all product enhancements and developments, we keep these three principles in mind as we look to the future of Gradescope and AI.