These last couple of months, I’ve been busily working with Seattle design firm WINTR to build the Assignment Design Wizard, a tool to help instructors create great Wikipedia assignments. The beta version of the wizard is now up and running at wizard.wikiedu.org, and my focus has turned to user testing in preparation for the official 1.0 release later this month.
What is user testing?
As the product manager for Wiki Education Foundation’s digital services, I’ve spent a lot of time thinking about how software can help us through our biggest challenges. I’ve talked with many instructors, students, and veteran Wikipedia editors to find out what the pain points are when it comes to Wikipedia assignments. For the Assignment Design Wizard, I’ve spent a lot of time with the talented designers at WINTR coming up with solutions for some of them. But I’ve also been immersed in these problems and the solutions we’ve been building. I know what this wizard is for, and I know how it’s supposed to be used. I can’t see it with the eyes of someone encountering it for the first time. I can’t discard all of my assumptions that got built into the wizard. That’s where user testing comes in.
The essence of user testing is just to let people try the software out and see what happens. It seems simple enough, but it’s an incredibly important part of the development process. When you watch someone try to use a piece of software for the first time, shortcomings and usability problems become painfully obvious. You may get to see them struggle with the basics. Where do I click? What am I supposed to do now? Why did it just do that? What did I do wrong? What is this thing for? Especially the first few times you do live user testing for a new piece of software, you often come away knowing exactly what you need to fix. Things you had never thought of suddenly become obvious and important.
Early results
The first user test, with a Canadian professor preparing for her first Wikipedia classroom assignment, revealed some major work that needed to be done. In particular, it was clear that the wizard wasn’t doing enough to explain what it was for and how it should be used. Now, the entry point to the wizard explains that while this tool is built around Wiki Ed’s best practices, the assignment plan you publish with it is meant as a starting point for further customization, not a rigid template.
That user test also showed that the explanation of “Did You Know” (DYK)– with the opportunity to include DYK as an ungraded option for your class — was pretty confusing. One of the ways we want to support classes is by providing hands-on help and advice with the DYK process, so that the students doing great work don’t miss out on the opportunity to showcase their articles on Wikipedia’s Main Page. Now the wizard’s DYK section explicitly covers the support Wiki Ed can provide to students interested in DYK.
Improvements to the wizard’s output– which I’ve just finished implementing today — also came directly out of my observations from user testing. The assignment plans generated by wizard are intended to be attractive and easy to read, but also easy to edit– even for someone new to wiki markup. But as the early tests showed, it wasn’t quite easy enough. Is the latest version is up to the task? The answer to that will have to await more user feedback.
On the whole, the user tests so far are showing what we hoped: the Assignment Design Wizard will be a beautiful, easy-to-use tool that will help dive right into planning their first Wikipedia assignments.
Want to try it out?
In this beta stage, it’s especially important to hear from as many people as possible. Want to give the wizard a try? Here’s what you can do:
- Put yourself into the mindset of a college professor — a reader of Wikipedia but not necessarily an experienced editor– who is interested in running their first Wikipedia classroom assignment. (If that’s really you, all the better!)
- Starting from the end of the training for educators, go through the assignment design wizard, and think through each step from the professor’s perspective. Take notes about any interface problems you encounter along the way.
- At the end, look through the assignment plan that gets posted to your userspace, and try editing it to customize your assignment plan even further.
- Let me know what you thought about the wizard, and any suggestions you have for making it better: sage@wikiedu.org
