Setting priorities for Programs & Events Dashboard

I want to offer a very big ‘Thank You’ to everyone who took the time to respond to the first-ever Programs & Events Dashboard user survey. It had 73 responses from a diverse cross-section of Wikimedia program leaders — roughly mirroring the usage patterns of the Dashboard, with less than half from English Wikipedia. In preparing the survey, I wanted to cover a wide variety of possible focus areas for Dashboard development and to get a sense for what users’ top priorities for improvement are. The results are helping me form a clear picture of what to work on — the start of a Programs & Events Dashboard roadmap that I’ll be publishing soon.

Top priorities

The most popular priority matches a request I’ve heard frequently in the last year: program leaders want “easier ways to manage overlapping campaigns and sets of events”. Especially for Wikimedia affiliates that run large numbers of programs, using the Dashboard’s campaigns feature doesn’t provide an easy way to group related sets of events while also getting accurate aggregate statistics for all their events (or all their events over a certain time period). Organizers are looking for something like a “campaign of campaigns” that they can use to get an overall picture of their programs. I’m not sure exactly how this will look, but this is something I’m going to work on. I’m planning to follow up with survey respondents to learn more about the specific use cases and what new features would meet their needs. Relatedly, Wiki Education is exploring the possibility of offering “Dashboards-as-a-service” to other organizations in the Wikimedia community; having a dedicated, branded website for all an organization’s programs (as Wiki Education has with dashboard.wikiedu.org) could be a simpler and more satisfying solution to the problem.

The second most popular priority — “better scoping of which edits are part of my event, and which edits are not” — represents a mix of problems. Many program leaders want to track edits — and statistics — for pages that are not in mainspace. For programs where most of the expected activity is happening in sandboxes or Draft space, the Dashboard just doesn’t offer much utility right now. Being able to explicitly choose which namespaces to count is a straightforward feature to describe — although implementing it won’t be trivial. For other program leaders, the problem of “better scoping” is about defining a topic area or set of articles to track. The Dashboard has some flexible features around scoping — you can track a large set of articles by category, talk page template, or by putting together a PetScan query or PagePile — but these scoping features aren’t intuitive or discoverable enough. I plan to work on both aspects — providing more namespace flexibility, and improving the user interface to make it easier set up a program that tracks a specific set of articles.

After that, the next two top priorites were a bit of a surprise to me: more detailed Wikipedia statistics, and more detailed Wikidata statistics. For Wikipedia statistics, this partly overlaps with the need to track specific namespaces, but I need to talk with users to learn more about what kinds of additional statistics they have in mind. For Wikidata statistics, the Dashboard has some tools for analyzing Wikidata edits to count the number of claims added, claims changed, descriptions added, and so on. These features aren’t readily available on Programs & Events Dashboard yet, but I knew that users running Wikidata projects wanted them; what surprised me is just how many survey respondents consider Wikidata statistics a high priority. Wiki Education’s Wikidata Program Manager, Will Kent, and I are hoping to mentor an Outreachy intern in December (and possibly another intern in mid-2022) to make advanced Wikidata stats available.

Other areas for future work

Some of the other more popular priorities will be part of the roadmap, but we probably won’t tackle them in 2021-2022:

• Alerts for on-wiki problems. These are already present for English Wikipedia, but they don’t function yet for most other languages, and they aren’t discoverable enough.
• Enabling more language-specific features. Related to the alerts for on-wiki problems, some other features — like Authorship Highlighting — are limited to one or a few languages. We’ll continue rolling out support as we can; in most cases, this will depend on adding support for additional languages on the ORES and WikiWho services.
• Better support for education programs. I’m not sure exactly what features are the most important right now for education programs, but there are key features that Wiki Education uses on dashboard.wikiedu.org — like the assignment wizard for creating a course timeline based on a series of choices from the instructor — that would be useful for education program organizers. The challenge here will be to ensure that each program can create their own blueprints for what a standard course timeline looks like.
• More training module translations. This one is ready for anyone to add translations. The Programs & Events Dashboard training modules all have source pages on Meta; some are already set up for translations, and others will need to be marked for translation first. (If you are interested in translating existing modules, or adding new ones, let me know, and I can guide you through the process.)
• Creating accounts for new users. This is a feature that is already available, allowing users to request their desired username, and allowing organizers to review the requests and create all the requested accounts with a single click (which automatically adds them as participants). However, the account request feature must be enabled for each program and may not be discoverable enough.
• Support for judging editing competitions. A substantial number of program organizers have used the Dashboard to support editing competitions, but so far there are no dedicated features for judging competitions or tracking the particular metrics that a given competition is focused on. We may try to improve support for competitions at some point, but it will be a big project.
• Adding video tutorials & Improving written documentation. While these were the least popular priorities from the survey, they were still ranked highly by quite a few respondents (with more people interested in video tutorials than written documentation). Video tutorials — focused on how to set up a particular type of program on the Dashboard and make the most of the available features — may be a good way for organizers to discover existing features. Working on video tutorials (and perhaps written documentation as well) could make a good internship project.

Common pain points

Many users have run into problems with the Dashboard. The most common one — statistics not getting updated quickly enough — was a major issue in late 2020 and early 2021 (and was closely connected to the problem of downtime). The Dashboard’s infrastructure was stretched too thin, and was being overwhelmed by a few long-running programs tracking extremely active editors, and the system’s single server on Wikimedia Cloud just couldn’t keep up. Since then, we’ve made major strides — both in making the Dashboard’s statistics update process more efficient and adding more server capacity. (Dashboard programs get sorted in different update queues based on how resource-intensive they are. Currently, about 4/5ths of the ~550 active programs are in the fast queue, getting updates approximately every 20 minutes. Most of the rest are in the middle queue, with updates every hour or so. The 10 most resource-intensive programs — typically long-running ones that track hundreds of thousands of edits — are in the slow queue, getting updates every 3 hours.)

The other most common problems cluster on tracking the relevant edits (and only the relevant edits), and on creating accounts and joining events. We have some clear areas to work on for both adding tracking options that don’t exist right now and for making the existing options more user friendly. For creating accounts and joining programs, I think the core features work smoothly, but it’s not a simple enough user experience to enable and use those features.

What do Wikimedians use the Dashboard for, and what problems do they face?

For this survey, we worked with the Wikimedia Foundation’s new Campaigns team to add some questions focused on what people use the Dashboard for, and what the biggest challenges are for program organizing in general.

The survey responses highlight the variety of ways program organizers use the Dashboard. A large portion of respondents (77%) have used it to organize edit-a-thons, and many use it for editing contests (48%), education programs (41%), and a wide mix of other things like workshops, hackathons, WikiProjects, or tracking the activity of a group of interns or affiliate staff members.

The target audience for programs is quite wide:

The most common “other” target audience is educators.

Here’s what survey respondents see as the biggest challenges with organizing editing events and programs:

That’s a big list of challenges — some are core challenges of the whole Wikimedia movement, in fact — but I’m excited to do what I can to tackle them. I’m hopeful that between Wiki Education’s work on the Dashboard and the new tools the WMF Campaigns team will be building, we can make a lot of progress in the coming year and beyond.

Image credit: Dmitry Barsky, CC BY 2.0, via Wikimedia Commons.