Sources and Verifiability in the Age of AI

Oliver Wunsch is an associate professor and Director of Undergraduate Study for Art History in the Art, Art History, and Film Department at Boston College.

A few months ago, a student stopped by my office at Boston College to discuss the sources for her Wikipedia Assignment research project. She was enrolled in one of the art history courses in which I ask students to revise and expand a Wikipedia article on an artwork related to our course’s themes. She had taken seriously my requirement that she base most of her research on physical books from the Boston College library, and she proved the point by stacking them on my desk when she arrived to talk. I asked whether the trip to the library had gone smoothly. She replied that it had gone well, even though it was the first time she had checked out any books during her time at college. She then said something that I found puzzling: “I felt like I was in a movie about college.” I asked what she meant, and she pointed out that nobody carried books around campus in real life, at least not anymore. She had only seen it in movies, presumably old or inaccurate ones.

Oliver Wunsch. Image courtesy Oliver Wunsch, all rights reserved.

Why do I make my students engage in this antiquated ritual of taking out books from the library? In part, I insist on it because many of the most important art historical sources remain books that, for the time being, are not fully accessible online. But in recent years, with the rise of artificial intelligence, I have come to see this practical necessity as secondary to a deeper goal: to show students that the process of acquiring a piece of information changes its value and meaning; knowledge gleaned from a scholarly book, full of footnotes and supporting evidence, carries a different significance from something generated by ChatGPT.  

It might seem paradoxical that I seek to impart this message while asking my students to write articles for Wikipedia, which has historically served as the preferred information source of underprepared students across the world. When I ask my students at the beginning of the semester what first comes to mind when I say Wikipedia, many of them recall teachers in high school who told them not to use it. Yet Wikipedia’s traditional association with poorly researched papers belies the platform’s sophisticated policies around citation practices and reliable sources. It is these policies that have turned Wikipedia into a powerful ally in my effort to demonstrate to students not only that books still matter in the age of AI, but also that how you learn a fact matters as much as the fact itself.

Among the Wikipedia policies that I have found useful in conversations with students about AI, the most salient one has been the requirement of verifiability. The policy is simple enough to explain to students: a reader should be able to verify that the facts and claims in a Wikipedia article come from reliable sources. Wikipedia editors therefore need to include inline citations to their sources, and when citing books or long articles, they should point readers to the specific page numbers where the information appears. I sometimes expand on the principle by saying to students that every Wikipedia article needs to be easy to reverse engineer. Anyone should be able to figure out how the editors of the article put it together from its source material. Such a principle, of course, runs directly counter to the nature of large language models, whose processes of text generation are hardly transparent. The power and peril of generative AI lie precisely in the fact that, unless explicitly connected to a retrieval system or source database, it does not typically answer questions by gathering information from a specific source; instead, it generates text by predicting likely sequences of words based on patterns in its training data and the prompt itself. This design makes LLMs excellent at producing plausible replies that are very often true yet impossible to verify by tracking information back to its source. For this reason, the logic of generative AI runs counter to the principle of verifiability, and it makes good sense that Wikipedia’s current policies prohibit using LLMs to generate or rewrite article content, apart from limited uses such as copyediting one’s own writing. 

Policies, of course, only get you so far. If students don’t understand the reason for a rule, or if they don’t believe you can enforce it, they are unlikely to take it too seriously. I therefore spend substantial time early in the semester showing students some of the limits of LLMs in generating verifiable research. For example, I recently demonstrated to students how the current version of ChatGPT would respond if I enlisted its help for research on Michelangelo’s Dying Slave, the topic of one student’s Wikipedia article. While ChatGPT provided a competent overview of the sculpture and an impressive bibliography composed mostly of real sources, it faltered when I asked it to connect information about the sculpture to specific sources. Rather than cite the most important books in the bibliography that it had just produced, it instead provided links to various websites, some more credible than others. When I then asked why it had not cited any of the authoritative books on the topic, which offer much more extensive analysis of the sculpture than the online sources, it acknowledged a limitation: “most major Michelangelo books… are not fully readable online, so I can’t responsibly ‘cite’ them.”

I wish I could say that these demonstrations have completely dissuaded students from using AI on the assignment. In truth, I still have several students each semester who turn to LLMs when drafting their articles. Here, I am grateful that Wikipedia allows students to work on drafts in “sandboxes,” where I can typically identify issues before students publish anything to Wikipedia’s public-facing “mainspace.” I also appreciate the Pangram AI detector built into the Wiki Education dashboard, which scans these sandbox drafts and notifies both the student and me of any suspicious text. When I follow up with students about these issues, I do not generally make direct accusations about AI use or seek confessions, especially since no AI detector is 100% accurate. Instead, I focus on the question of verifiability. In my experience, almost all the passages flagged by the AI detector fail the verifiability standard. When I check the sources that the student has cited, I usually cannot find the information in the source, at least not on the cited page. In the few cases where the cited source does correspond to the flagged text, then the source is almost always a website and not one of the more authoritative publications on the topic. I can then remind the student of our conversations earlier in the semester about the importance of verifiability and reliable sources for maintaining Wikipedia’s credibility. This approach serves several purposes. First, it allows me to avoid lengthy, antagonistic debates about whether the student used AI. Second, it ensures that we concentrate on ways to remedy the problem rather than on culpability. And most importantly, it shifts the focus to the underlying issue of what makes writing trustworthy. 

Working through these questions of evidence and credibility with students has had unexpected benefits beyond the Wikipedia Assignment itself. Students often draw upon our Wikipedia conversations about AI later in the semester when we discuss trust and truth in visual representation. Students made those connections this past spring toward the end of my course The Medium Shapes the Message: Materials and Technologies of Visual Communication. In the final unit, we turned to digital images in the age of AI, seeking to establish some techniques for evaluating the veracity of images today. One day, I provided my students with a group of about twenty pictures of Boston College students around campus, half of which came from our campus photographer and the other half of which I generated using AI. When students discussed how they might authenticate the images, one student made the connection to the principle of verifiability: just as we check the credibility of a Wikipedia article by tracing information back to its source, we might verify a photograph by comparing inconspicuous background elements against architectural details or landscape features actually present on campus. 

In that moment, it became clear that the conversation about AI use in the Wikipedia project amounted to something bigger than enforcing an anti-cheating policy. It pushed the class to think much more deeply about the relationship between knowledge, representation, and authenticity. Another student, I should note, drew upon the lessons of the Wikipedia project to arrive at an even simpler means of separating the real photographs from the fake ones: only in the AI-generated images of Boston College students did we see anyone carrying any books, presumably because the AI model had been trained on stock imagery from an earlier era. I imagine the models will catch up soon. Or perhaps, with renewed interest in verifiable sources, we may see a few students walking across campus with books in hand once again.

Google Gemini images of Boston College students carrying books on campus, used by Wunsch in a class discussion.

Interested in incorporating a Wikipedia Assignment into your course? Visit teach.wikiedu.org to learn more about the free resources, digital tools, and staff support that Wiki Education offers to postsecondary instructors in the United States and Canada. Priority deadline for fall 2026 courses: Wednesday, August 12, 2026

Categories

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.