One Year of Specs Grading: the Good, the Bad, and the Ugly

Posted on by Josh Nudell

Spring semester 2023 is in the books. It actually has been in the books for a few days, though I have spent the time since working on wrapping up its ragged ends.

In truth, this was a second consecutive brutally difficult semester, making the 2022/23 school year one of the most difficult of my career. In addition to a series of crises external to what happened in the classroom, I was also teaching three new classes: upper level surveys on Ancient Rome and Persia, and a first year seminar. The demographics of that first year seminar were particularly challenging such that I had to functionally re-write the syllabus midway through the semester and I had one student so difficult that I came to dread walking into that classroom.

I wrestled the semester into submission eventually. My Persia class might be my favorite class I have ever taught, in large part because of the mix of students, and my revised first year seminar syllabus along with a slightly different approach to discussion allowed my students to pick up on the themes and skills that are most important for the course. In each of these three classes I was also able to build trust with the students that we were able to largely weather the techpocalypse ransomware attack that took down the network two weeks before the end of the semester. The outpouring of comments from students in the last few weeks was enormously moving, but I also want to recognize how hard I had to work to get there.

However, by way of semester retrospective, I want to focus on one academic year using Specifications Grading. I adopted this system because it promised to make my life easier, and my spring changes like an UnGrading system to assess participation and taking attendance every day worked, but, one year in, I am left wondering whether a specs model is the right fit for most of my classes.

The Good

My favorite part of specs grading is not assigning grades to assignments. The obsession with grades is deeply rooted in students, but grades themselves are often a poor match for learning. Specifications, by contrast, clearly establishes my expectations and, at least in theory, gives the students guidance on how they can earn credit for an assignment. This is still a form of grading, but the expectations provide a framework within which the students can learn and my feedback can focus on whether the student has met the expectation for that assignment. Moreover, the grades are earned across categories, meaning that the students have to engage with each part of the course and the clear expectations for each grade tier can allow students to prioritize their efforts if, for instance, they have met the requirements for their target grade in my class and need to focus instead on passing a different one.

Moreover, by modifying the expectations up or down for either the overall grades or for individual assignments I can adjust what my expectations are for the students. Thus, when our tech issues struck, I could easily fulfill every learning objectives and still lower the expectations for several graded categories in my classes, much to the relief of my students.

I particularly found specifications grading effective for relatively small, repeated assignments like journals where partial credit is particularly arbitrary and missing the rubric on one or two assignments both teaches an important lesson about following the assignment guide and has a relatively minimal overall effect on the final grade. Whether or not I continue with Specifications Grading as an overarching grading scheme, I will definitely carry these aspects forward into what comes next.

The Bad

More of my students this semester than in the fall term seemed to embrace the spirit of the specs grading and understood how the grade tiers worked, but this still left me with some students who struggled to see the connection between the work that they were completing the grade tiers in the syllabus. A couple of these were unique cases with a confluence of circumstances, but others were more persistent and connected to another issue that frustrated me last semester.

One of the keys to Specifications Grading is transparency. Every assignment guide came with a detailed rubric that spelled out exactly how to earn credit for that assignment. These rubrics were prescriptive in that they articulated the formal characteristics that I was grading on, but they were deliberately open-ended so that the students could work within the guardrails to express themselves. For instance, the journal assignment specified a length, a mandate to include a date, title, and word count, and a set of prompts like “what was the most interesting thing you learned from class this week” or “how would something you learned this week change a paper you wrote earlier in the semester.” For responses to a class movie, the rubric might be that you need to answer each question with at least 2 complete sentences appropriate for the movie.

However, I often got the sense that the students weren’t checking their work against the rubric before submitting it. In the small repeated assignments one or two times being told that an assignment wasn’t accepted put the students back on the right track, but then in some of these cases the students would trip up in exactly the same way on the next assignment.

Even more worrying was that this also happened on bigger assignments like papers where students turned in sometimes two or more drafts that seemed to rely on little more than hope that it fulfilled the rubric, even after having the students use this exact rubric for the purposes of peer review. I allow students to revise their papers both as a matter of praxis for teaching writing and because not doing so would be too draconian a policy for a specs system (see below), but nevertheless getting rounds of papers that simply ignored the guidelines, and, in at least one case, introduced new ways that the paper missed the rubric on revision, made me ask in frustration why I provide the rubrics in the first place.

But for all of these frustrations, these are not the reasons I’m considering whether to keep a specifications model or adopt some sort of hybrid system.

The Ugly

Two semesters into using Specifications Grading, my biggest question is whether it is a good match for writing-enhanced classes.

I really like the rubric I designed for grading essays in this system. Unlike most specs rubrics that use a proficient/not-proficient binary, my rubric has two “pass” tiers, one for basic proficiency and another for advanced. The advanced tier I calibrated at roughly a low-A. Earning a C in this course required revising one of three papers to the advanced tier and just the first tier for the other two, a B required revising two, and the A required all three.

Despite the promises of specs grading, I have not found that this system saves me any time at all, especially when grading papers on the learning management system, which I do as a matter of equity (e.g. costs of printing), scheduling (e.g. not having things due at class time), and convenience (e.g. I can toggle between versions). Simply put, I found that a lot of students would not be able to write well enough to fulfill the advanced tier of the rubric on one paper, let alone three. Even when they looked at the scored rubric, which was not always the case, I felt like I had to give lots of direct and actionable feedback in the paper itself, in the rubric comments, and in the summary comments on the paper. Otherwise, I feared, the students might not be able to make the connections between whatever they wrote and the rubric scores.

Let me be clear here: the system works. As I told my students, my goal at this point in their college career is to help build good writing skills and habits so so that every student knows that they can revise a (relatively short) paper to a high quality before they get to the two research-centric classes that they take in their junior and senior year. I am also comfortable with the rubric calibration because each semester I had a few students who fulfilled the rubric with no or minimal revisions to their paper, and nearly every student improved dramatically from the start of the semester to the end.

But there were also some days when I felt like I was dragging two classes worth of students (46, at final count) toward writing proficiency, on top of being responsible for the course content, two sections of tag-along non-WE sections of these courses (6 students), and the first year seminar. It was a lot. Having two sections of this process of course magnified all of the issues, but it also left me wondering whether continuing down this path toward completely spec-ified writing-enhanced courses is sustainable. I don’t relish the prospect of going back to traditional points-based grading either, which makes me wonder if I can imagine some sort of hybrid grading scheme that does what I want it to do.

Weekly Varia no. 25, 05/06/23

Posted on by Josh Nudell

I never know quite what to do as a professor in the last week of the semester. When I was a teaching assistant in survey courses I dedicated this week to exam preparation, but my deep skepticism of that mode of assessment means that I almost requires students to sit final exams. I have students write papers instead, and, increasingly, I have moved key pieces of the assessment earlier in the semester so that the students are able to revise their work before the end of term. This change gives me more flexibility about what we cover later in the semester, but I go back and forth on what we should focus on given that the “new” content is not going to be assessed on a test. But neither do I want to leave a semester unfinished, so I have recently been doing two things to round out the semester.

First, I use the last couple of weeks of class to complete whatever thematic arcs we have followed from the start of the semester. I started thinking about my courses in these terms four or five years ago when I realized that doing so helped both me and my students approach the material as a discrete unit that layers and builds depth as the semester goes on. The last couple of weeks let us tie these themes together.

Second, I lean into reflection. For instance, in two of my classes this semester I showed the students their opening day Jamboard where I asked them to reflect on what they knew coming into the course. My Persian history class got a kick out of seeing how little they knew, while the Roman history course talked about how they had a lot of key terms or ideas, but as buzzwords absent context. In both classes, we then talked about what they learned and reinforced the themes for the semester. Predictably, my Persian history class ended up in a passionate discussion about the challenges of writing ancient history.

But when I step out of the classroom for the last time for the semester, whether tinged with sadness or relief that a particular class has concluded, I also feel like I’m stepping into an unsettling limbo. As much as I am often ready for a break at the end of a long semester, my work is not over. I will be meeting with students throughout the finals week and screening films for a couple of classes, and the “final” grading push has only just begun.

By next week, though, I might be ready to start looking ahead to the summer.

This week’s varia:

  • Paul Thomas has a good reflection on one of the core challenges of a good writing-intensive class: changing student perceptions about whose responsibility learning actually is. In his estimation, as it is mine, these challenges are systemic to American education, and COVID policies only exacerbated the issues where students often don’t avail themselves of the resources at their disposal and limit their revisions to the things specifically pointed out in the comments, even when those comments are representative of other issues. My hope is that because I’m doing this from my seat in a history department and frequently have the students in our major more than once I can help them break these habits even if it takes more than one iteration.
  • NAEP Civics and US History scores for eighth graders dropped last week, showing a modest drop from 2018 and 2014, but only back to the baseline for earlier years. There is some performative rending of clothes and tearing of hair about these scores (including on my campus), but these scores are deeply misleading. These are not good tests, to start with, and I could easily see how questions about “the rule of law” on a civics test might be shaped by the discourse filled with mass shootings, police violence, and attempts to overthrow the government. But even more damning is the data about social studies instruction that, even disregarding the frequently-true stereotype of the coach-teacher, the suggests that social studies education has lagged behind other disciplines in terms of time and resources. Unfortunately, this new data is being used to manufacture a crisis that can only have negative outcomes.
  • Stephen Chappell writes about his approach to digitally restoring the polychrome painting on the Apollo Belvedere for a French exhibition.
  • Pasts Imperfect highlights a history of philosophy podcast this week.
  • Modern Medieval features an article about a Carolingian coin bearing the name Fastrada, one of the wives of Charlemagne.
  • Baker Maurizio Leo, the author of The Perfect Loaf, asked ChatGPT to provide him a sourdough bread recipe. His assessment is that the AI produced a reasonable generic loaf, albeit with a particularly high baking temperature, but that the recipe lacked creativity. AI is a powerful tool and the pace of its development is truly impressive, but I also believe that even some of the “basic” tasks people are racing to offload onto the AI require more care, attention, and creativity to do well. In many ways, bread baking is a metaphor for life and “Great sourdough bread isn’t simply a pattern that can be detected and replicated; it requires a human touch to guide it in the right direction.”
  • AI machine learning translation tools that swapped singular for plural pronouns (and other little errors) put Afghan asylum claims at risk. I find these tools incredibly useful, but this sort of error underscores my primary concern with AI, namely that putting all your trust in the tools without any way to check or verify the accuracy will cause innumerable problems that will only be caught when it is too late if they’re caught at all.
  • The “Godfather” of AI left his job at Google and raised warnings about the future of the technology. His concerns are about ethics and bad actors, while mine lie more in the likelihood that most people are going to assume that AI can do more than it can in a way that is going to cause enormous disruption while eroding the imperative to learn the underlying skills and putting significantly more noise and misinformation out into the world.
  • Missouri’s Senate passed a proposal to raise the threshold for statewide constitutional amendments to 57% of voters (from the 50%+1 that in recent years rejected right to work, legalized marijuana, and approved medicare expansion, among others) or a simple majority in five out of eight heavily gerrymandered districts. The combination seems designed to curtail the power of voters unless they’re likely to vote the way that the Republicans want. His concerns seem rooted in
  • New reporting at ProPublica has revealed still more financial ties between Clarence Thomas and Harlan Crow, including more than $6,000 a month(!) in boarding school tuition for a grandnephew over whom Thomas had legal custody. Thomas did not report this financial relationship. At the same time, the Washington Post has a report about money that Leonard Leo funneled to Ginni Thomas the same year that the court was hearing Shelby County v. Holder, all the while directing her name be left off the receipts. There is not much more to say about this deep level of corruption in the Supreme Court, but it seems bad when ProPublica has a section dedicated to this series of stories. Sheldon Whitehouse is spearheading efforts to create accountability, so, naturally, Senators like Josh Hawley claim they are designed to intimidate justices.
  • Herschel Walker’s campaign appears to have violated campaign finance laws by acquiring hundreds of thousands of dollars for his private business.
  • This week in “there are too many guns,” a baseball player for Texas A&M-Texarkana was hit in the chest by a stray bullet while he was in a game.
  • Greg Abbott described five victims of a shooting last week as “illegal immigrants” alongside a reward for the shooter in a bit of casual cruelty. At least one of the victims appears to have been a legal resident, as though the residency status matters.
  • A passenger on a New York City subway killed Jeremy Neely after putting him in a choke hold. I almost didn’t include this story in this list I had a hard time bringing myself to read about it and, especially, the discourse around whether killing someone was somehow justified. These posts are a curated rundown of things I read about during the week, usually that I have some sort editorial comment about. When one topic seems to have captured a particular zeitgeist I have nothing of particular substance to add, except to note that the grotesque discourse about under what circumstances it is acceptable to murder people in public absolutely terrifies me.
  • The story that made me think about the gun violence epidemic in the United States was when I heard the BBC World Service do a feature on two mass shootings in Serbia this week, with the rarity of the violence making international news. The level of gun violence in the United States is not normal.
  • Belgian customs officials zealously enforced the complaint from the Comité Champagne by destroying a shipment of Miller High Life after the trade group objected to the slogan “the champagne of beers” on the grounds that it infringed on the designated place of origin label.
  • Astronomers observed a gas giant being eaten by a star for the first time, doing it in one big gulp. Pretty cool.
  • Tucker Carlson has thoughts about how white men fight (McSweeney’s).
  • McSweeney’s has an imagined short monologue: “If elected president, I promise to slaughter Mickey Mouse.”

Album of the week: Justin Townes Earl, The Saint of Lost Causes (2019)

Currently Reading: Adrian Tchaikovsky, Children of Time; Martin Hallmannsecker, Roman Ionia: Constructions of Cultural Identity in Western Asia Minor

Doing History 101

Posted on by Josh Nudell

Teaching history in an era of CRT panic means facing a constant barrage from the Nothing But the Facts Brigade who vocally assert that the only way to teach history is to offer nothing but the facts. Anything else, they say, is tantamount to injecting politics into the classroom. Left wing politics, mind you, because conservative politics in this country often get labeled as apolitical, especially when it comes to a traditional, triumphalist, narrative of US history. The hegemonically white narrative.

Now, these claims are on their face patently absurd, and the same people who insist that teachers stick to the facts also want them to omit facts like the massacre at Wounded Knee in 1890, the Tulsa Race Massacre in 1921, or Japanese Internment in the 1940s that are deemed “too divisive.” The reality is that teaching (and doing) history always relies on processes of analysis, selection, and omission, and the ramifications of these choices become particularly pronounced at the survey level.

Take, for instance, my Persian history course. I could have decided that my students would benefit from a detailed operational study of the wars between the Sasanian Empire and Rome that we have been studying recently, and thus lectured campaign season by campaign season, introducing my students to the characters of each general, the conditions of warfare, and the tactical considerations of each battle. Instead, we have focused on the institutional structures of empire, royal presentation and ideology, and how to critically assess the Greek and Roman sources, which is both better in keeping with how I taught the rest of the course and required less work on my part to master and synthesize this campaign data. Neither of these two approaches is less grounded in the historical facts of the Sasanian Empire, but the students receive a somewhat different understanding of the historical period depending on where we place our focus.

I found myself reflecting on these issues while listening to an episode of the Keith Law Show with David Grann about his new book, The Wager. Grann’s books are popular history, but I broadly enjoyed The Killers of the Flower Moon and would consider reading his other books.

After setting the stage for the book, which details an 18th century mutiny aboard the HMS Wager, Law floated a question about an aspect of the book that he found particularly remarkable: that Grann infused the book with a sense of uncertainty about what actually happened on the Wager and thus which of the accounts ought to be believed.

For his part, Grann responded with predictable answers. He described how that very uncertainty created a mystery for him to try to unravel by analyzing the competing narratives that came out in the trial that followed he return of some members of the crew (including Lord Byron’s grandfather) and by putting the event in the institutional, social, and cultural context in order to explain how the event likely unfolded and what its consequences were. To be sure, the mutiny on the Wager sounds like a particularly striking story an Grann is a talented writer, but I found the answers so simple that I actually opted not to listen to his interview on Fresh Air that came out the same day.

I have a lot of respect for Keith Law as both a reader and a writer even where our tastes diverge (e.g. on Arkady Martine’s A Memory Called Empire), and his personal blog was one of the reasons I started writing about novels in this space. But this line of questioning, which Law described as something he had never really seen so prominently in a book took me by surprise. I know that Law’s non-fiction reading skews toward science and social science and it may be that Grann’s approach in The Wager foregrounds these source question to a degree unusual in popular histories, but the answers were in a sense History 101.

However, this exchange also made me think again about the Nothing But the Facts Brigade online. I found this methodology discussion rather basic, but I have also spent a lot of years training as a historian and now have these conversations about methodology with students as my job. If anything, I suspect that the conversation reflects how tightly these pernicious ideas about history grip the public imagination. What I described as Grann’s History 101 answer about carefully analyzing historical sources within their context and then spinning out an explanation of what happened requires skills honed through years of practice. The facts denuded of interpretation both denies the importance of stories for making meaning and obscures that any choice is political. There are layers of complexity that one can add when it comes to methodological approaches for academic history, but Grann’s answer should be understood as the basic methodology of both doing and teaching history. The fact that it is not at best and actively under attack at worst is part of the problem.

Inventing Utopia

Posted on by Josh Nudell

This week in my speculative fiction first-year seminar we have been working through a mini-unit on Utopias and Utopian thinking.

On Monday, I gave in lecture a “brief history of Utopian thinking” (I tried to name as many daily topics as possible like they were episode titles from Community). We started with a breakdown of Thomas More’s Utopia (1516) and the society therein, but then explored both earlier examples like the Golden Age of Man in Hesiod’s Works and Days and Plato’s Republic, and historical attempts to create these communities like the Shakers and the Oneida Community. That day concluded with a discussion of what utopias do, both in terms of social critique of the present and imagining a better future. We haven’t yet talked about Atlantis and Atlantean-type stories as Utopias because I (mistakenly) put it at the end of this unit, but the next time I teach this class, I’m going to move that day to put it more directly in dialogue with this one.

Then, on Wednesday, we read Ursula Le Guin’s “The Ones Who Walk Away From Omelas.” This short story asks you to imagine a happy, pleasant city that can only exist because of the abject suffering of a single child. Everyone in the community is aware of this trade-off and the ones who walk away cannot live with that knowledge. The story prompted a lively discussion, drawing comparisons to the Trolley Problem and generally about the morality of Utopias that always require some sort of trade. Several students challenged whether the people walking away are any more moral than the ones who stay given that even though they are opting out of the benefit of the Utopia they are nevertheless still living with the knowledge of the child’s suffering. One student asked how the message changes if you can’t walk away, to which several responded that it suddenly becomes a dystopia. This was my favorite question, though, because Omelas can be read as allegory for modern society where the happiness of people in one part of the world comes at the expense of the suffering of people elsewhere, in which case individuals only have so much capacity to opt-out.

(We are going to return to this point in the class at the very end of the semester with N.K. Jemisin’s “The Ones Who Stay and Fight.” Now that I think about it, I might move the Utopia discussion to the end of the semester next time around.)

The assignment for this unit is a poster where the students work in groups to create their own utopia, as agreed upon by the group members.

This is a deceptively difficult assignment. It requires thinking through the consequences of the society that they set up and consider what makes it a Utopia. One of the things I stressed in our discussions is that a Utopia for one is not a Utopia for all (except in the fleeting moment of Hesiod’s Golden Age), so one of the tasks is to define who are the “in” group and who are the “out” group, with those definitions being entirely up to the group. The larger the society, the harder it is to think through the consequences of the rules, laws, and social norms. This is why it amuses me that one group is re-creating a Matrix to allow each person their own bespoke Utopia that exists only in their minds.

To be completely transparent, this assignment is my equivalent of the Kobayashi Maru, just with a different set of lessons that can be taken from it.

If I were completing this same assignment, I would start by considering the sources of human conflict, big and small. If we were able to eliminate scarcity, jealousy, and pain, that would eliminate most conflict. Something like the world imagined in Wall-E as a dystopian future after humans destroyed the world.

The issue is that the elimination of all of these needs strips away something essential to being human, I think. Put another way, I think it is not possible to both have humans and to have a true Utopia, thus short-circuiting the whole exercise. As Hesiod says in Works and Days, we live in an Iron Age where we are doomed to experience sickness and pain as our meat sacks move through the world. It is simply the price of being human. Thus, the best that we can hope for is to mitigate the suffering that comes from scarcity, jealousy, and pain rather than eliminate it altogether. And, to paraphrase a delirious priest in Brothers Karamazov, we already live in paradise, so we have all the tools of that mitigation if we’re willing to commit to the practice.

However, this impossibility is also why I really like this assignment, perhaps with some fiddling around the edges. Utopias are good to think with, and working through the potential issues as a group forces the students to focus on the process rather than skipping ahead to the product.

ChatGPT, Again

Posted on by Josh Nudell

I had no intention of returning to the ChatGPT discourse. I said my piece back in December and the conversation seemed to take a histrionic turn around the start of the semester when the course planning period coincided with a wave of op eds that treated AI-writing as an existential crisis.

Moral panics turn tedious in a hurry and I bore easily.

Happily, I think that the discussion is starting to move into a more compelling and, I hope nuanced, direction. In his column at Inside Higher Ed last week, for instance, Matt Reed suggested that there are two issues at play with AI-generated writing. The one, academic dishonesty and how to catch it, receives the bulk of the attention. The other, disinformation and inaccuracies, has to this point received much less attention. In other words, the practical considerations about the norms, expectations, and enforcement of academic transactions are taking precedence over the underlying principles. This sort of priority of course makes sense, as anyone who has worked within the institutions of higher education can tell you, but I also think that it misses that these two issues are inextricably intertwined.

Simply put, I am convinced that ChatGPT specifically, and AI more generally, is a digital and information literacy issue.

Now, I should acknowledge that the stakes involved are more profound outside of the semi-controlled academic context, and at least potentially herald fundamental disruption to existing economic models. Google, for instance, is reportedly treating the chatbot like an existential threat to their hegemony over access to information online. Likewise, AI-generated art is just the latest technology that will allow companies to cut labor costs—why pay artists to create cover-art for a book when you can have an intern churn out AI-generated images until you find one you like? As much as I maintain that AI is a tool and the person producing the art is an artist, companies are not likely to compensate the artist as such under these scenarios. But while both of these are ethical issues related to my point about digital literacy, neither are they wholly new.

When it comes to writing, AI is a tool, and tools are only as good as their users. A spell-Czech [sic] doesn’t provide any value if one doesn’t have the vocabulary to recognize when it misleads, just as gratuitous use a thesaurus can lead the writer astray. Predictive text is fine for email, but I find it distracting in other contexts because the program prompts me down particular lines of composition. And, as I put in the last post on this topic, citation generators will lead you astray if you are unwilling or unable to format the text that it generates.

In this sense, the danger with AI is that people are going to treat a tool for something as a replacement for that thing. But this does not constitute either an existential crisis or a fundamental disruption, despite groups of people treating it as one or the other.

There are a myriad reason that a student might submit an AI-generated essay. Most of these overlap with the reasons a student might purchase an essay or otherwise cheat on assignments, and need to be addressed as such. However, AI should give educators greater pause because, compared to the other forms of dishonesty, AI might give the impression to some students that they don’t need to learn the skill in the first place. Chatbots can give the appearance of engaging with a historical figure, but they do not actually let you converse with that person any more than the Metaverse can allow you to watch Mark Antony debate in Rome in 32 BCE. But that superficial engagement risks drawing people away from the actual substance that would allow the participant to see how the AI turns unredeemed racists into apologists for their heinous beliefs or to recognize that seeing Antony debate in Rome in 32 BCE would be quite a feat because he was in Egypt gearing up for war with Octavian at that time.

On a whim, I decided to ask ChatGPT why students should avoid using the AI to write papers. This was what it produced:

I followed that prompt with a question about whether AI could help students with their writing:

I received a slightly more enthusiastic response when I directly inverted the original prompt, but still as a tool that can make writing easier or more efficient. At my most cantankerous, I dislike several of these uses—text summarization assumes one viable reading that simply isn’t true, which is also my problem with services like Blinkist, and I think that text generation will create pathways that guide how the person writes and thinks about a topic—but I could make similar arguments for writing being shaped by whatever we’re reading and simple reliance on the the first definition of a word found in a dictionary. As I said in my original post, if someone were to use AI as a tool and produce a quality paper either without any further intervention or by editing and polishing the text until it met the standards, that paper would meet my criteria for what I want my students to achieve in the class. This process would not be my preference, but the student would have guided the program through numerous rounds of revision much as they would draft and re-draft any paper that they wrote themselves. So much so that it would be easier to just write the paper, in fact. I doubt that a truly revolutionary thesis could be developed that way, but the student would have demonstrated their mastery of the course material and a sensitive understanding of the writing practices to know that it met standards on my rubric—grammar might be easier to accomplish, but the other categories not so much.

In fact, the arrival of AI makes it all the more important for students to learn skills like reading, writing, and, especially in my discipline, historical literacy. To do this, though, I think it is a mistake to issue blanket prohibitions or build assessment as though it does not exist. Rather, I want students to understand both why AI is not a great choice and what its limitations are, which requires steering into AI, at least a little bit.

This semester I am planning two types of activities, both of which are similar to the suggestions made in an opinion piece published today in Inside Higher Ed.

I scheduled a week for my first year seminar to address their first big writing assignment. The students have no reading this week, during which they will be working on their drafts of their first paper that are due on Friday. In the two class periods earlier in the week, I am going to have them complete an exercise using ChatGPT in their groups for the semester. On Monday, the students will work with ChatGPT to produce papers about the readings that we have covered to this point in the class, sharing with the me the results of the exercise. Then they will be charged with offering a critical evaluation of the generated text, which we will spend time on Wednesday sharing and discussing the critiques with the class, which will segue into a discussion of what makes writing “good.”

Students in my upper-division courses will do a similar exercise. As their first essays approach, I am going to provide students essays produced by ChatGPT using the same prompts and my essay rubric. Their task will be to “mark” the ChatGPT.

The goal is the same in both cases: to remind students that AI has severe limitations that cannot replace their unique thoughts. Further, I aim to engage the students as both writers and editors since I see the latter skill as an essential part of the writing process.

I don’t want suggest a prescriptive advice in this given that my class sizes and teaching mandates allow me to pursue some of these options. But the ChatGPT discourse has made even more convinced that it is necessary to teach basic, foundational, transferrable skills that will empower students to engage responsibly with the world in which they live.

Specsitol: a semester reflection

Posted on by Josh Nudell

I submitted grades a little over a week ago and promptly withdrew, exhausted, into a little fort with curtain walls made of novels. At least that’s what it felt like. The specific details might be exaggerated.

Several times I tried to break through the fog that had settled over my mind, but succeeded only in producing a silly post about pizza TV shows and the weekly varia post that I start compiling as soon as the previous week’s goes up. I could barely think about the semester that had just ended, let alone put those thoughts into any sort of coherent discussion.

Simply put, I had an exceptionally difficult semester, and one that rates among the very toughest I have ever experienced. Some issues stemmed from causes external to my classes (e.g. not getting some much-needed rest this summer and early semester indexing and proofing a book manuscript that put me perpetually behind), while others stemmed from things that happened in the classes, most of which I don’t want to talk about in this space because I don’t like talking about specific student activity in a public forum even when identifying details have been redacted, especially when there is nothing of universal value that can be gleaned by doing so.

Not every problem stemmed from these issues, of course.

Dissatisfied with traditional forms of grading, I dove headlong into the world of Specifications Grading for most of my courses this semester. To stick with the metaphor, I liked these waters but they also sent me crashing into the rocks.

The formula varied a little bit, class by class, but, in general I came up with a system where the students earned credit across four or five categories of assignments (e.g. journal entries, small assignments/participation, papers). Every assignment was graded using a bespoke rubric and it either met the standard and thus earned credit, or it did not. More work, and higher quality essays (the essay rubric had two tiers, one for basic competency and another for advanced) earned higher grades. To meet these higher standards, I allotted virtual tokens that the students could use either to revise their papers or turn work in late, pegging the number of tokens to the number of papers.

I entered the semester thinking that I had worked out a reasonably simple system that would give students the agency to decide what grade they were aiming for, make my expectations for each grade level clear, and provide in-semester flexibility that would allow students to do their best work. However, I had not anticipated that putting these assignments and expectations up front in the course would lead to cognitive overload for a significant number of students. In fact, I had a conversation in the final week of class with a student who said that this semester was much harder than the course they had taken the semester before even though the workload in the two courses was identical except that I had swapped one short weekly assignment for another. While there are other explanations why this student might have struggled with my course, I’m inclined to take the sentiment at face value because I saw evidence of the same struggle from other students who were struggling to interface with the information that I had provided in a way that made it harder to complete the work itself.

The core of this problem, I think is that many students were used to traditional grading schemes that allow students to muddle through to a passing grade without too much effort. By contrast, the system I devised required students to complete assignments in each category to a specified level in order to earn the grade. Passing my general education courses last semester did not require too much work, unless you simply neglected a graded category.

I am treating this as a messaging problem for now. Traditional grading schemes remain stupid and I’m not ready to abandon my attempt to find something better just yet.

However, the issue of students neglecting grade categories dovetailed with the tokens and flexible deadlines to create absolute chaos on my end. Here there were several intertwined issues.

Several semesters ago I developed a system for deadlines where students could receive an automatic extension by filling out a Google form before the due date. This policy has proven incredibly popular with my students. However, while I intend to keep it intact in some form, I am starting to question whether the system is having the intended effect. Rather than providing students the space to do their best work, I am finding that whatever grace I provide is filled by other classes with stricter deadlines such that my students wind up writing their papers at the last minute anyway, just several days later, and I had so many students taking the extension that it became a challenge to return papers in a timely fashion.

However, it was the tokens that turned this semester into a logistical nightmare. I set up the tokens anticipating that most would be used for revisions, knowing full well that revisions coming in at any point would cause some chaos. What I did not anticipate is that some portion of students would use most or all of their tokens to turn work in late. This meant that I had not only revisions, but also new work being turned in on no particular schedule throughout the semester, and I had difficulty keeping tabs on students who hadn’t turned in assignments, some of whom I knew were working on things and some of whom I did not.

Compounding these issues was, I think, a consequence of having a significant number of first year students. Anecdotally, from talking with friends who teach in high school, some students have been conditioned to think that flexible deadlines and the like mean that an assignment is optional. Or that whatever make-up assignment gets offered will be easier than the original assignment. As one explained:

“I’ll allow X to be redone/revised/resubmitted” is increasingly being taken as “I don’t need to do X, I’ll do the makeup Y later which will be easier anyway.”

This was obviously not what had been intended, but this collision of expectations and conditioning meant that I spent a significant amount of time amid the chaos of trying to grade everything just trying to track down missing work so that the students wouldn’t fail on those grounds. Oh, and I had 50% more students than I had in either semester last year.

Then there was the grading itself. I adopted a specifications system because it promised to offload some portion of the grading onto explicit rubrics where I could check the appropriate box. I loved not assigning grades to papers, but I quickly discovered several things that meant the system created just as much work as the mystery black box of traditional grading, if not more. The issues started because, I discovered, many students simply did not complete the assignments with the rubrics in mind and did not use the rubrics to check the work before submission. This meant that I often received work that did not fulfill the simplest rubrics.

These problems were particularly acute on the written assignments with its long, detailed rubric that should have provided guidance for the papers. I quickly realized that many of my students did not have the writing background to achieve the higher proficiencies, so simply checking the rubric box was not going to provide adequate guidance or encouragement. At the same time, while some students were not going to be aspiring to those grade tiers, I also couldn’t in good conscience provide detailed feedback for some students and not for others until the very end of the semester when the possibilities of revision had passed. By the last two weeks of the term it was clear that I would not be able to get caught up, so I offered that any student who wanted to revise their work could come to office hours and have their paper(s) marked in person so that they could receive feedback on how to meet the next tier. These meetings gave any student meant that (I think) any student aiming for higher grade tiers reached them, but they also meant that those weeks were a whirlwind of paper conferences.

Finally, my small assignments policy put a cherry on top of this disaster sundae.

The policy was simple. There were some number of small papers, in-class activities, exit-tickets, one-minute essays, and other activities that took place in class. If you weren’t there, you couldn’t make up the work. Unless you were an athlete at a competition. Or you got sick. Or had other “excused” absences. Right from the start, I found myself litigating what counts as a legitimate absence, which is one of my least favorite parts about taking attendance. Then, like with non-completion of work, I found myself around the middle of the semester worried about the number of students who seemed liable to fail (or otherwise drop grade tiers) because they had failed to adequately participate in the class. Since the opportunities for these points often did not come at regular intervals, I found myself inventing “optional extra” opportunities that would allow the students to bring their grade in that category up, which, in turn, created confusion about what assignments students actually needed to complete. Often, the students who completed the optional assignments were not the ones I had in mind when I created them. And, of course, adding all of these small assignments created a flurry of paperwork that I had to manage.

Chaos.

I should point out that for a non-negligible percentage of my students this system worked exactly as I envisioned, giving them agency to achieve grades based on their goals for the semester. Had I not felt compelled to give the students aiming for the “C” the same level of feedback I gave to those aiming for an “A,” my grading might have even been manageable—but, of course, almost everyone said that they were aiming for an “A” back in August.

I am not ready to abandon this grading mode, just yet, but it needs to be modified in critical ways for it to become sustainable and productive. The changes I have in mind to this point are:

  • Streamline my messaging and expectations. This means not only being clear about my expectations in terms of earning credit across multiple categories, but also clarifying that this is a labor-based grading scheme. It is designed to be transparent and achievable, but not necessarily easy. At the same time…
  • I want to submerge the mechanics of the participation grade. Some of the chaos this semester was created by the various points that students earned for doing in-class activities, which meant that this was something I had to track. I am not planning to change the activities that I do for small assignments, but my current thought for this category is to take a page out of the “ungrading” playbook. Instead of me assigning grading, the students will complete three reflections, one at the start, one at the middle, and one at the end of the term. The first one will set expectations and think about where they are at the start of the course. The middle two reflections will both have the students assign themselves a percentile grade for their own engagement with the course material. I will then plug the final percentile grade into a formula that adds or subtracts points based on attendance and maybe what percentage of small assignments they complete where perfect participation and attendance adds to score, a range results in no change, and excessive missed classes and activities results in lost points. I see a number of ways that this could go horribly wrong and I’m still working out the kinks, but it would also relieve the demand for me to track so many different assignments or create “optional” work.
  • I am going to rewrite the longer rubrics both to make them easier to follow and so that the students can explicitly use them as checklists. Similarly, I am going to print these rubrics and distribute them directly to my students.
  • Ditto for handouts on things like writing. I provide a lot of resources for the skills that I ask the students to master in these classes, but I find that even when directing students to them via presentation in front of the class, they are not being used because most students forget that they are there. I remember sticking handouts into my backpack never to be seen again, but at least having been handed a physical copy of something might help jog memories.
  • I am changing the token system. Tokens will only be used for turning in assignments late and probably limited to just 2, with a reward to the participation grade for every token left unused. Revision will be limited to the papers, but allowed for every paper, albeit probably with firmer deadlines for when a first round of revisions need to be complete.
  • Since none of this addresses how much time I spent responding to individual papers this semester, I am also likely going to lean more heavily on the language in the rubric and invite students looking to revise their papers to higher levels of achievement to come for conferences earlier in the semester.

Looking over these changes, there are still parts of this system I am concerned about. The ungrading formula, for instance, is an awkward beast to explain in the syllabus and it could lead to uncertainty about how the various non-paper assignments contribute to their grade. But I also think that there is a real possibility that these changes might be able to preserve what I liked about last semester while also steering into the sorts of written and metacognitive exercises that I find particularly valuable for students in a way that will make it a more sustainable and productive learning environment for everyone involved.

What is “the college essay,” or ChatGPT in my classroom

Posted on by Josh Nudell

Confession: I don’t know what is meant by “the college essay.”

This phrase has been the shorthand for a type of student writing deployed over the past few weeks in a discussion about the relationship between college classes and AI programs like ChatGPT-3 that launched in November, which I touched on in a Weekly Varia a few weeks ago. These programs produce a block of unique text that imitates the type of writing requested in response to a prompt. In its outline, input/output mimics what students do in response to prompts from their professors.

The launch of ChatGPT has led to an outpouring of commentary. Stephen Marche declared in The Atlantic that the college essay is dead and that humanists who fail to adjust to this technology will be committing soft suicide, which followed on from a post earlier this year by Mike Sharples declaring that this algorithm had produced a “graduate level” essay. I have also seen anecdotal accounts of professors who have caught students using ChatGPT to produce papers and concern about being able to process this as an honor code violation both because the technology is not addressed explicitly in the school’s regulation and because they lacked concrete evidence that it was used. (OpenAI is aware of these concerns, and one of their projects is to watermark generated text.) Some professors have suggested that this tool will give them no choice but to return to in-class, written tests that are rife with inequities.

But among these rounds of worry, I found myself returning to my initial confusion about the nature of “the college essay.” My confusion, I have decided, is that the phrase is an amorphous, if not totally empty, signifier that generally refers to whatever type of writing that a professor thinks his or her students should be able to produce. If Mike Sharples’ hyperbolic determination that the sample produced in his article is a “graduate level” essay is any guide, these standards can vary quite wildly.

For what it is worth, ChatGPT is pretty sure that the phrase refers to an admissions personal statement.

When I finished my PhD back in 2017, I decided that I would never assign an in-class test unless there was absolutely no other recourse (i.e. if someone above me demanded that I do so). Years of grading timed blue-book exams had convinced me that these exams were a mismatch for what history courses were claiming to teach, while a combination of weekly quizzes that the students could retake as many times as they want (if I’m asking the question, I think it is worth knowing) and take-home exams would align better with what I was looking to assess. This also matched with pedagogical commitment to writing across the curriculum. The quizzes provided accountability for the readings and attention to the course lectures, as well as one or more short answer questions that tasked the students with, basically, writing a thesis, while the exams had the students write two essays, one from each of two sets of questions that they were then allowed to revise. Together, these two types of assignments allowed the students to demonstrate both their mastery over the basic facts and details of the course material and the higher-order skills of synthesizing material into an argument.

My systems have changed in several significant ways since then, but the purpose of my assignments has not.

First, I have been moving away from quizzes. This change has been a concession to technology as much as anything. Since starting this system on Canvas, I moved to a job that uses Blackboard and I have not been able to find an easy system for grading short answer questions. I still find these quizzes a valuable component of my general education courses where they can consist entirely of true/false, multiple choice, fill in the blank, and other types of questions that are automatically graded. In upper-level courses where I found the short-answer questions to be the most valuable part of the assignment, by contrast, I am simply phasing them out.

Second, whether as a supplement to or in lieu of the quizzes, I have started assigning a weekly course journal. In this assignment, the students are tasked with choosing from a standard set of prompts (e.g. “what was the most interesting thing you learned this week,” “what was something that you didn’t understand this week form the course material? Work through the issue and see if you can understand it,” “what was something that you learned this week that changes something you previously wrote for this course?”) and then writing roughly a paragraph. I started assigning these journals in spring 2022 and they quickly became my favorite things to grade because they are a low-stakes writing assignment that give me a clear insight into what the students have learned from my class. Where the students are confused, I can also offer gentle guidance.

Third, I have stopped doing take-home exams. I realized at some point that, while take home exams were better than in-class exams, my students were still producing exam-ish essay answers and I was contributing to this problem in two ways. First, two essays was quite a lot of writing to complete well in the one week that I allotted for the exam. Second, by calling it an exam most students were treating it as only a marginal step away from the in class exam where one is assessed on whether they have the recall and in-the-moment agility to produce reasonable essays in a short period of time.

What if, I thought, I simply removed the exam title and spread the essays out over multiple paper assignments?

The papers I now assign actually use some of the same prompts that I used to assign on exams, which were big questions in the field the sort that you might see on a comprehensive exam, but I now focus on giving the students tools to analyze the readings and organize their thoughts into good essays. Writing, in other words, has become an explicit part of the assignment, and every paper is accompanied by a meta-cognitive reflection about the process.

Given this context, I was more sanguine about ChatGPT than most of the commentary I had seen, but, naturally, I was curious. After all, Sharples had declared that a piece of writing it produced was graduate level and Stephen Marche had assessed it lower, but still assigned it a B+. I would have marked the essay in question lower based on the writing (maybe a generous B-), and failed it for having invented a citation (especially for a graduate class!), but I would be on firmer footing for history papers of the sort that I grade, so I decided to run an experiment.

The first prompt I assigned is one that will, very likely, appear in some form or another in one of my classes next semester: “assess the causes underlying the collapse of the Roman Republic and identify the most important factor.” I am quite confident in assigning the AI a failing grade.

There were multiple issues with ChatGPT’s submission, but I did not expect the most obvious fault with the essay. The following text appeared near the end of the essay.

Vercingetorix’ victory was, I’m sure, quite a surprise for both him and Julius Caesar. If I had to guess, the AI conflated the fall of the Roman Republic with the fall of the Roman Empire, thus taking the talking points for the Empire and applying them to the names from the time of the Republic. After all, ChatGPT produces text by assembling words without understanding the meaning behind them. Then again, this conflation also appears in any number of think-pieces about the United States as Rome, too.

But beyond this particular howler, the produced text has several critical issues.

For one, “Internal conflict, economic troubles, and military defeats” are exceptionally broad categories each of which could make for a direction to take the paper, but together they become so generic as to obscure any attempt at a thesis. “It was complex” is a general truism about the past, not a satisfactory argument.

For another, the essay lacks adequate citations. In the first attempt, the AI produced only two “citations,” both listed at the end of the paper. As I tell my students, listing sources at the end isn’t the same thing as citing where you are getting the information. Upon some revision, the AI did manage to provide some in-text citations, but not nearly enough and not from anything I would have assigned for the class.

A second test, using a prompt I did assign based on Rudyard Kipling’s The White Man’s Burden, produced similarly egregious results. The essay had an uninspired, but a mostly adequate thesis, at least as a starting point, but then proceeded to use three secondary sources, none of which existed in the format that they were cited. Unless the substantial C.V. of the well-published scholar Sarah C. Chambers is missing a publication on a topic outside her central areas of research, she hasn’t argued what the paper claims she did.

A third test, about Hellenistic Judea, cited an irrelevant section of 1 Maccabees and a chapter in the Cambridge History of Judaism, albeit about Qumram and neither from the right volume nor with the right information for the citation. You get the idea.

None of these papers would have received a passing grade from me based on citations alone even before I switched to a specifications grading model. And that is before considering that the AI does even worse with metacognition, for obvious reasons.

In fact, if a student were to provide a quality essay produced by ChatGPT that was accurate, had a good thesis, and was properly cited, and then explained the process by which they produced the essay in their metacognitive component, I would give that student an A in a normal scheme or the highest marks in my specs system. Not only would such a task be quite hard given the current state of AI, but it would also require the student to know my course material well enough to identify any potential inaccuracies and have the attention to detail to make sure that the citations were correct, to say nothing of demonstrating the engagement through their reflection. I don’t mind students using tools except when those tools become crutches that get in the way of learning.

In a similar vein, I have no problem with students using citation generators except that most don’t realize that you shouldn’t put blind faith in the generator. You have to know both the citation style and the type of source you are citing well enough to edit whatever it gives you, which itself demonstrates your knowledge.

More inventive teachers than I have been suggesting creative approaches to integrating ChatGPT into the classroom as a producer of counterpoints or by giving students opportunities to critique its output, not unlike the exercise I did above. I have also seen the suggestion that it could be valuable for synthesizing complex ideas into digestible format, though this use I think loses something by treating a complex text as though it has only one possible meaning. It also produces a reasonable facsimile of discussion questions, though it struggles to answer them in a meaningful way.

I might dabble with some of these ideas, but I also find myself inclined to take my classes back to the basics. Not a return to timed, in-class tests, but doubling down on simple, basic ideas like opening student ideas to big, open-ended questions, carefully reading sources (especially primary sources) and talking about what they have to say, and how to articulate an interpretation of the past based on those sources–all the while being up front with the students about the purpose behind these assignments.

My lack of concern about ChatGPT at this point might reflect how far from the norm my assessment has strayed. I suspect that when people refer to “the college essay,” they’re thinking of the one-off, minimally-sourced essay that rewards superficial proficiency of the sort that I grew frustrated with. The type of assignment that favors expedience over process. In this sense, I find myself aligned with commentators who suggest that this disruption should be treated as an opportunity rather than an existential threat. To echo the title from a recent post at John Warner’s SubStack, “ChatGPT can’t kill anything worth preserving.”

First Day Fragments: Fall 2022

Posted on by Josh Nudell

Each of the past four years I have written a post celebrating the start of the fall semester with quick hits on various topics that I’m mulling over going into the new academic year. You can find the earlier posts in the archive: 2021, 2020, 2019, 2018.

I spent most of this summer diligently making myself go out running several times a week. Like many of my summer goals this practice ended up being a mixed bag, and I ended up not hitting my arbitrarily-set target. Despite the aches and pains that have accompanied these runs, I have been pleased by my progress and consider this one of my best recent decisions. There is always another mile to run, just as there is always more that could be done. After a summer during which I neither got the rest I had hoped nor accomplished as much as I intended, it has been useful to just focus on the next step.

ΔΔΔ

The latest buzz about workplace culture is Quiet Quitting, aka working to contract. This is a radical concept that I have seen some of my friends who work in the UK talk about when they, *checks notes*, clock out for the weekend and take annual leave. In a general sense, this “trend” is a reminder that it often takes enormous amounts of uncompensated labor to make the current labor systems function.

This is no less true in higher education than anywhere else, and something that hits basically everyone involved. The diminishing portion of the overall faculty being tenure or tenure track means that the service burdens fall that much heavier on those who remain, while the low pay for adjuncts is sometimes justified by pretending that the hours spent working outside of the classroom don’t exist. At least in the former case there is a reasonable expectation that their job will remain semester after semester.

However, quiet quitting is especially disruptive in teaching. In some fields it means reclaiming one’s time from their employer, but in this one the consequences disproportionately affect the students. They might not exactly be customers, but they are paying for their education in some capacity, which is a difficult circle to square.

My situation is not nearly as bad as many people, and I have a department chair who is excellent about keeping most duties not in my contract off my plate. And yet, this trend of quiet quitting has been on my mind this last week of working (uncompensated) overtime to make sure that my courses were ready to go. Ultimately this is yet another reminder that the entire system needs to be reformed to make it more humane for everyone involved.

ΔΔΔ

One summer during college (summer 2007), I worked on the team that moved the college to a new learning management system. At one point we had a conversation about the functionality that allowed professors to track the time students spent on the website. As a student, I was stridently against making this function available to professors, or at least against highlighting it for them–I don’t actually recall the specific details of the discussion. My point at the time was that these tracking features were, but delicately, bullshit, and I didn’t want there to be any chance that such data would play a role in my grade when it provided, at best, a shoddy reflection of my engagement with the course.

I have modified my stance somewhat as a professor. I still strongly dislike the emphasis on time-on-task, even if I begrudgingly acknowledge that it can play a role in setting expectations. As for the LMS data, I never use it as a form of assessment but have found that I like having it as a diagnostic tool–one of several in my arsenal that can help me best help my students.

It was with this background that I read John Warner’s piece last week on the madness of productivity trackers. He excoriates these trackers as not only a bad way of assessing productivity, but also actively harmful to knowledge work that requires hours of work that can’t be tracked by these tools without rigging an artificial mechanism to keep the system occupied while you actually get the work done. As Warner puts it, ” we are more than our ability to produce according to metrics counted by an algorithm.” He also rightly points out how LMS policies in college can serve as a training ground to normalize this sort of surveillance. This makes me wonder whether the marginal benefit I gain in helping my students is worth contributing to our present dystopia.

ΔΔΔ

I found this semester to be particularly difficult to prepare for. In part my struggle was the growing pain of designing and preparing a new course that is unlike anything I have taught before while, simultaneously, having events conspire to keep me from get the amount of rest I had intended. Put simply, I was tired. But a substantial portion of my exhaustion was not so much physical as feeling emotionally drained from the fire hose of devastation that seems to be going on in the world, from historic drought to continuing pandemic, to vaccine denial bringing about the return of awful disease thought eliminated by modern technology, to the fever pitch of nasty politics, to the crisis of climate change. All systemic problems where the proposed solutions range from personal responsibility to nothing whatsoever. Not good.

In his book Radical Hope, Kevin Gannon writes about the importance of hope as a pedagogical obligation because this is not a job that one should do if they don’t have at least a glimmer of optimism about the future.

I spent most of this summer not feeling a whole lot of hope. I’m still not, in a lot of ways. But this is also not a job that one can do without hope and I often find that I can see that hope most clearly when I’m working with my students. This is very likely going to be a challenging semester, but one day into the semester and I can already start to see the glimmer of that hope rising like a phoenix from the ashes. The world might be burning, but that doesn’t mean we can just let it burn. Only by working together can we start to heal.

Three Things of Spring 2022

Posted on by Josh Nudell

Grades are submitted, another semester is in the books—along with my first year at Truman State.

Frankly, that end couldn’t come soon enough. A bunch of factors colluded to make the first two months of the semester one of the busiest stretches of my life. Most of that busy was good, but it also meant that I spent the back half of the semester—a period during which I organized a speaker coming to campus and prepared an hour-long talk for a student group, on top of teaching and meetings—triaging my commitments and trying to avoid burning out. I don’t recommend letting things get this busy.

With the semester just several days in the rear-view mirror and aided by an early heat wave that sent the temperatures into the 90s, I am starting to settle into a summer routine. I will have a post in the coming days that lays out some goals, but, first, I want to take a moment to survey the semester that just ended.

As you might guess from the title, I have three major thoughts.

First, I once again find myself considering burnout.

More than once in the past few months I asked myself how much of my exhaustion was particular to this semester, how much of it was residual anxiety from years spent as a part-time adjunct, and how it was the accretion of stress from teaching for two years during a global pandemic.

The answer, of course, is “all of the above.”

In my muddled thoughts from last semester, I framed this question in light of compassionate pedagogy and idly wondered whether flexible policies inadvertently prompts students to devalue a course. Despite those questions, my “flex” late policy that asked students to either submit their work by the time a checkpoint came due or fill out a form that would record their extension was easily one of the most popular course policies I have ever come up with so I naturally rolled it over to similar effect this semester. Despite my misgivings, my students reported exactly what I hoped, that this policy allowed them the flexibility to manage their schedules and do their best work. Transitioning to a specs grading system this fall will require some slight tweaks because there will no longer be points to deduct for late assignments, but the framework of this policy will be fixture of my courses going forward.

The popularity of this policy speaks to the stress that the students are facing and I came out of this semester more convinced than ever that the problems with burnout are structural. No amount of self care will resolve these issues and reminders to be mindful like the one I received last week from a textbook publisher border on the farcical.

In his recent book Four Thousand Weeks, Oliver Burkeman argues time management gurus have been focusing on exactly the wrong message. Basically, he says, we have a finite amount of time, which necessarily means that we will not be able to do everything in a single lifetime. Trying to do more by working more efficiently or cutting into time for rest will detract from the experience of all of the activities rather than lead feelings of accomplishment. His argument, then, is that we should do less so that the activities that we do do become more meaningful.

Burkeman was not writing about higher education specifically, but there is a useful lesson here. Bill Caraher on his blog has written a good bit about “slow” as an ideal and mentioned his concerns with the workload expected of both graduate and undergraduate students. I fear that I contributed a little bit to the heightened expectations such that I am going to scale back a little bit in the fall. At the same time, this is not a problem that I can solve on my own.

Second, this semester I optimistically incorporated peer review into several of my courses. My students had to bring paper drafts to class and submit a copy to Blackboard. We talked about papers and revision before the first peer review day, and I had hoped that the course material would give the reviewers the content background to critique the papers, while seeing different approaches to similar material, repeated practice at reviewing, and my feedback would make them better writers and reviewers over the course of the semester.

In short, this part of the course was a spectacular failure.

I should qualify that statement. Some of the students became pretty good reviewers and saw dramatic improvement in their work, but, for every one of those, there were at least two for whom it didn’t work. The causes varied. Some routinely brought such scant “drafts” that their peers had almost nothing to critique. Others reported only receiving grammar and spelling comments. Still others reported deep-seated anxiety over giving sharp feedback and being unwilling or unable to look at the comments. And those were substantive problems before considering that peer-review day saw by far the lowest attendance rates.

The requirement that drafts be submitted to Blackboard also allowed me to give some feedback at an early stage, but even minimal guidance magnified my workload to an unmanageable extent.

I think that peer review can work, but I need to come up with a new approach because what I did this semester ain’t it. I suspect that it would require significantly more time training students to give good feedback and reiterating the purpose of the exercise to make them more receptive to the comments. Guiding students to become better readers would also help, but there were enough different problems that I doubt this is the sole or even primary cause. I don’t want to abandon the peer-review entirely, but the question is how to make it useful.

Third, something of a PSA: when you have two assignments due close together, the one that needs timely feedback should be due first.

For reasons that defy understanding I reversed that order this semester and spent several weeks in grading purgatory as a result. I hadn’t finished grading the first set when the second arrived and required immediate attention, causing the first batch to languish for far longer than I intended. Simply changing the order in which these arrived would have made for a more satisfactory experience for everyone involved.

Distracted: Why Students Can’t Focus and What You Can Do About It

Posted on by Josh Nudell

Tell me if this sounds familiar.

You have just spent the last ten minutes doomscrolling through Twitter. Some of the posts made you laugh. Some made you anxious over the state of the world. Some made you insecure about what you are or are not doing. A couple made you think. Maybe you responded, but probably not. You might have clicked through a link, but, again, probably not. It is time to work. You close the Twitter app. Then, without so much as putting your phone down, you reflexively open the Twitter app and check out what is happening — if you’re anything like me, you didn’t even open another app in between.

Or maybe you went from the Twitter app on your phone to Twitter on a browser, or vice-versa.

Or, maybe, TikTok or Facebook are more your speed. Or maybe snapchat or a game. The specifics don’t matter because the end result is the same: people flit from one thing to another drawn like moths to a flame to advertisements, social media, and a host of other distractors carefully designed to harvest our attention.

This ubiquitous feature of modern life, naturally, leads to waves of hand-writing over the pace of life and how modern technology has entirely ruined the ability of people, but particularly young people, to focus for any length of time.

In an educational context, these fears has led to the question of how to best eliminate distractions from the classroom, whether through draconian technology bans or trying to convince students to treat class like a sanctuary where they should leave their concerns at the door for the duration. According to James Lang, however, these well-meaning impulses are asking the wrong questions. We can never eliminate distractions. Beyond the simple fact that our monkey minds are calibrated to look for distractions, it is too much to expect that students will be able to put out of mind a sick loved one, or a relationship problem, or a bodily pain, or any of an infinite variety of other concerns for a class that may or may not be all that important for them. If this wasn’t obvious before, it should be now given the ongoing COVID-19 pandemic.

That’s the bad news. The good news, as James Lang points out in the first chapter, is that latest round of laments for the prelapsarian days before distraction are strikingly myopic. That is, there was never a golden age when people were free from distraction and laments about its loss merely get updated to account for technology. In his posthumous novel “The City and the Mountains” (A Cidade e as Serras) from 1901, the Portuguese novelist José Maria de Eça de Queirós includes a dream sequence where the narrator is appalled by the frivolity of modern life:

“Leaning in His super-divine forehead which conceived the world, on the super-powerful han which created it—the Creator was reading and smiling. I dared, shivering with sacred horror, to peep over His radiant shoulder. The book was a popular edition, paper-covered. The Eternal was reading Voltaire in the new, three-franc, cheap edition, and smiling.”

Or one could look to the collection of quotes on the subject collected by Randall Munroe in XKCD:

In other words, to be distracted is to be human. Even as I write this, I am distracted by a kitten who doesn’t understand it is a problem for her to repeatedly leap onto my desk, chew on books, papers, and pens, and nuzzle my hands while I type. She is also fascinated by my fingers when I am touch-typing.

Lang’s thesis in Distracted is thus that we should not pursue the quixotic aim of eliminating distraction, but that we should be leaning in to strategies that cultivate attention. Sometimes this requires temporarily eliminating distractions — when I am doing my academic writing, for instance, I set a length of time during which I turn off my email and won’t check social media —but, more frequently, the strategies involve finding ways to redirect and renew attention when it flags over the course of a class and a semester. Learning is hard work and if you’re anything like me your attention span dips precipitously when you’re tired. The same thing applies to students.

This thesis might be simplicity itself, but actually pulling it off in a classroom setting requires practice and attention.

Like his earlier book Small Teaching, Distracted is not prescriptive. Lang mentions several times that he is generally agnostic about a lot of teaching methods because good teaching can take many forms. What works for one teacher — or student — won’t necessarily work for another. Rather, he lays out current research into the science of attention and uses numerous examples of activities and practices to establish principles that any teacher can adapt to their class.

I concluded of Small Teaching that its simplicity was the greatest sign of its success. Distracted tackles thornier issues and Lang dedicates the entire third chapter (~35 pages) to the tech ban debate that couches his suggestions in the awareness that his own policies have changed quite dramatically over the years. This and other portions of the book take a more process-oriented approach that encourage the teacher to be conscientious of how the policies affect the classroom atmosphere.

Other portions of Distracted are more like Small Teaching. The book’s second part offers six “practices” of attention and how they can help draw students toward the material you have to offer. These range from the simple — cultivating a community through the use of names and modeling the behavior you want to see by leaving your phone in your office — to engaging student curiosity to techniques for focusing attention by switching between activities or with quick attention renewal devices in which he gave the example of a preacher asking an audience for an “amen” when they start to drift. Lang also makes the case that assessments are a critical component of attention because they work to direct students toward the material that you believe is important in the course. Sometimes this means crafting assessments with attention in mind since many students will never be more focused on your material than when writing a big test, but other times it involves no- or minimal-grading on repeated assignments that ask the students to connect what they’re learning in the class to life today. Students might find the practices unfamiliar at first, but with practice and attention on the part of the teacher they can pay dividends in the classroom.

Much of what Lang writes in Distracted echoes the direction I have been moving my courses over the past few years in terms of building community and keeping the classroom fresh, particularly on low energy days. It doesn’t always work, of course, but each of the chapters in Part 2 offers a wealth of ideas to help draw students back in. For this reason I fully expect that I will return to Distracted for inspiration and found that it was an ideal book to read while putting together my courses for the semester. In fact, I often would read something that inspired me to put down the book mid-chapter to modify language in a syllabus or tweak an assignment. It is possible to quibble with a small individual observation or policy or suggestion, and I did at times, but for every one where that happens two more will land home.

Distracted is not necessarily where I would start for a new teacher looking for tips on teaching (my current recommendation is David Gooblar’s The Missing Course), but it is both one of the two books I would suggest after that (along with Jay Howard’s Discussion in the College Classroom) and a book with a lot worth considering for even the most experienced teachers.