This week in my speculative fiction first-year seminar we have been working through a mini-unit on Utopias and Utopian thinking.
On Monday, I gave in lecture a “brief history of Utopian thinking” (I tried to name as many daily topics as possible like they were episode titles from Community). We started with a breakdown of Thomas More’s Utopia (1516) and the society therein, but then explored both earlier examples like the Golden Age of Man in Hesiod’s Works and Days and Plato’s Republic, and historical attempts to create these communities like the Shakers and the Oneida Community. That day concluded with a discussion of what utopias do, both in terms of social critique of the present and imagining a better future. We haven’t yet talked about Atlantis and Atlantean-type stories as Utopias because I (mistakenly) put it at the end of this unit, but the next time I teach this class, I’m going to move that day to put it more directly in dialogue with this one.
Then, on Wednesday, we read Ursula Le Guin’s “The Ones Who Walk Away From Omelas.” This short story asks you to imagine a happy, pleasant city that can only exist because of the abject suffering of a single child. Everyone in the community is aware of this trade-off and the ones who walk away cannot live with that knowledge. The story prompted a lively discussion, drawing comparisons to the Trolley Problem and generally about the morality of Utopias that always require some sort of trade. Several students challenged whether the people walking away are any more moral than the ones who stay given that even though they are opting out of the benefit of the Utopia they are nevertheless still living with the knowledge of the child’s suffering. One student asked how the message changes if you can’t walk away, to which several responded that it suddenly becomes a dystopia. This was my favorite question, though, because Omelas can be read as allegory for modern society where the happiness of people in one part of the world comes at the expense of the suffering of people elsewhere, in which case individuals only have so much capacity to opt-out.
(We are going to return to this point in the class at the very end of the semester with N.K. Jemisin’s “The Ones Who Stay and Fight.” Now that I think about it, I might move the Utopia discussion to the end of the semester next time around.)
The assignment for this unit is a poster where the students work in groups to create their own utopia, as agreed upon by the group members.
This is a deceptively difficult assignment. It requires thinking through the consequences of the society that they set up and consider what makes it a Utopia. One of the things I stressed in our discussions is that a Utopia for one is not a Utopia for all (except in the fleeting moment of Hesiod’s Golden Age), so one of the tasks is to define who are the “in” group and who are the “out” group, with those definitions being entirely up to the group. The larger the society, the harder it is to think through the consequences of the rules, laws, and social norms. This is why it amuses me that one group is re-creating a Matrix to allow each person their own bespoke Utopia that exists only in their minds.
To be completely transparent, this assignment is my equivalent of the Kobayashi Maru, just with a different set of lessons that can be taken from it.
If I were completing this same assignment, I would start by considering the sources of human conflict, big and small. If we were able to eliminate scarcity, jealousy, and pain, that would eliminate most conflict. Something like the world imagined in Wall-E as a dystopian future after humans destroyed the world.
The issue is that the elimination of all of these needs strips away something essential to being human, I think. Put another way, I think it is not possible to both have humans and to have a true Utopia, thus short-circuiting the whole exercise. As Hesiod says in Works and Days, we live in an Iron Age where we are doomed to experience sickness and pain as our meat sacks move through the world. It is simply the price of being human. Thus, the best that we can hope for is to mitigate the suffering that comes from scarcity, jealousy, and pain rather than eliminate it altogether. And, to paraphrase a delirious priest in Brothers Karamazov, we already live in paradise, so we have all the tools of that mitigation if we’re willing to commit to the practice.
However, this impossibility is also why I really like this assignment, perhaps with some fiddling around the edges. Utopias are good to think with, and working through the potential issues as a group forces the students to focus on the process rather than skipping ahead to the product.
I had no intention of returning to the ChatGPT discourse. I said my piece back in December and the conversation seemed to take a histrionic turn around the start of the semester when the course planning period coincided with a wave of op eds that treated AI-writing as an existential crisis.
Moral panics turn tedious in a hurry and I bore easily.
Happily, I think that the discussion is starting to move into a more compelling and, I hope nuanced, direction. In his column at Inside Higher Ed last week, for instance, Matt Reed suggested that there are two issues at play with AI-generated writing. The one, academic dishonesty and how to catch it, receives the bulk of the attention. The other, disinformation and inaccuracies, has to this point received much less attention. In other words, the practical considerations about the norms, expectations, and enforcement of academic transactions are taking precedence over the underlying principles. This sort of priority of course makes sense, as anyone who has worked within the institutions of higher education can tell you, but I also think that it misses that these two issues are inextricably intertwined.
Simply put, I am convinced that ChatGPT specifically, and AI more generally, is a digital and information literacy issue.
Now, I should acknowledge that the stakes involved are more profound outside of the semi-controlled academic context, and at least potentially herald fundamental disruption to existing economic models. Google, for instance, is reportedly treating the chatbot like an existential threat to their hegemony over access to information online. Likewise, AI-generated art is just the latest technology that will allow companies to cut labor costs—why pay artists to create cover-art for a book when you can have an intern churn out AI-generated images until you find one you like? As much as I maintain that AI is a tool and the person producing the art is an artist, companies are not likely to compensate the artist as such under these scenarios. But while both of these are ethical issues related to my point about digital literacy, neither are they wholly new.
When it comes to writing, AI is a tool, and tools are only as good as their users. A spell-Czech [sic] doesn’t provide any value if one doesn’t have the vocabulary to recognize when it misleads, just as gratuitous use a thesaurus can lead the writer astray. Predictive text is fine for email, but I find it distracting in other contexts because the program prompts me down particular lines of composition. And, as I put in the last post on this topic, citation generators will lead you astray if you are unwilling or unable to format the text that it generates.
In this sense, the danger with AI is that people are going to treat a tool for something as a replacement for that thing. But this does not constitute either an existential crisis or a fundamental disruption, despite groups of people treating it as one or the other.
There are a myriad reason that a student might submit an AI-generated essay. Most of these overlap with the reasons a student might purchase an essay or otherwise cheat on assignments, and need to be addressed as such. However, AI should give educators greater pause because, compared to the other forms of dishonesty, AI might give the impression to some students that they don’t need to learn the skill in the first place. Chatbots can give the appearance of engaging with a historical figure, but they do not actually let you converse with that person any more than the Metaverse can allow you to watch Mark Antony debate in Rome in 32 BCE. But that superficial engagement risks drawing people away from the actual substance that would allow the participant to see how the AI turns unredeemed racists into apologists for their heinous beliefs or to recognize that seeing Antony debate in Rome in 32 BCE would be quite a feat because he was in Egypt gearing up for war with Octavian at that time.
On a whim, I decided to ask ChatGPT why students should avoid using the AI to write papers. This was what it produced:
I followed that prompt with a question about whether AI could help students with their writing:
I received a slightly more enthusiastic response when I directly inverted the original prompt, but still as a tool that can make writing easier or more efficient. At my most cantankerous, I dislike several of these uses—text summarization assumes one viable reading that simply isn’t true, which is also my problem with services like Blinkist, and I think that text generation will create pathways that guide how the person writes and thinks about a topic—but I could make similar arguments for writing being shaped by whatever we’re reading and simple reliance on the the first definition of a word found in a dictionary. As I said in my original post, if someone were to use AI as a tool and produce a quality paper either without any further intervention or by editing and polishing the text until it met the standards, that paper would meet my criteria for what I want my students to achieve in the class. This process would not be my preference, but the student would have guided the program through numerous rounds of revision much as they would draft and re-draft any paper that they wrote themselves. So much so that it would be easier to just write the paper, in fact. I doubt that a truly revolutionary thesis could be developed that way, but the student would have demonstrated their mastery of the course material and a sensitive understanding of the writing practices to know that it met standards on my rubric—grammar might be easier to accomplish, but the other categories not so much.
In fact, the arrival of AI makes it all the more important for students to learn skills like reading, writing, and, especially in my discipline, historical literacy. To do this, though, I think it is a mistake to issue blanket prohibitions or build assessment as though it does not exist. Rather, I want students to understand both why AI is not a great choice and what its limitations are, which requires steering into AI, at least a little bit.
I scheduled a week for my first year seminar to address their first big writing assignment. The students have no reading this week, during which they will be working on their drafts of their first paper that are due on Friday. In the two class periods earlier in the week, I am going to have them complete an exercise using ChatGPT in their groups for the semester. On Monday, the students will work with ChatGPT to produce papers about the readings that we have covered to this point in the class, sharing with the me the results of the exercise. Then they will be charged with offering a critical evaluation of the generated text, which we will spend time on Wednesday sharing and discussing the critiques with the class, which will segue into a discussion of what makes writing “good.”
Students in my upper-division courses will do a similar exercise. As their first essays approach, I am going to provide students essays produced by ChatGPT using the same prompts and my essay rubric. Their task will be to “mark” the ChatGPT.
The goal is the same in both cases: to remind students that AI has severe limitations that cannot replace their unique thoughts. Further, I aim to engage the students as both writers and editors since I see the latter skill as an essential part of the writing process.
I don’t want suggest a prescriptive advice in this given that my class sizes and teaching mandates allow me to pursue some of these options. But the ChatGPT discourse has made even more convinced that it is necessary to teach basic, foundational, transferrable skills that will empower students to engage responsibly with the world in which they live.
I usually do “first day fragments” to mark the start of the fall semester, but here on the first day of the spring term I find that I also have a few topics rattling around that are also worth exploring. Only time will tell whether this is a one-off or a new spring-semester routine.
Course design is an exercise in omission. And the more of a survey the course is designed to be, the more this truism cuts close to the truth. This has been on my mind over the last week while preparing for the upcoming semester. Even before the pandemic I had begun adopting a “less is more” mantra in the classroom, and doubling down on core questions and fundamental skills. But I also like big and open-ended questions, both to structure the course and to set as assignment prompts.
This semester I will be teaching upper-division survey courses on Ancient Rome (Romulus to Romulus Augustulus, in theory), Ancient Persia (Achaemenid to Sassanid), and then a first-year seminar on speculative fiction. Enormous topics, all.
Adding material to these courses is the easy part. It would be easy, for instance, to have the students read Beowulf and Le Morte d’Arthur, skip forward to Lord of the Rings, and then do something contemporary. Or just watch the movies. Or I could have decided that we’re going to do an entire course on the thousands of pages in Ken Liu’s Dandelion Dynasty novels. But neither of these options fit with my objectives for the course.
The challenge is finding the right balance. The entire extent of Tolkien that we are going to read will be “On Hobbits” and two short pieces of commentary about Rings of Power. We’ll read Ken Liu’s brilliant short story “Paper Menagerie,” but nothing from his longer works. Ditto for N.K. Jemisin’s “The Ones Who Stay and Fight,” which I’m using both as a counterpoint to Le Guin’s “The Ones Who Walk Away From Omelas” and and as a way to close the semester on a note of optimism after an emotionally challenging set of readings.
ΔΔΔ
All three of my courses this semester are new preps. This is a double-edged sword.
On the one hand, new preps make for a lot of work. They require compiling materials as you go through the semester, writing presentation slides, and deciding on how you want to present the material, even when approaching topics that you know well. Some of the activities are going to flop, or maybe the scope of the course needs to be changed. The course wobbles a little, because it has not yet settled into its foundations. A graduate school professor told me once that he believes a course only reaches its mature form in its third iteration.
On the other hand, I sometimes find that certain in-class activities and readings work best the first time I assign them. This is in part because I am forced to spend more time with the readings and preparing the activities, which means that everything is fresher, but I also find something magic in the thrill of invention. The second and third time through I can adjust to how the students experienced the assignment, but this comes at a cost when the assignment becomes somewhat calcified or the pathways that the course discussion become a little more worn in.
ΔΔΔ
People have been talking on Twitter about when professors have an obligation to post the syllabus. My only thought is that the syllabus will go up when it is ready and the course website is minimally ready for use, usually a day or two before the semester starts. I’m happy to answer questions even when the syllabus is in the design phase, but there are a myriad of reasons why it is good to take right up until the last minute making changes even if the basic structure has been set for weeks.
ΔΔΔ
Most of my courses are what my university calls “Writing Enhanced,” which means that they fulfill the standards of that program—emphasis on product, cognition, and process. Nearly twenty years ago when I was an undergraduate student, a writing-enhanced course required a certain number of pages, some of which had to be revised, but my guidance here is more flexible. I have another course design post (yes, I know that this is turning into a teaching-heavy blog) in mind for the near future that engages with the models we use when designing new courses, but, every semester, I have a momentary pang of concern that I’m not having my students write enough. For instance, I have never assigned a long 15–20+ page final paper. Instead, my students write multiple shorter papers (5–7 page) that they revise to a high standard, with the thinking that learning to polish a concise argument in a short paper is a prerequisite for writing a good longer paper when taking research classes. Besides, even without a long research paper to conclude the semester, my students write a lot. By my rough tally, I find that many of my students write nearly twice as much as I did for any class I took as an undergraduate student. Which then sends a flare of concern in the other direction: how much writing is too much?
ΔΔΔ
I wrote about Chat-GPT last semester and stand by everything I wrote there. But the new semester has brought out another round of hand-wringing and panic about how this tool means for higher education. This semester I’ll be leaning into AI writing in some classes with an “AI-essay critique” exercise and otherwise just incorporating it into the conversations we have when we talk about writing. But as the topic du jour, I’m bored by the conversation now. Moral panics turn tedious in a hurry.
I submitted grades a little over a week ago and promptly withdrew, exhausted, into a little fort with curtain walls made of novels. At least that’s what it felt like. The specific details might be exaggerated.
Several times I tried to break through the fog that had settled over my mind, but succeeded only in producing a silly post about pizza TV shows and the weekly varia post that I start compiling as soon as the previous week’s goes up. I could barely think about the semester that had just ended, let alone put those thoughts into any sort of coherent discussion.
Simply put, I had an exceptionally difficult semester, and one that rates among the very toughest I have ever experienced. Some issues stemmed from causes external to my classes (e.g. not getting some much-needed rest this summer and early semester indexing and proofing a book manuscript that put me perpetually behind), while others stemmed from things that happened in the classes, most of which I don’t want to talk about in this space because I don’t like talking about specific student activity in a public forum even when identifying details have been redacted, especially when there is nothing of universal value that can be gleaned by doing so.
Not every problem stemmed from these issues, of course.
Dissatisfied with traditional forms of grading, I dove headlong into the world of Specifications Grading for most of my courses this semester. To stick with the metaphor, I liked these waters but they also sent me crashing into the rocks.
The formula varied a little bit, class by class, but, in general I came up with a system where the students earned credit across four or five categories of assignments (e.g. journal entries, small assignments/participation, papers). Every assignment was graded using a bespoke rubric and it either met the standard and thus earned credit, or it did not. More work, and higher quality essays (the essay rubric had two tiers, one for basic competency and another for advanced) earned higher grades. To meet these higher standards, I allotted virtual tokens that the students could use either to revise their papers or turn work in late, pegging the number of tokens to the number of papers.
I entered the semester thinking that I had worked out a reasonably simple system that would give students the agency to decide what grade they were aiming for, make my expectations for each grade level clear, and provide in-semester flexibility that would allow students to do their best work. However, I had not anticipated that putting these assignments and expectations up front in the course would lead to cognitive overload for a significant number of students. In fact, I had a conversation in the final week of class with a student who said that this semester was much harder than the course they had taken the semester before even though the workload in the two courses was identical except that I had swapped one short weekly assignment for another. While there are other explanations why this student might have struggled with my course, I’m inclined to take the sentiment at face value because I saw evidence of the same struggle from other students who were struggling to interface with the information that I had provided in a way that made it harder to complete the work itself.
The core of this problem, I think is that many students were used to traditional grading schemes that allow students to muddle through to a passing grade without too much effort. By contrast, the system I devised required students to complete assignments in each category to a specified level in order to earn the grade. Passing my general education courses last semester did not require too much work, unless you simply neglected a graded category.
I am treating this as a messaging problem for now. Traditional grading schemes remain stupid and I’m not ready to abandon my attempt to find something better just yet.
However, the issue of students neglecting grade categories dovetailed with the tokens and flexible deadlines to create absolute chaos on my end. Here there were several intertwined issues.
Several semesters ago I developed a system for deadlines where students could receive an automatic extension by filling out a Google form before the due date. This policy has proven incredibly popular with my students. However, while I intend to keep it intact in some form, I am starting to question whether the system is having the intended effect. Rather than providing students the space to do their best work, I am finding that whatever grace I provide is filled by other classes with stricter deadlines such that my students wind up writing their papers at the last minute anyway, just several days later, and I had so many students taking the extension that it became a challenge to return papers in a timely fashion.
However, it was the tokens that turned this semester into a logistical nightmare. I set up the tokens anticipating that most would be used for revisions, knowing full well that revisions coming in at any point would cause some chaos. What I did not anticipate is that some portion of students would use most or all of their tokens to turn work in late. This meant that I had not only revisions, but also new work being turned in on no particular schedule throughout the semester, and I had difficulty keeping tabs on students who hadn’t turned in assignments, some of whom I knew were working on things and some of whom I did not.
Compounding these issues was, I think, a consequence of having a significant number of first year students. Anecdotally, from talking with friends who teach in high school, some students have been conditioned to think that flexible deadlines and the like mean that an assignment is optional. Or that whatever make-up assignment gets offered will be easier than the original assignment. As one explained:
“I’ll allow X to be redone/revised/resubmitted” is increasingly being taken as “I don’t need to do X, I’ll do the makeup Y later which will be easier anyway.”
This was obviously not what had been intended, but this collision of expectations and conditioning meant that I spent a significant amount of time amid the chaos of trying to grade everything just trying to track down missing work so that the students wouldn’t fail on those grounds. Oh, and I had 50% more students than I had in either semester last year.
Then there was the grading itself. I adopted a specifications system because it promised to offload some portion of the grading onto explicit rubrics where I could check the appropriate box. I loved not assigning grades to papers, but I quickly discovered several things that meant the system created just as much work as the mystery black box of traditional grading, if not more. The issues started because, I discovered, many students simply did not complete the assignments with the rubrics in mind and did not use the rubrics to check the work before submission. This meant that I often received work that did not fulfill the simplest rubrics.
These problems were particularly acute on the written assignments with its long, detailed rubric that should have provided guidance for the papers. I quickly realized that many of my students did not have the writing background to achieve the higher proficiencies, so simply checking the rubric box was not going to provide adequate guidance or encouragement. At the same time, while some students were not going to be aspiring to those grade tiers, I also couldn’t in good conscience provide detailed feedback for some students and not for others until the very end of the semester when the possibilities of revision had passed. By the last two weeks of the term it was clear that I would not be able to get caught up, so I offered that any student who wanted to revise their work could come to office hours and have their paper(s) marked in person so that they could receive feedback on how to meet the next tier. These meetings gave any student meant that (I think) any student aiming for higher grade tiers reached them, but they also meant that those weeks were a whirlwind of paper conferences.
Finally, my small assignments policy put a cherry on top of this disaster sundae.
The policy was simple. There were some number of small papers, in-class activities, exit-tickets, one-minute essays, and other activities that took place in class. If you weren’t there, you couldn’t make up the work. Unless you were an athlete at a competition. Or you got sick. Or had other “excused” absences. Right from the start, I found myself litigating what counts as a legitimate absence, which is one of my least favorite parts about taking attendance. Then, like with non-completion of work, I found myself around the middle of the semester worried about the number of students who seemed liable to fail (or otherwise drop grade tiers) because they had failed to adequately participate in the class. Since the opportunities for these points often did not come at regular intervals, I found myself inventing “optional extra” opportunities that would allow the students to bring their grade in that category up, which, in turn, created confusion about what assignments students actually needed to complete. Often, the students who completed the optional assignments were not the ones I had in mind when I created them. And, of course, adding all of these small assignments created a flurry of paperwork that I had to manage.
Chaos.
I should point out that for a non-negligible percentage of my students this system worked exactly as I envisioned, giving them agency to achieve grades based on their goals for the semester. Had I not felt compelled to give the students aiming for the “C” the same level of feedback I gave to those aiming for an “A,” my grading might have even been manageable—but, of course, almost everyone said that they were aiming for an “A” back in August.
I am not ready to abandon this grading mode, just yet, but it needs to be modified in critical ways for it to become sustainable and productive. The changes I have in mind to this point are:
Streamline my messaging and expectations. This means not only being clear about my expectations in terms of earning credit across multiple categories, but also clarifying that this is a labor-based grading scheme. It is designed to be transparent and achievable, but not necessarily easy. At the same time…
I want to submerge the mechanics of the participation grade. Some of the chaos this semester was created by the various points that students earned for doing in-class activities, which meant that this was something I had to track. I am not planning to change the activities that I do for small assignments, but my current thought for this category is to take a page out of the “ungrading” playbook. Instead of me assigning grading, the students will complete three reflections, one at the start, one at the middle, and one at the end of the term. The first one will set expectations and think about where they are at the start of the course. The middle two reflections will both have the students assign themselves a percentile grade for their own engagement with the course material. I will then plug the final percentile grade into a formula that adds or subtracts points based on attendance and maybe what percentage of small assignments they complete where perfect participation and attendance adds to score, a range results in no change, and excessive missed classes and activities results in lost points. I see a number of ways that this could go horribly wrong and I’m still working out the kinks, but it would also relieve the demand for me to track so many different assignments or create “optional” work.
I am going to rewrite the longer rubrics both to make them easier to follow and so that the students can explicitly use them as checklists. Similarly, I am going to print these rubrics and distribute them directly to my students.
Ditto for handouts on things like writing. I provide a lot of resources for the skills that I ask the students to master in these classes, but I find that even when directing students to them via presentation in front of the class, they are not being used because most students forget that they are there. I remember sticking handouts into my backpack never to be seen again, but at least having been handed a physical copy of something might help jog memories.
I am changing the token system. Tokens will only be used for turning in assignments late and probably limited to just 2, with a reward to the participation grade for every token left unused. Revision will be limited to the papers, but allowed for every paper, albeit probably with firmer deadlines for when a first round of revisions need to be complete.
Since none of this addresses how much time I spent responding to individual papers this semester, I am also likely going to lean more heavily on the language in the rubric and invite students looking to revise their papers to higher levels of achievement to come for conferences earlier in the semester.
Looking over these changes, there are still parts of this system I am concerned about. The ungrading formula, for instance, is an awkward beast to explain in the syllabus and it could lead to uncertainty about how the various non-paper assignments contribute to their grade. But I also think that there is a real possibility that these changes might be able to preserve what I liked about last semester while also steering into the sorts of written and metacognitive exercises that I find particularly valuable for students in a way that will make it a more sustainable and productive learning environment for everyone involved.
Confession: I don’t know what is meant by “the college essay.”
This phrase has been the shorthand for a type of student writing deployed over the past few weeks in a discussion about the relationship between college classes and AI programs like ChatGPT-3 that launched in November, which I touched on in a Weekly Varia a few weeks ago. These programs produce a block of unique text that imitates the type of writing requested in response to a prompt. In its outline, input/output mimics what students do in response to prompts from their professors.
The launch of ChatGPT has led to an outpouring of commentary. Stephen Marche declared in The Atlantic that the college essay is dead and that humanists who fail to adjust to this technology will be committing soft suicide, which followed on from a post earlier this year by Mike Sharples declaring that this algorithm had produced a “graduate level” essay. I have also seen anecdotal accounts of professors who have caught students using ChatGPT to produce papers and concern about being able to process this as an honor code violation both because the technology is not addressed explicitly in the school’s regulation and because they lacked concrete evidence that it was used. (OpenAI is aware of these concerns, and one of their projects is to watermark generated text.) Some professors have suggested that this tool will give them no choice but to return to in-class, written tests that are rife with inequities.
But among these rounds of worry, I found myself returning to my initial confusion about the nature of “the college essay.” My confusion, I have decided, is that the phrase is an amorphous, if not totally empty, signifier that generally refers to whatever type of writing that a professor thinks his or her students should be able to produce. If Mike Sharples’ hyperbolic determination that the sample produced in his article is a “graduate level” essay is any guide, these standards can vary quite wildly.
For what it is worth, ChatGPT is pretty sure that the phrase refers to an admissions personal statement.
When I finished my PhD back in 2017, I decided that I would never assign an in-class test unless there was absolutely no other recourse (i.e. if someone above me demanded that I do so). Years of grading timed blue-book exams had convinced me that these exams were a mismatch for what history courses were claiming to teach, while a combination of weekly quizzes that the students could retake as many times as they want (if I’m asking the question, I think it is worth knowing) and take-home exams would align better with what I was looking to assess. This also matched with pedagogical commitment to writing across the curriculum. The quizzes provided accountability for the readings and attention to the course lectures, as well as one or more short answer questions that tasked the students with, basically, writing a thesis, while the exams had the students write two essays, one from each of two sets of questions that they were then allowed to revise. Together, these two types of assignments allowed the students to demonstrate both their mastery over the basic facts and details of the course material and the higher-order skills of synthesizing material into an argument.
My systems have changed in several significant ways since then, but the purpose of my assignments has not.
First, I have been moving away from quizzes. This change has been a concession to technology as much as anything. Since starting this system on Canvas, I moved to a job that uses Blackboard and I have not been able to find an easy system for grading short answer questions. I still find these quizzes a valuable component of my general education courses where they can consist entirely of true/false, multiple choice, fill in the blank, and other types of questions that are automatically graded. In upper-level courses where I found the short-answer questions to be the most valuable part of the assignment, by contrast, I am simply phasing them out.
Second, whether as a supplement to or in lieu of the quizzes, I have started assigning a weekly course journal. In this assignment, the students are tasked with choosing from a standard set of prompts (e.g. “what was the most interesting thing you learned this week,” “what was something that you didn’t understand this week form the course material? Work through the issue and see if you can understand it,” “what was something that you learned this week that changes something you previously wrote for this course?”) and then writing roughly a paragraph. I started assigning these journals in spring 2022 and they quickly became my favorite things to grade because they are a low-stakes writing assignment that give me a clear insight into what the students have learned from my class. Where the students are confused, I can also offer gentle guidance.
Third, I have stopped doing take-home exams. I realized at some point that, while take home exams were better than in-class exams, my students were still producing exam-ish essay answers and I was contributing to this problem in two ways. First, two essays was quite a lot of writing to complete well in the one week that I allotted for the exam. Second, by calling it an exam most students were treating it as only a marginal step away from the in class exam where one is assessed on whether they have the recall and in-the-moment agility to produce reasonable essays in a short period of time.
What if, I thought, I simply removed the exam title and spread the essays out over multiple paper assignments?
The papers I now assign actually use some of the same prompts that I used to assign on exams, which were big questions in the field the sort that you might see on a comprehensive exam, but I now focus on giving the students tools to analyze the readings and organize their thoughts into good essays. Writing, in other words, has become an explicit part of the assignment, and every paper is accompanied by a meta-cognitive reflection about the process.
Given this context, I was more sanguine about ChatGPT than most of the commentary I had seen, but, naturally, I was curious. After all, Sharples had declared that a piece of writing it produced was graduate level and Stephen Marche had assessed it lower, but still assigned it a B+. I would have marked the essay in question lower based on the writing (maybe a generous B-), and failed it for having invented a citation (especially for a graduate class!), but I would be on firmer footing for history papers of the sort that I grade, so I decided to run an experiment.
The first prompt I assigned is one that will, very likely, appear in some form or another in one of my classes next semester: “assess the causes underlying the collapse of the Roman Republic and identify the most important factor.” I am quite confident in assigning the AI a failing grade.
There were multiple issues with ChatGPT’s submission, but I did not expect the most obvious fault with the essay. The following text appeared near the end of the essay.
Vercingetorix’ victory was, I’m sure, quite a surprise for both him and Julius Caesar. If I had to guess, the AI conflated the fall of the Roman Republic with the fall of the Roman Empire, thus taking the talking points for the Empire and applying them to the names from the time of the Republic. After all, ChatGPT produces text by assembling words without understanding the meaning behind them. Then again, this conflation also appears in any number of think-pieces about the United States as Rome, too.
But beyond this particular howler, the produced text has several critical issues.
For one, “Internal conflict, economic troubles, and military defeats” are exceptionally broad categories each of which could make for a direction to take the paper, but together they become so generic as to obscure any attempt at a thesis. “It was complex” is a general truism about the past, not a satisfactory argument.
For another, the essay lacks adequate citations. In the first attempt, the AI produced only two “citations,” both listed at the end of the paper. As I tell my students, listing sources at the end isn’t the same thing as citing where you are getting the information. Upon some revision, the AI did manage to provide some in-text citations, but not nearly enough and not from anything I would have assigned for the class.
A second test, using a prompt I did assign based on Rudyard Kipling’s The White Man’s Burden, produced similarly egregious results. The essay had an uninspired, but a mostly adequate thesis, at least as a starting point, but then proceeded to use three secondary sources, none of which existed in the format that they were cited. Unless the substantial C.V. of the well-published scholar Sarah C. Chambers is missing a publication on a topic outside her central areas of research, she hasn’t argued what the paper claims she did.
A third test, about Hellenistic Judea, cited an irrelevant section of 1 Maccabees and a chapter in the Cambridge History of Judaism, albeit about Qumram and neither from the right volume nor with the right information for the citation. You get the idea.
None of these papers would have received a passing grade from me based on citations alone even before I switched to a specifications grading model. And that is before considering that the AI does even worse with metacognition, for obvious reasons.
In fact, if a student were to provide a quality essay produced by ChatGPT that was accurate, had a good thesis, and was properly cited, and then explained the process by which they produced the essay in their metacognitive component, I would give that student an A in a normal scheme or the highest marks in my specs system. Not only would such a task be quite hard given the current state of AI, but it would also require the student to know my course material well enough to identify any potential inaccuracies and have the attention to detail to make sure that the citations were correct, to say nothing of demonstrating the engagement through their reflection. I don’t mind students using tools except when those tools become crutches that get in the way of learning.
In a similar vein, I have no problem with students using citation generators except that most don’t realize that you shouldn’t put blind faith in the generator. You have to know both the citation style and the type of source you are citing well enough to edit whatever it gives you, which itself demonstrates your knowledge.
More inventive teachers than I have been suggesting creative approaches to integrating ChatGPT into the classroom as a producer of counterpoints or by giving students opportunities to critique its output, not unlike the exercise I did above. I have also seen the suggestion that it could be valuable for synthesizing complex ideas into digestible format, though this use I think loses something by treating a complex text as though it has only one possible meaning. It also produces a reasonable facsimile of discussion questions, though it struggles to answer them in a meaningful way.
I might dabble with some of these ideas, but I also find myself inclined to take my classes back to the basics. Not a return to timed, in-class tests, but doubling down on simple, basic ideas like opening student ideas to big, open-ended questions, carefully reading sources (especially primary sources) and talking about what they have to say, and how to articulate an interpretation of the past based on those sources–all the while being up front with the students about the purpose behind these assignments.
My lack of concern about ChatGPT at this point might reflect how far from the norm my assessment has strayed. I suspect that when people refer to “the college essay,” they’re thinking of the one-off, minimally-sourced essay that rewards superficial proficiency of the sort that I grew frustrated with. The type of assignment that favors expedience over process. In this sense, I find myself aligned with commentators who suggest that this disruption should be treated as an opportunity rather than an existential threat. To echo the title from a recent post at John Warner’s SubStack, “ChatGPT can’t kill anything worth preserving.”
I have had an abiding love of speculative fiction for about as long as I can remember. I have a memory of my father reading the stories of Tolkien and Lewis to me and my younger brothers, and, at some point, I started reading ahead on my own to complete my first of many read-throughs of The Lord of the Rings. I started reading The Wheel of Time in elementary school and was deeply disturbed by descriptions of the blight. I picked up A Song of Ice and Fire sometime in middle school, mostly because I was drawn to the cover art. I also read a lot of bad speculative fiction in those days and am retroactively pleased with my youthful dissatisfaction with certain books.
I say all of this by way of prologue.
First year students at Brandeis (my undergraduate institution) took a “University Seminar in Humanistic Inquiries” course. These courses are designed as seminars on a coherent topic that begins establishing transferable skills and lays a foundation for further progression in college. If I’m being honest, I don’t recall my section of this course being particularly successful (I got into my third choice, after my top choice taught by my future adviser filled up before I enrolled), but I like the idea of the course.
Truman State offers “Self and Society” seminars that work toward the same end while also promoting multi- and inter-disciplinary thinking. One of the myriad of things that has been consuming my time this semester is that I was offered an opportunity to design and offer a course. The remit of these courses have to meet a certain level of disciplinary background, they are also a space that can allow for professors to create courses based on their areas of interest, outside the usual disciplinary constraints. The course I pitched, and that I am now designing to be taught next semester is “Historicizing Speculative Fiction.”
I read speculative fiction as a historian, which makes sense given my professional training and areas of expertise. One of my pet peeves about speculative fiction is when the world itself is undeveloped, while, by contrast, I will often overlook narrative or character issues if I have fallen in love with a creative world. When I proposed the course, I explained that while these genres of literature have their roots in myths and legends, these invented worlds are reflections of real world issues. Thus, the course description:
In this section, we will use speculative fiction—particularly science fiction and fantasy stories—to approach the issues of Self and Society. Once framed as niche interests, these stories make up some of the biggest pieces of intellectual property in the world today. Such stories might seem like simple entertainments featuring wizards and elves and dragons, but these worlds and the ideas we bring with us to talk about them reflect very present concerns about society and our place in them. So step through the wardrobe with me and let’s see how we can use these stories to better understand ourselves.
My idea for the course is to build a series of thematic units each built around one novel, or a primary and a back-up that could be substituted in future iterations. These novels are supplemented with short stories, essays about popular culture, and selections from other authors. These units are interchangeable by design, such that each time the course is offered I can swap units in and out. For the first iteration, I have chosen four units: World-building and historicism (P. Djèlí Clark, A Master of Djinn), Power, Language and Authority (Ursula le Guin, The Tombs of Atuan), The Environment (Octavia Butler, Parable of the Sower), Self, Society, and the Worlds we Create (Susanna Clarke, Piranesi). You can see the core reading list at the end of this post.
The core reading list is intentionally diverse, with the intent being to break students away from the expected canon for a course like this and to introduce them to the range of creative stories that exist. I won’t say that it was easy to craft this reading list. My personal tastes in fantasy stories run to very long books and extended series, and I can’t reasonably set the four books of The Dandelion Dynasty and expect the students to actually read them all, even though it might be the most perfect series for this course. However, I have also been greatly enjoying the excuse to read short stories in preparation for the course—more than once this semester you might have found me weeping in my office because of something I had just read. But I also have more to do still, since I would like at least one short story to fill out the unit on the environment.
The other work-in-progress for this course is the list of assignments. Some of these are going to be straightforward (e.g. book reviews and a course journal), but I am also concocting some creative assignments designed to get students to make students engage with the course themes in different ways. For instance, one assignment is going to be an “Inventing Utopia” group project where the students will work in groups to design their own utopias and present them as a poster presentation.
One thing I want to be particularly careful about with this course is striking a balance between sharing with the students all of these things that I think are particularly great without overloading the students who are in their first or second semester of college. I am beyond excited to be teaching this course, but if my enthusiasm leads to a course that is packed to the gills with amazing books and stories, then it won’t allow any space for the analysis and reflection where the actual learning happens.
Core Reading List
Introduction
Excerpts from Arthur stories and Beowulf
Nibedita Sen, “Ten Excerpts from an Annotated Bibliography on the Cannibal Women of Ratnabar Island”
Unit 1: World-building and historicism
P. Djèlí Clark, A Master of Djinn
Selections of J.R.R. Tolkien and George R.R. Martin
Adam Serwer, “Fear of a Black Hobbit” (the Atlantic)
Ursula Le Guin, “The Ones Who Walk Away From Omelas”
Unit 2: Power, Language, and Authority
Ursula le Guin, The Tombs of Atuan
Ken Liu, “Paper Menagerie”
Octavia Butler, “Speech Sounds”
Unit 3: The Environment
Octavia Butler, Parable of the Sower
Appendices to Dune
Unit 4: Self, Society, and the Worlds We Create
Susanna Clarke, Piranesi
Rebecca Roanhorse, “My Authentic Indian Experiencetm“
N.K. Jemisin, “The Ones Who Stay and Fight”
Alternate Units
Orientalism: Saladin Ahmed, Throne of the Crescent Moon
Colonialism: undecided
Epic Journeys: Neil Gaimon, Ocean at the End of the Lane
Noodlings 