The Altar of Academic Overproduction

Do publications define someone as an academic success? More to the point, do publications demonstrate one’s potential for scholarship?

These are both pressing questions for contingent faculty members who almost never have research as part of their contract and yet are assessed on their potential for research when being considered for tenure-track jobs. I have written here several times how I approached my research as a second job, though my current position allows me to integrate teaching and research even though the latter is not technically part of my contract.

With this in mind, one might ask when is the right time to publish. Job ads generally look for the potential for high-impact publishing, which should certainly include promising graduate students who have yet to publish anything, but it also might mean that because I just published my dissertation book and work on teaching contracts I might need to show some progress on my next book project before I’ll be seriously viable for research positions given that as soon as I am hired I am on a clock to finish the next thing or lose my job. My last book should prove that I can publish, but that track record is only good if it can support a trajectory toward future potential.

But the words that I often think about in those ads are “high impact.” These words mean different things to different people, and even more different things in different fields. When I was a newly-minted PhD my institution was just getting caught up with impact metrics, so it of course surprised me to learn that most history publications simply weren’t indexed at the institution. And probably for good reason. You can find impact metrics for classics journals, but this will only take you so far. According to one, the two journals with the two highest impact factors are Ancient Philosophy and the Journal of Roman Studies, neither of which are viable publication venues for my research, and I would dispute the quality of some of the other highly-ranked journals. This is not a field with Nature or The New England Journal of Medicine.

Moreover, I have heard differing opinions about when students ought to start publishing original research. I know of people who discourage junior graduate students from trying to publish prematurely. The work of publishing extend the time of completion for publications that might not move the needle in a job search at best, and, at worst, “immature” publications might attach the student’s name to subpar work. My advisor, by contrast, had me send off a revised version of a seminar paper as a second year MA student—a paper, I might add, that was duly rejected with a multi-page reviewer critique. I agreed with the rejection enough that I scrapped that paper and didn’t submit my own research again until I published a note with Classical Quarterly midway through my PhD that I had accepted with revisions upon first submission. I don’t personally subscribe a hard and fast rule, but, based on my own experiences, I generally think that students reach a point where their research is ready for publication later in graduate school than earlier.

Much like early-career graduate students, some undergraduates might have the command of both the primary evidence and scholarship to publish peer-reviewed scholarship and there are some fields where students can be sufficiently involved in research projects that their names end up on publications, but neither of these should be an expectation before graduate school. Learning how to do this work is literally one of the purposes of graduate school. Making admissions decisions based on these publications rewards students with structural advantages in their undergraduate institutions more than selecting students based on their research potential. No system will level the playing fields in this respect, but publications strikes me as a particularly insidious factor. Admittedly, I have not seen much evidence of this happening, but I am opposed to it in any field. Certainly, I would not have been admitted to graduate school if that were the expectation.

So you can imagine my reaction to this Pro Publica report on new companies like Scholar Launch designed to help get high school students published in “peer reviewed journals” by having them work with renowned scholars whose names are noticeably absent from the program forms that only list their academic rank and affiliation. All for the low, low fee of several thousand dollars.

The article opens with the story of a high school student who went through the program, a sophomore who explains that “Nowadays, having a publication is kind of a given.” Then comes a description of her project, a marketing strategy analysis for Chick-fil-A that was “published” on a journal’s online pre-print platform.

Schemes to get children into the “right” schools are nothing new, but I have to say that I prefer the old-fashioned grift like paying for a building or bribing the water polo coach to have your student be classified as a “recruit.” I find these programs much more insidious, by contrast. While those of us at most schools are hearing only bleak prognostications about the impending demographic cliff, the application numbers are soaring at the most elite schools, while only further fuels the aura of exclusivity. The result is programs like Scholar Launch that mask a pay-for-play system that rewards families with the means behind a cloak of merit all for a chance at getting access to the most elite schools.

In this arms race only the appearance of merit matters. Internships and educational opportunities are never going to be equally distributed, but I would never begrudge students legitimate opportunities for learning or engagement. After all, this is the premise behind taking a challenging course load and engaging in extra-curricular activities or internships. Publications, though, are fundamentally different from these other forms of engagement.

What is the point of a publication? Or, even more broadly, what is the point of writing?

To paraphrase Umberto Eco, the purpose of writing is to be read. Secondarily, to paraphrase John Warner, writing is a form of thinking. So you write to put your thoughts in order so that someone else may read them. Peer-reviewed academic articles are in their purest form a contribution to an ongoing discussion about a field of study. They are meant to be read, considered, and responded to by other people working or interested in the same field of study.

Whereas, the point of these publications is to be listed on the application, to give the appearance that your child has what it takes to “change the world” in some nebulous, ill-defined way because they were able to publish an article in high school. Who cares if anyone reads it? Or whether it is great? It was published in a “peer-reviewed” journal, which makes it sound substantial and scholarly, just like the journals that academics publish their work in. Never mind what “peer-reviewed” means for high school students or that these journals exist solely to showcase high school work.

I understand that both students and parents place an enormous importance on getting into the “right” school, by which they usually mean the most prestigious school that will unlock every door, but this attitude is deeply toxic. In addition to moving cycles of anxiety and burnout younger and younger, it internalizes two counter-productive ideas.

First, it underscores the false notion that students need to already have these skills and ideas. College, for instance, should be a time when students are developing many foundational skills. Setting an apparent expectation like this about what they should know creates an environment with heightened anxiety for students who think they’re behind frustration for students who discover a gap between what they think they know and how they perform in class.

Second is a sense that seeming is more important than being. If you seem smart or accomplished or charitable, then whether or not you are those things is almost besides the point. Obviously it is possible to wash out after gaining admission, but many of these systems are more set up to exclude people at the point of entry than to drive them out once they’re in. Thus being at the “right” school is more important than being at the school that will give you the best education. Besides, the former can be judge by the school’s prestige, while the latter can be hard to assess before you arrive on campus—or even until years after you graduate.

In truth, these paper mills are responding to a system that sacrifices at the altar of academic overproduction. A system that identifies publications as the highest virtue and often rewards short-term impact of ideas. And academic administration that would prefer to flatten the differences between fields so that the apples and oranges can be evenly judged as indistinct fruit pulp—how else can you compare scholars in one field where people write numerous co-authored publications based on lab work with those in a field where a productive scholar might put out a single sole-authored article a year and a book every few years? In this context it only to be expected that admissions officers might reward a student whose application shows that they already have a publication, irrespective of its topic or quality.

There is no easy panacea. These programs are a metastatic cancer that formed deep in how we assess academic success. But I also think that this makes it all the more important for people in a world increasingly defined by anxiety and burnout to collectively resist its acceleration. For academics to resist the cult of productivity, for students to focus on the process of learning over the appearance of achievement, for admissions officers to resist the allure of purchased credentials, and for numerous places in the wider culture to stop setting Harvard and other similarly exclusive schools as the benchmark for success in higher education.

Easier said than done.

Weekly Varia no. 22, 04/15/23

Spring arrived in force in Northeast Missouri this week. The world is starting to turn green, but the leaves around town have largely been preceded by an explosion of flowering things. I can’t complain about the views and the rising temperatures have drawn students out into the quad outside my office, making campus generally feel more alive than it does throughout the winter.

However, spring also comes at a cost. I have never been one to suffer from allergies in the past, but one of these flowering things causes my sinuses to go haywire each spring in Kirksville, which has made teaching classes a bit of an adventure this week. This phase only lasts a couple of weeks, fortunately, and the nice weather almost cancels out the temporary pain. Besides, I’ll be complaining about the heat again soon enough.

This week’s varia:

Album of the Week: Counting Crows, August and Everything After (2007)

Currently Reading: Julie Schumacher, The Shakespeare Requirement, Caroline Winterer, The Mirror of Antiquity

ChatGPT, Again

I had no intention of returning to the ChatGPT discourse. I said my piece back in December and the conversation seemed to take a histrionic turn around the start of the semester when the course planning period coincided with a wave of op eds that treated AI-writing as an existential crisis.

Moral panics turn tedious in a hurry and I bore easily.

Happily, I think that the discussion is starting to move into a more compelling and, I hope nuanced, direction. In his column at Inside Higher Ed last week, for instance, Matt Reed suggested that there are two issues at play with AI-generated writing. The one, academic dishonesty and how to catch it, receives the bulk of the attention. The other, disinformation and inaccuracies, has to this point received much less attention. In other words, the practical considerations about the norms, expectations, and enforcement of academic transactions are taking precedence over the underlying principles. This sort of priority of course makes sense, as anyone who has worked within the institutions of higher education can tell you, but I also think that it misses that these two issues are inextricably intertwined.

Simply put, I am convinced that ChatGPT specifically, and AI more generally, is a digital and information literacy issue.

Now, I should acknowledge that the stakes involved are more profound outside of the semi-controlled academic context, and at least potentially herald fundamental disruption to existing economic models. Google, for instance, is reportedly treating the chatbot like an existential threat to their hegemony over access to information online. Likewise, AI-generated art is just the latest technology that will allow companies to cut labor costs—why pay artists to create cover-art for a book when you can have an intern churn out AI-generated images until you find one you like? As much as I maintain that AI is a tool and the person producing the art is an artist, companies are not likely to compensate the artist as such under these scenarios. But while both of these are ethical issues related to my point about digital literacy, neither are they wholly new.

When it comes to writing, AI is a tool, and tools are only as good as their users. A spell-Czech [sic] doesn’t provide any value if one doesn’t have the vocabulary to recognize when it misleads, just as gratuitous use a thesaurus can lead the writer astray. Predictive text is fine for email, but I find it distracting in other contexts because the program prompts me down particular lines of composition. And, as I put in the last post on this topic, citation generators will lead you astray if you are unwilling or unable to format the text that it generates.

In this sense, the danger with AI is that people are going to treat a tool for something as a replacement for that thing. But this does not constitute either an existential crisis or a fundamental disruption, despite groups of people treating it as one or the other.

There are a myriad reason that a student might submit an AI-generated essay. Most of these overlap with the reasons a student might purchase an essay or otherwise cheat on assignments, and need to be addressed as such. However, AI should give educators greater pause because, compared to the other forms of dishonesty, AI might give the impression to some students that they don’t need to learn the skill in the first place. Chatbots can give the appearance of engaging with a historical figure, but they do not actually let you converse with that person any more than the Metaverse can allow you to watch Mark Antony debate in Rome in 32 BCE. But that superficial engagement risks drawing people away from the actual substance that would allow the participant to see how the AI turns unredeemed racists into apologists for their heinous beliefs or to recognize that seeing Antony debate in Rome in 32 BCE would be quite a feat because he was in Egypt gearing up for war with Octavian at that time.

On a whim, I decided to ask ChatGPT why students should avoid using the AI to write papers. This was what it produced:

I followed that prompt with a question about whether AI could help students with their writing:

I received a slightly more enthusiastic response when I directly inverted the original prompt, but still as a tool that can make writing easier or more efficient. At my most cantankerous, I dislike several of these uses—text summarization assumes one viable reading that simply isn’t true, which is also my problem with services like Blinkist, and I think that text generation will create pathways that guide how the person writes and thinks about a topic—but I could make similar arguments for writing being shaped by whatever we’re reading and simple reliance on the the first definition of a word found in a dictionary. As I said in my original post, if someone were to use AI as a tool and produce a quality paper either without any further intervention or by editing and polishing the text until it met the standards, that paper would meet my criteria for what I want my students to achieve in the class. This process would not be my preference, but the student would have guided the program through numerous rounds of revision much as they would draft and re-draft any paper that they wrote themselves. So much so that it would be easier to just write the paper, in fact. I doubt that a truly revolutionary thesis could be developed that way, but the student would have demonstrated their mastery of the course material and a sensitive understanding of the writing practices to know that it met standards on my rubric—grammar might be easier to accomplish, but the other categories not so much.

In fact, the arrival of AI makes it all the more important for students to learn skills like reading, writing, and, especially in my discipline, historical literacy. To do this, though, I think it is a mistake to issue blanket prohibitions or build assessment as though it does not exist. Rather, I want students to understand both why AI is not a great choice and what its limitations are, which requires steering into AI, at least a little bit.

This semester I am planning two types of activities, both of which are similar to the suggestions made in an opinion piece published today in Inside Higher Ed.

I scheduled a week for my first year seminar to address their first big writing assignment. The students have no reading this week, during which they will be working on their drafts of their first paper that are due on Friday. In the two class periods earlier in the week, I am going to have them complete an exercise using ChatGPT in their groups for the semester. On Monday, the students will work with ChatGPT to produce papers about the readings that we have covered to this point in the class, sharing with the me the results of the exercise. Then they will be charged with offering a critical evaluation of the generated text, which we will spend time on Wednesday sharing and discussing the critiques with the class, which will segue into a discussion of what makes writing “good.”

Students in my upper-division courses will do a similar exercise. As their first essays approach, I am going to provide students essays produced by ChatGPT using the same prompts and my essay rubric. Their task will be to “mark” the ChatGPT.

The goal is the same in both cases: to remind students that AI has severe limitations that cannot replace their unique thoughts. Further, I aim to engage the students as both writers and editors since I see the latter skill as an essential part of the writing process.

I don’t want suggest a prescriptive advice in this given that my class sizes and teaching mandates allow me to pursue some of these options. But the ChatGPT discourse has made even more convinced that it is necessary to teach basic, foundational, transferrable skills that will empower students to engage responsibly with the world in which they live.

Weekly Varia no. 11, 01/28/23

This was one of those weeks when it felt as though I got nothing done. Everything takes too much time, and then I am pulled in too many directions at once. This is the story of most semesters, if I’m being honest. So I didn’t manage to finish either my academic book for the week or any of the four draft posts in various stages of completion for this site, and I am trying to resist adding anything else to my plate. At this point I would like to focus on making more time for the things that I’m already doing. After all, as Oliver Burkemann argued in Six Thousand Weeks and the late Randy Pausch talks about in his time management lecture, our time is finite so we should pay more attention to how we spend it. Squeezing every last ounce of efficiency or sacrificing sleep (as I have done in the past) on the altar of rat race culture is both not sustainable and means enjoying life less in the meantime.

Admittedly, I am very bad at this. I have too many interests and a bad habit of saying yes to things before considering how much time they will take, but I now recognize this as an issue. I have more thoughts on these issues and their intersection with academic hobbies and living to work, but I’ll save them for a subsequent post. For now, just a range of links from the week.

This week’s varia:

Album of the week: Amanda Shires, My Piece of Land

Currently Reading: Brandon Sanderson, Tress of the Emerald Sea; Rabun Taylor, Roman Builders

Some additional information about my first book

Hello, again, bolded alter-ago.

I saw online that you received a physical copy of your book!

Right to the point, I see. A pre-print copy arrived with some other book deliveries yesterday. It was quite a surprise.

You promised me news! And here you are just Tweeting it out. What do you think your name is, Donald, or something?

Not a chance. What would you like me to start with?

I didn’t expect you to be so accommodating. How about the title?

I could give you the title, but what about if I show you the cover at the same time?

Fine.

Pretty. How do I get it?

The book is available for pre-order on the University of Michigan Press website. The book is scheduled for release in March and an electronic book will be available at or around the same time.

Since this is an academic book, I assume that this will cost me an arm, a leg, a kidney, and the deed to my firstborn child. Did I get that right?

Do children come with deeds?

You know what I mean.

I do. This is perhaps the most exciting piece of news. The book will be coming out with University of Michigan Press as a hard cover volume at their normal price point (about $75), but I was offered an option for my book to be included in a new open-access program. The book will still be found in the catalog and available for purchase, but, in effect, I agreed to forgo a paperback version of the book and instead make the e-book open-access.

So you volunteered to sell fewer books. Why?

A few reasons. First, there is very little chance that this book will sell enough to earn me meaningful royalties, with or without a paperback run. I tried to write my book to be approachable and hope that it sells well for an academic book, but I read the contract and am under no illusions that academic publishing will make me rich. Second, open access makes it possible for more people to read my work and that could, at least in theory, open more doors for me. The third reason is more philosophical. I have benefited enormously from scholars and organizations that make their work available for free. I am always looking for opportunities to pay that forward by publishing open access work where I can, even if I generally haven’t been successful with my articles. Given this opportunity, I took it.

Very noble of you.

It is also practical. I have reservations about the sustainability of open-access publishing over the long term and it is not going to resolve the issues of a crumbling higher-ed infrastructure for the humanities and social sciences, but I’m also intimately familiar with the many difficulties that come with publishing as a contingent faculty member. If making my work open access makes the life of any contingent scholar or graduate student a little easier, then it’ll have been worth it.

When I tentatively raised my concerns about sustainability, my editor told me to have that conversation about my next book. Her answer didn’t really assuage my concerns, but I guess I’ll need to write another book.

So, how’s the next book coming?

Patience. I have a few book projects in mind that I am starting to work on, but each of them is likely multiple years out at this point.

Slacker.

Call it what you will. Book writing takes time under the best circumstances and I am one of many professors who don’t receive research leave. I will likely write more books because I want to write more books—in fact, I already have outlines for three more history books and a novel. But what I write and how quickly will depend enormously on how the other parts of my career develop over the next few years.

I’m excited to be moving on to new work after spending the better part of a decade with this one, I’m also going to enjoy seeing this book out in the world.

First Day Fragments: Spring 2023

I usually do “first day fragments” to mark the start of the fall semester, but here on the first day of the spring term I find that I also have a few topics rattling around that are also worth exploring. Only time will tell whether this is a one-off or a new spring-semester routine.

Course design is an exercise in omission. And the more of a survey the course is designed to be, the more this truism cuts close to the truth. This has been on my mind over the last week while preparing for the upcoming semester. Even before the pandemic I had begun adopting a “less is more” mantra in the classroom, and doubling down on core questions and fundamental skills. But I also like big and open-ended questions, both to structure the course and to set as assignment prompts.

This semester I will be teaching upper-division survey courses on Ancient Rome (Romulus to Romulus Augustulus, in theory), Ancient Persia (Achaemenid to Sassanid), and then a first-year seminar on speculative fiction. Enormous topics, all.

Adding material to these courses is the easy part. It would be easy, for instance, to have the students read Beowulf and Le Morte d’Arthur, skip forward to Lord of the Rings, and then do something contemporary. Or just watch the movies. Or I could have decided that we’re going to do an entire course on the thousands of pages in Ken Liu’s Dandelion Dynasty novels. But neither of these options fit with my objectives for the course.

The challenge is finding the right balance. The entire extent of Tolkien that we are going to read will be “On Hobbits” and two short pieces of commentary about Rings of Power. We’ll read Ken Liu’s brilliant short story “Paper Menagerie,” but nothing from his longer works. Ditto for N.K. Jemisin’s “The Ones Who Stay and Fight,” which I’m using both as a counterpoint to Le Guin’s “The Ones Who Walk Away From Omelas” and and as a way to close the semester on a note of optimism after an emotionally challenging set of readings.

ΔΔΔ

All three of my courses this semester are new preps. This is a double-edged sword.

On the one hand, new preps make for a lot of work. They require compiling materials as you go through the semester, writing presentation slides, and deciding on how you want to present the material, even when approaching topics that you know well. Some of the activities are going to flop, or maybe the scope of the course needs to be changed. The course wobbles a little, because it has not yet settled into its foundations. A graduate school professor told me once that he believes a course only reaches its mature form in its third iteration.

On the other hand, I sometimes find that certain in-class activities and readings work best the first time I assign them. This is in part because I am forced to spend more time with the readings and preparing the activities, which means that everything is fresher, but I also find something magic in the thrill of invention. The second and third time through I can adjust to how the students experienced the assignment, but this comes at a cost when the assignment becomes somewhat calcified or the pathways that the course discussion become a little more worn in.

ΔΔΔ

People have been talking on Twitter about when professors have an obligation to post the syllabus. My only thought is that the syllabus will go up when it is ready and the course website is minimally ready for use, usually a day or two before the semester starts. I’m happy to answer questions even when the syllabus is in the design phase, but there are a myriad of reasons why it is good to take right up until the last minute making changes even if the basic structure has been set for weeks.

ΔΔΔ

Most of my courses are what my university calls “Writing Enhanced,” which means that they fulfill the standards of that program—emphasis on product, cognition, and process. Nearly twenty years ago when I was an undergraduate student, a writing-enhanced course required a certain number of pages, some of which had to be revised, but my guidance here is more flexible. I have another course design post (yes, I know that this is turning into a teaching-heavy blog) in mind for the near future that engages with the models we use when designing new courses, but, every semester, I have a momentary pang of concern that I’m not having my students write enough. For instance, I have never assigned a long 15–20+ page final paper. Instead, my students write multiple shorter papers (5–7 page) that they revise to a high standard, with the thinking that learning to polish a concise argument in a short paper is a prerequisite for writing a good longer paper when taking research classes. Besides, even without a long research paper to conclude the semester, my students write a lot. By my rough tally, I find that many of my students write nearly twice as much as I did for any class I took as an undergraduate student. Which then sends a flare of concern in the other direction: how much writing is too much?

ΔΔΔ

I wrote about Chat-GPT last semester and stand by everything I wrote there. But the new semester has brought out another round of hand-wringing and panic about how this tool means for higher education. This semester I’ll be leaning into AI writing in some classes with an “AI-essay critique” exercise and otherwise just incorporating it into the conversations we have when we talk about writing. But as the topic du jour, I’m bored by the conversation now. Moral panics turn tedious in a hurry.

My 2022 in Writing

I have decided to rethink my year-in-review series this year. Where I have traditionally provided separate posts for anything published and anything published here, I am combining those two posts into one as a way to better address my writing as a coherent whole. This post thus includes a status update on projects, a list of things published, the best* posts of the year, and some raw stats from the blog.

Status update

2022 was a year of booms and busts for my writing. I started tracking how much time I spent on academic writing back in 2017 and this year marked the second lowest total of the past five years (2019 was a deep nadir for reasons of employment). But my writing this year also swung between periods of exceptional stamina, like a three week period in February where I averaged almost twenty hours each week, punctuated by periods when I didn’t write anything. Even the success of the writing group I started with Vicky Austen couldn’t keep me on track as my semester spun wildly out of control.

The state of my writing projects also contributed to the stop-and-start nature of my writing since the bursts often coincided with imminent deadlines. For instance, every few months this year I had a new deadline while moving my first book through the phases of production. That book is due out in March 2023. The same thing happened on a smaller scale with respect to an article accepted for Classical Quarterly that I am optimistic will appear next year and a book review, and I have also been wrapping up some smaller projects. By contrast, I had to do very little work on the only piece I had come out this year because it had been caught up in the production pipeline.

Finishing, or nearly finishing, these projects, many of which I once thought would be my final academic publications, has also left me thinking through my research pipeline. I have ideas in the works and at least one commitment for 2023, but one of my tasks over the next few months will be to put this in order and figure out where I want to spend my energy.

Perhaps not coincidentally, then, I also did less public writing and fewer presentations in 2022. I still worked on the SCS Blog’s contingent faculty series, but I was not the lead editor for either of the features that we produced this year. (I particularly recommend Kristina Chew’s two part essay.) I also delivered just one conference paper, connecting the mass of people not from Athens on the Sicilian Expedition to the revolt the following year. My favorite piece of writing of the year was a talk about bread baking for a student group on campus that offered “a family and social history of bread.”

It was a similar story on this blog. I wrote somewhat less frequently, but I produced more words than I ever have before because the average post length ballooned enormously.

Publications

“Remembering injustice as the perpetrator?: Orators, Cultural Memory, and the Athenian Conquest of Samos,” in The Orators and their Treatment of the Recent Past, ed. A. Kapellos (Berlin: De Gruyter, 2022), 447–63.

Previous years: 2021; 2020; 2018

I have a complete list of my publications, with links to everything available online, here. If you are interested in reading any of my work and do not have access to it, email me for a pdf or off-print.

Best* Posts

Previous years: 2021; 2020; 2019; 2018; 2017; 2016

Blog stats to date, with a few days left to go

  • Posts: 65
  • Words: 69,482
  • Av. length: 1,069
  • Visitors: 7,941
  • Views: 10,916

What is “the college essay,” or ChatGPT in my classroom

Confession: I don’t know what is meant by “the college essay.”

This phrase has been the shorthand for a type of student writing deployed over the past few weeks in a discussion about the relationship between college classes and AI programs like ChatGPT-3 that launched in November, which I touched on in a Weekly Varia a few weeks ago. These programs produce a block of unique text that imitates the type of writing requested in response to a prompt. In its outline, input/output mimics what students do in response to prompts from their professors.

The launch of ChatGPT has led to an outpouring of commentary. Stephen Marche declared in The Atlantic that the college essay is dead and that humanists who fail to adjust to this technology will be committing soft suicide, which followed on from a post earlier this year by Mike Sharples declaring that this algorithm had produced a “graduate level” essay. I have also seen anecdotal accounts of professors who have caught students using ChatGPT to produce papers and concern about being able to process this as an honor code violation both because the technology is not addressed explicitly in the school’s regulation and because they lacked concrete evidence that it was used. (OpenAI is aware of these concerns, and one of their projects is to watermark generated text.) Some professors have suggested that this tool will give them no choice but to return to in-class, written tests that are rife with inequities.

But among these rounds of worry, I found myself returning to my initial confusion about the nature of “the college essay.” My confusion, I have decided, is that the phrase is an amorphous, if not totally empty, signifier that generally refers to whatever type of writing that a professor thinks his or her students should be able to produce. If Mike Sharples’ hyperbolic determination that the sample produced in his article is a “graduate level” essay is any guide, these standards can vary quite wildly.

For what it is worth, ChatGPT is pretty sure that the phrase refers to an admissions personal statement.

When I finished my PhD back in 2017, I decided that I would never assign an in-class test unless there was absolutely no other recourse (i.e. if someone above me demanded that I do so). Years of grading timed blue-book exams had convinced me that these exams were a mismatch for what history courses were claiming to teach, while a combination of weekly quizzes that the students could retake as many times as they want (if I’m asking the question, I think it is worth knowing) and take-home exams would align better with what I was looking to assess. This also matched with pedagogical commitment to writing across the curriculum. The quizzes provided accountability for the readings and attention to the course lectures, as well as one or more short answer questions that tasked the students with, basically, writing a thesis, while the exams had the students write two essays, one from each of two sets of questions that they were then allowed to revise. Together, these two types of assignments allowed the students to demonstrate both their mastery over the basic facts and details of the course material and the higher-order skills of synthesizing material into an argument.

My systems have changed in several significant ways since then, but the purpose of my assignments has not.

First, I have been moving away from quizzes. This change has been a concession to technology as much as anything. Since starting this system on Canvas, I moved to a job that uses Blackboard and I have not been able to find an easy system for grading short answer questions. I still find these quizzes a valuable component of my general education courses where they can consist entirely of true/false, multiple choice, fill in the blank, and other types of questions that are automatically graded. In upper-level courses where I found the short-answer questions to be the most valuable part of the assignment, by contrast, I am simply phasing them out.

Second, whether as a supplement to or in lieu of the quizzes, I have started assigning a weekly course journal. In this assignment, the students are tasked with choosing from a standard set of prompts (e.g. “what was the most interesting thing you learned this week,” “what was something that you didn’t understand this week form the course material? Work through the issue and see if you can understand it,” “what was something that you learned this week that changes something you previously wrote for this course?”) and then writing roughly a paragraph. I started assigning these journals in spring 2022 and they quickly became my favorite things to grade because they are a low-stakes writing assignment that give me a clear insight into what the students have learned from my class. Where the students are confused, I can also offer gentle guidance.

Third, I have stopped doing take-home exams. I realized at some point that, while take home exams were better than in-class exams, my students were still producing exam-ish essay answers and I was contributing to this problem in two ways. First, two essays was quite a lot of writing to complete well in the one week that I allotted for the exam. Second, by calling it an exam most students were treating it as only a marginal step away from the in class exam where one is assessed on whether they have the recall and in-the-moment agility to produce reasonable essays in a short period of time.

What if, I thought, I simply removed the exam title and spread the essays out over multiple paper assignments?

The papers I now assign actually use some of the same prompts that I used to assign on exams, which were big questions in the field the sort that you might see on a comprehensive exam, but I now focus on giving the students tools to analyze the readings and organize their thoughts into good essays. Writing, in other words, has become an explicit part of the assignment, and every paper is accompanied by a meta-cognitive reflection about the process.

Given this context, I was more sanguine about ChatGPT than most of the commentary I had seen, but, naturally, I was curious. After all, Sharples had declared that a piece of writing it produced was graduate level and Stephen Marche had assessed it lower, but still assigned it a B+. I would have marked the essay in question lower based on the writing (maybe a generous B-), and failed it for having invented a citation (especially for a graduate class!), but I would be on firmer footing for history papers of the sort that I grade, so I decided to run an experiment.

The first prompt I assigned is one that will, very likely, appear in some form or another in one of my classes next semester: “assess the causes underlying the collapse of the Roman Republic and identify the most important factor.” I am quite confident in assigning the AI a failing grade.

There were multiple issues with ChatGPT’s submission, but I did not expect the most obvious fault with the essay. The following text appeared near the end of the essay.

Vercingetorix’ victory was, I’m sure, quite a surprise for both him and Julius Caesar. If I had to guess, the AI conflated the fall of the Roman Republic with the fall of the Roman Empire, thus taking the talking points for the Empire and applying them to the names from the time of the Republic. After all, ChatGPT produces text by assembling words without understanding the meaning behind them. Then again, this conflation also appears in any number of think-pieces about the United States as Rome, too.

But beyond this particular howler, the produced text has several critical issues.

For one, “Internal conflict, economic troubles, and military defeats” are exceptionally broad categories each of which could make for a direction to take the paper, but together they become so generic as to obscure any attempt at a thesis. “It was complex” is a general truism about the past, not a satisfactory argument.

For another, the essay lacks adequate citations. In the first attempt, the AI produced only two “citations,” both listed at the end of the paper. As I tell my students, listing sources at the end isn’t the same thing as citing where you are getting the information. Upon some revision, the AI did manage to provide some in-text citations, but not nearly enough and not from anything I would have assigned for the class.

A second test, using a prompt I did assign based on Rudyard Kipling’s The White Man’s Burden, produced similarly egregious results. The essay had an uninspired, but a mostly adequate thesis, at least as a starting point, but then proceeded to use three secondary sources, none of which existed in the format that they were cited. Unless the substantial C.V. of the well-published scholar Sarah C. Chambers is missing a publication on a topic outside her central areas of research, she hasn’t argued what the paper claims she did.

A third test, about Hellenistic Judea, cited an irrelevant section of 1 Maccabees and a chapter in the Cambridge History of Judaism, albeit about Qumram and neither from the right volume nor with the right information for the citation. You get the idea.

None of these papers would have received a passing grade from me based on citations alone even before I switched to a specifications grading model. And that is before considering that the AI does even worse with metacognition, for obvious reasons.

In fact, if a student were to provide a quality essay produced by ChatGPT that was accurate, had a good thesis, and was properly cited, and then explained the process by which they produced the essay in their metacognitive component, I would give that student an A in a normal scheme or the highest marks in my specs system. Not only would such a task be quite hard given the current state of AI, but it would also require the student to know my course material well enough to identify any potential inaccuracies and have the attention to detail to make sure that the citations were correct, to say nothing of demonstrating the engagement through their reflection. I don’t mind students using tools except when those tools become crutches that get in the way of learning.

In a similar vein, I have no problem with students using citation generators except that most don’t realize that you shouldn’t put blind faith in the generator. You have to know both the citation style and the type of source you are citing well enough to edit whatever it gives you, which itself demonstrates your knowledge.

More inventive teachers than I have been suggesting creative approaches to integrating ChatGPT into the classroom as a producer of counterpoints or by giving students opportunities to critique its output, not unlike the exercise I did above. I have also seen the suggestion that it could be valuable for synthesizing complex ideas into digestible format, though this use I think loses something by treating a complex text as though it has only one possible meaning. It also produces a reasonable facsimile of discussion questions, though it struggles to answer them in a meaningful way.

I might dabble with some of these ideas, but I also find myself inclined to take my classes back to the basics. Not a return to timed, in-class tests, but doubling down on simple, basic ideas like opening student ideas to big, open-ended questions, carefully reading sources (especially primary sources) and talking about what they have to say, and how to articulate an interpretation of the past based on those sources–all the while being up front with the students about the purpose behind these assignments.

My lack of concern about ChatGPT at this point might reflect how far from the norm my assessment has strayed. I suspect that when people refer to “the college essay,” they’re thinking of the one-off, minimally-sourced essay that rewards superficial proficiency of the sort that I grew frustrated with. The type of assignment that favors expedience over process. In this sense, I find myself aligned with commentators who suggest that this disruption should be treated as an opportunity rather than an existential threat. To echo the title from a recent post at John Warner’s SubStack, “ChatGPT can’t kill anything worth preserving.”

#AcWriMo 2022 Recap

Back at the start of November, I set for myself writing targets for #AcWriMo. In the spirit of accountability, this post reviews those targets.

1. Finish and submit my three outstanding short pieces. They just need to be off my plate so that I can focus on something else.

I completed two of the three, but I chose to hold off on sending them off until I have finished the third. I’m optimistic that this can happen by the end of the year.

2. Spend at least one hour each week writing on one of my new academic projects. For this goal I’m going to set an absurd (for me) target of 500 words an hour, for a minimum of 2,000 fresh words on top of whatever else I write this month.

I spent an hour writing on new projects in two of the four weeks. Four would have been better, of course, but these projects are in a state such that any progress is good progress.

3. Write one book review blog post per week.

I published two review posts in November, one on The Final Strife, one on The Medieval Crossbow.

4. Write one other blog post per week. Writing begets writing, as they say.

Aided in no small part by the decision to start publishing a weekly roundup of news and stories that I read in a given week, I achieved this target. I published four posts, a What is Making Me Happy on my new tea infuser, a post about Twitter, and two weekly varia posts.

5. Continue writing in my journal every night. In particular: November has 30 days: write 28 entries.

I wrote 26 entries in 30 days. One of those might have been in a morning, rather than an evening.

6. Write a recap blog post for December 1 that reviews the targets and reflects on my month in writing.

I missed this by a couple of days but met it in spirit.

If you’re a stickler for such things, I successfully completed one of my six writing goals in November. Spiritually, though, this was a wildly successful #AcWriMo. I set my targets with the understanding that this is the busiest time of my year and that good writing habits are the secret to a good writing routine. (The secrets to good writing, on the other hand are a little more arcane and involve reading, attention to the poetry of language, and learning to edit, but you can’t write well until you write.) While I didn’t hit these targets, I made demonstrable progress on every one of them and, in so doing, primed the pump for more writing just as soon as I finish the final grading push of the semester.

#AcWriMo2022

It is November first, which means that it is once against AcWriMo, an academic writing challenge inspired by National Novel Writing Month.

I read through my blog archive in preparation for this post, as I often do when I sit down to write this sort of annual post. After all, I get frustrated with myself when it seems that I am writing the same things over and over. This tag first appeared in 2012, just one year after PhD2Published launched the challenge. I was a second year PhD student at the time, just starting to send ill-fated article manuscripts off for review and preparing for my comprehensive exams with not even the slightest inkling what my dissertation project would end up being.

(How I came to that project is a curious story that points to my atypical journey through graduate school.)

The tag then fell dormant for six years only to begin an annual appearance in 2018, a year and a half after I received my PhD and at a time when I was working on my book proposal. I wrote four posts that year, following a series of prompts created by Margy Thomas of Scholarshape that were designed to inspire metacognitive reflection on the writing process.

2019 saw just one post that was quite gloomy and frustrated because I felt that I was nearing the end of the road in academia. 2020, year one of the pandemic, was more of the same, except now with an attempted return to the goal-setting mandate. I did not hit my goals. By November 2021 I had started my current job and I was starting to acclimate to my schedule and established a single goal of a month-long metacognitive exercise about my writing…that I also did not hit.

So where does that leave me for 2022?

2022 has been a good year for my writing overall, if also more boom-and-bust than is ideal. I started the year with an article that had been rejected a couple of times getting accepted at Classical Quarterly and submitting the final manuscript for my first book at University of Michigan Press. That book has now also gone through copy edits and proof. Between these stages I also turned in five of the eight small pieces that I had outstanding between the pandemic and conditions of my employment, as well as a delivering a conference paper and a book review. The progress has mostly been confined to projects years in the making, though, and I’m having more trouble creating the space for new writing projects.

I have also recently returned to writing in a journal more or less nightly, both as a quiet, cathartic way to wind down before bed and as an extension of my writing discipline. Once upon a time I wrote in that space most days, often as a way of settling my mind before jumping into work on my dissertation. I fell out of that habit in the past few years, but I find that I maintain better equilibrium when I giving myself the space to write in my journal.

The other way that 2022 has been good for my writing is that I started a virtual writing group with Vicky Austen. I have participated in these in the past run by people in the UK, but I’m not in a place right now where I can reasonably wake up at 3am to write, so I suggested that we start one for those of us in this hemisphere. The practice of setting aside two hours twice a week to work in a communal, supportive environment has been enormously helpful as I am trying to re-establish a regular writing habit rather than one that means working feverishly to hit deadlines and then slumps because I’m forced to set aside that work in order to catch up on everything else that I fell behind on because I was writing.

This year I am setting for myself six targets for AcWriMo:

  1. Finish and submit my three outstanding short pieces. They just need to be off my plate so that I can focus on something else.
  2. Spend at least one hour each week writing on one of my new academic projects. For this goal I’m going to set an absurd (for me) target of 500 words an hour, for a minimum of 2,000 fresh words on top of whatever else I write this month.
  3. Write one book review blog post per week. These posts have been a casualty of the general chaos of my life recently, but I want to get back in the habit of writing them for some, if not all, of the books I read. First up is Stuart Ellis-Gorman’s The Medieval Crossbow which I promised to review after I won it in his online giveaway.
  4. Write one other blog post per week. Writing begets writing, as they say.
  5. Continue journaling every night. In particular: November has 30 days: write 28 entries.
  6. Write a recap blog post for December 1 that reviews the targets and reflects on my month in writing.

I see two potential complications with this set of targets.

First, one might reasonably ask whether these targets are suitably academic—which one might ask about so much of what I end up doing. The first two goals clearly fit the bill, while the back three are more about using this month to re-establish good writing habits. Basically, when I write more in general I end up writing more on my academic projects.

Second, I am curious whether this is yet another instance of unreachable targets that will be counter-productive when it comes to building the sustainable habits that I claim to want. This is of particular note because four of these five goals are set on top of whatever other writing I do. I guess there is only one way to find out.