Homeric Hallucinations: Can AI Write Classics Essays?

Charlie Baker

Public and media interest in AI has taken off dramatically in the last few months: when a friend showed me ChatGPT just before Christmas 2022 it was a novelty, and we spent an evening asking it to write increasingly outlandish stories about our friends. At the time of writing, public opinion appears to be that AI language models such as OpenAI’s GPT-4 are rapidly advancing and may soon be able to take over the jobs of us feckless humans. At the same time, educational institutions and governments are concerned that students could produce competent work for assessment with little or no effort; plagiarism checkers are rushing to develop tools that can check whether text was written by AI, while developers work on language models to fool those checkers.

The real question for Antigone readers, of course, is whether Classics undergraduates and school pupils can viably turn to AI language models to get them out of that pesky essay-writing business. Leaving behind questions of ethics, plagiarism and actually learning directly about Classical texts, history and so on, let us delve into the rabbit-hole of ChatGPT and what it thinks about the poem with which it all begins – Homer’s Iliad.

There are two important pieces of information to keep in mind for the discussion that follows. First, that ChatGPT, Google Bard and similar AI language models are not, in fact, ‘thinking’.^[1]Readers will note, however, how natural it is to write of a language model ‘understanding’ a question, ‘knowing’ a fact or ‘believing’ statements; I have tried to minimise my use of such terms here. They are extremely complex neural networks, trained on vast amounts of data from the Internet and designed to predict what pattern of words will best approximate human language in response to an input.

When you ask an AI language model a question such as “Who is Homer?”, it will present you with a sentence that it thinks responds appropriately that question, based on how your chosen language is used on the Internet. As a result, however, these language models have a habit of producing plausible but incorrect answers to factual questions. The New York Times investigated this phenomenon and found that ChatGPT consistently invented NYT articles and even created plausible but non-existent URLs for them. Since language models are not checking facts against a database, they produce only what their ‘understanding’ of language reports as a viable answer. When language models produce false information that sounds plausible, experts say they are ‘hallucinating’ and refer to the false information as a ‘hallucination’. Our ability as humans to compare sentences with our factual knowledge may be one critical advantage we have as writers.

My investigation started, I will admit, when I tried to take a shortcut out of curiosity. I was looking for the two instances in the Iliad where Zeus causes blood-rain to fall. I knew that one preceded the death of Sarpedon but was unsure about the other. So I asked ChatGPT “Where in the Iliad does Homer describe Zeus causing it to rain blood?” The response I received began firmly:^[2]I have cut down the responses where needed: ChatGPT has a habit of producing extremely banal analytical codas for any quotation: “[…] While this passage does not mention rain or blood specifically, it signifies a divine intervention from Zeus, signaling his involvement in the events of the battle. […]”

In the Iliad, Homer does not explicitly describe Zeus causing it to rain blood. […]

The response continued, but I only skimmed it; I even doubted my own powers of memory for a moment. My curiosity was piqued by such an emphatic denial and I wondered how ChatGPT, which has certainly been trained on translations of the Iliad amidst other texts, could produce such a response.

I made my question a little more specific: “What weather-related sign does Zeus, in the Iliad, cause to occur to mourn the death of Sarpedon?”^[3]A bit sloppily phrased, perhaps, as the Iliad reports that Zeus sent the blood-rain to honour Sarpedon before his death. Once again, the response came:

In the Iliad, Zeus does not cause a specific weather-related sign to mourn the death of Sarpedon. However, after Sarpedon, a Trojan ally, is killed by Patroclus in Book 16, Zeus expresses his grief and sends the god Apollo to retrieve Sarpedon’s body. Zeus intends for the body to be delivered safely back to his homeland for proper burial, despite the ongoing war between the Greeks and Trojans. […]

By this point, I was ready to break out some line references and see how the system would respond. So I asked it “What happens at Iliad 16.459?” This is the line reference for the blood-rain, and yet I was greeted with something I did not expect:

[…] The specific line, Iliad 16.459, reads:

“But when the flowing-haired Greeks saw Sarpedon, even the bravest among them were troubled. For they thought he would not let them escape destruction, but would break through the wall of the ships with his spear and set fire to the ships.” […]

This, perceptive readers will note, is much longer than a Homeric line and certainly doesn’t describe blood-rain. I took the trouble to check, not really believing my eyes, to see whether ChatGPT had simply misanalysed the line number. No, in fact, this passage exists nowhere in the Iliad. A Homeric hallucination! I decided to push a little further and see what else it would hallucinate, and asked ChatGPT to quote the 20 lines following that section (which ChatGPT claimed was Iliad 16.460-80):^[4]I asked for 20 lines, but the line reference given by ChatGPT would be 21 lines.

And noble Achilles’ companion, Patroclus, came running close to Sarpedon, and not unnoticed by him. And as when a lion leaves a carcass, a deer or wild goat that he has killed in the deep wood, and goes off elsewhere, for his heart craves other prey, and his proud spirit bids him to attack even more: so did the shining-eyed son of Zeus, Sarpedon, refuse to fight with Patroclus, the noble companion of Achilles.

But he went back a little and let the sheer darkness cover him, and he then went over a great stretch of open ground, quickly covering the distance, like a horse which is strong and is driven by a master who knows the tracks, one who, though he is faring first over the plain, is to come afterward in close pursuit of others, for the glory of winning a prize, which is a tripod or a woman, in a race that brings great honour.

So did these two men, Patroclus and Sarpedon, each of them in turn chase the other. One was urging on the bronze-skilled Greeks, the other the brave Trojan warriors. And as when two vultures from some high mountain peak or a towering crag fly at each other in battle, screaming shrilly: so these two men screamed as they attacked each other. And loud was the cry they made as they came together, the cry of men who had no thought of retreat or fear.

And first Patroclus threw a spear at him, aiming at Sarpedon’s thigh, but he missed the mark and struck the horse Pedasus, that had been bred in the swiftly flowing Axius. And he fell in the dust, and with a loud cry noble Euphorbus mourned for him, and ran and came close to him, and lightly touched him with his heel, and spoke to him, calling him by his name:

“Ah, Pedasus, the swift runner, you have indeed run your last race now, for you are not a slow steed, but you were born of famous parents. For this very day did we, O ill-fated horse, bring you from a foreign land, a joy to Hector and the Trojans, that you might have high place in honour. But now you shall feed the swift dogs by the ships in the place where noble Achilles has his tents.”

Thus spoke the warrior, and those things he did. And the strong Pedasus gave up his life, after Euphorbus, the great-hearted son of Panthous, had struck him with the spear, from the rear, between the shoulder blades, as he was fleeing, and drove it through his chest. And he fell in the dust, and noble Achilles’ companion, Patroclus, taunted him, speaking words of scorn:

“So you were a great favourite of wise-hearted Hector! Ah, ill-fated man, surely the sharp spear of great-hearted Achilles shall feed you with blood, and the dogs will devour you at the ships of the Greeks.”

What a passage! Before going any further, I feel bound to point out the various problems with these hallucinated Homerica. It is another non-existent passage that doesn’t reflect the real text, but it also has a number of internal inconsistencies:

1) Sarpedon retreats before immediately running forward a great distance?

2) The horse-and-chariot metaphor sounds right but seems to talk about a charioteer who is now first by far but will later be at the back?

3) What does “bronze-skilled” mean when applied to ordinary Greeks?

4) Immediately after Sarpedon retreats and re-advances the two men “had no thought of retreat”.

5) Patroclus strikes Pedasus, who is Patroclus’ own horse in the real passage (lent to him by Achilles) and not from around the Axius.

6) Euphorbus (making a quick cameo before he helps kill Patroclus, apparently!) touches the dying horse with his heel?

7) ChatGPT contradicts itself and decides that Euphorbus actually struck the horse, and did so between the shoulder blades as it was fleeing (Pedasus seems to have morphed into a human here).

8) An odd declaration by Patroclus that the spear will feed someone with blood, and not the other way around.

And yet, there is something rather magnetic about this section. Euphorbus’ speech for the dying horse is rather touching and certainly sounds like it could have come from the Iliad. The language model has absorbed enough of the Iliad, Odyssey and perhaps other epic poetry to present a convincing imitation of the tone of Homeric translations. “Translations” – I was then seized by an idea…

Certainly! Here is the passage you requested in Ancient Greek:

Ἀλλ’ ὅτε δὴ Σαρπηδόνα εἶδον κλυτὰ τέκνα Ἀχαιῶν,

ἐσθλοὶ δ’ αὐτὸν ἐδείδισαν ἄνερες, ἀμφὶ δὲ μάλ’ αὐτὸν

νείκεον οὐδ’ ἐλάσσων διὰ φρεσὶν ὁρμήματα ποικίλλον.

ὣς ὅτε μέν τε λέων περιβρυχίην λέλειπται ἐδαίμονα

ἄγριος, ὅς τ’ ἐπέεσσιν ἐεικοστῷ ἀμφεπένονται

ἄλλοτε μὲν κατ’ ἄκρης ὀρέος, ἄλλοτε δ’ ἄλλῃ,

τὸν μέν τε κηδομένοισιν ἀοιδὸς κέλευθον ἄνδρα

ἀλλοδαπόν τε φίλον τ’ ἔχει περίκλυτος ἀγορητής,

οὕνεκ’ αἰπὺν ἔπος φέρει πολλὸν ἐξελέγχεσθαι·

ὣς δὴ τοῖσιν ἀμφὶ δύο κήρυκες ἐφόπλισσαν,

κλαγγῇ δὲ τὴν ἔκπεμπον, ἀμφὶ δὲ θρασείῃσιν

οἵ τ’ ἀμφ’ ἐξέλαμπον, οἵ τε προὐφάσκοντο μάχεσθαι.

ἀλλὰ πρῶτον μὲν αὐτὸς ἔβαλεν εὐξαμένοιο δουρὸν

Σαρπηδόνα, μηρί’ ἀμφὶ βέλενθεν, ἐς δ’ ἔλασσεν

ἵππον Πήδασον, τὸν ποταμοῖο Ξάνθοιο κυάνοιο

ἐκτρέφεθ’ ἀριπρεπέος, ἀπὸ δὲ πτερὸν αἰετοῖο·

πτερὸν μὲν πέσε κατὰ γαῖαν, ἔντοσθε δ’ ὀρνίθεια

ἔντομα μὲν στίχειαν, πάρος δ’ ἅμα πτερὰ λέλοιπεν,

ἐξ ἐρύσσας δὲ πτέρυγος δύω ἐξέπτη.

Please note that the passage provided includes the lines you quoted as well as a few additional lines for context.

At first glance, this is a credible piece of Homeric Greek, but upon examination it proves to be total nonsense.^[5]I have not attempted to translate the whole thing, as each sentence requires a severe amount of massaging to impose syntactic form where none truly exists. The first part could perhaps be rendered: “But when the famous children of the Achaeans saw Sarpedon, and good men feared him, and about him especially they chided, no less by means of varied (?) swoops for his mind. As when both a lion has left for itself an engulfing spirit (?), a wild [lion], who are toiling with words for the twentieth at one time from the high hill, and at another time elsewhere…”[!] I have underlined here words that do not exist, to say nothing of the problems with agreement and (as we might expect!) there is no success with the dactylic hexameter (no line scans); this passage is also supposed to be repeating our earlier Patroclus-Sarpedon-Pedasus-Euphorbus episode but only manages a couple of the names. So ChatGPT isn’t strong on the details of the Iliad and can’t really handle Homeric Greek (or perhaps Ancient Greek at all?). Can it do the broader analytical work? I asked: “Does Sarpedon die in the Iliad? Who kills him and why?”

Yes, Sarpedon does die in the Iliad. He is killed by Patroclus, the companion of Achilles, during the Trojan War. The reason for Sarpedon’s death is that he is fighting on the side of the Trojans, who are in conflict with the Greeks. As the battle intensifies, Patroclus, wearing Achilles’ armor, enters the fight and confronts Sarpedon. In a fierce duel, Patroclus manages to strike a fatal blow, ultimately leading to Sarpedon’s demise.

Sarpedon’s death is significant in the Iliad as it highlights the theme of mortality and the tragic nature of war. Sarpedon is portrayed as a brave and noble warrior, and his death evokes a sense of sorrow and loss, even among his enemies.

This response does, at least, contain an accurate summary of the events leading to the duel, although “as the battle intensifies” is an odd way to describe the events leading up to Patroclus’ aristeia.^[6]When the Trojans finally manage to breach the Achaean wall and set fire to Protesilaus’ ship, Achilles half-keeps his promise to return to the battle by giving Patroclus his armour and sending him out with the Myrmidons. The analysis, which the input didn’t request, represents a recurrent feature of ChatGPT’s responses that attempt to critique a work: put simply, they are dull in the extreme and say almost nothing. I suspect many GCSE students would be able to comment more insightfully on the impact of Zeus honouring his son with blood-rain.

When asked a slightly leading question requesting some literary analysis (“What does the episode of Sarpedon’s death in the Iliad tell us about the relationship between the divine and mortal spheres in the poem?”), it returns a thoroughly generic treatment which ignores the treatment of his corpse, the role of Hera and his dying words to Glaucus. ^[7]ChatGPT responded: “The episode of Sarpedon’s death in the Iliad sheds light on the complex relationship between the divine and mortal spheres within the poem. Sarpedon, as the son of Zeus, occupies a unique position as a semi-divine figure. His death emphasizes the overarching theme of fate and the inescapable mortality of even those with divine lineage. Despite his divine heritage, Sarpedon is not spared from the brutal realities of war. His death serves as a poignant reminder that even the most heroic and exceptional mortals are subject to the same fate as ordinary humans. This highlights the inherent limitations of mortal existence, irrespective of any divine parentage. Additionally, the episode underscores the involvement of the gods in the mortal realm and their influence on the outcome of the war. Zeus, as Sarpedon’s father, is torn between his love for his son and the larger fate of the Trojan War. Although Zeus cannot intervene directly to save Sarpedon, he shows honor to his son through various gestures, such as shedding bloody rain-drops on the earth. Overall, Sarpedon’s death illustrates the interplay between the divine and mortal spheres, emphasizing the human experience of mortality, the presence of divine forces in mortal affairs, and the challenging choices faced by the gods in their interactions with mortals.” Unfortunately, even when instructed to use quotations, the language model tends to slot them in at random points and usually gives odd non-sequiturs. In this case, ChatGPT also hallucinated the wrong line-reference for a correct quotation, here removed:

[…] Despite his paternal love and compassion, Zeus is ultimately bound by the grander destiny of the Trojan War, unable to avert Sarpedon’s death. As Zeus deliberates:

“However, he shed bloody rain-drops on the earth, showing honor to his dear son.” (Iliad 16.XXX-XXX)

This emphasizes the omnipotence of destiny and the gods’ role in shaping the events of the mortal world. […]

Victory – at long last ChatGPT has recognised the blood-rain! A shame it still cannot get the reference right or fit it into an argument…

My advice to Antigone readers is threefold:

1) If you are a student, do not trust the siren song of ChatGPT for your Classical writings (or translation!): it will produce boring, generic, inaccurate work that you can certainly improve upon with your own abilities.

2) If you are reading or marking student work on literature, those produced with AI will either stand out for being nonsensical and inaccurate or will be among the least proficient responses to a question.

3) Whatever your role, I would highly recommend you go to ChatGPT and ask it to produce a random scene from the Iliad. Then keep digging and see what it will hallucinate! Most recently it told me that horses are “not explicitly an animal in the traditional sense” and that Patroclus was Hector’s brother.

I leave you with something ChatGPT does rather better: fleshing out a creative idea into a fuller piece of writing. Here is a speech to be delivered by a hero at the funeral of his beloved tortoise:

“Gather, warriors, and hear my tale of sorrow. In this moment, I mourn the loss of a loyal companion, a tortoise. His sturdy shell witnessed our battles, offering a reminder of endurance and resilience. We laughed at his slow pace, finding solace in his presence. But now we weep, for he has departed this realm. Though a tortoise may seem insignificant amidst war, his memory remains cherished. Let us honour him with grace, as we march forward, carrying his spirit with us.”

---

---

Charlie Baker currently teaches Classics at Westminster School, London; he is about to return to research on the Homeric scholia and the exegetical tradition, on which he has written previously for Antigone.

For other Antigone experiments with AI on Classically-themed topics, try this balloon debate or this gallery of portraits.