I Stumped ChatGPT . . . Or So I Thought
Human and AI compete to write a sonnet
Just like almost everyone else, my initial encounter with ChatGPT left me in amazement.
I started off by asking the chatbot all the usual questions: Is God a Sadist? Can you tell me a werewolf story? Could you help me understand the Chinese Room experiment?
Without hesitation, robust and nuanced answers came streaming out of OpenAI’s much-vaunted chatbot, and I was staggered by the sight. ChatGPT marks a truly extraordinary invention that will revolutionize education and content creation going forward.
But I fancy myself a bit of an iconoclast. I can’t let the hot new thing that everyone’s talking about get off with such an easy pass. When my dizziness dissipated, I decided I was going to up the difficulty and really challenge the limits of ChatGPT.
Sure, ChatGPT can spit out encyclopedic answers and generate narratives in prose form on all topics from astrophysics to popular culture, but can the AI handle the complex emotion and metered expression of poetry?
Thus, I made the following request of ChatGPT: Write a Petrarchan sonnet about James Caan.
I hit enter and waited. And waited. Instead of the usual stream of text, I got nothing. Then an error message appeared in red letters. (You’ll have to take my word for it; I’m currently kicking myself for not taking a screenshot of the output.)
Complex poetic forms, it appeared, had left ChatGPT flummoxed. I closed my browser feeling like a badass for stumping the hot new tech, thoughts of chalk one up for humanity running through my head. On the other hand, I felt a bit crestfallen. I was kind of hoping to see a Petrarchan sonnet about James Caan composed by AI. Apparently, ChatGPT was not all-powerful. That made the AI feel almost . . . human.
Within minutes, I started to sympathize with ChatGPT. I began to see things from its perspective, and then I questioned my own pro-human pride. Sure, I could fault ChatGPT for what it couldn’t do, but that didn’t say anything about my own personal capabilities. If I was going to fault the technology for not being able to write a Petrarchan sonnet about James Caan, I should at least try to write one myself.
And so, over the next couple days, I labored to come up with what follows:
This isn’t an especially great Petrarchan sonnet about James Caan, but it has its merits, if I may say. I chose to go with the ABBAABBA CDDECE rhyme scheme, as per Wordsworth’s “London, 1802”, and managed to stick with it. In the end, I wound up with a sonnet that reads serviceably. Soon, my feelings of superiority over ChatGPT returned.
But my self-doubt started to creep in again, and, a few days later, I went back to ChatGPT. I wanted to make sure that I still held the edge over the software as a sonneteer. I typed in the same request I had made before regarding a Petrarchan sonnet about James Caan just to be sure I hadn’t done something wrong.
And as soon as I pressed enter, the text started rifling out:
Now, this poem comes with a few caveats. What appears above is not a Petrarchan sonnet. It has three quatrains and a couplet, as per the Shakespearean sonnet, but it doesn’t follow Shakespeare’s rhyme-scheme (ABAB CDCD EFEF GG). Rather, it follows the non-standard AABBCCDDEEGGGG format. Effectively, it can only be called a sonnet insofar as it has 14 lines.
More egregiously, some of the lines don’t make a lot of sense. I’ll forgive ChatGPT for not knowing that James Cann is deceased (given that it runs on data from 2021 and earlier), but I don’t think James Caan’s “dedication to his craft [was] small”. This line certainly doesn’t fit in a poem that is otherwise praising the actor. Also, I’m not sure what it means to refer to James Caan as “a great human race”, which seems like a non-sequitur born of a desperate stretch to keep a rhyme.
To ChatGPT’s credit, it has managed to keep the lines to 10 syllables, as per the iambic pentameter that typifies English sonnets. However, not all the lines follow the unstressed-stressed pattern of the iambic meter (see, for instance, “From ‘The Godfather’ to ‘Misery’”).
That said, scansion of my own sonnet above will reveal the same inconsistencies in meter. In practice, poets as great as Shakespeare were known to take some license with their iambic pentameter, so neither ChatGPT nor I have committed an unforgivable literary sin here.
In fact, we shouldn’t hold anything against ChatGPT in terms of its poetic abilities. It whipped out a pseudo-sonnet on a rather esoteric topic (James Caan) in a matter of seconds. This is a mind-boggling achievement in AI development, and immense praises are due to the programmers. With updates in the years and months to come, we can only imagine what kind of sonnets ChatGPT will soon be capable of, to say nothing of so many other, more crucial applications that will doubtlessly enrich the world.
For now, though, the world is richer than it was before by two sonnets of questionable quality about James Caan.
