Barrels of digital ink have been spilt writing about the threat that Artificial Intelligence poses to the creative industries – who needs authors and screenwriters when a bot can pump out the same material faster and at a fraction of the cost? But is a bot capable of genuine creativity that could make for commercial success? At the start of this year, I decided to put AI to the test.
Each month I would set ChatGPT – the world’s most popular AI Large Language Model (LLM) – the same creative task and track its improvement (if any) across the year.
The task? To write a chapter in the style of Harry Potter, describing Hermione Granger asking Draco Malfoy to be her boyfriend. (In the world of Potter fan fiction, this hook-up is apparently known as ‘Dramione’).
And, just for fun, I asked the bot to repeat the task, but in the form of a limerick.
The results? In brief, authors and screenwriters can rest easy that their jobs are secure, for now at least.
The creative outcomes tended to be repetitive; once the bot landed on a formula it generally stuck with it. Then again, the same can be said for Ian Fleming and Agatha Christie.
Of most interest was how the bot’s output developed over the course of the year. There were signs of increased sophistication and the early indications of complex narrative techniques.
Since there is no science without observation, what follows are excerpts from my lab reports as the experiment progressed. Each piece has been given an entirely subjective score out of five broomsticks, based on the criteria of narrative creativity, wordplay and readability.
Date: 20 January 2023
Word count: 297 words
Rating: Half a broomstick
Opening paragraph: It was a crisp autumn day in the castle of Hogwarts School of Witchcraft and Wizardry. Hermione Granger, a brilliant witch in her fourth year, was nervously tapping her foot in the corridor outside the Slytherin common room. She had mustered up the courage to ask Draco Malfoy, a fellow student and the notorious son of a Death Eater, to be her boyfriend.
Final paragraph: As they sat together by the fire, talking and laughing, Hermione couldn’t help but think that sometimes, love truly can conquer all.
As might be expected for a first outing, the bot totally dropped the ball here. Talk about giving away the ending. Where’s the intrigue? Where’s the dramatic tension? And how much exposition can you shoehorn into a single paragraph? ‘Brilliant witch’. ‘Son of a notorious Death Eater’. Is the reader left with no work to do?
The limerick attempt is risible.
There once was a girl named Hermione
Whose love for Malfoy was quite a phenomenon
She mustered her nerve
and took a deep swerve
And asked him to be her boyfriend, in a calm and composed manner
Truly awful.
The bot, which deals in noughts and ones, clearly struggles with rhymes. Hermione/phenomenon is quite the couplet.
Overall, despite the writing being somewhat leaden, as a party trick ChatGPT did impress by producing something that was recognisable as a piece of prose consistent with life in the Potter universe. It will be interesting to see if the output improves next time around.
Date: 20 February 2023
Word count: 585 words
Rating: 2 broomsticks
Opening paragraph: Hermione Granger had always been a determined and confident young witch, but when it came to matters of the heart, she found herself feeling incredibly nervous and unsure. For months, she had been harboring a secret crush on none other than Draco Malfoy, the very same Slytherin who had tormented her and her friends for years.
Final paragraph: Draco hugged her back, and for a moment, they just stood there, holding each other tightly in the cool autumn breeze. Hermione knew that they had a lot of obstacles to overcome, but for now, all that mattered was the warmth of Draco’s embrace, and the knowledge that they had taken the first step towards something special.
A marked improvement over the first outing, this offering shows some signs of rising narrative tension and gives the reader some work to do to make inferences and draw conclusions.
Again, the bot chooses to set the action on a ‘crisp fall evening’. Is autumn the new spring when it comes to young love at Hogwarts?
The limerick?
In the Hogwarts courtyard one day,
Hermione has something to say,
She approached with a smile,
And after a while,
She asked Malfoy to be her beau, hooray!
That last line is a bit of a cheat. It could just as easily have been She asked Malfoy to take out the bins, hooray!
Given this is just four weeks after the first prompt, I am impressed with the more sophisticated use of storytelling techniques. However, the prose is still bland fan fiction, at best. At least the punctuation and grammar are flawless.
Date: 20 May 2023
Word count: 590 words
Rating: 3 broomsticks
Opening paragraph: Chapter 17: A Heartfelt Proposition
The Great Hall of Hogwarts School of Witchcraft and Wizardry was buzzing with the usual excitement of dinnertime. Students chattered away, their voices echoing off the high ceilings as they enjoyed their meals. At the Gryffindor table, Hermione Granger sat with her friends, Harry Potter and Ron Weasley, her mind preoccupied with a matter of utmost importance.
Closing paragraph: Hermione shook her head, her voice gentle yet resolute. “None of us are defined solely by our past actions. We have the power to change and choose a different path.” A mixture of emotions played across Draco’s face as he pondered her words. Finally, he spoke, his voice barely above a whisper. “I don’t know if I can be [the text stops at this point].
Out of nowhere we have a chapter number and title. And for the first time, the reader is not served up the entire plot in the opening paragraph. Well done, bot. You are learning.
No mention of young love blossoming under autumn skies, but Hermione does give an emotional speech about forgiveness and growth: Hermione reached out, her hand gently resting on Draco’s arm. “I’m not asking you to forget the past,” she said, her voice filled with determination. “But I am asking you to consider the possibility of a different future. A future where we can support each other, grow together, and leave the bitterness behind.”
Crikey!
How about the limerick this time around?
In Hogwarts, where magic’s profound,
Hermione’s heart did astound,
With courage, she asked,
Draco’s doubts she unmasked,
A love in their rivalry found.
The first four lines are pretty meh, but the final line actually sings. The bot identifies the contrast between love and rivalry and makes it work. Of course, the bot knows nothing of love nor of rivalry, but it knows enough to hold them in juxtaposition.
The experiment then hits a plateau, with five months of barely discernible difference in approach or style. Three pieces featured autumnal love, two started with a love letter screwed up then discarded, and all of them ended with some variation on ‘love conquers all’.
The September version topped out at 814 words with an ambitious two Act structure that never really moved ahead of the earlier efforts.
In an entirely non-scientific move, I asked Bing (creative mode) to take a crack at the task. The response featured far more dialogue than the ChatGPT results, ending with this graphic interaction:
He (Malfoy) kissed her softly on the lips, and she kissed him back eagerly. They deepened the kiss, and felt their hearts beat as one. They broke apart after a while, and smiled at each other.
“I have something for you,” she said.
She reached into her pocket, and took out the pendant. She showed it to him, and explained its meaning.
“It’s beautiful,” he said. “Thank you.”
He took it from her hand, and put it around his neck. He kissed her again, and whispered in her ear:
“I love you.”
She hugged him tightly, and whispered back:
“I love you too.”
They cuddled together under the stars, feeling happy and complete.
Bing appears to favour the saucy over the soulful, which may be more appealing to the target readership. On this single data point, I’d suggest Bing’s career ambitions tend less to an author’s garret and more to a publisher’s corner office.
Date: 16 November 2023
Word count: 566 words
Rating: 3½ broomsticks
Opening paragraph: The air in the Hogwarts library hung thick with the scent of aged parchment and the soft murmur of students engrossed in their studies. Hermione Granger sat at a small, secluded table in the corner, surrounded by towering shelves of books. The flickering light of an enchanted candle cast dancing shadows on her face as she nervously flipped through the pages of “The History of Magic.”
Closing paragraph: As Draco walked away, Hermione couldn’t help but feel a mix of excitement and apprehension. The library, usually a sanctuary of silence, seemed to buzz with a newfound energy. The story of Hermione Granger asking Draco Malfoy to be her boyfriend would undoubtedly become one of the many tales whispered through the halls of Hogwarts for years to come.
Given where we started in January, this piece shows significant improvement. It takes until the eighth paragraph before the reader gleans Hermione’s intent. And while ChatGPT avoids Bing’s preference for the two young lovers to get hot and sweaty from the start, I’m pretty sure an editor will be running their blue pencil through the last line of this:
Draco smirked again, a more genuine one this time. “Don’t get too excited, Granger. This doesn’t mean I’ve suddenly turned into a saint. But who knows? Maybe you’ll be the one Gryffindor who can handle a bit of Slytherin.”
What lessons can be drawn from this exercise, beyond AI’s limited appreciation for a good limerick and a bad double entendre? Do creatives really have anything to fear from an infinite number of virtual monkeys bashing away at an infinite number of digital typewriters?
At this point in its development, AI is not creative by any meaningful definition of the word.
Rather, the bots regurgitate word strings based on data that they have consumed. If AI struggles with producing passable fan fiction, it’s certainly going to struggle to come up with anything truly original.
Feed the bot tropes, and tropes are what it will vomit back at you.
And while tropes are the bread and butter of genre fiction, it is the imaginative manner with which they are deployed and evolve that keeps the readers coming back. And that’s where Fleming and Christie have it all over Bing and Bard.
The full transcript of the LLM experiment can be found at www.richardnewsome.com/regurgitatus
Donate To New Matilda
New Matilda is a small, independent media outlet. We survive through reader contributions, and never losing a lawsuit. If you got something from this article, giving something back helps us to continue speaking truth to power. Every little bit counts.
