People couldn't tell AI from Shakespeare and often preferred AI, finds study

In the age of oil lanterns and handwritten journals, poets poured their emotions into verses that captured the beauty of the world and the depths of their souls. Fast-forward to today, where even as artificial intelligence (AI) revolutionizes countless industries, poetry remains a cherished form of deeply human expression. However, a new study published in Scientific Reports suggests that AI is beginning to encroach on this traditionally human domain. The study found that readers often favor AI-generated poetry over works by human poets, challenging the idea that human emotion is irreplaceable in creative expression.

Researchers used OpenAI’s ChatGPT-3.5 to create poems mimicking the styles of iconic poets such as Walt Whitman, Geoffrey Chaucer, T.S. Eliot, Sylvia Plath, Allen Ginsberg, Emily Dickinson, and William Shakespeare. A total of 1,634 participants read ten poems—five crafted by humans and five generated by the AI. Surprisingly, readers could correctly identify whether a poem was human-written or AI-generated only 46% of the time. Even more striking, AI-generated poems were rated more favorably, leading researchers to conclude: “AI-generated poetry is indistinguishable from human-written poetry and is rated more favorably.”

The apparent indistinguishability of AI-generated poetry wasn’t surprising, given the chatbot's advanced features and capabilities. What did come as a surprise, however, was that readers overwhelmingly preferred bot-generated poetry over human-written verses. Researchers randomly assigned sets of poems to 696 participants, asking them to evaluate 14 qualities, including rhyme, meter and originality, without revealing the origins of the poems. When the ratings were analyzed, the results were startling: most participants had rated AI-generated poems higher than those written by humans.

One reason for this preference could be complexity, as Brian Porter, lead study author from the University of Pittsburgh, told Forbes. Human-written poems often brim with intricate layers of meaning that can be challenging for most readers to decipher without deep contemplation and intellectual effort. In contrast, the poems generated by AI turned out to be less codified, more accessible and simpler in terms of context, rhyme, meaning and writing, as Porter noted.

“You can pretty easily understand what the AI-generated poems are saying on your first read, with limited context. Whereas to appreciate a poem by Chaucer or Plath, you may need to know some context and you may need to spend more time with it,” Porter said. However, the researchers also noted that the responses of readers still gravitated toward human poets when they were told that the poem they were reading was generated by a robot. "AI-generated poems felt like a deception to them," said Porter.

Simply put, poets need not fear losing their craft to AI, despite the study’s findings. As Porter explains, great poetry is not a mimicry but an expression of an experience that is beyond the limit of AI’s grasp. AI can mimic a poet’s expression of grief but AI can’t experience what grief feels like and hence, can’t express the emotion in a new way. Essentially, a robotic poet can capture the scientific side of poetry but never the artistic or creative side.

Experts still believe that AI can augment human poetry writing, but not replace it completely. “While I have no doubt that AI can generate an almost perfect Fibonacci sequence of poetry by reverse engineering well-known works, humanity is at the core of what a poem is,” poet Joelle Taylor, who was not involved in the study, told The Guardian. “A poem is more than an algorithm. It is meaning, empathy, revelation, inversion, dissidence, passion and surprise: poetry is what happens in the space between logic and chaos.” So when one sees poetry adorning coffee cups, greeting cards or Instagram scrapbook posts, it may seem like the era of human poets is fading. However, this is far from true. The innate human emotions that inspire poetry remain untouchable, beyond the reach of algorithms. Human-written poetry, with its depth and authenticity, continues to thrive and will endure far beyond the constraints of time and technology.