If you're interested in such things, novelist Curtis Sittenfeld wrote a few days ago in the New York Times about a writing competition with ChatGPT. Based on reader input, she chose a few themes (lust, regret, kissing) and details (middle-aged protagonist, flip-flops) and then set out to write a short story. A Times editor worked with ChatGPT to write a competing story.
As it happens, I was pretty sure I could tell which was which, and I was right. Oddly, there were two things that stood out to me. The first is obvious: the Sittenfeld story included odd, unexpected quirks that made it lively, while the ChatGPT story was sort of dull and predictable. But the second reason is that the Sittenfeld story felt awkward in places and made me want to break out my blue pencil. The prose in the ChatGPT story was merely workmanlike but hardly ever seemed objectionable. Sittenfeld basically agrees:
Overall I found it to be proficient on a sentence level but clichéd, and also shallow in sentiment. To me, there’s just something missing — like the literary equivalent of fat-free cookies or a Ken doll’s genitals.
Of course, one might say it's remarkable that ChatGPT could even come close to the level of an accomplished real writer. It's also fair to point out that ChatGPT is very sensitive to its prompts. A different editor that offered different prompts and iterated a few times might have produced something better.
And, as Sittenfeld acknowledges, there's another big difference: she took a couple of weeks to write her story and then had to spend more time paring it down to 1,000 words (the agreed length). ChatGPT wrote its story in 17 seconds and hit the length requirement right on target.
For now, this experiment shows that AI still isn't close to the best humans in creative fields. But it also shows that AI isn't all that far behind. Bewarned.