Skip to content

AI can turn mediocre writers into pretty good writers.

In a paper published last summer, a pair of researchers looked at how well ChatGPT improves writing. They rounded up several hundred knowledge workers with average salaries around $70,000 and paid them to write short, typical business memo type things. After the first writing task, half the group was told to sign up for ChatGPT and use that to help out. Here are the results:

With ChatGPT to help out, the time taken for the writing task decreased nearly 40%. When a human graded the writing, grades with ChatGPT increased about 20% on a scale of 1-7.

Roughly speaking, what happened was that bad writers became good writers and good writers became (slightly) better and (slightly) faster writers. Poor writers closed a big chunk of the original gap against good writers, though not all of it.

This doesn't mean that writers will be put out of work by ChatGPT. Not yet, anyway. But it does suggest that the pay premium for being a good writer is likely to go down. Why pay extra for a great writer when a mediocre writer can be made pretty good with a little bit of AI help?

25 thoughts on “AI can turn mediocre writers into pretty good writers.

  1. D_Ohrk_E1

    I suggest asking GPT to rephrase a paragraph at a time when you feel something isn't quite perfect, then selectively use the parts that GPT did better. It helps get around a type of writer's block where you know you've written something badly but can't figure out how to fix it. Skip the (human) editor.

    1. MarissaTipton

      Make $280 per hour. Getting a job is not easy. In any case, you have access to a wealth of resources to help you with your work style. Become motivated to promote hundreds of jobs through job boards and vx05 career websites.

      Take a look at this………………………. https://careershome74.blogspot.com/

  2. ScentOfViolets

    Actually, all this shows is that fair to middlin' business memo writers become middlin' to good business memo writers. Since I don't read business memos to either a) inform myself, or b) for pleasure, this strikes me as a significant distinction.

    1. Joseph Harbin

      Right. The best writers aren’t being paid to write memos.

      KD: “But it does suggest that the pay premium for being a good writer is likely to go down. “

      More likely the writers of memos will become an increasingly commoditized group. (It’s not clear how many are getting a premium for being good at it today.)

      The best writers will likely continue to earn a premium because AI will be implemented to replace the vast amount of writing where mediocre is good enough (memos, news reporting, potboilers, Ivy League dissertations, ransom notes).

  3. Narsham

    Business leaders, by and large, have never been good writers: many can't manage basic spelling and grammar. Some were good or great writers, perhaps, but those skills weren't particularly valuable in most business communications, however much they may have made employees' lives better. Before e-mail, executives and managers had secretaries who took correspondence, and they were the people who could spell and arrange a sentence properly.

    E-mail eliminated that for the most part--now, as the many e-mail leaks we've seen from various corporations demonstrate, people making a lot more than $70,000 a year are exposing themselves as bad writers in every communication they make.

    What's driving AI "assistance" for these kinds of communications aren't principles of good and skillful writing. It's automating the kinds of routine office communications that take time for people who can't spell or assemble words into a sentence without help or effort. "Thank you for your response" doesn't call for a good writer; minutes of a business meeting usually don't call for a good writer. In fact, most business communications don't call for good writers, although well-written documents in a business context can be valuable.

    Things like clarity, brevity, and precision matter a lot more in law or government contexts (or scientific/academic contexts), and those communications frequently go outside of what "autocomplete" can manage. But even in those fields, I'd wager a quarter or more of e-mails could receive "autocomplete" responses and they'd be fine, even without AI.

    AI may be great for language translation in real time, and for basic communications. Not only is it a long way from being, itself, a "good writer," I've seen little indication that it can manage long-form writing, whether it's a 40 page research paper or a 300 page history book. That's not what its creators are interested in right now, anyway. Tech support, diagnostic assistance for physicians, and basic communications are a lot less complex than a history of the American Civil War.

  4. GMF

    This is the only use I've found so far for AI & it's great. The problem I'm having with it is that the more I learned how to coax it into saying what I want, the lazier I got with my own writing. I have the same problem with spelling now that spellcheck is everywhere.

  5. Toofbew

    Only a fairly good writer would recognize improvements in AI generated text over what the writer drafted. That's basically the job of an editor, and good editors generally have writing and reading skills far above average.

    The idea that a C or B writer will automatically become an A writer with AI assistance seems unlikely. Of course, some not so good writers may actually learn from AI drafting and up their game. I guess we'll soon see.

  6. Leo1008

    I don’t think this is accurate:

    “Roughly speaking, what happened was that bad writers became good writers and good writers became (slightly) better and (slightly) faster writers.”

    There seem to be some details missing, but, as best I can tell, what happened is that AI did some of the writing while the human writers didn’t improve at all. And, rather than something to celebrate, that strikes me as a shame.

    Are we finally just going to give up on our decades if not centuries of trying to educate humans to be better writers?

    And, if so, what happens to the quality of our thinking? And, ultimately, this raises questions regarding our humanity.

    Just how much of the job of being a 21st century human are we ready, willing, and able to offload onto seemingly intelligent machines? And what are the inevitable ramifications?

    No doubt these are questions that have echoed through the centuries in the face of technological progress. But that does not mean these questions have always been or still are irrelevant. If anything, these questions need to be updated just as frequently as our tech.

    1. Special Newb

      Ideally all of it. If we can play all day, that's the ultimate goal. Well that an immortality because if we can play all day no one will want to go through the hell of raising kids.

  7. jambo

    As long as AI knows the difference between “your” and “you’re” and between “there” “their” and “they’re” it will be a real step forward for much of the writing I come across. “Less” vs “fewer” will just be a bonus, as will subject/verb agreement.

  8. megarajusticemachine

    I still ponder how trustworthy these are for accuracy rather than readability. And if they're not a great writer, they might not realize that the accuracy isn't in the final product. I know we all hate paying people, but you really need to have a set of eyes run over stuff before it goes out.

    That said, I work with folk who really could use some help writing, so maybe that's not going to actually affect much.

  9. Kit

    Plenty of comments are giving off a Planck’s Principle vibe: we advance one funeral at a time.

    At a minimum, I see AI as the next step from spell checkers to grammar checkers and now to style checkers. Perhaps it will become embedded in email clients and present people with proposed changes either in real time or when they click send. Hell, why not when writing comments? I don’t think I’ve ever managed to write one without at least one error creeping in.

    AI will change the nature of some writing, mostly for the better. But the scolds will be around for a few more generations, lamenting the past golden age of office memos, much like some praise paper maps.

  10. azumbrunn

    Seriously: Is there a pay premium for good writers of memos? That would surprise me.
    Anyhow: This needs a lot more detail: How is good writing defined? How was the grading done? By AI?

  11. Goosedat

    Is the compensation paid to knowledge workers based on the quality of their written short, typical business memos?

  12. jwc123

    I'm 80 and writing a history of the community where I live. One of the problems I was having was getting other people to help "edit" what I had written. A friend who is a writer, mentioned Chat and so I took a look. I does make you better. I'm not a bad writer by any means, but that is not my strength, finding and organizing information is. The size of the eventual book will be limited by costs, and I tend to be wordy when I write. With multiple sources of information getting the correct tense consistently was a problem. Some times I pop in a section and tell Chat it is too long. Some times I tell it the text needs an initial edit with sentence structue, punctuation, and tense. Some times I tell it I just need a final edit. I've had to learn how to do instructions. I do small sections at a time. My friend the writer was ghost writing a book and she said Chat cut the time it took her in half. Sjhe also said you need to be careful and edit what they give you.

  13. jeffreycmcmahon

    What does "help out" mean in this scenario? It doesn't sound like "mediocre writers are becoming good writers", it sounds like an algorithm turned mediocre copy into slightly less mediocre copy. "Business memos" are exactly the type of writing that an algorithm can replicate without trouble because there's very little "content" in them, it's just duplicating the form and replicating the faint ideas of a million other business memos. An algorithm can write PR copy for a movie or a new business with the same kind of ease because these things are basically Mad Libs using the same formatting and structure every single time because that's what the industry's expectations and conventions are. Actual creative "writing" is a whole other thing.

    In other words, I think Mr. Drum is extrapolating improperly. Like saying that the advent of 3-D printers means that sculptors will soon be replaced.

  14. rrhersh

    This blog entry would be far more useful if it gave us some hint of the criteria used to judge better or worse writing. Are we talking about presenting information is a way that is concise and clear? Or are we talking about getting "its" and "it's" right?

  15. illilillili

    The speed seems more important than the quality. If you can fire the slowest 50% of your writers, without decreasing the amount of work produced, while slightly improving the quality of the work, why not?

Comments are closed.