Drowning in a Sea of Mediocre AI-Generated Content

A few weeks ago, my colleague Laura Rich ran an experiment. She trained ChatGPT to write a blog post for a client based on a detailed brief of the story¹, and then hired a human freelancer to do the same. Once both versions were in, she asked the rest of our editorial team to guess which was AI-generated. Among those who guessed wrong were three veteran journalists with decades of experience. Humbling, to say the least.

But upon further reflection, we realized this was the wrong question. The point wasn’t whether ChatGPT was marginally better or worse than the human. The takeaway is that for content to truly stand out in the AI era, humans have to do A BETTER JOB OF DOING what humans are uniquely good at doing.

Now, let’s give ChatGPT its due. It produced a perfectly passable article in seconds, compared to seven days for the writer. But it succeeded mostly in its ability to sound generically human. Like many a rushed or lazy human writer, it fell into marketing-speak (describing a new way to use emojis as: “It's quirky, it's unique, and it's so ‘you.’"), and employed overused rhetorical devices, such as starting a paragraph on a new product feature with: “Imagine this: it’s a chilly morning and…”

But the human writer also fell into some all-too-human traps. The piece was somewhat repetitive, and lacked oomph. Laura ended up editing what had been a longish throat-clearing opening paragraph into a pithy one-sentence lede to quickly engage the reader.

In a world awash in AI-generated content, standing out is table stakes

In today’s world, the author had not committed any content mortal sin. But soon enough, the bar will have risen. Mediocre will become the new bad, because, as Laura’s experiment suggests, ChatGPT and its ilk will render actually awful content truly worthless. With 73% of B2B and B2C companies already using generative AI to help create content (and 49% using it to produce final copy), a torrent of mediocre, forgettable AI-generated content is coming our way.

Mediocre is the new bad, because ChatGPT and its ilk will render actually awful content truly worthless.

But here’s the good news: Content organizations can also use generative AI to overachieve in ways that we believe will reset expectations of what’s possible. It’s early yet, but it’s already clear to us that generative AI can improve almost every part of the editorial process. With the right curiosity, boldness, and prompt engineering skills, we’ll be able to consistently produce content that is more original, more compelling, and more timely.

Here are some of the opportunities we see so far:

Getting smart enough to be dangerous I’ve covered enterprise technology for 30 years, but still had a lot to learn when I was asked to write about software “observability” last year. I wouldn’t hazard a guess at how many hours of Googling it took to feel equipped to write something of value, but it was more than I felt comfortable billing for. I probably had 20 hours to write the first piece, according to the official budget. The real number of hours was no doubt far more than that.

I won’t spend that many hours Googling again. My get-smart process now starts with ChatGPT or Perplexity.ai², my preferred AI chatbot (in part because it clearly labels the sources it drew from, easing my concerns about hallucinations or bias). I haven’t tracked the time spent to see how it compares to the “before times,” but I’m clearly learning more per hour with generative AI — and am more confident that I can quickly get answers as new questions come up.

Finding unique angles, and avoiding tired ones Once the research is done, the job is to identify the best story to tell. Generative AI is a helpful assistant here, too, as you brainstorm with it around different topics. Ask it for angles that have not been covered, and it will deliver dozens in an instant. Don’t like any of them? Ask follow-ups to unpack the topic or bring another fresh aspect to it.

Restoring gumshoe reporting as a competitive advantage While your competitors are cranking out automated content that has few differentiators, the original reporting in your content shows actual authority on a subject. The traditional skills of journalists, such as the ability to land interviews with important sources and the skill to ask incisive questions in the heat of the moment, will be more important than ever. When anyone can get a Reader’s Digest version of any story in nanoseconds, you need real scoops and real stories to stand out.  

A reliable first-draft generator This may well be the most transformational role of generative AI in the editorial process. Even if these tools never match humans’ ability to create compelling final copy, it’s easy to imagine a time when time-crunched writers will start from an AI-generated first draft. Right now, it’s still a labor intensive job to provide the training necessary to create a useful first draft. But it won’t be long before writers get in the habit of letting their AI co-pilot “take the keyboard” for the first version.

AI’s role in the editorial process

As speed is generative AI’s leading attribute, it will most certainly impact the pace at which corporate content is produced. For example, a venture firm client recently asked us to drop what we were doing and create a feature-length article (the subject was…you guessed it…generative AI). Using Perplexity.ai, I quickly researched the topic and came up with a set of themes that had not gotten much attention. The client very quickly lined up great interviews with partners and portfolio companies, and worked with us to get approvals in days or even hours. I used Dall-E and Midjourney to propose some design ideas to Stephanie Cuenca, our designer on the project. While she didn’t ultimately use them, it helped us quickly get mind-meld on how we wanted the package to look.

In the end, it was a successful project for all involved. The article felt fresh, and helped the client achieve its content goals. And it left our team feeling invigorated by the idea that we could on occasion match the speed and intensity of a newsroom.

Where to put the brakes on generative AI

Of course, there are plenty of tasks that we will not leave to ChatGPT and its ilk. We will not outsource the final editing of anything, both to ensure the quality of the storytelling and writing. Nor will we put our credibility at risk by relying on generative AI technologies to get the facts right. In fact, one potential long-term impact of generative AI could be a return to the prominence of the fact-checker. I’m old enough to remember when every self-respecting magazine or newspaper result had fact-checkers on staff — in Jay McInerney’s 1984 novel ''Brights Lights, Big City,” the narrator works at an esteemed literary magazine’s Department of Factual Verification. As more parts of the editorial process are automated, it’s conceivable that there will be more money — and more of a competitive need — to bring this important job back.

¹ This included a summary of the story, examples of other stories written for the client, and related marketing materials from the client.

² Perplexity includes sources and citations along with its answers, raising our confidence in the accuracy of its findings.