LLMs Are Surprisingly Bad Blog Authors
Nobody likes reading AI slop. (If they did, it wouldn't be slop!) Slop comes in many forms, but one of the most common is the output of asking an LLM to author a blog post. You have an idea, but writing is hard! Your friend Claude is here to help, taking your outline or rough notes and spitting out 750 words of polished prose. Publish, kick up your feet, and wait...until the inevitable Hacker News comment comes back: "I'd rather read the prompt!"
Blame information theory. A major part of how LLMs work is optimized for writing bad blog posts.
Surprise!
In information theory, information is measured in bits. The information content of a message is called its surprise. Surprise is a measure of how unexpected an event's outcome is given your prediction. Let's say I ask you to complete this sentence:
The quick brown fox jumps over the lazy...
The most predictable completion is "dog". In this case, it's such a common completion that typing those three characters adds nearly no information. Nearly. Consider if you ask somebody to complete that sentence, and they say "lizard"! That would be unusual! Surprise!
Now of course, nobody uses a pangram to tell you anything useful, so let's take a real message:
Title: An Update on Our Team.
Body: After careful consideration, we've made the difficult decision...
Raise your hand if you know where this is going already. If the rest of the post is about layoffs, those words are conveying very little information beyond the title. But what if it continues with "...to raise each employee's salary by 20%"? The information, the actual part worth reading, lives only in the words that you can't predict. Anything else is slop (whether from AI or a PR team).
If you want to convey an idea, some amount of surprise is necessary, lest readers be faced with a block of completely unsurprising and therefore information-less text.
Surprise is quantifiable
Imagine a game show where a prize is placed behind one of 1000 doors. Our contestant Connor can ask the host yes-or-no questions to identify the prize. Connor first asks "Is the prize behind Door Number 1?" (Connor is not very good at this game.) The host says... "No!" Connor is not very surprised. He has only gained a small amount of information (999 out of 1000 possible doors remain).
Connor next asks "Is the prize behind Door Number 650?" (C'mon Connor!) The host says... "Yes!" Connor is now very surprised! He has gained the maximum amount of information and knows exactly where the prize is.
If more surprise conveys more information, should we aim for maximum surprise? Well, no... maximum surprise is achieved with random white noise. As authors, we want to focus our use of surprise on conveying our idea. Lizards are not useful surprises unless you're trying to poison Spock.
Surprise is also sticky
The concept of surprise also comes up in another form relevant to blog authorship: memorability. Made to Stick argues that ideas are easier to remember when they are unexpected, when they violate a prediction we didn't know we were making. From Sketchy to KanjiDamage, we see that the more visceral, odd, or shocking mnemonics are, the easier they are to memorize. Jeopardy contestants use similar techniques to memorize vast amounts of trivia.
I'm no cognitive psychologist, but I do wonder if these concepts of surprise and information transfer are more related than we might think.
LLMs don't add (the right) surprise
An LLM's job is to take a prompt and predict the next most likely token. This is exactly the opposite of generating surprise! They are optimized to sample the most likely of their potential output tokens. Completing "The quick brown fox jumps over the lazy..." will almost certainly result in "dog" every time.
By increasing the temperature parameter, LLMs can be guided to choose less likely outcomes. At a high enough temperature, you may start to see lizards. That is surprise, but it comes from the RNG, not the prompt. It isn't information you as an author want to convey. Increasing the temperature in the hopes that an LLM will inject meaningful details into your message is a fool's errand; you're only going to get hallucinations. If you're lucky, they'll be extraneous inventions. Often they'll outright conflict with the message you're trying to convey.
To take another example, ask an LLM to rewrite a sentence like a pirate or a 1940s gangster. This is just a lateral move from one voice to another. Apart from the noise of mistranslated details, the information you put in roughly matches the information you get out.
Where LLMs can help
Rewriting with an LLM doesn't change the information content of your input. Your idea is worth some fixed number of bits. Prompting an LLM to smear those bits across 750 words instead of 100 doesn't make it any more compelling. Instead it flattens your nugget of surprise into droning predictability. Readers call it slop, and take in your idea less for it. This is where the desire to "just show me the prompt" comes from.
On the other hand, what if the original information is already spread across a verbose context? That's where LLMs can shine, pulling that down and consolidating it into a concise, structured form. Again, this adds no new information, but now it makes it more consumable rather than less. LLMs do best when they are seeking out these sources of context or helping you to edit and refine your prose, not generating from scratch.
Don't sound like an LLM
So let's go back to the beginning: you have an idea, but writing is hard! Write anyways; write poorly. Use an LLM for research and feedback. Do not simply hand it your idea and expect it to generate something enjoyable or effective to read. And while you're at it, lean into the imperfections, because even if they aren't information-theoretical depictions of your ideas, they help make your writing interesting, unexpected, and yes, surprising. And that is what makes it fun to read.
Cheers!