Chapter 8. Variety Versus Slop

Requisite Variety — book coverThe book on LeanpubRequisite VarietyThe book is written in the open. Buy it once and get every new chapter as PDF and EPUB as it’s published.Get the book on Leanpub →

We recognize slop before we finish reading. A README that opens with “This project is a…” and goes on to Installation, Usage, Contributing. A LinkedIn post of three paragraphs with a call to action at the end. An architecture diagram with a separate service, a queue, and three workers. There are usually no mistakes in these texts. So what is it we recognize?

That anyone could have written this. And therefore — no one did.

In Chapter 6 I stopped at the point where the model picks from a distribution, and without your intervention the answer lands at its peak, the way a marble rolls to the bottom of a dimple. Now — what that does to quality, and how slop grows out of the middle of the distribution.

The resting state

The model’s language is statistics. It takes every next word from a distribution of likely continuations, and that distribution has a densest spot: whatever appeared most often in the training data. For the request “a recipe for a Ukrainian dish,” the densest spot is borscht. For “the architecture of a notification service” — a queue and workers. For “write a README” — Installation, Usage, Contributing.

Picture the bell of a normal distribution. Its peak is the model’s most stable state: that’s where the most weight sits, and that’s where every step taken without outside intervention pulls. And the state sustains itself: a typical word makes the typical next word more likely; a typical first paragraph drags a typical second one behind it. The longer a text runs on its own, the fewer options it has left.

This is slop: output from the peak of the distribution that is indistinguishable from the output for any other request of the same class. It is a drift toward the average with no outside push at all. For the reader, slop is zero information: everything in it they have already seen, and what they opened the text for never made it in.

Hence an unpleasant consequence: slop is not cured by a better model. A better model usually knows more precisely what is typical, so its peak is sharper, and the answer lands there with even more confidence. When we wait for the next version to “finally write less gray,” we are waiting for a sharper bell.

Regression to gray

A camera in automatic mode does the same thing, and there you can see it with the naked eye.

Inside the camera sits a light meter. It measures the light coming from the scene and sets shutter speed and aperture so that the shot comes out, on average, middle gray — because that’s how it was calibrated: the world, on average, is gray. For most scenes that’s true, and auto mode gives a decent shot.

Picture snow in the sun. The automatic sees a lot of light, decides the scene is overexposed, and darkens the shot: the snow comes out gray. A night street: the automatic sees darkness, decides there isn’t enough light, and pulls the scene up toward the average: the night comes out gray, with smeared streetlights. The device isn’t broken; it works exactly as designed: it returns any scene to the average.

The photographer knows what the device cannot know: “this is snow, and it should be white.” One dial — exposure compensation at “+2” — and the snow is white. That is the photographer’s variety, fed into the automatic: the device distinguishes only the amount of light; the human distinguishes snow.

From a viewing platform, auto mode gives every tourist the same shot: the same spot, the same tower, the same averaged exposure. Nobody made a mistake. There was simply nothing in the shot from the person who pressed the button.

And where the photographer steps in, a price appears. Pull up the shadows, raise the sensitivity — and noise appears in the shot: grain that wasn’t there on auto. And good shots from manual settings don’t come on the first press: the photographer takes two hundred and selects three.

The one thing not in the distribution

Now Ashby’s law from Chapter 2, applied to quality.

Everything in the model’s distribution is in it for everyone. It is shared: the same articles, the same READMEs, the same landing pages. When your request contains nothing but the name of the task, the model has only the shared and returns the shared. A result that differs from the typical can come only from what the distribution lacks. And it lacks exactly one thing — you: your context (this team, this reader, this server), your constraints, and your taste — what you’ll reject without explaining why.

The arithmetic is the same as in Chapter 2, only now it’s about quality: the result cannot differ from the typical by more than the variety you brought in on top of what the model already has. Bring in zero — get the peak.

IN THE DISTRIBUTION — shared by all articles · READMEs · landing pages · "best practices" the peak: slop the output when you bring in nothing YOURS context constraints taste the one thing the distribution lacks — and the one thing that shifts the output
Figure 8.1. Everything in the distribution is shared; only what it lacks can shift the output

Averaging has a literal picture, too. Mix every paint on the palette and you get a muddy brown. The paints aren’t bad: each one is bright on its own. Brown is what all the colors have in common. A text assembled from everything ever written on the topic is brown for the same reason.

There’s a temptation to get around this by asking: “be original,” “make it more interesting,” “write something unconventional.” But the request to be original has a peak of its own — typical originality: the unexpected metaphor you’ve already seen ten times, “imagine a world where…,” an ellipsis for intrigue. You asked the model to bring in variety in your place and got the most probable variety.

That’s why slop is a diagnosis of the operator, not of the output. The operator here is you together with how you assembled your pair — that is, the pair in action: what you put into the context, which constraints you set, what you selected by. From the output you can reconstruct whoever made it. If you as an operator can be replaced by anyone, the output will be anyone’s. When your README is no different from the README of the project next door, it isn’t the README that’s gray. What was gray was the prompt, with nothing of yours in it.

One working question remains: what does “better” mean? “Good” and “bad” exist only for someone, from some position, and for the sake of some goal. Without those, in any set of solutions only one stands out by itself — the most stable. When there are no criteria, the distribution makes the default choice, and “better” means “more typical.” Where criteria come from and how they add up into a system of values is Chapter 9.

Accept, cut, shift

There are three ways to work with the peak, and choosing between them is your job.

Accept the peak. Sometimes the typical is the goal. Translating an interface: you want the most familiar phrasing. A migration script, a server config, a commit message, a template letter about a price change — everything where a surprise costs more than grayness. Here slop is a virtue: predictable and cheap. One condition: the peak is chosen; it doesn’t just happen while you aren’t looking. Before the task, a question to yourself: do I need the peak here?

Cut the peak. When the typical doesn’t fit, you narrow the zone, and you can cut from two sides.

The first side is to forbid. A list of banned phrases in the style guide. “Not borscht and not varenyky” in a request for a recipe. “No separate service and no queue” in a request for an architecture. A ban removes the peak itself, and the model is forced to climb down from it to where it’s less at home.

The second side is to specify exactly what you need, and thereby pick one narrow slice of the bell. Instead of “a Ukrainian dish” — “Poltava-style borscht, on beet kvass, with halushky dumplings.” Instead of “the user” — the on-call engineer at three in the morning who needs to restart the service and find the logs. Instead of “in our style” — a set of samples to measure against. An exact specification leaves one piece of the whole bell, and inside that piece the model again gives the most typical: the most ordinary Poltava borscht. Often that is exactly what’s needed, and then an exact specification is the cheapest way to get the result.

All of these are attenuators from Chapter 2, and the rule for them is the same: wherever you can, do it structurally. A linter for banned phrases will check them even when you’ve forgotten about them; a template with ready-made blocks won’t let the model slide back to “the user.” The price is the same in both cases: the zone gets narrower, but you’re still on the same bell, and no surprises will come from there.

Shift the bell. The third way is, instead of cutting the peak, to move the whole distribution so far that the peak itself lands where no one has been. “A recipe for a Ukrainian dish” gives borscht. “A Ukrainian dish for astronauts on the Moon” — no borscht there anymore: weightlessness, no crumbs, a long shelf life. “A Ukrainian dish they would have eaten on Magellan’s ship in the Drake Passage” — salt meat, hardtack, whatever keeps for half a year. “A Ukrainian dish for lemurs on Sumatra in the sixties” — here I can’t even guess, and that is the point. The crossing of several contexts is where the statistics are thin: there is nothing ready-made there, and the model has to assemble from parts instead of pulling out a whole.

possible answers probability specify exactly: one narrow slice forbid: peak cut off the peak: slop frame a new peak where no one has been accept when typical is the goal translation, config, commit cut forbid the typical ("not borscht") or pick one slice shift a frame from crossed contexts; thin statistics — more checks
Figure 8.2. The bell of the distribution and three ways to work with the peak: accept, cut, shift

The set of conditions you put up is a frame, and it’s yours. The Moon, the Drake Passage, the lemurs — that’s your variety, and it covers everything the model can “think up” inside the frame: whatever it assembles, you know why the Moon is there, and you can judge whether it worked. The model’s variety is working for you.

There’s a price, and it’s the same one the photographer pays for raised sensitivity: where statistics are thin, there is noise. The model can assemble confidently even where there’s nothing to assemble from, so a shifted bell needs more checking afterwards. It’s the same dependency between freedom and control from Chapter 2: the farther from the peak, the more attention verification takes. And there is one action common to all three ways.

Select, don’t accept. All three ways rest on one thing: you choose from several variants. Ask for three instead of one, and the spread between the model’s answers — which in a single variant looked like chance — becomes a scale for comparison. Selection is where taste enters the work: you reject two variants, and your criterion lives in exactly what you rejected them for. The photographer keeps three shots out of two hundred — that’s how he works.

The First Shot prompt variant 1 from the peak publication no criterion Selection prompt variant 1 rejected variant 2 rejected variant 3 chosen criteria why two were rejected becomes the criterion next time taste enters here
Figure 8.3. The First Shot and selection: a criterion appears only where there is something to choose from

The same technique rescues you when it’s you who lacks the variety, because the field isn’t yours. Then ask the model for a map instead of an answer: “what approaches are there to this task — name five, and for each, the conditions under which it fits.” The model will show more than a single peak, though these five will be the most typical five. The choice stays with you — with your goal, which the model doesn’t know. You borrowed variety for the map, but not for the choice.

And a separate word about text. Editing is a separate component of activity (Chapter 10), and the prompt “improve it” doesn’t replace it. Generate and verify are different components, and when you ask the model to “make it better,” you’re once again handing it what should have been yours: the criterion. An editing pass with a checklist of your own — banned phrases, the first sentence, an ending without a summary — is your manual settings, like the photographer who doesn’t trust the automatic.

Borscht in architecture

A scenario from software design — one of the fields where slop has lived for years.

First attempt. The task: the product needs notifications — emails and pushes about events in the account. I opened Claude Code and wrote: “Propose an architecture for a notification service.” The answer came in a minute, and I could have written it myself without reading it: a separate microservice, a message queue between it and the main app, workers reading the queue, retries with a growing pause, a separate queue for failures, templates, a separate database, a delivery dashboard. All correct. Every item is an element from a thousand articles on notification-system architecture, and together they are borscht: the dish you serve when you need to be sure you’ve shown Ukrainian cuisine.

What’s wrong? With the architecture — nothing. What’s wrong is the operator: there was none of us in the request. How many of us there are, what we already have, how many notifications a day, what happens if one gets lost. The model filled the blank with the most ordinary thing, the same way it filled the blank in “fix the test” in Chapter 6.

Second attempt. A frame. There are two of us engineers. There is Postgres, and we add nothing else: no queue, no separate service. Ten thousand notifications a day, not ten million. One criterion, but a hard one: if the process dies midway, no notification may be lost or go out twice. And a request: three variants, starting from the simplest.

The simplest variant became the solution: a notifications table with a status column in the same database; a worker in the same process that once a minute takes a few dozen unsent rows with a lock, so that two copies of the app don’t grab the same row; an idempotency key on every record, checked by the email and push provider, so that a resend after a crash doesn’t go out twice. No new infrastructure. This scheme isn’t on the peak: articles about notifications are written for ten million. And for ten thousand it’s better than the canonical one. A separate service and a queue have to be deployed, monitored, and fixed when they go down — and the two of us would be the ones fixing them.

Same model, same task. One thing changed: the second request had our context and our real needs in it.

The same mechanics in design

In design, slop is even easier to see than in code, because you don’t have to read it.

“Make a landing page for our product” gives a page you recognize in a second: a big headline, a gradient, three cards with icons, a row of “trusted by” logos, a button. That’s a landing page from the peak. It isn’t bad; it’s the same as everyone’s.

Shifting the bell here means composing a frame out of things that haven’t yet been on one landing page together. Is the product a service for small workshops? Then: “a landing page as a spread from the manual of a seventies film camera: numbered diagrams, one accent color, no cards and no gradients, three steps instead of benefits.” The combination looks strange, and that is exactly why the very first variant resembles no landing page you’ve seen. Is it good? This is where the frame does its work: you composed it, you know why the manual and why three steps, so you have something to measure with. Then selection: three variants, reject two, and in the third fix the thing you nearly rejected it for.

In design, the unexpected combination is a familiar technique: a style multiplied by a material and by an era. The model only makes trying out combinations cheap. The variety that at the peak gave “every landing page” gives, at the crossing of contexts, what has been nowhere. But only the one who composed the frame can judge it.

The First Shot

This anti-pattern grows out of trust in the model.

It looks like this: one prompt, the first variant, publication. A post, a document, a page, a deck — whatever the model produced on the first try goes out into the world, sometimes with two words corrected. And the author sincerely believes they worked with the model.

By construction, the first variant is the most probable one. It is the peak itself: what the model would have given anyone who typed these same words. Without selection and without criteria, the result belongs to the distribution. There was nothing of yours in the prompt, and your name under the text is the only thing that distinguishes it from the statistical average.

This anti-pattern has a second stage that looks like a fix. The result seems gray, and the author writes: “make it more interesting.” The second variant really is different — from the peak of typical originality: a punchy first line, an unexpected comparison you’ve already seen, a call to action at the end. The author publishes the second variant and considers the job done. Yet no variety entered the system: both times they asked the model to add what they should have added themselves.

The cure is selection. Choose from three. Name what you rejected two for, and that becomes the criterion next time. Run the editing checklist. And ask yourself the simplest question: if anyone else had written this text, what in it would be left of me? If nothing — it isn’t your text yet.

The peak at every level

The bell doesn’t disappear with scale. Three agents from the same context, one writing, the second reviewing, the third approving, are the same peak three times: in Chapter 14 this is called a committee of copies. A team whose design documents all look alike, because everyone starts from the same template and the same prompt. A product that looks like every product in its category, because it was assembled from best practices with nothing of its own added. At every level the diagnosis is the same: if the operator can be replaced by anyone, the output will be anyone’s.

In your craft

Text. Your voice is the one thing the distribution lacks. Pin it down in text: a set of samples, a list of banned phrases, three rules of rhythm. That cuts the peak. Do the editing pass separately from generation, with your own checklist. Don’t publish the first variant.

Analytics. A “market overview” from the peak is the most plausible overview in the world, and it helps no decision. The frame here is the decision the overview has to support, plus what you already know yourself. If the field isn’t yours, ask for a map of approaches, not a conclusion. Choose the conclusion yourself.

Design. “Make it beautiful” is the shortest road to the peak. Instead of the judgment “beautiful,” give a direction. Add a stack of constraints: a grid, a palette, banned moves. An unexpected combination of contexts shifts the bell. And selection from variants is the main creative act.


To close — three questions for your practice.

— Recall the last text you published from the first variant: if anyone else had written it, what in it would be left of you?

— Where in your work is the peak the right choice — and did you choose it deliberately, or did it just happen?

— What frame could you put up tomorrow so that the peak lands where no one has been — and will your variety be enough to judge what came out of it?

Key Takeaways