LLMs are good at code you can verify, and bad at code you cannot

The usual way people divide up what an LLM is good at is by difficulty. Fine for boilerplate, unreliable for hard problems. I think that framing is wrong, and following it leads you to delegate the wrong work.

The axis that predicts outcomes is not difficulty. It is whether you can cheaply tell that the result is correct.

Two piles

Put a task in the first pile if being wrong is loud. A parser with a table of inputs and expected outputs. A pure transformation between two data shapes. A refactor the type checker will reject if it breaks. A regex with a fixture file. Query construction you can run against a snapshot and diff. In all of these, wrongness announces itself in seconds and the cost of a bad answer is the time it took to run the check.

Put it in the second pile if being wrong is quiet. Concurrency, where the bug appears under load on a Tuesday. Anything security relevant, because you cannot write a test asserting the absence of a vulnerability. A data migration, which is destructive and runs once. Performance work, where you need measurements to know if you helped. Error handling paths that only execute during an incident. Cache invalidation, where the failure is stale data that looks like data.

Difficulty cuts across both piles. A tricky bit-manipulation routine is hard and belongs firmly in the first pile, because you can test it exhaustively. A three-line change to a permission check is trivial and belongs in the second, because nothing will tell you it is wrong until it matters.

The confidence is flat, which is the trap

A model produces code with the same fluency in both piles. There is no tremor in the output when it moves from a well-trodden transformation to a subtle memory-ordering question. The prose in the explanation is equally assured. Variable names are equally sensible. It compiles either way.

Human collaborators leak signal here in a way that models do not. A colleague who is unsure tends to say so, or hedges, or writes a comment asking whether this is right. That signal is a real part of how review works, and its absence is easy to underestimate. Generated code arrives with no uncertainty attached, so the reader supplies the confidence themselves, and the reader is usually in a hurry.

What this changes in practice

Three things, and none of them are about writing better prompts.

Spend the assistance where the loop is tight. If a task has a check that runs in under a minute, delegate generously and iterate against the check rather than reading closely. Reading generated code line by line is slow and, for this pile, largely redundant, because the check is a better reviewer than you are.

In the second pile, use it for a draft you intend to rewrite. Not for the answer. A generated first attempt at a concurrency fix is useful the way a colleague’s whiteboard sketch is useful: it surfaces the shape and the cases you had not considered. Shipping it because it looks reasonable is the failure. The verification cost has not gone anywhere just because the writing cost dropped.

Move work from the second pile to the first. This is the part I find genuinely interesting. Checkability is not a fixed property of a task. It is a property of your codebase.

A permission check with an exhaustive table-driven test moves piles. A migration with a dry-run mode and a reversible path moves piles. Concurrency logic pulled behind an interface you can deterministically exercise moves piles. A performance-sensitive path with a benchmark you actually run in CI moves piles.

Every one of those was already good engineering, and every one of them was easy to skip. What has changed is the return: investment in checkability now directly increases how much work you can safely hand off. That reframes test and type discipline from hygiene into leverage, which is a better argument than the one we had before.

The failure I keep seeing

Somebody delegates a task from the second pile, the output looks right, the tests that exist pass, and it goes out. The tests that exist pass because they were written for the code that used to be there. Nothing has verified the new behaviour, and the code carries no signal that it needs verifying more than usual.

This is not an argument for using these tools less. I use them constantly and they have changed how much I get done, particularly in that first pile, which turns out to be a larger share of real work than the framing about boilerplate suggests. It is an argument for being deliberate about which pile you are in, because the tool will not tell you and the output looks identical from the outside.

The question worth asking before delegating anything is not whether the model can do it. It is how you will know if it did.