Task Complexity: Why Makers May Out-Think Scientists

October 14, 2026
blog image

The claim is that creative people are the smart ones and scientists are coasting on method. It is wrong, but it is wrong in an interesting way — and what it gets at turns out to be the single most important thing about how human fields work

Two people reach the end of a decade of work.

One has a theorem. She stands on four centuries of verified structure — Newton, Euler, Gauss, Hilbert — none of which she re-derived, all of which she received as free, load-bearing bricks, each one certified by a process she can trust without inspecting it. Her personal contribution is one new result at the top of an enormous inherited tower.

The other has a novel. She inherited technique — structure, scene, point of view, the accumulated craft of the form — but nobody handed her an answer to the only question that matters, which is whether the thing is any good. She had to build that answer inside herself, from scratch, over about ten years. And when she finally had it, she could not give it to anyone. It will die with her.

Now rank them. That is the question this article is about, and the reason it cannot be answered the way people try to answer it is that the two women are not doing the same kind of thing at all. One is climbing a ratchet. The other is working without one.

The hypothesis I set out to test was the folk one, stated sharply: that composers, novelists, choreographers and architects exercise more raw intelligence than physicists and engineers, because creative work forces the mind to hold open-ended ambiguity, cultivated taste and tacit judgment that formal science is explicitly built to offload onto codified method and peer-reviewed procedure. Put bluntly: the creators are the smart ones, and the scientists have infrastructure.

To test it I scored eighty real tasks across twenty-five dimensions of difficulty and read a library of 104 primary sources on task complexity, expertise, tacit knowledge, creativity, uncertainty and feedback. The scores came out close to tied, and I will say up front that I do not think the scores settle anything — they are one reasoner’s structured judgment, not measurement, and a result I happen to like is the kind I should trust least.

What the research did produce was a mechanism. One property of a field determines whether its knowledge compounds, whether its novices can be trained, whether its masters can teach, how long its apprenticeships run, how trustworthy its experts’ confidence is, and what machines will take from it first. That property is not how hard the field is. It is whether anyone can tell you that you are wrong.

Sixteen principles.


The 16 principles, in brief

  1. Difficulty is a relationship between a problem and the representations available for it, not a property of the problem.

  2. Task complexity and practitioner intelligence are different quantities, and the folk debate conflates them.

  3. One property decides a field’s entire character: whether it comes with a verifier.

  4. A verifier turns patience into competence, because it makes search work.

  5. A verifier is what makes knowledge cumulative — it is the mechanism of the ratchet.

  6. A verifier is what makes novices trainable, which is why some fields have prodigies and others cannot.

  7. Feedback you cannot attribute produces confidence rather than skill.

  8. Without a verifier, volume stops being progress.

  9. Science is leveraged and art is not — the asymmetry is inheritance, not difficulty.

  10. Altitude is not climb, and comparing fields without subtracting inherited scaffolding measures the wrong quantity.

  11. Verifiability is independent of stakes, so unverifiable does not mean unserious.

  12. What produces a verifier is a fast, attributable causal link — not the severity of consequences.

  13. Taste is a verifier you have to grow inside yourself, which makes it the most expensive knowledge there is.

  14. Compression has no decompression algorithm, which is why masters so often cannot teach.

  15. The real line runs through science rather than around it: paradigm-founding work is verifier-poor.

  16. We built a meritocracy out of the legible half of human capability, and that half is now becoming abundant.


1. Difficulty is a relationship, not a property

Nothing is hard in itself. Difficulty is a relation between three things — a problem, a mind, and the representations that mind has been handed — and changing any one of the three changes the difficulty without touching the problem at all. The word handed is doing most of the work in that sentence, and we will come back to it.

Notation is the clearest case. A notation looks like a way of writing a problem down, a neutral description, and it is nothing of the kind: it is part of the machinery that solves the problem. Positional notation with a zero performs most of the work the Roman system forces a skull to do unaided. Swap the notation and you have not described the task differently. You have redistributed it between the page and the head.

Multiply CXLVII by XXIII. Sit with it for a moment — not for the answer, for the feeling of a mind reaching for a handhold and finding none. There is no column to carry into and no place value to lean on, so every partial product has to be held whole and recombined from memory. This was a profession. It had manuals, and a merchant with a ledger to close hired a specialist to do it for him.

Now multiply 147 by 23. Identical quantities, identical operation. A nine-year-old does it in forty seconds with a pencil, and the pencil is not incidental — the paper holds the partial products so the child does not have to. Nothing about the world changed between those two paragraphs.

Physics tells the same story at the frontier. Computing how two electrons scatter in quantum electrodynamics once meant pages of algebra in which the physical content was invisible; a correct calculation taught you almost nothing about what was happening. Feynman’s diagrams turned the algebra into a picture. A line is a particle, a vertex is an interaction, and every drawing you can sketch corresponds to a term you have to compute. A graduate student now reads off in an afternoon what cost a senior theorist a week, and reads off the physics as well as the number.

The history of human capability is largely a history of such swaps. Arabic numerals, algebraic symbolism, the Cartesian plane, the calculus, double-entry bookkeeping, staff notation, the periodic table, Feynman’s diagrams. In each case a class of problems that had required hired specialists became routine, not because anyone got cleverer but because someone found a shape that did the thinking.

There is direct evidence for the mechanism. Chi, Feltovich and Glaser asked PhD physicists and undergraduates to sort twenty-four mechanics problems into groups. Novices sorted by what the problems looked like — inclined planes here, springs there, pulleys in the third pile. Experts sorted by the principle that cracks them: conservation of energy, Newton’s second law. Both groups were internally consistent. They were simply sorting different objects. The finding is not that experts know more; it is that the same printed problem is a different problem for the two of them, because the expert’s representation already contains most of the solution. Simon pushed this to its edge in 1973, arguing that ill-structuredness is not a property of a task but an artifact of the solver.

So “how hard is X” has no answer in the abstract, and every argument that opens by comparing the difficulty of two fields has skipped the only question that matters. Ask instead which representations this person was handed, by whom, and at what age. The difficulty you observe is the residue left after the inherited machinery has taken its share — and the shares are wildly unequal.

None of this makes difficulty arbitrary, and it does not make it subjective. Some problems have resisted every representation anyone has found: the behaviour of turbulent fluids, whether checking is fundamentally cheaper than solving, the path of a particular economy eighteen months out. Grünwald and Vitányi’s survey of algorithmic information theory states the hard version of this. By a simple counting argument almost all objects are incompressible — they admit no short description and no simplifying rule at all. Most of the space is genuinely structureless. A better notation is not always waiting to be found, and assuming one is is its own species of error.

2. Task complexity and practitioner intelligence are different quantities

The folk debate about whose work is harder fuses two claims that come apart cleanly: that a kind of work is more complex, and that the people doing it are more intelligent. Those are different quantities with different units. All four combinations exist, and the two mismatches are the interesting ones.

They come apart for a structural reason. An instrument that scores tasks scores tasks. It can tell you how much indeterminacy a piece of work contains, how many constraints collide inside it, whether anything can check the answer — and it knows nothing whatsoever about the distribution of minds that happen to walk into that work. Difficulty is a property of a problem plus a representation. Intelligence is a property of a person. Reading one off the other requires an assumption nobody states aloud: that fields recruit in proportion to their difficulty.

Consider a postdoc running a standard immunoassay inside a validated protocol. Plate layout fixed, dilutions fixed, controls specified, acceptance criteria written down by somebody else years ago. The work is demanding in the way careful work is demanding — fumble the pipetting and the plate is wasted — but it is close to zero on indeterminacy. There is a right answer and a written procedure for reaching it. A brilliant person doing that assay is doing low-indeterminacy work. The hard thinking happened once, upstream, and now lives in a binder.

Then a songwriter in a bedroom, self-taught, trying to get a chorus to land. No protocol, no acceptance criteria, nothing that can be consulted to establish whether the fourth version beats the second. The problem is genuinely open: the space of possible choruses is unbounded and the only available arbiter is an instrument the writer has to grow inside her own head. That is high-indeterminacy work, and the absence of a credential does not reduce it by a point.

Something else muddies every comparison of this kind, and it has nothing to do with the work. Fields with long, verifiable training ladders select hard on a particular measurable profile, because they can. A physics department admits on the strength of exactly the operations physics will later test. Songwriting has no entrance exam, no qualifying paper, no gate at all. The result is that physicists look more uniformly able than songwriters — and probably are — without that being a fact about the comparative difficulty of physics and songwriting. It is a fact about the door, and about who was allowed to stand in front of it at seventeen.

The cleanest evidence that the two dials turn independently comes from chess, where the task is fixed and the people vary enormously. Gobet and Campitelli surveyed 104 Argentinian players from amateur to grandmaster and found that reaching master strength took a mean of 11,053 hours of practice — with a range from 3,016 to 23,608, roughly eight to one. For solitary practice alone the spread was twenty-two to one. Several players logged more than 25,000 hours and never made master. Hold the task perfectly constant and the human cost of competence still varies by an order of magnitude. Whatever that variation is, the difficulty of chess cannot be it, because chess did not change.

What follows is a discipline rather than a belief. Stop reading a field’s prestige as a verdict on the people inside it, and stop reading an individual’s brilliance as a verdict on their field. When hiring, score the task and the candidate on separate sheets — because the commonest error in senior hiring is to credit a person with the structure they inherited. The analyst who looked exceptional inside a firm with an exceptional model, and who is ordinary without it, is a type, and everyone has met one.

None of this licenses the comfortable conclusion that intelligence is unreal, unmeasurable, or merely a social artifact. Gobet and Campitelli’s own data cuts the other way too: practice was necessary, and twenty-five thousand hours without mastery says something about the player and not only about the hours. The honest claim is narrower and harder to wriggle out of. Task complexity is a real quantity, individual capability is a real quantity, and an instrument built to measure the first is simply not evidence about the second.

3. One property decides a field’s entire character: whether it comes with a verifier

Of all the ways one piece of work can differ from another, a single property matters more than the rest combined. Does the task come with a verifier? A verifier is any procedure that can look at a candidate answer and tell you whether it is good — without you already knowing the right answer. That last clause is the whole definition. Drop it and everything becomes verifiable, trivially and uselessly.

The idea is borrowed, and it is worth knowing from where. In 1971 Cook proved that every problem whose candidate solutions can be checked quickly reduces to Boolean satisfiability, founding the complexity class NP and making a mathematical object out of an everyday asymmetry: some problems are agonising to solve and cheap to check. A sudoku takes an hour to fill and twenty seconds to score. The P-versus-NP question asks whether checking is fundamentally cheaper than doing. Everyone believes it is. Nobody can prove it.

Carried out of computing and into human work, the property predicts far more than it was built for. The five principles that follow are its consequences — how a field trains people, whether its knowledge accumulates, what volume of effort buys, and which of its tasks machines take first.

Start with the contrast at its sharpest. A compiler is a verifier and it answers in seconds. It does not care who wrote the code, does not get tired, and cannot be argued with. A test suite is a verifier that answers in minutes. Now look for the equivalent on the other side: nothing in the world takes a novel as input and returns a verdict. An editor is not a verifier. Editors disagree, and the same manuscript’s fate turns on which desk it lands on, which is a fact about desks rather than about the book.

The same gap opens in work with enormous stakes. A transport planner deciding whether to run a tram line down a particular street is choosing between options no available procedure can rank. Rittel and Webber named this class in 1973 and listed its properties: there is no definitive formulation of the problem, because formulating it is the problem; there is no stopping rule; solutions are good-or-bad rather than true-or-false; and there is no immediate and no ultimate test of a solution. The tram helped or it did not, twenty years later, tangled with everything else that happened to that street, with no counterfactual city to compare against.

What that absence does to a practitioner shows up in the protocol studies. Goel and Pirolli watched expert architects, mechanical engineers and instructional designers work, and extracted twelve invariants common to all three. Two are the ones to keep: no right or wrong answers, only better and worse — and personalised stopping rules and evaluation functions. Read the second one slowly. With no correctness criterion and no feedback, the designer has to supply her own termination condition. That is why two excellent architects stop at different places on the same brief and both are finished, and why “it’s done” is a sentence only one person in the room can say.

The consequence is a habit worth installing before any substantial piece of work. Name the verifier out loud, specifically, before planning anything. Not “the market will tell us” — when, how precisely, and will you be able to tell which decision it was answering? If you cannot name it, you are in the other regime, and every instinct imported from the verified half of your career is about to be wrong.

The limit here is large. Almost no real work is purely one thing or the other. Most tasks are a stack with a different verification regime at every layer: does the code compile, do the tests pass, is this architecture any good, is this the right product to be building. The bottom two layers answer in minutes and the top one may never answer. Teams routinely run a flawless loop at the bottom of the stack and mistake it for progress at the top — green pipelines, an elegant refactor, a beautifully built thing nobody wanted. Confusing which layer you are on is a distinct failure with its own signature, and it is commoner than having no verifier at all.

One more caution against over-reading the formalism. Fortnow’s survey of P versus NP stresses that NP-hardness is a worst-case statement, and that modern solvers routinely crack industrial instances with hundreds of thousands of variables. Having a verifier does not make a task easy, and being provably hard in the limit does not make the instance on your desk intractable.

4. A verifier turns patience into competence, because it makes search work

Here is what a verifier buys. If you can rank candidates, you can generate a great many of them badly and let the ranking do the work. Search plus a verifier is a machine that manufactures competence out of patience, and it requires no genius at any step — which is precisely why its presence or absence matters more than the talent of the people involved.

The mechanism is that the intelligence relocates. It stops living in the generator and takes up residence in the filter. Evolution has no mind at all and produced the eye. Drug screening, engineering optimisation, chess engines, finding a decent restaurant in an unfamiliar city: in every case the generating step is dumb, repetitive and cheap, and all the discrimination happens in the test.

A materials team hunting a compound that stays conductive at a given temperature does not need anybody who can predict the winner. They need a synthesis route cheap enough to run thousands of times and a measurement honest enough to trust. The chemistry intuition in the room gets spent pruning the space, not picking the answer, and the furnace settles it. If the measurement is good, the team’s competence is largely a function of how many plates they can run before the funding ends.

The same logic at the opposite scale: an engineer at two in the morning with one failing integration test. Deliberation is the wrong tool here and it feels like the right one. Each hypothesis costs thirty seconds to check — comment out the middleware, pin the dependency, bisect the commit range, flip the feature flag — so generate-and-test beats thinking by a wide margin, every time. The engineers who are fast at this are not the ones with the best theory of the bug. They are the ones who stopped theorising soonest.

Evidence that patience genuinely substitutes for insight comes from where you would least expect it. Satisfiability is NP-complete; the theory says it is hard. Fortnow’s survey notes that modern SAT solvers nevertheless dispatch industrial instances with hundreds of thousands of variables as a matter of routine. The wall is real and practical problems went over it anyway, because a cheap checker plus relentless structured search plus compute is an engine that does not get tired. Read that as a claim about strategy rather than about logic: given a verifier, grinding wins against problems insight cannot touch.

What to do with it is unambiguous, and most organisations do the opposite. Where a verifier exists, volume and delegation are correct and agonising is merely slow. Let juniors and machines generate; let the verifier discriminate, because it is better and faster at discriminating than your judgment is. Stop reviewing line by line what the test suite already answers. Ship the experiment instead of debating it, when the experiment costs a day and the debate costs a week.

Hold that beside the inverse case, which arrives in the eighth principle. Where no verifier exists, every sentence in the paragraph above becomes dangerous advice, because volume without a ranking function is motion rather than progress, and delegation without a check cannot be audited at all.

The limit is the one that destroys companies. Search needs the verifier to be aligned with what you actually want. Optimise a proxy hard enough and you get a technically excellent product nobody wanted — the team that maximised time-on-page and built an application unpleasant to leave, hitting its target quarter after quarter. A cheap wrong verifier is more dangerous than no verifier at all, because it industrialises the error, reaches it faster, and wraps it in a dashboard that makes doubt look like innumeracy.

A verifier also makes search possible without making the space navigable. Franco, Doroc, Yadav, Bossaerts and Murawski showed that complexity measures taken from theory — chiefly how close an instance sits to the satisfiability phase transition — predict both human accuracy and time spent across three canonical hard problems, and do so after controlling for instance size. Difficulty tracks the structure of the particular instance rather than how large it looks. Some searches are cheap to check and still hopeless to run, which is why “generate more candidates” is an instruction and not yet a method.

5. A verifier is what makes knowledge cumulative — it is the mechanism of the ratchet

If I can verify your result, I can build on it without redoing it. That single move is the most consequential thing our species has ever worked out. I take your theorem as a brick and put my floor on top of it, and the brick does not have to be fired again.

A ratchet is a mechanism that permits motion in one direction and forbids slippage back. Verification is the pawl. Without it a field can still move, but every generation re-establishes the ground it stands on, and the ground is never higher than one person can reach in one lifetime.

Mathematics is a four-thousand-year cathedral built from nothing but that move. A graduate student this morning used the spectral theorem, the fundamental theorem of algebra, Lebesgue’s construction of the integral and a dozen results from the 1970s whose proofs she has never read and never will. This is not laziness and nobody pretends it is. Verification happened once, the result was cemented into the structure, and the entire point of cementing it was that nobody would pay that cost again.

The price of the first turn of the ratchet is occasionally visible. Klein and colleagues’ formal verification of the seL4 microkernel — 8,700 lines of C and 600 of assembler, proved correct against an abstract specification — consumed about twenty person-years of proof work against 2.2 to write the kernel. An order of magnitude, paid once. Every system since that sits on seL4 inherits the proof for free and does not know it is inheriting anything. That is what a brick costs and what a brick is worth.

It is not only a mathematical habit, which is why the law generalises rather than describing one discipline’s good luck. A structural engineer sizing a steel beam this afternoon opens a code table and reads off a load factor. That number is a compressed century of tested failures — collapses investigated once, by people long dead, whose conclusions were checked, argued over and then cemented into a figure she multiplies by. She does not revisit the collapses. She is not expected to. The verification was the work, and the table is where the work went to live.

Now the other side, and it is not a complaint about art. A novelist inherits technique, and does so freely: free indirect style, the way a scene turns into its own consequence, how to put a gun on a wall in act one. Craft is real and it transmits through books, workshops and apprenticeship. What does not transmit is the evaluation function. Nobody can hand a writer a working answer to “is this good,” because that answer was never externalised into anything checkable in the first place. It has to be grown privately, over roughly a decade, and when it finally works it cannot be given away.

So two fields that look comparable in difficulty diverge wildly in what their practitioners inherit. A twenty-five-year-old physicist operates at an altitude four centuries of verified structure carried her to, and her own contribution can be modest while the view is enormous. A twenty-five-year-old novelist starts at sea level, as every novelist has, because the shoulders are not available to her — not because sand is harder terrain than rock.

The practical consequence runs in two directions. When judging a person or a field, separate inherited altitude from personal climb, since the folk ladder quotes the first while claiming to report the second. And when you want your own work to compound rather than evaporate, externalise the evaluation criteria: write down what would count as good before you look at the result, so that next year’s version of you inherits a brick instead of a mood.

The limit comes in two parts and both matter. Cumulation is not the same as progress toward truth — a verifier cements a shared error as efficiently as a correct result, and a paradigm shift is precisely the discovery that a long-ratcheted structure was built on a bad foundation, with the ratchet having made the error load-bearing in the meantime. And art does accumulate. Technique accumulates, vocabulary accumulates, the available moves multiply with every generation. What art does not accumulate is settled verdicts, which is a far narrower claim than the one usually made on its behalf.

6. A verifier is what makes novices trainable, which is why some fields have prodigies and others cannot

With a verifier you can hand a beginner a problem, let them fail, and tell them immediately and unambiguously that they failed. That loop is the entire engine of skill acquisition. Where it is missing, training does not merely slow down — it becomes a different and much weaker activity, and no quantity of institutional seriousness compensates.

Deliberate practice, as Ericsson defined it, presupposes a verifier and cannot be built without one. The requirement is effortful work at the edge of current ability, with immediate informative feedback and repetition. Remove the feedback and the phrase stops meaning anything. You cannot stay at the edge of your ability if nothing tells you where the edge is; you drift to wherever it feels productive, which is reliably somewhere comfortable.

This is why chess has thirteen-year-old prodigies. A junior who loses a game runs it through an engine the same evening and receives a per-move verdict, with the exact move where the position turned marked for her. The engine does not care that she is thirteen, does not soften the assessment, and will do it again tomorrow night. Ten thousand of those verdicts, compressed into pattern recognition, is a master.

Now the same age in a different world. There are no teenage general counsels, and the reason is not that law is harder than chess. A twenty-six-year-old who has drafted a dozen settlement agreements has no way to learn which clauses saved the client, because the disputes that never happened leave no trace in the file. The feedback she gets is the deal closing: one terminal bit, arriving months late, confounded with everything else in the negotiation. There is nothing for an engine to score.

Notice what her firm does instead, because every low-feedback profession does the same thing. It substitutes time. Seven years to partner, a fixed sequence of rotations, promotion gates spaced by the calendar rather than by any demonstration. That is not stupidity. It is what an institution builds when it cannot test, and it has the texture of training without the loop that makes training work.

The gradient this produces is measurable and stark. Macnamara, Hambrick and Oswald’s meta-analysis of 88 studies found deliberate practice explained 26% of performance variance in games, 21% in music, 18% in sports, 4% in education — and under 1% in the professions. The figure fell as domains became less predictable and less structured. Interpret the bottom of that list properly: in professional work, more than 99% of the difference between practitioners is something other than accumulated practice. Set that beside how we actually talk about seniority, where years serve as a near-sufficient proxy for capability, and the result is devastating.

The gradient is not a ranking of those fields by difficulty. Games sit at the top because the game tells you, instantly and without negotiation. The professions sit at the bottom because the world answers late, vaguely, and about something other than what was asked.

What to do about it is to stop paying for years and start manufacturing the missing loop. Inside a four-year geopolitical forecasting tournament, Chang, Chen, Mellers and Tetlock found that a cognitive-debiasing tutorial lasting under an hour improved accuracy by 6 to 11% in Brier score over control, with the benefit persisting across years. An hour of structure beat decades of unstructured experience, in a domain famous for teaching nobody anything. Forecasts with resolution dates, a decision journal that records the reasoning before the outcome, a pre-mortem: these are manufactured verifiers, and they are cheap.

The gradient is about the availability of the learning loop, not about whether skill exists in those fields. Professional skill is real — a good general counsel is worth a great deal and a bad one is a catastrophe — it is simply not acquired the way chess skill is acquired, and it therefore does not arrive at thirteen. And the underlying literature is arguing with itself: Macnamara and Maitra’s double-blind replication of the original violinist study failed to reproduce its core claim that practice differentiates every level of skill. Hours matter less than the canonical story says, in both directions. Resist any reading of this principle that makes practice decisive, and any that makes it irrelevant.

7. Feedback you cannot attribute produces confidence rather than skill

Three things get lumped together under the word feedback and they need separating. Latency: how long until the world answers. Granularity: whether the answer is a rich signal or one terminal bit. And credit assignment: whether you can tell which of your actions caused the outcome. The third is the one nobody tracks, and it is the one that decides whether experience teaches anything at all.

Credit assignment fails in specific, identifiable ways. Pignatelli and colleagues’ survey of temporal credit assignment sets out the structure: long delays between action and consequence, stochastic outcomes, sparse reward, and transposition — many different action sequences arriving at the same state — all dilute the learning signal per decision, independently of how large the reward is. Their summary of real-world decision problems is that feedback is typically noisy, delayed, and carries little or no information about causes. That is not a description of a bad environment. It is a description of most environments.