Four Experts, One Algorithm

The problem

Jigsaw reading usually fails in the second half. The expert groups work — students genuinely learn their own text — and then the mixed groups turn into four consecutive summaries delivered to people with no reason to listen, because nothing depends on hearing them. The structure promises interdependence and delivers turn-taking.

Statistics teaching in TOK has its own version of the same problem. Students can recite that correlation is not causation and that samples can be unrepresentative, and then apply none of it to an actual claim.

What I changed

The second half is a fictional government proposal. PredictaGov would shut down the economic advisory board and replace it with an algorithm reading GPS, card transactions and market data to set regulation, healthcare funding and public spending automatically.

It has four provisions, and each one is broken in a different way:

• P1 runs 40,000 correlation tests nightly and acts on anything at p < 0.05 within 24 hours — which only Fry's reader can take apart
• P2 ranks all 3,141 counties monthly and funds the best and worst fifty — and 3,141 is not a coincidence; it is the exact figure from Kahneman's kidney-cancer map
• P3 closes the board and optimizes against one index of GDP and headline inflation — Luyendijk's territory
• P4 licenses traces for 38 million citizens and pushes predictions back as alerts — the Nature editorial's

A student who read Fry cannot explain what is wrong with P2. That is what makes the mixed group work: the panel physically cannot evaluate the proposal unless all four people speak, and everyone can feel it.

Read your flag aloud, thirty seconds each, experts one to four in order, no interruptions.

Every step is timed, which matters more than it sounds. Thirty seconds each stops the confident student running the table, and because the flags are written before anyone speaks, the round-robin is four arguments rather than four people thinking aloud.

Sentence stems, not questions

Each expert completes three stems rather than answering three questions. "Provision 1 will raise false alarms because…" "The counties at both extremes of the ranking will mostly be…" "Once citizens receive the alerts, the system is really measuring…"

A question invites a summary of the reading. A stem forces a claim about the proposal, and it has to be finished in writing before the discussion starts. Students who cannot complete it discover that they have not understood their text yet — at the point where that is still fixable.

The panel then has to reject PredictaGov or approve it with conditions, and write the one condition all four would sign. Four flags, one sentence: the synthesis is forced into a single line, and a panel that writes a condition addressing only the loudest flag has visibly not done the work.

The answer key does the harder job

For each expert it carries model answers, the condition that would genuinely answer that flag, a listen for list of the phrases that show real understanding — one in twenty, the word artifact, hubris and humility — and the misconception to expect.

Those misconceptions are the sharpest part. That a very small p-value means an important finding, and that more data would fix it — when more data makes P1 worse, because it raises the number of tests and shrinks the effect needed to pass. That the rural counties are genuinely unhealthier, when the data is accurate and the ranking built from it is what misleads. That Luyendijk thinks economics is useless, when his target is the prestige. That P4 is a privacy problem, when privacy is the least interesting of its three flaws.

Each of those is a plausible, articulate, wrong answer — the kind a good student gives and a teacher can easily accept in the moment.

What's in it

• Panel sheets — the shared proposal page plus a sheet per expert, each carrying the provision they answer, the point from their text, their three stems, and the knowledge question
• Teacher answer key — model answers, conditions, listen-fors and misconceptions for all four roles
• Check-for-understanding questions — for the text-alike stage, one set per text
• Running instructions — timings for independent reading, expert groups and the panel

The four readings are Hannah Fry in the New Yorker, Kahneman's "The Law of Small Numbers" from Thinking, Fast and Slow, Joris Luyendijk on economics and the Nobel prize, and a Nature editorial on computational social science. All four are findable online or on a shelf; supply them yourself rather than expecting them here.

Who it's for

DP Year 1 Theory of Knowledge, in the Human Sciences. One lesson for the jigsaw and panel, with the reading set the class before — twelve to fifteen minutes each, except Kahneman, which runs longer and can be handed deliberately to students who need the stretch.

The knowledge question on every sheet is the official one about how quantitative instruments affect the way knowledge in the human sciences is valued, so the panel feeds straight into an exhibition or an essay.

It follows naturally from What a Questionnaire Can Know, where students meet the instrument problem at personal and national scale. This one asks what happens when nobody is left to check the instrument.