What a Questionnaire Can Know

The problem

The human sciences unit where students learn that surveys have limitations is the easiest lesson in TOK to teach badly. Everyone already agrees. Questionnaires are simplistic, people lie, samples are unrepresentative — a class can produce the whole critique in four minutes and believe they have understood something.

What they have actually learned is to be dismissive, which is the opposite of the point. The census still produces knowledge. Governments still allocate money on it and they are not being foolish. A student who can only say the instrument is imperfect has no way to explain why we use it anyway.

What I changed

Students are the data first. They take the Big Five — a hundred and twenty questions, ten minutes — and then have to do something harder than criticize it: identify three to five features specifically designed to make it reliable. Before any complaint is allowed, they have to reconstruct why the instrument is built the way it is.

Then the flaws, then the language questions, and then the item that makes it personal: which domain was your highest, which your lowest, and how well do they reflect your personality? A student who has just decided the test got them slightly wrong is holding exactly the problem the module is about.

Do these categories measure reality or construct reality?

The BuzzFeed quizzes arrive as the control condition, and the question asked of them is unusual. Not what's wrong with this — students can do that — but what validity, if any, might this type of quiz have? That inverts the expected move, and it forces students to notice how structurally similar the two instruments are: forced choices, a categorical output, a flattering narrative explaining you to yourself.

Then it scales up

Lesson two moves to the census, and the pivot is the interesting part. Census questions are more concrete — how many people live here, what work do they do — and the data really is treated as fact, and policy really is built on it. The slide says so plainly before asking why we should still look critically.

The companion site carries that half: two readings on why a statistical agency addresses its mail to a building rather than a person and what happens when only 61.3% of households answer; two short films on how sampled answers become a claim about a country; Dollar Street, where households are organized by income rather than nation and photographed; and a slide deck tracing one figure — a household earning $51–142 a month — through its whole construction.

Every resource carries its own guiding questions, and the best of them is the one a student cannot answer quickly: if two official poverty measures describe the same people in the same year, in what sense is either one true?

Two scales, one instrument

The final reflection asks for three hundred words on whether a questionnaire can produce valid knowledge about people, given that it must simplify complex lives into questions, answers and categories — drawing on both halves.

The structure is what makes that answerable. A student who has only met the census can argue about statistics abstractly. A student who has just been told they are 34% Ravenclaw and mildly low in conscientiousness, and who has decided the second one is about right, is arguing about an instrument they have personally submitted to. The census is the same machine at a scale where no individual gets to say it was wrong about them — and that is what they have to reckon with.

What's in it

• The personality test worksheet — Big Five analysis in five parts, then two BuzzFeed quizzes of the student's choosing with a structured comparison
• The lesson deck — thirteen slides carrying both lessons, from the Big Five discussion through the census pivot to the final reflection
• The census resource site — two readings, two films, Dollar Street and a twelve-slide deck, each with guiding questions, on a page students can work through at their own pace

Who it's for

DP Year 1 Theory of Knowledge, in the Human Sciences area of knowledge. Two lessons, plus the Big Five as homework before the first — it takes ten minutes and students will do it willingly, which is not true of most TOK homework.

The external tests are free and need no accounts. The census materials use Brazilian data, which works anywhere as a case study; swap in your own national statistics agency if you'd rather teach the version your students live inside.

It pairs with Discovered, or Merely Agreed Upon?, where the question is whether mathematical categories are found or chosen. Here the same question arrives about human ones, and the stakes are more obviously political.