Survey Item Discrimination

The bug this exists to prevent

Survey batteries grow by addition. Every stakeholder has a question they want in, each one reads sensibly, and the instrument reaches forty items nobody will finish.

The intuition for cutting them is wrong in a specific way: people remove items that seem redundant and keep the ones that feel important. But an item where 95% of respondents pick the same answer contributes almost nothing regardless of how central its topic is — it cannot distinguish between two people, which is the only thing an item is for.

Meanwhile the item that looks trivial and splits the population cleanly is doing most of the work. Judged by reading, both look the same. Judged by distribution, they are opposites.

What it scores

• Zero when everyone answers identically — no information, full stop.
• Near 1 when responses split evenly between the scale endpoints — maximally discriminating.
• In between for a real, moderately spread distribution.
• Deterministic: identical inputs always score identically, so a battery can be re-scored in
• CI and a change in the number means a change in the data.

Both Likert and forced-choice items are handled, each with its own zero case.

What this does NOT do

No factor analysis, no IRT, no reliability coefficients, no validity testing. This is one statistic answering one question: does this item separate people?

It says nothing about whether an item measures the thing you care about. A question that splits respondents perfectly on something irrelevant scores highly here and is still worthless — item discrimination is necessary, not sufficient, and treating it as a sole criterion optimises a battery into noise.

No sampling correction. Discrimination computed over thirty responses is a shakier number than the same figure over three thousand, and this returns the same type either way.

Verified

10 tests: zero discrimination for a unanimous Likert item, near-1 for an even endpoint split, mid-range for a realistic spread, determinism across repeated runs, and the equivalent zero and split cases for forced-choice items.

Not covered: no test over real survey data, and no confidence interval — the score has no attached uncertainty, which matters most at small sample sizes.