Statistics and Charts in the News

Numbers feel objective, but the way they are chosen and drawn can completely change the story. These checks will help you read headlines, charts and health news with confidence.

Correlation is not causation

• Two things can move together (correlation) without one causing the other (causation).
• Classic example: cities with more ice cream sales also have more drownings. Ice cream does not cause drowning; hot weather increases both (Mathwords, LibreTexts).
• A hidden factor like the weather is called a confounding or lurking variable. Another example from LibreTexts: in children, shoe size is linked to spelling ability, because older children have bigger feet and spell better.
• Also ask whether the cause could run the other way, or whether the link is pure coincidence in a large dataset.
• LibreTexts notes that a properly designed, controlled experiment is needed to show causation. Be cautious when a headline says "X causes Y" from an observational study.

Relative versus absolute risk: a worked example

• Absolute risk is the actual chance of something happening. Relative risk compares two groups (Cancer Research UK).
• Example: suppose a condition affects 2 in every 1,000 people. A headline says a habit "raises the risk by 50%". That means 3 in 1,000 instead of 2 in 1,000.
• The relative increase is 50% (from 2 to 3 is one and a half times). The absolute increase is 1 extra case per 1,000 people, or 0.1 percentage points.
• Real example from Cancer Research UK: childhood CT scans were reported as tripling the risk of some cancers (a "200 per cent increase"), but the extra risk was about one additional case of each cancer per 10,000 children scanned.
• Cancer Research UK's advice: "beware of headlines bearing relative risk". Always look for the starting number and compare risks using the same denominator, such as "per 1,000".

How charts can mislead

• Truncated axis: a bar chart whose axis starts above zero makes small differences look huge (FlowingData).
• Cherry-picked time frames: choosing a convenient start or end date can create a trend that disappears over a longer period.
• Totals instead of rates: a big city will have more incidents than a small one simply because more people live there. Compare per person (FlowingData).
• Dual axes: two lines with separate scales can be stretched to look connected when they are not.
• Pie charts whose slices add up to more than 100%, and 3D effects that distort sizes, are other warning signs.
• FlowingData notes that many misleading charts come from carelessness rather than deliberate deception, but a chart that looks surprisingly dramatic deserves a closer look.

Questions to ask about any number

• Compared with what? A number on its own ("5,000 cases") means little without a baseline or comparison.
• How many people were studied, and who were they? A survey of 20 people, or only of one app's users, may not represent everyone.
• Who collected the data and who paid for it?
• Is it a percentage or a percentage point? Going from 10% to 15% is a 5 percentage point rise but a 50% relative rise.
• Does the headline match what the study actually found? Use SIFT's Trace move to find the original.