Regression Coefficients Are Not Causal

A health insurance model comes back with a coefficient of 24,330 on smoking status. Smokers pay about $24,000 more than non-smokers of the same age, sex, BMI, family size, and region. The p-value is effectively zero.

Somebody turns that into a slide: a smoking cessation program would save $24,000 per participant.

That sentence does not follow, and the gap between it and the coefficient is the most expensive misreading in applied analytics.

What the coefficient actually says

Two sentences that sound the same and are not:

A coefficient describes differences between groups that already exist in your data. A causal claim describes what happens when you intervene. Those coincide only under conditions your data has to earn. No amount of statistical significance supplies them. A p-value tells you the difference is unlikely to be noise. It says nothing about whether the difference would follow you into a world where you changed something.

Three specific reasons the $24,330 wouldn't follow someone who quit:

The coefficient answers "how do these two groups differ in this dataset." That is a genuinely useful question. It is not the intervention question, and the two get conflated constantly.

"Holding all else fixed" is doing less work than it sounds like

This is the phrase that creates the illusion. It sounds like a controlled experiment. It is not.

What it actually means: statistically adjusted for the variables I happened to put in the model.

It does not mean all else. It means the handful of columns that were in your dataframe. Every variable you did not measure is still varying freely, and if one of them drives both your predictor and your response, your coefficient has quietly absorbed it.

It also does not mean held fixed by you. Nobody intervened. The data is observational, the groups formed themselves, and the model is describing a pattern it found.

How badly can this go? In a concrete-strength model, a coefficient on coarse aggregate in concrete runs at −0.0054 on its own and +0.0154 with seven other ingredients in the model. Both significant past p < 1e-8. The predicted probability for the same mix moves from 99.8% to 36.1%. Same data, same variable, opposite conclusions, entirely because of which other columns were present.

If which variables you include can flip the sign, then "holding all else fixed" is a claim about your variable list, not about the world.

What earns a causal interpretation

Design, not modeling.

• A randomized experiment. Randomization is what makes the treated and untreated groups comparable on everything, including what you did not measure. This is why A/B tests are worth their cost.
• A natural experiment with a defensible identification strategy: difference-in-differences, instrumental variables, regression discontinuity. These earn their conclusions from a specific argument about how the assignment happened, and that argument is the deliverable, not the regression output.
• An explicit set of causal assumptions you are willing to write down and defend, which is what a causal diagram is for. The point is not that the assumptions are certain. The point is that they are visible and arguable.

Regression is the estimation step in all three. It is never the justification. You bring the identification; the model just does the arithmetic.

Correlation vs causation: why this costs real money

Two sentences that look almost identical:

Customers who use feature X churn 30% less, adjusting for plan and tenure.

Shipping feature X to everyone will cut churn 30%.

The first is a real finding and a legitimate reason to investigate further. The second is a budget request, and the regression did not authorize it.

The obvious failure mode: the people who adopted feature X may simply be the people who were already staying. Engaged customers try more features and renew. Push the feature to disengaged customers and you may move nothing at all, having spent a quarter of engineering time to find out.

This same shape shows up everywhere. Training programs that "increase retention" among the people who volunteer for training. Sales tools that "lift quota attainment" among reps who choose to adopt them. Marketing channels that "drive conversion" among people already intending to buy. In each case the coefficient is real and the causal reading is unearned.

Three habits that keep it straight

Write the counterfactual out loud. "If we changed this one thing and nothing else, the outcome would move by β." Say the actual sentence. If it makes you uneasy, your model does not support it, and the discomfort is information.

Name what you did not measure. Ask what would have to be true for the coefficient to be causal, then ask which of those things you actually know. This takes two minutes and it is the highest-value two minutes in most analyses.

Use associational verbs on purpose. "Is associated with." "Differs by." "Predicts." These are not hedging or false modesty. They are accurate, and they are what stops a slide from quietly becoming a strategy three meetings later.

Six things worth remembering

1. A coefficient describes differences between groups already present in your data.
2. A causal claim describes what happens when you intervene. Significance never bridges the two.
3. "Holding all else fixed" means adjusted for the columns you happened to have.
4. Which variables you include can flip a coefficient's sign, so the phrase is a claim about your variable list.
5. Randomization, a credible natural experiment, or written-down assumptions earn a causal reading. Regression does the arithmetic.
6. Say the counterfactual out loud before you put it on a slide.

This is a companion to a six-part series on regression analysis. If you want the basics first, start here. If you want the failure modes, the trap index collects all of them in one table.