15 Ways to Misread Your Own Regression Model

Regression rarely fails loudly. It returns a clean summary table with stars next to the coefficients, and the mistake is in how you read it.

Every trap below came out of a real fit. Most of them I walked into myself. The pattern is always the same: the software did exactly what it was told, the output looked reasonable, and the conclusion was wrong.

This is the index. Every row links to the post that works it through with the actual numbers, so you can start anywhere. New to this? Start with what "we ran a regression" actually means.

Reading the output

#	What you see	What you conclude	What is actually true
1	A variable is wildly significant, then isn't	The data changed your mind	Nothing changed but your column list. Engine size read p near zero alone and p = 0.48 once cylinders joined, because the two are 93 percent correlated. Full post
2	A coefficient is not significant	The variable does not matter	It adds nothing given the others in the model. Drop a correlated predictor and it can become significant. Part 3
3	R² went up when you added a variable	The model improved	R² never decreases when you add anything, including random noise. Use adjusted R². Part 3
4	Two dummies for one categorical are both non-significant	Drop the variable	Test the whole factor with a partial F-test. Jointly it can be highly significant. Part 3
5	Coefficient changed when you added predictors	The model is unstable	It is answering a different question. Marginal and conditional effects are different quantities. Part 3

Intervals and predictions

#	What you see	What you conclude	What is actually true
6	`interval = "confidence"` gives [24,395, 27,335]	This customer will pay in that range	That is the interval for the group average. The interval for one person was [10,211, 41,519], more than ten times wider. Part 3
7	A coefficient of 24,330 on `smoker`	Quitting would save them $24,330	It describes a difference between groups that already exist. Intervention requires a design, not a p-value. Full post
8	`exp(predict(glm_model, newdata))` = 1.60	160%? A probability of 1.6?	`predict()` on a GLM returns the link scale. That is odds, not probability. The probability is 0.616. Part 5

Diagnostics

#	What you see	What you conclude	What is actually true
9	`GVIF = 13.23` on a factor	Multicollinear, drop it	Raw GVIF is not on the VIF scale for multi-df terms. Square the `GVIF^(1/(2·Df))` column: 1.29² = 1.68. Fine. Part 4
10	Cook's distance flags 53 points	You found 53 outliers	The sorted plot showed about 6 worth investigating. `4/n` slices a smooth continuum. Part 4
11	Removing outliers raised R² from 0.818 to 0.856	Cleaner model	Held-out MSPE got worse, 11.469 to 11.563. Any in-sample statistic improves when you delete the worst-fitting rows. Part 4
12	AIC dropped from 5302 to 4720 after cleaning	Much better model	Different row counts. AIC and BIC are only comparable across models fit to identical data. Part 4
13	Funnel-shaped residual plot	The model is invalid	Coefficients are still unbiased. Your standard errors and p-values are what broke. Curvature is the serious violation. Part 2

GLMs and model selection

#	What you see	What you conclude	What is actually true
14	Deviance GOF p = 1, Pearson GOF p = 0	Pick the one you like	Neither is valid. That disagreement is the signature of insufficient replication. With continuous predictors, use Hosmer-Lemeshow. Part 5
15	Ridge, lasso, and elastic net return identical MSPE	Suspicious, but they agree	They were used for selection, then OLS was refit on the survivors. The shrinkage was discarded. Part 6

Two that deserve their own explanation

A coefficient flipped sign and stayed significant. Coarse aggregate went from -0.0054 to +0.0154 when seven other predictors entered, both significant past p < 1e-8, and the predicted probability for the same concrete mix moved from 99.8% to 36.1%.

The instinct is to find the confounder. There isn't one. No single predictor flips the sign, and adding the obvious candidate makes it more negative. Add the seven in two different orders and both paths land on the same endpoint with the big jump falling on whichever variable enters last. The reversal belongs to the whole covariate set, and the set is something you chose. Full post

Selecting variables by their p-values. Keeping only the coefficients significant at 95% and dropping the rest is the most natural thing in the world and it fails on four counts: every p-value is conditional on the others, six tests at α = 0.05 gives a 26% chance of a false positive, large samples make trivial effects significant, and selecting on p-values invalidates the p-values you then report. On this data, a partial F-test on the three "insignificant" variables came back at p = 0.0496. Part 6

The one habit that catches most of them

Look at traps 1, 3, 10, 11, and 12. Every one is a metric computed on the data the model was fit to, and every one can be improved by making the model worse.

Held-out data is the only check none of them survive. It is also the cheapest thing in this entire list to add.

Start the series

	Post	What it covers
1	What "We Ran a Regression" Actually Means	Coefficients, standard errors and the three objectives, in plain language
2	What Each Regression Assumption Buys You	Which violations kill your estimates and which only kill your p-values
3	How to Read a Regression Coefficient	Conditional effects, baselines, partial F-tests, CI versus PI
4	The R-Squared Trap	VIF, GVIF, Cook's distance, and why cleaning data backfired
5	How to Read Logistic Regression Output	Odds versus probability, the `predict()` trap, goodness of fit, overdispersion
6	Variable Selection, Ridge, Lasso, Elastic Net	Cp, AIC, BIC, stepwise, and when regularization does nothing
+	Regression Coefficients Are Not Causal	The one that costs the most money
+	The Coefficient That Flipped Sign	When conditioning reverses a significant result

About the data

Every result in these posts is reproducible. Each post links its own dataset; here they all are:

Used in	File
Assumptions, coefficients, diagnostics	beer_new.csv
Coefficients, causality	insurance.csv
Diagnostics	laptop_data.csv
Logistic regression	retention_data.csv
Sign flip	Concrete_Data.csv
Variable selection	market_data.csv

One caveat worth stating plainly. These are teaching versions of public datasets: relabelled, rescaled, and in some cases resampled. The file I call beer data has 1,599 rows and shares six column names with the UCI Wine Quality (red) set, which has exactly 1,599 rows, so it is near-certainly that data wearing different labels. The concrete file has 3,090 rows against UCI's 1,030, expanded so the goodness-of-fit tests have repeated rows to work with.

Treat the domain labels as scaffolding. The statistical behaviour is the point, and that part is real: these are genuine fits to genuine data, and the traps they expose are the ones you will hit on your own.