Model diagnostics

One train/test split can make one model look better than it is and another look worse, in the same experiment. Here's what that looks like with real numbers.

Fifteen ways regression output gets misread, with the symptom, what it actually means, and how to check. Every one came from a real model.

Alone it reads -0.0054. With seven other predictors, +0.0154. Both significant. Why hunting for the guilty variable is the wrong instinct.

The smoker coefficient is $24,330. That does not mean quitting saves anyone $24,330, and no amount of significance will make it mean that.

I removed 53 outliers. R-squared rose from 0.818 to 0.856 and the model got worse. Every number that improved ignored the rows I deleted.