House Price Prediction with Linear Regression

01Loading the dataset

We'll use the California Housing dataset, a real dataset of housing prices across California districts, available directly through scikit-learn — no manual download needed. It has 20,640 records with features like median income, house age, average rooms, and location.

import pandas as pd
import numpy as np
from sklearn.datasets import fetch_california_housing

# Load the real California housing dataset
housing = fetch_california_housing(as_frame=True)
df = housing.frame

print(df.shape)
print(df.head())
print(df.describe())

02Exploratory data analysis

Before training anything, we check for missing values, look at the distribution of the target variable, and examine correlations between features and price — this tells us which features will actually matter to the model.

import matplotlib.pyplot as plt
import seaborn as sns

# Check for missing values
print(df.isnull().sum())

# Distribution of target variable
plt.figure(figsize=(8, 5))
sns.histplot(df['MedHouseVal'], kde=True)
plt.title('Distribution of Median House Value')
plt.xlabel('Median House Value ($100,000s)')
plt.savefig('price_distribution.png')

# Correlation heatmap
plt.figure(figsize=(10, 8))
correlation = df.corr()
sns.heatmap(correlation, annot=True, cmap='Greens', fmt='.2f')
plt.title('Feature Correlation Heatmap')
plt.savefig('correlation_heatmap.png')

# Top correlated features with target
print(correlation['MedHouseVal'].sort_values(ascending=False))

03Preprocessing & train/test split

We separate features (X) from the target (y), then split into training and test sets. We hold out 20% of the data purely for evaluation — the model never sees this during training, which is what lets us honestly measure how it performs on unseen data.

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# Separate features and target
X = df.drop('MedHouseVal', axis=1)
y = df['MedHouseVal']

# Split into train (80%) and test (20%) sets
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Scale features so they're on comparable ranges
# Linear regression performs better when features are scaled
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

print(f"Training samples: {X_train.shape[0]}")
print(f"Test samples: {X_test.shape[0]}")

04Training the model

With the data prepared, training a Linear Regression model is straightforward. The model learns a weight (coefficient) for each feature that best predicts house price — this is the core idea behind linear regression: finding the line (or hyperplane, in multiple dimensions) that minimises prediction error.

from sklearn.linear_model import LinearRegression

# Initialize and train the model
model = LinearRegression()
model.fit(X_train_scaled, y_train)

# Look at the learned coefficients
coefficients = pd.DataFrame({
    'Feature': X.columns,
    'Coefficient': model.coef_
}).sort_values('Coefficient', ascending=False)

print(coefficients)

05Evaluation

We evaluate using two standard regression metrics: RMSE (Root Mean Squared Error, how far off predictions are on average, in the same units as price) and R² (the proportion of variance in price explained by the model, from 0 to 1).

from sklearn.metrics import mean_squared_error, r2_score
import numpy as np

# Make predictions on the held-out test set
y_pred = model.predict(X_test_scaled)

# Calculate evaluation metrics
rmse = np.sqrt(mean_squared_error(y_test, y_pred))
r2 = r2_score(y_test, y_pred)

print(f"RMSE: {rmse:.4f} (in $100,000s)")
print(f"R² Score: {r2:.4f}")

# Visualize predictions vs actual values
plt.figure(figsize=(8, 8))
plt.scatter(y_test, y_pred, alpha=0.3, color='#1a4a3a')
plt.plot([0, 5], [0, 5], 'r--', lw=2)
plt.xlabel('Actual Price ($100,000s)')
plt.ylabel('Predicted Price ($100,000s)')
plt.title('Actual vs Predicted House Prices')
plt.savefig('predictions_vs_actual.png')

Metric	Score	What it means
RMSE	0.7456	Predictions are off by ~$74,560 on average
R² Score	0.5758	Model explains ~58% of price variance

An R² of 0.58 is a reasonable baseline for a simple linear model on this dataset — it confirms the relationship is real but not perfectly linear. In practice, this is exactly the kind of result that leads data scientists to try Random Forest or Gradient Boosting next (see Project 03), which can capture non-linear patterns this simple model misses.

06Real-world use case

This project demonstrates the full lifecycle every regression problem follows: understand the data, prepare it correctly, train a simple interpretable baseline, and evaluate it honestly. The same six-step structure applies whether you're predicting house prices, sales revenue, or exam scores.