The Impact of Algorithms on Our World

Data science is an intriguing field, addressing complex issues such as autonomous vehicles and artificial intelligence (AI). Recently, the media spotlighted Zillow Group's decision to halt its home buying program, which relied on predictive analytics to estimate home prices months ahead. Despite the potential for fine-tuning their algorithms, the risks were deemed too significant compared to the benefits.

Predicting home values months in advance poses a considerable challenge, even for advanced algorithms. Fundamentally, estimating a home's price is a regression problem, with price being a continuous variable influenced by various factors like the number of rooms, location, and year built. Even basic linear regression, one of the simplest algorithms, can effectively estimate home prices.

In this guide, we will provide a brief overview of regression analysis and explore examples of linear regression. Initially, we will construct a straightforward linear regression model using Microsoft Excel, followed by a more intricate model utilizing Python code. By the end, you will grasp the principles of regression analysis, linear regression, and its practical applications.

Defining Linear Regression

Linear regression is one of the most prevalent forms of regression analysis, categorized as a supervised learning algorithm. When a linear regression model uses one dependent variable and one independent variable, it is termed simple linear regression. In contrast, models with multiple variables are referred to as multiple linear regression.

The linear regression algorithm identifies the best fit line through the data by determining the regression coefficients that minimize the overall error. This is typically achieved using Ordinary Least Squares (OLS) method.

Understanding the basics of linear regression is essential for explaining predictions. The equation for a simple linear regression model is:

prediction = intercept + slope * independent variable + error

Where:

• y is the predicted value of the dependent variable,
• a is the intercept (where the line crosses the y-axis),
• B is the slope of the line,
• x is the independent variable,
• e represents the error or variation in the estimate of the regression coefficient.

While this may appear complex, it can be executed with a few clicks in a spreadsheet or lines of code in Python.

A Practical Example

To illustrate simple linear regression, consider a scenario where you are selling a house and need to determine its price. If the finished square footage of the home is 1280, you can examine the sale prices of similar homes in the area.

When plotting these prices against square footage, an upward trend typically emerges, indicating that larger homes generally command higher prices.

Using Microsoft Excel, you can input this data into two columns and create a chart via Insert > Chart. This process can also be replicated in Google Sheets.

To estimate the price of your 1280-square-foot home, fit a trendline to the chart data. Select Chart Elements > Trendline, ensure the Linear option is activated, and observe the resulting trendline.

Using this trendline, one can approximate the price for the 1280-square-foot home to be around $245,000. By accessing the Data Analysis tools in Excel and selecting Regression, a summary of statistics will populate in a new sheet. This summary includes the intercept and X Variable (slope) values, which can be plugged into the regression equation for a price prediction:

y = -466500 + 555 * 1280

= -466500 + 710400

= 243900

This example illustrates how simple linear regression can effectively estimate a home's price.

Using Python for Advanced Regression

While Excel is suitable for handling a few variables and smaller datasets, Python becomes indispensable when dealing with numerous variables and vast amounts of data, akin to what Zillow Group faced.

Continuing with the house price example, let's incorporate additional variables using a modified version of the Kaggle House Pricing dataset.

You can obtain the data via two methods:

1. Downloading the raw data from Kaggle and cleaning it yourself,
2. Accessing the cleaned example from my GitHub repository.

Data Overview

In this tutorial, the data has been condensed to 11 numeric columns from the original 81, excluding categorical and string data for simplicity.

First, import necessary libraries (Pandas, Plotly Express, and Scikit-Learn) and load the data from the train.csv file:

import pandas as pd

import plotly.express as px

from sklearn.model_selection import train_test_split

from sklearn.linear_model import LinearRegression

from sklearn.metrics import mean_absolute_error

# Read training data

train = pd.read_csv("path/to/train.csv")

print(train.shape)

train.head()

The dataset comprises 11 columns and 1431 rows. Here’s a brief description of the columns:

• SalePrice: Target variable, the property's sale price.
• LotArea: Lot size in square feet.
• GrLivArea: Above-grade living area in square feet.
• BsmtFullBath: Full bathrooms in the basement.
• BsmtHalfBath: Half bathrooms in the basement.
• FullBath: Full bathrooms above grade.
• HalfBath: Half baths above grade.
• BedroomAbvGr: Bedrooms above the basement.
• KitchenAbvGr: Number of kitchens.
• Fireplaces: Number of fireplaces.
• GarageCars: Garage size in car capacity.

Assumptions for Linear Regression

Linear regression is most effective when certain assumptions are met. Prior to modeling, I addressed missing values and outliers, but it’s essential to verify data suitability for linear regression by following this checklist:

• The dependent and independent variables should exhibit a linear relationship.
• The independent variables should not exhibit high correlation (check multicollinearity with a correlation matrix).
• Outliers must be managed, as they can significantly affect results.
• The data should ideally follow a multivariate normal distribution.

Exploratory data analysis is crucial before modeling. Use .describe() to review statistics that may indicate outliers or standard deviation:

train.describe()

Employ Plotly Express histogram to check the distribution of SalePrice, aiming for a normal distribution. If necessary, a log transformation can be applied to adjust skewed data.

px.histogram(train, x='SalePrice')

Utilize scatter plots to identify outliers and examine relationships between variables, such as SalePrice and LotArea:

px.scatter(train, x='LotArea', y='SalePrice')

If the statsmodels library is installed, Plotly Express enables performing simple linear regression in a single line:

# Perform simple linear regression

px.scatter(train, x='LotArea', y='SalePrice', trendline='ols', trendline_color_override="red")

Additionally, the scatter_matrix can be used to visualize multiple scatter plots simultaneously:

px.scatter_matrix(train[['SalePrice', 'LotArea', 'GrLivArea']])

Finally, assess correlations using the .corr() function, often visualized with a heatmap:

print(train.corr())

px.imshow(train.corr())

Building a Linear Regression Model with Scikit-Learn

Creating a linear regression model and generating predictions is straightforward with Scikit-Learn. However, it is essential first to separate the target variable (SalePrice) from the features and split the dataset into training and testing sets:

# Create target variable

y = train['SalePrice']

# Create features array

X = train.drop(columns='SalePrice')

# Split data into train and test sets

xtrain, xtest, ytrain, ytest = train_test_split(X, y, test_size=.55, random_state=42)

Next, instantiate a linear regression model and fit the data:

model = LinearRegression()

model.fit(xtrain, ytrain)

With the model trained, you can now predict prices. It's common to evaluate the model's performance before deploying it for real-world predictions. Generate predictions using the xtest dataset and compare them against the ytest values:

# Predict prices

pred = model.predict(xtest)

# Compare top predictions with actual values

print(pred[:5])

print(ytest[:5])

This will reveal how accurate the predictions are, though they may not be perfect. Understanding the model's error is vital for comparing different models to select the best one.

Evaluating the Linear Regression Model

To gauge the model's accuracy, utilize evaluation metrics such as Mean Absolute Error (MAE), which measures the average absolute difference between predicted and actual values:

mean_absolute_error(y_true=ytest, y_pred=model.predict(xtest))

The resulting MAE provides insight into the prediction accuracy, with lower values indicating better performance. Other metrics like Mean Squared Error (MSE), Mean Absolute Percentage Error (MAPE), and R-squared are also commonly employed, as they capture different aspects of model performance.

Linear Regression Code Review

Congratulations! You have successfully built a multiple linear regression model in Python, predicted house prices, and evaluated the model's accuracy using just a few lines of code:

# Create target variable

y = train['SalePrice']

# Create features array

X = train.drop(columns='SalePrice')

# Split data into train and test sets

xtrain, xtest, ytrain, ytest = train_test_split(X, y, test_size=.55, random_state=42)

# Build the model

model = LinearRegression()

# Fit the model

model.fit(xtrain, ytrain)

# Predict prices

pred = model.predict(xtest)

# Compare top predictions with actual values

print(pred[:5])

print(ytest[:5])

# Calculate mean absolute error

mean_absolute_error(y_true=ytest, y_pred=model.predict(xtest))