Machine Learning - Linear Regression with Multiple Variables

Table of Contents

This article explains what Linear Regression with Multiple Variables is on Machine Learning.

What is Linear Regression with Multiple Variables? #

Linear Regression with Multiple Variables in Machine Learning involves extending the concept of linear regression from a single variable to multiple features. This approach allows us to model more complex relationships between the input features and the output variable. Here’s a clearer explanation:

Features #

Consider a dataset with multiple features related to housing prices, such as:

Size in feet$^2$ $x_1$	Number of bedrooms $x_2$	Number of floors $x_3$	Age of home in years $x_4$	Price ($) in $1000’s $x_5$
2104	5	1	45	460
1416	3	2	40	232
1534	3	2	30	315
852	2	1	36	178
…	…	…	…	…

Each feature, denoted as $x_j = j^{th}$ contributes to predicting the house price. The number of features is represented by $n$.

Training Examples #

For each training example $i$, the features are represented as a vector $ \vec{x}^{(i)} = $, with $x_j^{(i)} = $ being the value of feature $j$ in the $i^{th}$ example. For instance, the features of the second training example can be expressed as $ \vec{x}^{(2)} = [1416 \ 3 \ 2 \ 40]$.

Representation of linear regression model for multiple variables #

In a simple linear regression model with a single variable, the prediction formula is $f_{w,b}(x) = wx+b$, where $w$ is the weight, and $b$ is the bias. In contrast, the linear regression model for multiple variables is formulated as:

$f_{w,b}(x) = w_1x_1 + w_2x_2 + … + w_nx_n + b$

Here, $ \vec{w} = [w_1 \ w_2 \ w_3 \ … \ w_n] $ represents the weights for each feature, and $b$ is the bias term.

Model Equation #

The equation for a linear regression model with multiple variables can be succinctly represented as:

$ f_{\vec{w},b}(\vec{x}) = \vec{w} ・ \vec{x} + b $

This equation indicates that the prediction $ f_{\vec{w},b}(\vec{x}) $ is the dot product of the weights vector $ \vec{x} $ and the features vector $ \vec{x} $, plus the bias $b$. By incorporating multiple features into the linear regression model, we can capture more complex relationships and patterns in the data, leading to more accurate predictions.