Imagine you're trying to draw a line to connect a buch of dots on a piece of paper. If all the dots are kind of in a straight line, you just draw a straight line, right? If stright line goes through all the dots or majority of them and if these dots are kined of in a stright line then that's a linear regression.
But what to do when the dots form a curve, like a shape of a hill or a rollercoaster? A straight line won't fit very well. Instead, we will need a bendy line that can go up and down to match the cruve. That's where polynomial regression comes in!.
A Polynomial Regression is like upgrading from a stright ruler to a flexible ruler that can ben. Instead of just fitting a straight line (\(y = mx + c\)), you can use a formula that can be written as:
\begin{equation}
y = a_0 + a_1 x + a_2 x^2 + a_3 x^3 + \cdots + a_nx^n
\end{equation}
where:
The application of the polynomial regression will be shown on the following example:
But what to do when the dots form a curve, like a shape of a hill or a rollercoaster? A straight line won't fit very well. Instead, we will need a bendy line that can go up and down to match the cruve. That's where polynomial regression comes in!.
What is Polynomial Regression?
A Polynomial Regression is like upgrading from a stright ruler to a flexible ruler that can ben. Instead of just fitting a straight line (\(y = mx + c\)), you can use a formula that can be written as:
\begin{equation}
y = a_0 + a_1 x + a_2 x^2 + a_3 x^3 + \cdots + a_nx^n
\end{equation}
where:
- \(x\) - this is the input (like dots on the paper)
- \(y\) - This is the output (the line you're drawing)
- \(a_0, a_1,a_2,...,a_n\) . These are numbers (coefficients) that the math figures out to make the line fit the dots.
- \(x^2,x^3,...,x^n\) - These make the line bend. The higher the power (n), the more bendy the line can be.
- The data doesn't fit a straight line but follows a curve
- You notice atterns like ups and downs (e.g. growth trends, hills, valleys)
- You want a model that's simple but flexible enough to capture curves.
How to use the polynomial regression?
The application of the polynomial regression will be shown on the following example:
- Look at the data - Suppose you're measuring how fast a toy car rools down a hill over time. The speed might increase slowly at first, then zoom up fast. The graph of this data could like a curve.
- Pick a polynomial degree (\(n\)) - The idea is to start from the lowest degree (\(n=2\)) ( a simple bendy line, a parabola). If that's not curvy enough, try \(n=3\), \(n=4\), etc. But don't make it too bendy, or it might wiggle too much and ift random noise instead of the real pattern.
- Fit the equation - Use a computer to calculate the coefficients (\(a_0\), \(a_1\), \(a_2\),...) that make the line match your data as closely as possible.
- Check the fit - Does the line match the dots? IF not, adjust the degree of the polynomial.
Key Things to Remember
- Don't overdo it: If you make the polynomial too bendy (\(n\) too high), it will try to fit every single dot perfectly, even the random little bumps (noise). That's bad because it won't work very well on the new data due to overfitting.
- Balanved simpicity and accuracy - find the lowest degree \(n\) that fits the curve well.
Example 1 - Estimation of the plants growth based on the exposure to sunlight.
You’re trying to figure out the relationship between the number of hours a plant gets sunlight (x) and how tall it grows (y). Your measurements are:
| \(x\) (hours of sunlight) | \(y\) (height in cm) |
|---|---|
| 1 | 2 |
| 2 | 6 |
| 3 | 10 |
| 4 | 18 |
| 5 | 26 |
Step 1: Set up the equation
For degree 2, the equation in general form can be written as:
\begin{equation}
y = a_0 + a_1 x + a_2 x^2
\end{equation}
In the previous equaiton we have to find \(a_0\), \(a_1\), and \(a_2\) which are called intercept, linear term, and quadratic term.
Step 2: Organize the data
| \(x\) | \(y\) | \(x^2\) |
|---|---|---|
| 1 | 2 | 1 |
| 2 | 6 | 4 |
| 3 | 10 | 9 |
| 4 | 18 | 16 |
| 5 | 26 | 25 |
Step 3: Write the system of equations
To solve for \(a_0\), \(a_1\), and \(a_2\), we use normal equaitons derived from least squares:
- Sum of \(y\): \begin{equation} \sum y = na_0 + a_1\sum x + a_2\sum x^2 \end{equation}
- Sum of \(xy\): \begin{equation} \sum xy = a_0\sum x + a_1\sum x^2 + a_2\sum x^3 \end{equation}
- Sum of \(x^2y\): \begin{equation} \sum x^2y = a_0\sum x^2 + a_1\sum x^3 + a_2\sum x^4 \end{equation}