Linear regression

By Wituz, 08/27/2022, Typescript

Linear regression finds a line equation that fits some data points. Here I show what the least squares method is, and how it works.

Least squares method

Okay, so you have a bunch of 2D points that you want to turn into a function.

In order words, draw the most fitting line through.

// Our points
const points = [
    {x: 1, y: 1}, 
    {x: 2, y: 2.5}, 
    {x: 3, y: 2.5}, 
    {x: 4, y: 4.2}, 
    {x: 5, y: 4.8}
]

Component needs update: react-vis

How to

First we find the sum of all x values, all y values, all x * x values and all x * y values

// The needed sums
const sumX = points.reduce((acc, point) => acc + point.x, 0)
const sumY = points.reduce((acc, point) => acc + point.y, 0)
const sumXX = points.reduce((acc, point) => acc + point.x * point.x, 0)
const sumXY = points.reduce((acc, point) => acc + point.x * point.y, 0)

Find the slope

Now we can find the slope of the line

// The slope
const slope = (points.length * sumXY - sumX * sumY) / (points.length * sumXX - sumX * sumX)

Find the intercept

And finally we can find the intercept of the line

// The intercept
const intercept = (sumY - slope * sumX) / points.length

Putting it all together

// Least squares function
const leastSquares = (points: {x: number, y: number}[]) => {

    // Calculate sums
    const sumX = points.reduce((acc, point) => acc + point.x, 0)
    const sumY = points.reduce((acc, point) => acc + point.y, 0)
    const sumXX = points.reduce((acc, point) => acc + point.x * point.x, 0)
    const sumXY = points.reduce((acc, point) => acc + point.x * point.y, 0)

    // Slope and intercept
    const slope = (points.length * sumXY - sumX * sumY) / (points.length * sumXX - sumX * sumX)
    const intercept = (sumY - slope * sumX) / points.length
    return {slope, intercept}
}

// Points
const points = [
    {x: 1, y: 1}, 
    {x: 2, y: 2.5}, 
    {x: 3, y: 2.5}, 
    {x: 4, y: 4.2}, 
    {x: 5, y: 4.8}
]

// Calcualte the line
const {slope, intercept} = leastSquares(points)
console.log(slope, intercept)

Results

The slope is 0.93 and the intercept is 0.21

This makes our line function f(x) = x * 0.93 + 0.21

Let's see if it fits by plotting it.

Component needs update: react-vis

It does, hoo-ray.