Linear Regression from Scratch (Ordinary Least Squares)
Ordinary Least Squares (OLS) finds the best-fit line $y = mx + b$ through data points by minimizing squared residual errors. You can compute slope and intercept in pure Python without SciPy 📉.
import statistics
xs = [1, 2, 3, 4, 5]
ys = [2, 4, 5, 4, 5]
mean_x = statistics.mean(xs)
mean_y = statistics.mean(ys)
# Slope (m) = sum((x - mx) * (y - my)) / sum((x - mx)^2)
m = sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / sum((x - mean_x) ** 2 for x in xs)
b = mean_y - m * mean_x
print(f"Regression line: y = {m:.1f}x + {b:.1f} 📉") # y = 0.6x + 2.2
# 48. Linear Regression from Scratch (Ordinary Least Squares) Ordinary Least Squares (OLS) finds the best-fit line $y = mx + b$ through data points by minimizing squared residual errors. You can compute slope and intercept in pure Python without SciPy 📉. ```python import statistics xs = [1, 2, 3, 4, 5] ys = [2, 4, 5, 4, 5] mean_x = statistics.mean(xs) mean_y = statistics.mean(ys) # Slope (m) = sum((x - mx) * (y - my)) / sum((x - mx)^2) m = sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / sum((x - mean_x) ** 2 for x in xs) b = mean_y - m * mean_x print(f"Regression line: y = {m:.1f}x + {b:.1f} 📉") # y = 0.6x + 2.2 ``` Building simple linear regression from scratch proves that introductory machine learning is just basic statistical geometry! 📉 --- ### Question What mathematical quantity does Ordinary Least Squares (OLS) regression minimize? 🧠- [ ] a. The sum of squared vertical differences between observed points and the fitted line 📉 - [ ] b. The number of features in the dataset - [ ] c. The execution time of CPU loops - [ ] d. The absolute distance to the origin (0, 0) --- ### Challenge Given points `xs = [1, 2, 3]`, `ys = [2, 4, 6]`, calculate slope `m`. Print round(m, 1). 🎯 <details> <summary>Solution</summary> ```python import statistics xs = [1, 2, 3] ys = [2, 4, 6] mx = statistics.mean(xs) my = statistics.mean(ys) m = sum((x - mx) * (y - my) for x, y in zip(xs, ys)) / sum((x - mx) ** 2 for x in xs) print(round(m, 1)) ``` </details>