Linear Regression by Gradient Descent

Fit y ≈ Xw + b by batch gradient descent on the mean squared error.

loss = (1 / n) * sum_i (x_i . w + b - y_i)^2

The gradients are

dw = (2 / n) * X^T (pred - y)
db = (2 / n) * sum(pred - y)

Start from w = zeros(d) and b = 0.0, and run exactly epochs full batch updates of w -= lr * dw and b -= lr * db. Update w and b from the same predictions inside one epoch, not one after the other.

Return the tuple (w, b) where w has shape (d,) and b is a float.

Function signature

def fit_linear_regression(X, y, lr, epochs):

Example

Input

X = np.array([[1.0], [2.0], [3.0]])
y = np.array([3.0, 5.0, 7.0])          # y = 2x + 1
fit_linear_regression(X, y, lr=0.05, epochs=2000)

Output

(array([2.]), 1.0)   # to 4 decimal places

Reasoning

The data lies exactly on y = 2x + 1, so the loss can reach 0. With lr = 0.05, 2000 full batch steps bring w and b to 2 and 1.

Constraints and notes

• Start from w = 0 and b = 0.
• Update w and b from the same predictions in each epoch.
• Return w with shape (d,) and b as a float.