Single Neuron with Backpropagation

Train one sigmoid neuron end to end. This is the smallest complete training loop there is: forward pass, loss, backward pass, update, repeat.

Forward:

z = X w + b
p = sigmoid(z)

Loss is the mean binary cross entropy against labels y in {0, 1}.

For the backward pass, the sigmoid derivative and the cross entropy derivative cancel into something much simpler than either one alone:

dz = (p - y) / n
dw = X^T dz
db = sum(dz)

That collapse is not a coincidence. It is why sigmoid is paired with cross entropy and not with squared error, where the p * (1 - p) factor survives and stalls learning whenever the neuron saturates.

Start from w = zeros(d) and b = 0.0 and run exactly epochs full batch updates. Return (w, b, losses) where losses is a list of the loss computed before each update, so it has exactly epochs entries.

Clip the probabilities into [1e-12, 1 - 1e-12] inside the log so the loss stays finite.

Function signature

def train_neuron(X, y, lr, epochs):

Example

Input

X = np.array([[1.0], [2.0], [3.0], [4.0]])
y = np.array([0, 0, 1, 1])
w, b, losses = train_neuron(X, y, lr=0.5, epochs=200)
w, b, losses[0], losses[-1]

Output

(array([2.39040644]), -5.705970791587991, 0.6931471805599453, 0.1500420059088744)

Reasoning

At the start w = 0 and b = 0, so every p is 0.5 and the first loss is log 2 = 0.6931. Training pushes w up and b down until the boundary x = -b / w sits near 2.4, and the loss falls on every step.

Constraints and notes

• Record the loss before each update. losses has epochs entries.
• Clip probabilities into [1e-12, 1 - 1e-12] inside the log.