I re-instantiated the neural network because, otherwise, the online training would be starting with a fully trained network. I used the same learning rate, 0.01, but because online and batch training ...

Coding a neural network from scratch allows you to fully understand exactly what's going on, and allows you to experiment with the code. The downside is the extra time and effort required. Listing 1: ...