next up previous
Next: Quickprop Up: Choosing the Learning Previous: Choosing the Learning


Back-propagation is the most widely used learning algorithm since it is very simple to implement and, most importantly, it often outperforms other methods in spite of its simplicity. We emphasize, though, that this statement refers to the ``on-line'' variant of BP, where the weights are updated after presentation of only a small subset of training data. On-line BP is much faster than batch mode BP.

For networks with more than one hidden layer it is beneficial to use the Langevin updating variant (eq. (gif)), where noise is added to the BP equations [6]. This is because the Hessian matrix easily becomes ill-conditioned with a flat subspace where the random search in Langevin updating is very efficient as compared to other alternatives.

System PRIVILEGED Account
Fri Feb 24 11:28:59 MET 1995