OPTIMAL ONLINE LEARNING: A BAYESIAN APPROACH
Sara A Solla and Ole Winther
A recently proposed Bayesian approach to online learning
is applied to learning a rule defined as a noisy single layer
perceptron. In the Bayesian online approach, the exact
posterior distribution is approximated by a simple parametric
posterior that is updated online as new examples are incorporated
to the dataset. In the case of binary weights, the approximate
posterior is chosen to be a biased binary distribution. The
resulting online algorithm is shown to outperform several other
online approaches to this problem.
LUTP 99-44, Computer Physics Communication, 121-122, 94-97 (1999)