The choice of activation function can change the behaviour of the ANN network considerably.
Hidden units
The standard choice is the sigmoid function, eq. (
), either in
symmetric 
or asymmetric 
form. The sigmoid function is
global in the sense that it divides the feature space into two halves,
one where the response is approaching 1 and another where
it is approaching 0 (-1). Hence it is
very efficient for making sweeping cuts in the feature space.
Other choices are the Gaussian bar [41], which replaces the sigmoid function with a Gaussian, and the radial basis function [42]. These are examples of local activation functions that can be useful if the effective dimension of the problem is lower than the actual number of variables, or if the problem is local.
Output units
For classification tasks, the standard choice is the sigmoid. The outputs
can also be normalized, such that they sum to one, by using so-called
Potts or softmax output
where 
is the summed signal arriving at output i.
For function fitting problems the output should be chosen linear.
Of these, JETNET 3.0 implements all possibilities except for the Gaussian bar and radial basis function.
It is sometimes suggested to use piecewise linear functions instead of the more complicated hyperbolic tangent for the sigmoid, in order to speed up the training procedure. We have not found any speedup whatsoever when the simulations are run on RISC workstations. It might however be relevant if the simulations are run on small personal computers.
