Symmetries in the problem can and should be exploited to reduce the connectivity and complexity of the network. For translational symmetries one can use so-called ``receptive fields'', in which the input field is divided into subfields with shared weights [1]. Also, if it is known that the important feature only occupies a small part of the input field, then one can use ``selective fields'', which is essentially the same as ``receptive fields'' without the shared weights property. If possible, the most robust and time saving technique is to preprocess the data such that it is presented to the network in an invariant form [56].
If it is suspected, but unknown, that the problem has a symmetry, then it is possible to use ``soft weight sharing'' [57], which clusters the weight values by adding a complexity term to the usual error measure (see the section on pruning below).