Neural Network `Surgery': Transplantation of Hidden Units (bibtex)
by Axel Pinz, Horst Bischof
Abstract:
We present a novel method to combine the knowledge of several neural networks by replacement of hidden units. Applying neural networks to digital image analysis, the underlying spatial structure of the image can be propagated into the network and used to visualize its weights (WV-diagrams). This visualization tool helps to interpret the behaviour of hidden units. We notice a process of specialization of certain hidden units, while others remain apparently useless. These units are cut out of one network and replaced by units taken from other networks trained for the same task using different parameters. We achieve better prediction accuracies for the new, combined network than for any of the two original ones. This constitutes a special kind of information fusion in image understanding. We give an application example from the field of remote sensing, where neural networks are used to interpret the species of trees in aerial photographs. The interpretation accuracy is raised from 85\% to 90\%.
Reference:
Neural Network `Surgery': Transplantation of Hidden Units (Axel Pinz, Horst Bischof), Technical report, PRIP, TU Wien, 1992.
Bibtex Entry:
@TechReport{PP-Pinz92f,
  author =	 "Axel {Pinz} and Horst {Bischof}",
  institution =	 "PRIP, TU Wien",
  number =	 "PRIP-TR-015",
  title =	 "Neural {N}etwork `{S}urgery': {T}ransplantation of
                  {H}idden {U}nits",
  year =	 "1992",
  url =		 "ftp://ftp.prip.tuwien.ac.at/pub/publications/trs/tr15.ps.gz",
  abstract =	 "We present a novel method to combine the knowledge
                  of several neural networks by replacement of hidden
                  units. Applying neural networks to digital image
                  analysis, the underlying spatial structure of the
                  image can be propagated into the network and used to
                  visualize its weights (WV-diagrams). This
                  visualization tool helps to interpret the behaviour
                  of hidden units. We notice a process of
                  specialization of certain hidden units, while others
                  remain apparently useless. These units are cut out
                  of one network and replaced by units taken from
                  other networks trained for the same task using
                  different parameters. We achieve better prediction
                  accuracies for the new, combined network than for
                  any of the two original ones. This constitutes a
                  special kind of information fusion in image
                  understanding. We give an application example from
                  the field of remote sensing, where neural networks
                  are used to interpret the species of trees in aerial
                  photographs. The interpretation accuracy is raised
                  from 85\% to 90\%.",
}
Powered by bibtexbrowser