You are on page 1of 1

A CLASS OF NEYMAN-PEARSON AND BAYES LEARNING ALGORITHMS FOR NEURAL CLASSIFICATION

Dimims Pados, Tel.: (804)982-2376, E-mail: dap5b@virginia.edu and P. Papantoni-Kazakos, Tel.: (804)924-6102, E-mail: pp0r@virginia.edu Department of Electrical Engineering Thornton Hall University of Virginia Charlottesville, Virginia 22903-2442

Abstract -- In this paper we focus our attention on neural networks whose objective is hypothesis testing. The term hypothesis testing refers to a very broad class of problems, including classification, detection and pattern recognition. There is not much doubt about the way we can measure the performance of a classifier or detector. When the prior probabilities are known, we use the probability of error. When no priors are given, and the hypothesis testing problem is binary, we express the performance in terms of the power and false alarm probabilities. Unfortunately, the existing learning algorithms do not seem to have a lot in common with these performance criteria. Back propagation [l] is a least squares algorithm. Least squares have been the standard approach in estimation theory for a long time, but they are on shaky grounds when applied to hypothesis testing [2]-[3]. On the other hand, the Perception Learning Rule [4],an algorithm that has been proven very successful in classification applications, may have some connection with the probability of error criterion. However, a formal investigation of this issue has not been carried out yet.
Driven by the above observations, we develop two new classes of learning algorithms, specially designed for hypothesis testing and applied to feed forward binary-output neural networks. The first class of algorithms provides optimization in the Neyman-Pearson sense. The second class deals with the probability of error or arbitrarily defined cost

functions, and optimizes the network in the Bayesian sense.

REFERENCES
[l] D.E. Rumelhart, G.E. Hinton and R.J. Williams, "Learning Internal Representations by Error Propagation", in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. I, Foundations. Cambridge, MA: MIT Press,

1986. [2] E. Barnard and D. Casasent, "A


Comparison Between Criterion Functions for Linear Classifiers, with an Application to Neural Nets", IEEE Trans. Syst., Man, Cybern., Vol. 19,pp. 1030-1041, Sept./Oct.

1989. [3] M. Brady, R. Raghavan, and J. Slawny,


"Back Propagation Fails to Separate Where Perceptrons Succeed", IEEE Trans. Circuits and Syst., Vol. 36, pp. 665-674, May 1989. [4] F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Washington, DC: Spartan, 1962.

This work was jointly supported by the NSF grant MSS-9216372 the EPRI contract RP8030and

08.
0 7803-2015-8/94/$4.00 01994 IEEE

- 218 -

Authorized licensed use limited to: Univ Carlos III. Downloaded on March 4, 2009 at 09:29 from IEEE Xplore. Restrictions apply.

You might also like