JDMP

David J. C. MacKay
Cambridge University Press 2003

After three introductory chapters, the book is made up of the following seven parts. I Data Compression. II Noisy-Channel Coding. III Further Topics in Information Theory. IV Probabilities and Inference. V Neural networks. VI Sparse Graph Codes. VII Appendices.

Christopher M. Bishop
Oxford University Press 1995

This book provides a solid statistical foundation for neural networks from a pattern recognition perspective. The focus is on the types of neural nets that are most widely used in practical applications, such as the multi-layer perceptron and radial basis function networks. Rather than trying to cover many different types of neural networks, Bishop thoroughly covers topics such as density estimation, error functions, parameter optimization algorithms, data pre-processing, and Bayesian methods.

Richard O. Duda, Peter E. Hart, David G. Stork
Wiley & Sons 2000

Bayesian Decision Theory.
Maximum-Likelihood and Bayesian Parameter Estimation.
Nonparametric Techniques.
Linear Discriminant Functions.
Multilayer Neural Networks.
Stochastic Methods.
Nonmetric Methods.
Algorithm-Independent Machine Learning.
Unsupervised Learning and Clustering.
Appendix.
Index.

John A. Hertz, Anders Krogh, Richard G. Palmer
Perseus Books 1991

This book comprehensively discusses the neural network models from a statistical mechanics perspective. It starts with one of the most influential developments in the theory of neural networks: Hopfield’s analysis of networks with symmetric connections using the spin system approach and using the notion of an energy function from physics. Introduction to the Theory of Neural Computation uses these powerful tools to analyze neural networks as associative memory stores and solvers of optimization problems. A detailed analysis of multi-layer networks and recurrent networks follow. The book ends with chapters on unsupervised learning and a formal treatment of the relationship between statistical mechanics and neural networks.