Probability and Information: An Integrated Approach

This updated textbook is an excellent way to introduce probability and information theory to new students in mathematics, computer science, engineering, statistics, economics, or business studies. Only requiring knowledge of basic calculus, it starts by building a clear and systematic foundation to the subject: the concept of probability is given particular attention via a simplified discussion of measures on Boolean algebras. The theoretical ideas are then applied to practical areas such as statistical inference, random walks, statistical mechanics and communications modelling. Topics covered include discrete and continuous random variables, entropy and mutual information, maximum entropy methods, the central limit theorem and the coding and transmission of information, and added for this new edition is material on Markov chains and their entropy. Lots of examples and exercises are included to illustrate how to use the theory in a wide range of applications, with detailed solutions to most exercises available online for instructors.