Information Theory, Pattern Recognition, and Neural Networks

David MacKay, University of Cambridge

A series of sixteen lectures covering the core of the book "Information Theory, Inference, and Learning Algorithms (Cambridge University Press, 2003)" which can be bought at Amazon, and is available free online. A subset of these lectures used to constitute a Part III Physics course at the University of Cambridge.

Introduction to information theory
* The possibility of reliable communication over unreliable channels. The (7,4) Hamming code and repetition codes.

Entropy and data compression
* Entropy, conditional entropy, mutual information, Shannon information content. The idea of typicality and the use of typical sets for source coding. Shannon's source coding theorem. Codes for data compression. Uniquely decodeable codes and the Kraft-MacMillan inequality. Completeness of a symbol code. Prefix codes. Huffman codes. Arithmetic coding.

Communication over noisy channels
* Definition of channel capacity. Capacity of binary symmetric channel; of binary erasure channel; of Z channel. Joint typicality, random codes, and Shannon's noisy channel coding theorem. Real channels and practical error-correcting codes. Hash codes.

Statistical inference, data modelling and pattern recognition
* The likelihood function and Bayes' theorem. Clustering as an example

Approximation of probability distributions
* Laplace's method. (Approximation of probability distributions by Gaussian distributions.)
* Monte Carlo methods: Importance sampling, rejection sampling, Gibbs sampling, Metropolis method. (Slice sampling, Hybrid Monte Carlo, Overrelaxation, exact sampling)
* Variational methods and mean field theory. Ising models.

Neural networks and content-addressable memories
* The Hopfield network.

  • Свободный график
Характеристики онлайн курса:
  • Бесплатный:
  • Платный:
  • Сертификат:
  • MOOC:
  • Видеолекции:
  • Аудиолекции:
  • Email-курс:
  • Язык: Английский Gb


Пока никто не написал отзыв по этому курсу. Хотите быть первым?

Зарегистрируйтесь, чтобы оставить отзыв

Входит в подборки курсов:
Small-icon.hover Machine Learning
Machine learning: from the basics to advanced topics. Includes statistics...
Ещё курсы на эту тему:
Small-icon.hover Fundamentals of Electrical Engineering
This course probes fundamental ideas in electrical engineering, seeking to understand...
Informationtheory Information Theory
This course is an introduction to information theory, which emphasizes fundamental...
Mooc_statistical_mechanics_2609 Statistical Mechanics: Algorithms and Computations
In this course you will learn a whole lot of modern physics (classical and quantum...
15-450f10 Analytics of Finance
This course covers the key quantitative methods of finance: financial econometrics...
6-451s05 Principles of Digital Communication II
This course is the second of a two-term sequence with 6.450. The focus is on...
Ещё из рубрики «Компьютерные науки»:
Maxresdefault CS 282: Principles of Operating Systems II: Systems Programming for Android
Developing high quality distributed systems software is hard; developing high...
Banner_ruby Ruby on Rails Tutorial: Learn From Scratch
This post is part of our “Getting Started” series of free text tutorials on...
Logo-30-128x128 NYU Course on Deep Learning (Spring 2014)
Lectures from the NYU Course on Deep Learning (Spring 2014) This is a graduate...
Cppgm C++ Grandmaster Certification
The C++ Grandmaster Certification is an online course in which participants...
Umnchem Computational Chemistry (CHEM 4021/8021)
Modern theoretical methods used in study of molecular structure, bonding, and...

© 2013-2019