Information Theory Concepts | Unsupervised Learning for Big Data

Information theory, originally Shannon's mathematical description of communicating via telegraph, has emerged as one of the most consequential ideas of the 20th century. This lecture covers some of the key concepts. This is a part of a series of lectures from the Yale class "Unsupervised Learning for Big Data", taught by Professor Smita Krishnaswamy. Unsupervised learning is perhaps the most beautiful and most frequently astonishing area of machine learning. It doesn't need to guzzle tons of labeled data to solve problems by brute force. Instead, it uses elegant mathematical principles to understand (in some sense) the data itself and the patterns underlying it. Because this is a young field, there's no established textbook. The field of unsupervised learning is a collection of methods, and this course is an introduction to several of the most useful techniques, grounded in an intuitive understanding of the principles underlying them. The tools from this class have been applied to an incredible range of problems, from molecular biology, to financial modeling, to medicine and even astrophysics. We're making these lectures publicly available in an effort to make it easier for anyone to make use of these powerful and elegant techniques in their own research. To learn more about the Krishnaswamy Lab's work, visit krishnaswamylab.org