EECS 769: Information Theory
  Fall 2018



Schedule:
Lecture period: MWF 12:00-12:50, 3153 Learned Hall

Instructor: Prof. Erik Perrins
Office: 2054 Eaton Hall
Office hours: MWF 10:00-12:00. Drop-ins at other times are always welcome but
are best scheduled via e-mail.
Office phone: 864-7770
E-mail: e s p "a t" k u . e d u



Text: Elements of Information Theory, by T. M. Cover and J. A. Thomas, Wiley-Interscience, Second Edition, 2006.

Syllabus
Day
 Date
 Topic
 Assignment
Mon
 		08/20
 		Course Introduction/Syllabus
Wed 08/22
 		Course Overview (Ch 1)
Fri 08/24 Entropy, Relative Entropy, Mutual Information (Ch 2, pp. 13-25) Compute entropy of English
alphabet
Mon 08/27 Entropy, Relative Entropy, Mutual Information (Ch 2, pp. 13-25)
Wed 08/29 Inequalities: Jensen's, Data Processing, Fano's (Ch 2, pp. 25-41)
Fri 08/31 Inequalities: Jensen's, Data Processing, Fano's (Ch 2, pp. 25-41) 2.5, 2.12, 2.18, 2.21, 2.29, 2.35, 2.43
(due 09/07 at the start of class)
Mon 09/03
 		NO CLASS, LABOR DAY
Wed 09/05 AEP (Ch 3)
Fri 09/07 AEP (Ch 3) 3.2, 3.13
(due 09/14 at the start of class)
Mon 09/10 Guest Lecture
Wed 09/12 Entropy Rate (Ch. 4)
Fri 09/14
 Entropy Rate (Ch. 4)
Mon 09/17 Entropy Rate (Ch. 4)
Wed 09/19 Entropy Rate (Ch. 4)
Fri 09/21 Entropy Rate (Ch. 4) 4.2, 4.7, 4.12
(due 09/26 at the start of class)
Mon 09/24 Data Compression (Ch. 5)
Wed 09/26 Data Compression (Ch. 5)
Fri 09/28 Data Compression (Ch. 5)
Mon 10/01 Data Compression (Ch. 5) TAKE-HOME EXAM BEGIN
Wed 10/03 Data Compression (Ch. 5) TAKE-HOME EXAM DUE
Fri 10/05 Data Compression (Ch. 5)
Mon 10/08 Data Compression (Ch. 5)
Wed 10/10 Data Compression (Ch. 5)
Fri 10/12 Data Compression (pp. 440-443) Data Compression Project
(due 10/24)
Mon 10/15 NO CLASS, FALL BREAK
Wed 10/17 Chapter 7
Fri 10/19 Chapter 7
Mon 10/22 NO CLASS
Wed 10/24 Chapter 7
Fri 10/26 Chapter 7
Mon 10/29 Chapter 7
Wed 10/31 Chapter 7
Fri 11/02 Chapter 7
Mon 11/05 NO CLASS
Wed 11/07 NO CLASS
Fri 11/09 Chapter 7
Mon 11/12 FEC codes 7.4, 7.8, 7.13, 7.29 (due 11/19)
Wed 11/14 Digital and Wireless Communications
Fri 11/16 Symmetric Information Rate (SIR)
Mon 11/19 Symmetric Information Rate (SIR) Symmetric Information Rate Project
(due 11/30)
Wed 11/21 NO CLASS, THANKSGIVING BREAK
Fri 11/23 NO CLASS, THANKSGIVING BREAK
Mon 11/26 SNOW DAY: NO CLASS
Wed 11/28 Gaussian Channel
Fri 11/31 Gaussian Channel
Mon 12/03 Gaussian Channel
Wed 12/05 Gaussian Channel