Problems
Everything below is available both as Postscript and pdf. If you are using a Unix machine, Postscript will probably open faster and look better on the screen, otherwise, you should click on the pdf version. This hasn't been update to 2011-12 yet.
Problem Sheet 1: ps, pdf and source. Solution: ps, pdf, short pdf and source. Entropy. C&T 2.1 and 2.3.
Problem Sheet 2: ps, pdf and source. Entropy and mutual information. Cover and Thomas 2.6, 2.7, 2.10 and 2.12.
- Solution: ps, pdf, short pdf and source.
- This is an audit of the information after each weighing in q2(b), the 12 coin problem, Cover and Thomas problem 2.7: ps, pdf, short pdf and source.
Problem Sheet 3: ps, pdf and source. Solution: ps, pdf, short pdf and source. More entropy, including entropy of functions. Cover and Thomas 2.2, 2.4, 2.8 and 2.14.
Problem Sheet 4: ps, pdf and source. Solution, except q2: ps, pdf, short pdf and source, Solution, q2: ps, pdf, short pdf and source. The data processing inequality, some other inequalities. Cover and Thomas 2.16, 2.43b, and 2.29.
Problem Sheet 5: ps, pdf and source. Solutions: ps, pdf, short pdf and source. Fano. Cover and Thomas 2.32, 2.36, 2.35 and 2.37: relative entropy. The lambda and Jensen Shannon divergences.
Problem Sheet 6: pdf and source. Solutions q1 to q3 ps, pdf, short pdf and source. Cover and Thomas 3.2, 3.3, 3.6 and instructions to go and look at 3.13: AEP.
Problem Sheet 7: pdf and source. Solutions q1 to q2 ps, pdf, short pdf and source. Solution to q3 pdf of a scan. Kraft inequality Cover and Thomas 5.2 and 5.4 and Huffman codes.