Estimation theory gives one approach to characterizing random variables. This was based on building parametric models and describing the data by the parameters. An alternative approach is given by information theory. Here the emphasis is on coding. We want to code the observations. The observations can then be stored in the memory of a computer, or transmitted by a communications channel, for example.