Here, H(X) = - \textstyle\sum_{i=1}^n p_i \log_2 p_i is the entropy, and Shannon's source coding theorem says that any code must have an average length of at least H(X). Hence we see that the Shannon–Fano code is always within one bit of the optimal expected word length.
