Before you start
Build a Huffman code and return the length in bits of the codeword for each symbol.
Entropy is a lower bound on the average number of bits per symbol that any lossless code can reach. Huffman's greedy algorithm builds a prefix code whose average length is the best possible for codes made of whole bits, and it is always within 1 bit of the entropy.
The algorithm:
The codeword length of a symbol is the number of merges above it, which is its depth in the final tree.
def huffman_lengths(freqs):
Input
huffman_lengths([0.5, 0.25, 0.125, 0.125])
Output
[1, 2, 3, 3]
Reasoning
First the two 0.125 symbols merge into a node of weight 0.25. That node
merges with the 0.25 symbol into a node of weight 0.5, which then merges
with the 0.5 symbol. The last two symbols sit three merges deep, the second
two deep, and the first one deep. The average length is 1.75 bits, which here
equals the entropy exactly.
freqs is a list of positive numbers. They do not have to sum to 1.2^-length
is 1) and that the average length is the optimum.1.freqs.Check runs the visible cases against a fresh kernel. Submit runs the hidden ones too.
⇧↵run cell