Why AI Uses Different Number Precisions

Every lesson so far has ended at the same bottleneck: moving bytes. So here’s a tempting idea. What if each number simply took up fewer bytes?

Computers store decimal numbers as floating point, a kind of scientific notation in binary. Each number has three parts: a sign (plus or minus), an exponent (how big: is this 0.001 or 1,000,000?) and a mantissa (the digits themselves). The classic format, FP32, spends 32 bits, 4 bytes, on every number.

Neural networks turn out to be remarkably relaxed about rounding. A weight of 0.4183 or 0.42 barely changes the answer, so modern AI mostly uses 16 bits, and increasingly 8 or even 4. Halve the bits and you halve the memory, halve the bytes to move (straight relief for the memory wall) and, on the right hardware, double the maths speed.

The trick is choosing which bits to give up. Exponent bits buy range: with too few, big numbers overflow into “infinity” and tiny ones vanish to zero. Mantissa bits buy precision: with too few, nearby numbers blur together. Look at the two 16-bit formats in the picture. FP16 keeps more digits but tops out at 65,504. BF16, designed at Google for deep learning, keeps FP32’s full range and gives up digits instead, because training is wrecked by an overflow but shrugs off a little blur. That’s why BF16 became the workhorse.

Go below 8 bits and floating point runs out of room, so models switch to integers with a scale factor: the quantization you met at the end of Track 2. The next lesson shows the special hardware that turns all these smaller numbers into raw speed.