FP32 (Single Precision) | Floating Point Format Guide

Bit Layout

An FP32 number uses 32 bits divided into three fields:

The sign bit determines positive (0) or negative (1). The exponent is stored with a bias of 127. The mantissa (also called significand or fraction) stores the fractional part of the number, with an implicit leading 1 for normal numbers.

Overview

FP32, also known as single precision or float, is defined by the IEEE 754 standard. It has been the default floating-point type in most programming languages and hardware for decades.

With 8 exponent bits, FP32 can represent numbers spanning roughly 80 orders of magnitude (from about 10^-38 to 10³⁸). The 23 mantissa bits provide approximately 7.2 decimal digits of precision, which is sufficient for most scientific computing, graphics, and general-purpose applications.

FP32 is used everywhere: CPU arithmetic, GPU shaders, physics simulations, audio processing, and as the default training precision for many neural networks. It is the float type in C/C++, Java, and C#, and numpy.float32 in Python.

Encoding Rules

Normal Numbers

For most values, the number is encoded as:

value = (-1)^sign × 2^{(exponent - 127)} × (1 + mantissa / 2²³)

The leading 1. before the mantissa is implicit: it doesn't need to be stored, giving you an extra bit of precision for free. The stored exponent is biased by adding 127, so an actual exponent of 0 is stored as 127. The bias of 127 is calculated as 2^(e-1) - 1, where e is the number of exponent bits (8).

Subnormal Numbers (Denormals)

When the stored exponent is all zeros (biased exponent = 0), the number is subnormal:

value = (-1)^sign × 2^-126 × (0 + mantissa / 2²³)

Subnormals have no implicit leading 1, and use a fixed exponent of -126. They allow "gradual underflow," where numbers can get closer to zero than the smallest normal number, at the cost of losing precision.

Special Values

Zero: Exponent = 0, Mantissa = 0. Both +0 and -0 exist (sign bit can be 0 or 1).
Infinity: Exponent = 255 (all ones), Mantissa = 0. Positive or negative infinity based on sign bit.
NaN (Not a Number): Exponent = 255, Mantissa ≠ 0. Results from undefined operations like 0/0 or √(-1).

Tip for beginners Think of the exponent as a "zoom level" and the mantissa as the "detail" at that zoom level. A larger exponent lets you represent bigger (or smaller) numbers, while more mantissa bits give you finer granularity between consecutive values.

Interactive Value Visualizer

Click any bit to flip it, drag the slider, or enter a decimal or hex value. The graphs show how values are distributed across the encoding space.

Decimal:

Hex:

Dynamic Range & Precision

Special Values & Bit Patterns

Key values and their exact binary representations:

Format Comparison

See how FP32 compares to related floating-point formats:

Where FP32 Is Used

GPU and CPU compute: FP32 (.f32) is the most heavily optimized arithmetic type across all GPU and CPU architectures. The PTX ISA uses .f32 as the default MMA accumulator output for FP16, BF16, TF32, and FP8 Tensor Core inputs.
Mixed-precision training: NVIDIA's mixed-precision guide defines FP32 as the baseline accumulator and master-weight type. Optimizer states and weight updates are kept in FP32 to preserve fidelity across many small gradient steps.
ML frameworks: FP32 is the default dtype for tensor creation in PyTorch (torch.float32). The ONNX specification defines it as FLOAT = 1, the first entry in the data type enum.
Compiler infrastructure: The MLIR arith dialect uses f32 as a core type for arithmetic operations. In Triton, lower-precision divisions are automatically upcast to tl.float32 to prevent precision loss.
Accumulation for low-precision ops: The ml_dtypes library recommends FP32 as the accumulation upgrade path for BF16 and FP8 operations (e.g., vals.sum(dtype='float32')).