FP32 (Single Precision) | Floating Point Format Guide

Bit Layout

An FP32 number uses 32 bits divided into three fields:

Sign

S

1 bit

Exponent

E

8 bits

Mantissa

23 bits

The sign bit determines positive (0) or negative (1). The exponent is stored with a bias of 127. The mantissa (also called significand or fraction) stores the fractional part of the number, with an implicit leading 1 for normal numbers.

Overview

FP32, also known as single precision or float, is defined by the IEEE 754 standard. It has been the default floating-point type in most programming languages and hardware for decades.

With 8 exponent bits, FP32 can represent numbers spanning roughly 80 orders of magnitude (from about 10^-38 to 10³⁸). The 23 mantissa bits provide approximately 7.2 decimal digits of precision, which is sufficient for most scientific computing, graphics, and general-purpose applications.

FP32 is used everywhere: CPU arithmetic, GPU shaders, physics simulations, audio processing, and as the default training precision for many neural networks. It is the float type in C/C++, Java, and C#, and numpy.float32 in Python.

Encoding Rules

Normal Numbers

For most values, the number is encoded as:

value = (-1)^sign × 2^{(exponent - 127)} × (1 + mantissa / 2²³)

The leading 1. before the mantissa is implicit: it doesn't need to be stored, giving you an extra bit of precision for free. The stored exponent is biased by adding 127, so an actual exponent of 0 is stored as 127. The bias of 127 is calculated as 2^(e-1) - 1, where e is the number of exponent bits (8).

Subnormal Numbers (Denormals)

When the stored exponent is all zeros (biased exponent = 0), the number is subnormal:

value = (-1)^sign × 2^-126 × (0 + mantissa / 2²³)

Subnormals have no implicit leading 1, and use a fixed exponent of -126. They allow "gradual underflow," where numbers can get closer to zero than the smallest normal number, at the cost of losing precision.

Special Values

Zero: Exponent = 0, Mantissa = 0. Both +0 and -0 exist (sign bit can be 0 or 1).
Infinity: Exponent = 255 (all ones), Mantissa = 0. Positive or negative infinity based on sign bit.
NaN (Not a Number): Exponent = 255, Mantissa ≠ 0. Results from undefined operations like 0/0 or √(-1).

Tip for beginners Think of the exponent as a "zoom level" and the mantissa as the "detail" at that zoom level. A larger exponent lets you represent bigger (or smaller) numbers, while more mantissa bits give you finer granularity between consecutive values.

Interactive Value Visualizer

Click any bit to flip it, drag the slider, or enter a decimal or hex value. The graphs show how values are distributed across the encoding space.

Decimal:

Hex:

Dynamic Range & Precision

Property	Value
Max Positive (Normal)	3.402823466e+38
Min Positive (Normal)	1.175494351e-38
Max Subnormal	1.175494211e-38
Min Positive (Subnormal)	1.401298464e-45
Machine Epsilon (at 1.0)	1.192092896e-7
Approx. Decimal Digits	~6.9
Exponent Bias	127
Exponent Range	2^-126 to 2^127
Total Bit Patterns	4,294,967,296
Supports Infinity	Yes
Supports NaN	Yes

Special Values & Bit Patterns

Key values and their exact binary representations:

Value	Bit Pattern	Decimal
+0	0 00000000 00000000000000000000000	0
-0	1 00000000 00000000000000000000000	-0
Min Subnormal	0 00000000 00000000000000000000001	1.401298464e-45
Max Subnormal	0 00000000 11111111111111111111111	1.175494211e-38
Min Normal	0 00000001 00000000000000000000000	1.175494351e-38
Max Normal	0 11111110 11111111111111111111111	3.402823466e+38
+Infinity	0 11111111 00000000000000000000000	+Infinity
-Infinity	1 11111111 00000000000000000000000	-Infinity
NaN	0 11111111 10000000000000000000000	NaN

Format Comparison

See how FP32 compares to related floating-point formats:

Property	FP32	FP64	FP16	BF16
Total Bits	32	64	16	16
Sign Bits	1	1	1	1
Exponent Bits	8	11	5	8
Mantissa Bits	23	52	10	7
Bias	127	1023	15	127
Max Value	3.402823466e+38	1.797693135e+308	65504	3.389531389e+38
Min Normal	1.175494351e-38	2.225073859e-308	0.00006103515625	1.175494351e-38
Epsilon	1.192092896e-7	2.220446049e-16	0.0009765625	0.0078125
~Decimal Digits	~6.9	~15.7	~3.0	~2.1
Has Infinity	Yes	Yes	Yes	Yes
Has NaN	Yes	Yes	Yes	Yes
Has Subnormals	Yes	Yes	Yes	Yes

Where FP32 Is Used

GPU and CPU compute: FP32 (.f32) is the most heavily optimized arithmetic type across all GPU and CPU architectures. The PTX ISA uses .f32 as the default MMA accumulator output for FP16, BF16, TF32, and FP8 Tensor Core inputs.
Mixed-precision training: NVIDIA's mixed-precision guide defines FP32 as the baseline accumulator and master-weight type. Optimizer states and weight updates are kept in FP32 to preserve fidelity across many small gradient steps.
ML frameworks: FP32 is the default dtype for tensor creation in PyTorch (torch.float32). The ONNX specification defines it as FLOAT = 1, the first entry in the data type enum.
Compiler infrastructure: The MLIR arith dialect uses f32 as a core type for arithmetic operations. In Triton, lower-precision divisions are automatically upcast to tl.float32 to prevent precision loss.
Accumulation for low-precision ops: The ml_dtypes library recommends FP32 as the accumulation upgrade path for BF16 and FP8 operations (e.g., vals.sum(dtype='float32')).