fp8
also: 8-bit floating-point, FP8 precision
FP8 is an 8-bit floating-point data type used in machine learning to reduce memory usage and improve computational speed compared to 32-bit or 64-bit precision.
FP8 (8-bit floating-point) is a low-precision numeric format that packs floating-point numbers into just 8 bits of memory. This contrasts with standard float32 (32 bits) or float64 (64 bits), making it highly efficient for GPU and AI accelerator workloads.
Modern GPUs and ML frameworks (PyTorch, TensorFlow) support FP8 to enable faster inference and training with reduced memory bandwidth. For example, a neural network layer using FP8 weights instead of FP32 uses one-quarter the memory, allowing larger batch sizes and faster matrix multiplications.
FP8 trades some numerical precision for performance; it's typically acceptable for inference and quantization tasks where slight accuracy loss is tolerable, but less suitable for high-precision scientific computing. Several FP8 formats exist—the most common are E4M3 (4 exponent, 3 mantissa) and E5M2 (5 exponent, 2 mantissa) as defined by industry standards.