safetensors
A safe, efficient file format for storing and loading machine learning model weights, designed as a safer alternative to pickle-based formats like PyTorch's .pth files.
safetensors is a serialization format developed by Hugging Face for storing tensor data (multi-dimensional arrays) used in machine learning models. Unlike Python's pickle format, which can execute arbitrary code during deserialization, safetensors only deserializes data without running code, making it significantly safer.
The format stores tensors efficiently in a binary layout with a JSON header describing the structure. Example: a language model's weights can be saved as model.safetensors instead of model.pth. Loading is faster and resistant to malicious code injection attacks that could occur with untrusted pickle files.
It's widely used in the machine learning community, particularly with Hugging Face's transformers library and model repositories. The format is language-agnostic, so weights saved in safetensors format can be loaded by different frameworks and programming languages.