Date: 01.09.2025

Instruction how to convert PyTorch binary weights from bin to safetensors

Why

It required by PyTorch community, because safetensors will not allow to execute any code inside weights, but it is possible in old binary format

How

General instruction

Convert mannualy pytorch_model.bin to model.safetensors with torch.load() and safetensors.torch.save_file()

 import torch
from safetensors.torch import save_file

src = "/home/sysadmin/llm/bark/pytorch_model.bin"
dst = "/home/sysadmin/llm/bark/model.safetensors"

# важно: weights_only=True (см. предупреждение torch)
sd = torch.load(src, map_location="cpu", weights_only=True)

new_sd = {}
seen = {}  # ключ: (data_ptr, size, dtype, shape, stride)
for k, t in sd.items():
    if not isinstance(t, torch.Tensor):
        continue
    stg = t.untyped_storage()
    sig = (stg.data_ptr(), stg.size(), t.dtype, tuple(t.size()), tuple(t.stride()))
    if sig in seen:
        # if tensor uses shared memory copy it
        new_sd[k] = t.clone()  # or t.contiguous().clone()
    else:
        seen[sig] = k
        new_sd[k] = t  # original one keep without changes

# safe safetensors without shared memory with metadata 
save_file(new_sd, dst, metadata={"format": "pt"})
print("Saved:", dst)