Date: 28.08.2025
BitsAndBytes ROCm Compatibility
The ROCm support still in beta and exist only in one specific branch
Few latest BitsAndBytes releases 0.43.3 0.44.1, 0.45.5, 0.46.0 - NOT INCLUDES ROCM support.
To build and install BitsAndBytes for ROCm please use AMD BitsAndBytes fork AMD/ROCm/bitsandbytes
Matrix for ROCm backend
| BNB Version | ROCm Toolkit | Min AMD GFX | PyTorch |
|---|---|---|---|
| rocm_enabled_multi_backend (start from version 0.43.3) | 6.1 + | gfx906 | 2.2 + |