Introducing FlashPack: Lightning-Fast Model Loading for...
Introducing FlashPack: Lightning-Fast Model Loading for PyTorchThe FlashPack package dramatically speeds up PyTorch model loading by flattening all weights into a single contiguous stream, memory-mapping the file, and overlapping disk, CPU, and GPU operations with CUDA streams. This approach yields...