Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I don’t think so, the HF weights are bf16 which means 24GB + cache/overhead.

It sounds like marketing spin where the performance claims are based on BF16 and the “runs in 16GB” claim is on a totally different quantized version.





Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: