Their latency measurements comparing Mamba-2 and Mamba-3 are done with a batch s... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

yorwba 4 months ago | parent | context | favorite | on: Mamba-3

Their latency measurements comparing Mamba-2 and Mamba-3 are done with a batch size of 128. It doesn't seem like Mamba-2 was compute-bound even at that batch size.

jychang 4 months ago [–]

Well, Deepseek batch sizes are something like 8192, so 128 isn't much.

https://arxiv.org/html/2412.19437v1 "the batch size per expert is relatively small (usually within 256 tokens)"

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact