Open-source chip design runs real LLMs on FPGAs with bit-exact precision

You can now run a real language model on an FPGA using an open-source chip design that makes zero math errors.

Sigmantic AI released APEX, an open-source silicon design that runs Qwen2.5-0.5B directly on FPGA hardware. Every hardware calculation matches a Python reference model bit for bit. On real hardware, it reached a measured 0.56 tokens per second—a 140× performance climb over initial tests.

Why it matters: Most hardware treats conversation memory—the KV cache—as a software problem. APEX bakes memory compression directly into the hardware pipeline, so reading speeds stay steady even as prompt length grows.

The design focuses on a single transformer layer that can scale up to 7B-parameter models. The project gathered over 600 GitHub stars in its first week.

It is still early, but exact beats fast when proving a chip design actually works.