No, Engrams won't let you run 1T models locally. It does something even better.

Key points

1. Contrary to viral claims, N-gram speculative decoding does not allow running 1 trillion parameter models on a single server by offloading 980 billion parameters to SSD.
2. The technique reduces latency at normal model sizes but does not change the fundamental memory requirements for large models.