Homelabber Runs Massive 2.8-Trillion-Parameter AI Model On Discontinued Intel Optane Memory, Without A GPU
AI-generated summary A hardware enthusiast ran Moonshot AI's Kimi K3, a 2.8-trillion-parameter Mixture-of-Experts model, on a server using discontinued Intel Optane persistent memory and a CPU without GPU acceleration. The experiment achieved initial speeds of 0.85 tokens per second with 1-bit quantization, which increased to 1.50 tokens per second after disabling Direct Access.