Garlic LLM Runs 9B Parameter AI on PS4 at 43 Tokens/Sec
AI-generated summary Software developer Leonardo Alegría released Garlic LLM, an open-source inference engine that runs Alibaba's 9-billion-parameter Qwen3.5-9B model offline on a jailbroken Sony PlayStation 4 at up to 43 tokens per second. The engine bypasses conventional graphics drivers to communicate directly with the console's AMD GPU and uses mixed precision quantization to fit the model into the system's shared GDDR5 memory.