A Forgotten 1988 NASA Design, And The Model That Reads Ten Million Tokens In 25 Megabytes
AI-generated summary Researchers from Google and the University of Texas at Austin posted a preprint about a model named Muir that reads ten million tokens. The preprint describes how the 1.34-billion-parameter model keeps a working state of 25 megabytes of float32 numbers.