A Forgotten 1988 NASA Design, And The Model That Reads Ten Million Tokens In 25 Megabytes
brianroemmele.substack.com — The Memory That Never Grows
Pretraining
Nace AI has published a 51-page report on how it trained Drex, its family of open-weight AI models that pick the most l…
AI learns from the products of thinking, not thought itself. That's a problem for machines—and for us
Tech Xplore — When you walk through a gallery, admiring the paintings on the walls, what you are seeing are finished products. They are the result of countless hours of learning and practice and trial and error, but none of that is on display.
GSLM2: Multi-Stream Speech Representation Learning and Generative Spoken Language Modeling
AI learns to predict the world without ever seeing it
Tech Xplore — Almost every artificial intelligence system we know begins in the same way: it is exposed to vast amounts of information from the real world—books, websites, images and recordings—and trained to identify patterns within it. A new study involving researchers from Tel Aviv University and Stanford University raises another possibility: Can a system develop general learning capabilities even when all the data it trains on is generated entirely from scratch within the system itself?
LiberAI Unveils Liber-0, a World Action Model That Grounds Human Experience in Robot Action
The Robot Report — LiberAI introduces Liber-0, a world action model pretrained on more than 2.2 million hours of human experience to support zero-shot robot manipulation.
OpenAI plans to train models on Daybreak Blue cybersecurity session data that was previously off-limits
The Decoder — Starting November 29, OpenAI will use data from Daybreak Blue sessions to train its models, even when users have turned off training. After backlash, a follow-up email promised a separate opt-out. The move revives concerns from the dispute with mathematicians who suspected OpenAI used their unpublished research for training without consent.
From Tokens to Bytes II: Learning from Raw Data
Cartesia — We provide evidence that byte-level language models fundamentally understand the data better, as measured by how pretraining performance translates to downstream tasks.
Dismiss email signup
Subscribe to 
The top AI news in your email delivered each morning. Choose categories