Pretraining

AIAI-generated summary Researchers from Google and the University of Texas at Austin posted a preprint about a model named Muir that reads ten million tokens. The preprint describes how the 1.34-billion-parameter model keeps a working state of 25 megabytes of float32 numbers.

AIAI-generated summary Nace AI has released a report detailing the training process for Drex, which is a family of open-weight AI models designed to select the most likely answer from a fixed list. The model called Drex v1.5 was trained using verified answers for tasks including contract reading and phishing detection, and it can be tested within Nace Console.

AIAI-generated summary A new paper published on the arXiv preprint server argues that artificial intelligence models are trained on the traces of thought rather than actual thinking processes. The authors state that this focus on final traces hobbles current artificial intelligence systems and can create similar problems for human users relying on them.

AIAI-generated summary ai.meta.com introduces GSLM2, a generative spoken language model designed to learn linguistic structure purely from speech. ai.meta.com states that the model uses SpidR 2.0 and a multi-stream SLM to improve linguistic competence and scale efficiently with data.

AIAI-generated summary The study by researchers from Tel Aviv University and Stanford University is available on the arXiv preprint server. The research shows that a model can acquire general predictive skills even when it is trained only on data generated within the learning process itself.

AIAI-generated summary LiberAI announced Liber-0, a world action model pretrained on more than 2.2 million hours of human experience to support zero-shot robot manipulation. LiberAI says the model achieved an overall success rate of 36.0% across 60 tasks evaluated in 600 trials without robot demonstrations collected specifically for those target tasks.

AIAI-generated summary OpenAI plans to use Daybreak Blue session data to train its models starting November 29 according to an email sent to Trusted Access program members. The company followed up with a second email promising a separate opt-out after facing swift backlash from researchers regarding the policy change.

AIAI-generated summary Cartesia releases a blog post exploring raw data processing methods and tokenization tradeoffs in language models. Cartesia says BPE tokenization noticeably improves compute-to-loss efficiency compared to byte-level models.