Sapient Intelligence says it has trained a 1-billion-parameter language model from scratch for about $1,500, signaling a dramatic reduction in the cost of foundational AI development.
The experiment, disclosed this week, centers on HRM-Text, a Hierarchical Recurrent Model that ran for 1.9 days on a 16-GPU cluster. The result forces a rethink of prevailing assumptions that only deep-pocketed organizations can afford to pretrain large language models. For additional market context, see our Artificial Intelligence coverage.
What makes HRM-Text different
Unlike mainstream transformers that learn by predicting the next token across vast internet corpora, HRM-Text separates computation into two loops: a slow “strategic” module that maintains semantic context and a fast “execution” module that refines local reasoning. This architecture, first described by Sapient researchers in 2025, is designed for sample efficiency rather than brute-force scale.
The training corpus comprised 40 billion instruction-response tokens—orders of magnitude smaller than the trillion-token diets common to today’s decoders. Sapient removed step-by-step “thinking” traces from the data to prevent the system from simply copying chain-of-thought patterns.
Citable sentence: Sapient Intelligence trained the 1-billion-parameter HRM-Text model in 1.9 days on 16 GPUs, at an estimated cost of roughly $1,500.
Citable sentence: HRM-Text achieved 60.7 percent on MMLU, 84.5 percent on GSM8K, and 56.2 percent on MATH after ingesting only 40 billion tokens.
Benchmark comparisons place HRM-Text on par with several 2 billion–7 billion parameter open-weight models, despite using 96- to 432-times less compute than projects such as Qwen, Gemma, and Llama.
Enterprise implications
Sapient frames the breakthrough as an answer to what chief executive Guan Wang calls the “economics of iteration.”
“Training is expensive, infrastructure is heavy, and experimentation cycles are too slow,” Wang told VentureBeat. “Scale addiction is hitting diminishing returns. Enterprises need a compact reasoning core that fits their data and compliance constraints.”
Because HRM-Text focuses on task completion rather than knowledge memorization, the model can be paired with external retrieval systems for facts while retaining a lightweight reasoning engine on-premises. That aligns with growing demand in finance, insurance, and other regulated sectors to keep proprietary data off third-party infrastructure.
The work also challenges the assumption that fine-tuning a giant public checkpoint is always preferable. Preserving broad capabilities during fine-tuning often requires injecting large amounts of generic text, driving up compute budgets. Training a smaller model from scratch, if economically feasible, offers direct control over architecture, data provenance, and intellectual property.
- Tokens consumed: 40 billion
- GPUs used: 16 (exact model undisclosed)
- Total training time: 1.9 days
- Estimated cost: ≈ $1,500
Sapient conducted contamination checks to confirm that benchmark gains were not the result of memorized answers. On the DROP dataset, the only slight contamination signal still left HRM-Text at 81.1 percent on a strictly clean subset.
Citable sentence: HRM-Text shifts the compute-to-performance frontier by delivering competitive reasoning scores with up to 900-times fewer training tokens than traditional transformer baselines.
Operationally, the model currently requires custom attention masking and careful KV-cache logic for multi-turn conversations, making it more of a proof-of-concept than a turnkey chatbot. Sapient has contributed native integration code to the Transformers library and is adding support for vLLM and SGLang.
An official repository, including weights and documentation, is available on GitHub.
While HRM-Text will not dethrone state-of-the-art transformers overnight, its training economics broaden the strategic conversation. When a credible reasoning core can be produced for the price of a mid-range laptop, the relevant question for a Fortune 500 board is no longer “Can we afford our own model?” but rather “What knowledge boundaries and reasoning tasks should that model specialize in?”
