New:Online KL Shampoo - 1.45–1.59× parameter efficiency over Muon

Train 1.5× larger models on the same compute

Breve is the Efficient Training Network - the open, production-grade layer from Tilde Research that turns their world-class optimizers and architectures into the default way the world trains AI models.

Open DashboardRead the code

Built by Tilde Research - the lab behind OKLS and Wall Attention

train.py
# drop in OKLS - zero-staleness, KL-optimal
from breve import OKLS

optimizer = OKLS(
    model.parameters(),
    lr=0.02,
    mup=True,  # zero-shot LR transfer across scales
)
$ breve run train.py --params 8B --cluster shared
step  450 · loss 2.931 · 98.2% Muon throughputstep  900 · loss 2.714 · ≈ Muon @ 12B paramsstep 1350 · loss 2.588 · lr transfer ✓
1.45–1.59×
param efficiency
98%
throughput kept
0
staleness
  • Khosla Ventures
  • Bain Capital Ventures
  • Forgepoint
  • Soma Capital
  • Timeless

Backed by - alongside individual researchers from OpenAI and Anthropic

The problem

Frontier training runs on legacy optimizers.

Most AI labs still train with AdamW - an optimizer from 2017 - or Muon-class baselines that only loosely approximate curvature-aware updates. The result is structural waste: 30–40% more compute burned to reach the same loss.

Tilde Research has repeatedly shown that better optimizers and co-designed architectures deliver 1.45–1.59× parameter efficiency while keeping nearly the same training speed. The science exists. What's missing is the production layer that makes it usable at scale - by everyone.

Read the research
AdamW · 2017

Diagonal preconditioning. Ignores cross-parameter curvature entirely - the default simply because it is old, not because it is good.

Muon · strong baseline

Matrix-orthogonalized momentum. Better - but still a stale, lossy approximation of the true curvature signal.

OKLS · Tilde Research

Zero-staleness, KL-optimal Kronecker-factored approximation of full-matrix AdaGrad. Matches Muon models ~1.5× larger.

The solution

A specialized efficiency layer.
Not another compute marketplace.

Breve packages Tilde Research's breakthroughs into battle-tested libraries and kernels, runs them on shared high-performance infrastructure, and closes the loop with community contributions that get verified and merged into the official stack.

Production optimizer kernels. OKLS and its successors, hardware-aligned and drop-in for PyTorch training stacks.

Automatic scaling tools. First-principles muP rules - optimal hyperparameters transfer zero-shot across model sizes.

Verified runs & leaderboards. Shared training infrastructure scored purely on efficiency - loss per FLOP, not FLOPs spent.

A continuous improvement loop. Community kernels, numerical methods, and co-designed architectures get verified and integrated.

Proven results

Published, measured, open. Built by the people who actually invent the methods.

1.45–1.59×

Parameter efficiency vs. Muon - an OKLS-trained model matches a Muon model roughly 1.5× larger

%

Training throughput retained - better models at nearly identical wall-clock speed

4k → k+

Length extrapolation with Wall Attention - trained at 4k, generalizes beyond 160k

Flagship optimizer

Online KL Shampoo (OKLS)

A zero-staleness, KL-optimal Kronecker-factored approximation of full-matrix AdaGrad, powered by the hardware-aligned Scaled CANS Coupled Newton–Schulz algorithm. Ships with a first-principles muP scaling rule for zero-shot learning-rate transfer.

Ashley Zhang*, Ben Keigwin*, Dhruv Pai*, Alec Dewulf - *Core Contributors

Blog Code
Architecture

Wall Attention

A new attention mechanism that generalizes diagonal forget gates to softmax attention. Dramatically better length extrapolation, outperforming RoPE and FoX - with production-ready Triton kernels.

Ashley Zhang - Core Contributor

Blog Code
How it works

From paper to production.
One network, five primitives.

Ready-to-use kernels. High-performance OKLS kernels - Scaled CANS Coupled Newton–Schulz, hardware-aligned, numerically verified.

// scaled-cans coupled newton–schulz
X ← G / ‖G‖F
for t = 1…5:
X ← a·X + b·(XXᵀ)X + c·(XXᵀ)²X
bf16 fusedzero-stalenessKL-optimal
OKLS
98%
Muon
100%

training throughput, relative

Automatic shape & LR scaling. muP-style tooling and beyond - tune once at small scale, transfer optimal hyperparameters zero-shot to frontier runs.

2565121k2k4k8klrwidthoptimal lr - transferred zero-shot (muP)naive scaling drifts

Seamless integration. Drop-in for existing PyTorch training frameworks - change three lines, keep your stack.

Memory-optimized primitives. Ownership-based distributed training and memory optimizations developed by Tilde.

Early research access. New Tilde releases - optimizers, architectures, kernels - land on Breve first.

Bounties & verification. Contribute kernels, numerical methods, or scaling rules - accepted work earns rewards.

Token utility

One token, aligned with efficiency.

Core optimizers and kernels stay fully open source. The token powers the network layer on top - compute, priority access, and governance.

  • 01

    Pay for training runs

    Spend tokens on training runs and priority compute across the network’s shared infrastructure.

  • 02

    Access premium kernels

    Unlock the latest Tilde kernels and tools before they hit the open-source releases.

  • 03

    Stake for priority & governance

    Stake for priority access and a voice in research priorities across the network.

  • 04

    Earn by contributing

    Accepted improvements - kernels, numerical methods, scaling rules - earn token rewards.

  • 05

    Compete & verify

    Participate in verification and efficiency competitions scored on real training metrics.

Built by

The people who actually invent the methods.

Breve is built by Tilde Research - a moonshot AI lab advancing mechanistic interpretability, new architectures, and pretraining science. Small, elite team. San Francisco / Palo Alto.

AZ

Ashley Zhang

Research Scientist, Tilde Research

Next-gen LLM architecture and optimizer research. Corresponding author and Core Contributor on Online KL Shampoo; Core Contributor on Wall Attention.

@ashley11829 ashleyzhang@tilderesearch.com

Tilde Research

Moonshot AI research lab

“We build foundational understanding of models to advance the frontier of intelligence.” Backed by Khosla Ventures, Bain Capital Ventures, Forgepoint, Soma Capital, Timeless, and researchers from OpenAI and Anthropic.

tilderesearch.com @tilderesearch LinkedIn

Open source foundations

Every core method ships as open code first. Breve builds the verified, production-grade layer on top.

github.com/tilde-research

8 major releases and counting - optimizers, attention mechanisms, and kernels.

Online KL Shampoo

Zero-staleness, KL-optimal Kronecker-factored optimizer.

Wall Attention

Forget-gate attention with 4k → 160k+ extrapolation.

Aurora

Next-generation optimizer from Tilde Research.

MoMoE

Memory-optimized Mixture-of-Experts.

NSA Kernels

Native Sparse Attention kernels.

Nitrobrew

Fast distillation for production models.

Compositional Muon

Composable building blocks on the Muon baseline.

One Layer Deeper

The open efficiency competition - onelayerdeeper.ai.

Frequently Asked Questions

Can't find what you're looking for? Contact us at join@tilderesearch.com

Train at the frontier, for less.

Join the waitlist for early access - or start with the open-source kernels today.

Open DashboardFollow @tilderesearch