İvme-Conversate — Tiny Language Models
A family of sub-130M-parameter language models from IvmeLabs: autoregressive base models, a Python-only coder model, and experimental masked-diffusion models.
Autoregressive base model, general text. Deeper/narrower architecture than v2 with QK-Norm + XSA attention changes and a digit-atomic tokenizer. Improves coherence (BLiMP) and perplexity over v2; ARC-Easy knowledge recall is flat as a disclosed tradeoff. Not instruction-tuned — continues text, doesn't chat. Arithmetic is still frequently wrong in free-form generation.
Masked-diffusion sampling: the model denoises a fully-masked span over a fixed number of steps rather than decoding left-to-right.
Run the same prompt and settings through both autoregressive base models at once to see the effect of v3's deeper/narrower architecture and attention changes, plus the benchmark deltas below. (v1, Coder-v1, and the diffusion models aren't included here — try them individually in the Playground tab.)
Benchmark improvement, v2 → v3
WikiText-2 byte perplexity ↓ | 75.09 | 78.49 | +0.09 (+4%) |
Expİvme-DiffusionConversate-v1-Instruct
🧪 Experimental. A 130M-parameter masked-diffusion model, SFT'd for basic chat. Per the model card: output is not reliably grammatical — expect locally plausible word choice that often doesn't compose into coherent sentences. Included here in the spirit of the model card's own honesty about its limitations, not as a working assistant.