vllm v0.29.0
2mo
- Published
- Collected
v0.29.0 Highlights This release features 594 commits from 277 contributors (91 new)! Model Runner V2 is now the default for all models (53183), completing the rollout that began with pooling models (48290). MRV2 also gained CUDA graph memory profiling for KV cache auto-sizing (53306), batch-sharded sampling that cuts per-step logits memory by 1/TP (50465), prompt embeds (42963), extract_hidden_states speculation (49811)…
overfeed.news indexes and links. We publish a short excerpt — the full article stays at GitHub Releases (AI inference).
More from GitHub Releases (AI inference)
Log in to follow this sourceollama v0.40.16
Ripple Preview [Ripple Preview](https://ollama.com/library/ripple-preview) from Ricursive Intelligence is a 65B dense model built for agentic coding. sh ollama run ripple-preview What's Changed - Qwen3.8 Flash Next uses about 27 GiB less memory on Apple silicon - ollama launch claude always routes Claude Code to the selected model - ollama launch explains when a cloud model has been retired - /v1/responses…
transformers Patch release v5.22.2
Patch release v5.22.2 Some in between patch for security and distributed fixes. See the included commits :hugs: - [Distributed] Set device_id only for NCCL (50281) by @michaelbenayoun and @3outeille - Harden our Jinja sandboxing (50431) by @Rocketknight1 Full Changelog: https://github.com/huggingface/transformers/compare/v5.22.1...v5.22.2
ollama v0.40.15
MiMo V2.6 Xiaomi MiMo's [mimo-v2.6](https://ollama.com/library/mimo-v2.6), a 9B model built for code and agent workflows, is now supported in Ollama. sh ollama run mimo-v2.6 AutoTrust decision models [AutoTrust](https://ollama.com/library/autotrust-jev) decision models read text and return choices, yes or no probabilities, and ratings for the questions you supply. The family includes 9b, 35b, and 27b-vl sizes — the 35b and 27b-vl sizes can…
ollama v0.40.14
What's Changed ollama launch claude now applies the model's output token limit for Claude Code, so cloud models no longer run into the 128k default cap Cloud web search responses on /api/chat are now decoded correctly, including compressed responses OpenAI-compatible endpoints no longer override a model's packaged sampler defaults — clients that send no temperature/top_p get the model's own values…