Serving DeepSeek-V4: why million-token context is an inference systems problem
6mes
- Publicado
- Recolectado
DeepSeek-V4 makes million-token context a serving-systems problem. Together AI explores the inference work behind V4 on NVIDIA HGX B200, including compressed KV layouts, prefix caching, kernel maturity, and endpoint profiles for long-context workloads.
overfeed.news indexa y enlaza. Publicamos un extracto breve; el artículo completo sigue en Together AI Blog.
Más de Together AI Blog
Inicia sesión para seguir esta fuente
Together Compiler: Continuous Compilation for AI inference
Together Compiler uses agents and persistent engines to optimize open models in under 24 hours, delivering up to 3.7x higher throughput as traffic changes.
Polymorphic Inference: self-tuning inference for open models
Models, traffic and chips are constantly changing. Polymorphic Inference adapts to each, and Together Compiler uses AI agents to keep production inference tuned.
Together Compiler: Continuous Compilation for AI inference
Together Compiler uses agents and persistent engines to optimize open models in under 24 hours, delivering 1.3 to 3x higher throughput as traffic changes.
Auto repair for Together GPU Clusters: from GPU fault to healthy node
Together GPU Clusters auto repair detects GPU, host and fabric faults, picks the lightest fix, and lets you decide per fault and per Xid code which repairs run automatically. Here is how it works.