Serving DeepSeek-V4: why million-token context is an inference systems problem
5mês
- Publicado
- Coletado
DeepSeek-V4 makes million-token context a serving-systems problem. Together AI explores the inference work behind V4 on NVIDIA HGX B200, including compressed KV layouts, prefix caching, kernel maturity, and endpoint profiles for long-context workloads.
O overfeed.news indexa e aponta. Publicamos um trecho curto — o artigo completo fica em Together AI Blog.
Mais de Together AI Blog
Entre para seguir esta fonte
Auto repair for Together GPU Clusters: from GPU fault to healthy node
Together GPU Clusters auto repair detects GPU, host and fabric faults, picks the lightest fix, and lets you decide per fault and per Xid code which repairs run automatically. Here is how it works.
Together Post-Training: frontier-grade model shaping, the way that fits you
Post-train open LLMs like Qwen, GLM and Kimi into custom models you own, with SFT or RL. You write the training loop. Together runs the hard infrastructure.
Meta Muse Spark Open is live on Together AI
Meta Muse Spark Open, Meta's open-weight model for agentic coding, is live on Together AI with a 1M-token context window on serverless and dedicated infrastructure.
Expanding our enterprise inference capacity with IBM Cloud and NVIDIA
Enterprises can now run open models at production scale on a dedicated B300 inference cluster, built by Together AI, IBM Cloud, and NVIDIA