Stop guessing auto scaling targets for TTS on Amazon SageMaker AI
6d
- Published
- Collected
Teams deploying custom text-to-speech models on Amazon SageMaker AI often guess at autoscaling thresholds, and a wrong guess either wastes GPU spend or breaches a latency SLO. This post deploys Kokoro-82M as a custom container and derives a measured autoscaling policy from two live-endpoint measurements, Q_max and T_total, instead of guesswork.
A traffic spike hits your self-hosted text-to-speech (TTS) endpoint, and requests start queueing. Latency climbs past your service level objective (SLO) right when the most callers are listening. The auto scaling policy behind that endpoint was supposed to prevent this, but the target value it scales on was a guess, and guesses like that fail silently until real traffic proves them wrong. Get that number wrong, and it’s paid for in one of two ways, and neither shows up as an alarm. Guess too…
overfeed.news indexes and links. We publish a short excerpt — the full article stays at AWS Machine Learning Blog.
More from AWS Machine Learning Blog
Log in to follow this source
Building an Agentic AI Platform on Amazon Bedrock AgentCore
Stand up a governed, multi-account Agentic AI Platform on Amazon Bedrock AgentCore. This post walks through the architecture and an open-source reference blueprint: the operating model, control plane, per-Workstream cells, separate inference and tool gateways, and a CI/CD evaluation pipeline that gates each agent before production.
Distributed training with Ray Train DLCs and EFA on EKS
The Ray Train Deep Learning Container ships the full CUDA-through-EFA stack, so you build no image. This post walks through a reproducible sample that fine-tunes Qwen2.5-1.5B with PyTorch FSDP across eight GPUs on two EFA-enabled g6.12xlarge instances on Amazon EKS, using the container exactly as published.
Accelerating software delivery with agentic QA automation using Amazon Nova Act – Part 3
As QA test automation spreads across an organization, organizing tests and controlling who owns them becomes the real work. Part 3 of the QA Studio series on Amazon Nova Act adds applications that organize tests and report health metrics, plus team-based user management that controls who can see and change what.
AWS named a Leader in the 2026 Gartner Magic Quadrant for Contact Center as a Service
AWS has been named a Leader in the 2026 Gartner Magic Quadrant for Contact Center as a Service (CCaaS) for AWS Connect Customer. This recognition marks an important milestone for our team, and it reflects years of work pushing on what's possible in customer experience.