Groundhog Day for AI agents: Prompt caching patterns for using OpenAI models on Amazon Bedrock

17d

Antigüedad

Publicado: 26 oct 2026, 17:27 UTC
Recolectado: 26 oct 2026, 17:35 UTC

Every time an agent takes a step, it re-reads its entire context before acting. Here is how prompt caching for OpenAI models on Amazon Bedrock lets your agent remember, cutting input cost and latency without resending the same tools, instructions, and history every turn.

Extracto de la fuente

Every time you spin off your agent, it has to re-read its whole life story before performing an action. Here’s how to make it remember without having to feed it the same context again. In the movie Groundhog Day , Bill Murray’s character re-lives the same February 2 every day. He wakes up to the same song playing on the radio, in the same inn, and the same small town, again and again for what some movie fans estimate to be 34 years. Nothing he accomplished the day before carries over unless he…

Lee el artículo completo en aws.amazon.com

overfeed.news indexa y enlaza. Publicamos un extracto breve; el artículo completo sigue en AWS Machine Learning Blog.

Más de AWS Machine Learning Blog

Inicia sesión para seguir esta fuente

AWS Machine Learning Bloghace 59 minutos

Four Amazon Bedrock inference choices that affect cost

Your Amazon Bedrock inference bill can keep climbing even after you tune prompts and application logic. Four request-level choices (cache duration, service tier, reasoning effort, and maxTokens) change how many tokens the service processes and the rate applied. We show how we tested each and how to verify the effect.

AWS Machine Learning Bloghace 2 horas

Tokenomics at scale: Track and cap Amazon Bedrock spend per IAM principal with CUR 2.0 and AWS Budgets

Get per-identity Amazon Bedrock cost visibility and enforceable spend limits using only AWS-native features, with no application code for the common case. Attribute spend with IAM principal tags, report with CUR 2.0 or Cost Explorer, and enforce with AWS Budgets actions that apply a Deny policy when a threshold is crossed.

AWS Machine Learning Bloghace 3 horas

Optimizing cost per task and proving agent ROI on Amazon Bedrock

Comparing per-token prices hides the real cost of running a model on Amazon Bedrock. This post shows how to calculate the true cost per task across four hidden cost drivers, reduce it with caching, routing, and output constraints, and prove agent ROI with human baselines, holdout experiments, and a five-dimension scorecard.

AWS Machine Learning Bloghace 3 horas

Building a self-service LLM gateway with Amazon Bedrock and LiteLLM

Build a self-service LLM gateway with LiteLLM on AWS Fargate that puts Amazon Bedrock foundation models behind an OpenAI-compatible API. Developers get instant access through a single API key, while platform teams gain per-user cost controls, quota isolation, and observability.

Más de AWS Machine Learning Blog

Four Amazon Bedrock inference choices that affect cost↗

Tokenomics at scale: Track and cap Amazon Bedrock spend per IAM principal with CUR 2.0 and AWS Budgets↗

Optimizing cost per task and proving agent ROI on Amazon Bedrock↗

Building a self-service LLM gateway with Amazon Bedrock and LiteLLM↗

Four Amazon Bedrock inference choices that affect cost

Tokenomics at scale: Track and cap Amazon Bedrock spend per IAM principal with CUR 2.0 and AWS Budgets

Optimizing cost per task and proving agent ROI on Amazon Bedrock

Building a self-service LLM gateway with Amazon Bedrock and LiteLLM