Reduce RAG costs on Amazon Bedrock with query-aware compression

Extracto de la fuente

Input tokens sent to the foundation model (FM) on every call are often a meaningful part of the cost of running Retrieval Augmented Generation (RAG) at scale. Query-aware compression offers one way to reduce how many of them reach the model. Amazon Bedrock provides the foundation models and features to build RAG applications. RAG retrieval usually tunes for high recall, returning a broad set of potentially relevant chunks so the primary model has thorough source material to work with. This…