Fix evaluation bias to unlock better models on Amazon Bedrock
1d
- Published
- Collected
Property Finder found that 96.8% of their vision-AI evaluation labels were inherited from the incumbent model, hiding its true accuracy. See how they built unbiased ground truth, applied evolutionary prompt optimization, and selected a more accurate, cost-effective model on Amazon Bedrock, cutting cost by 86% across 11 image-quality attributes.
This post was co-authored with Khaled Thabit, Meriem Boughrara, and Mohamed Desouky from Property Finder. The engineering team at Property Finder , a digital real estate application, discovered that 96.8 percent of their evaluation labels had been inherited directly from a single model’s predictions rather than verified by human experts. Their ground truth was systematically skewed toward the incumbent model, making every alternative appear to underperform. The labels also had errors, but the…
overfeed.news indexes and links. We publish a short excerpt — the full article stays at AWS Machine Learning Blog.
More from AWS Machine Learning Blog
Log in to follow this source
Building an Agentic AI Platform on Amazon Bedrock AgentCore
Stand up a governed, multi-account Agentic AI Platform on Amazon Bedrock AgentCore. This post walks through the architecture and an open-source reference blueprint: the operating model, control plane, per-Workstream cells, separate inference and tool gateways, and a CI/CD evaluation pipeline that gates each agent before production.
Distributed training with Ray Train DLCs and EFA on EKS
The Ray Train Deep Learning Container ships the full CUDA-through-EFA stack, so you build no image. This post walks through a reproducible sample that fine-tunes Qwen2.5-1.5B with PyTorch FSDP across eight GPUs on two EFA-enabled g6.12xlarge instances on Amazon EKS, using the container exactly as published.
Accelerating software delivery with agentic QA automation using Amazon Nova Act – Part 3
As QA test automation spreads across an organization, organizing tests and controlling who owns them becomes the real work. Part 3 of the QA Studio series on Amazon Nova Act adds applications that organize tests and report health metrics, plus team-based user management that controls who can see and change what.
AWS named a Leader in the 2026 Gartner Magic Quadrant for Contact Center as a Service
AWS has been named a Leader in the 2026 Gartner Magic Quadrant for Contact Center as a Service (CCaaS) for AWS Connect Customer. This recognition marks an important milestone for our team, and it reflects years of work pushing on what's possible in customer experience.