How overfeed.news collects and publishes its data

The feed side of the site tracks curated sources: official company blogs, two technology newsrooms, arXiv, Hugging Face model releases and the AI threads on Hacker News. A collector polls each source about every fifteen minutes, around the clock. Curation happens at the source level, never per story: if a tracked source publishes it, it appears here, in the section its source belongs to.

Every item is deduplicated by its canonical URL and by a hash of its content, so a story syndicated to two feeds shows up once. The original payload of each fetch is kept unchanged, which is what lets us re-check an item against what the source actually sent.

We never republish full text. For press sources we keep the headline, the link and up to 25 words; for official company blogs up to 60 words; for research papers the abstract, which is what indexes have always shown; for Hacker News and model cards the title and link. Images are shown only where the source class allows it. This is a legal line as much as an editorial one: the writing belongs to the people who wrote it, and every entry links back to them. Publishers who want an entry removed can use the removal page.

Prices are collected once a day, not continuously. GPU rental prices come from RunPod's catalogue and Vast.ai's marketplace, stored per card, per provider and per tier, with the number of offers behind each marketplace median. LLM API prices come from OpenRouter's public model list, stored per model with input price, output price, context window and whether the weights are open. A snapshot is written once per day and never edited, so the history is what was true on each date.

Because there is no backfill from any provider, our series start on the day we started collecting. A card or model that appears on our pages for the first time has a history of one point until the next day.

If a number on the site is wrong, we want to know. Write to the contact address on the about page with the page and the figure, and we will check it against the stored snapshot and correct or remove the page.