How AI training scales
7y
Age
- Published
- Collected
We’ve discovered that the gradient noise scale, a simple statistical metric, predicts the parallelizability of neural network training on a wide range of tasks. Since complex tasks tend to have noisier gradients, increasingly large batch sizes are likely to become useful in the future, removing one potential limit to further growth of AI systems. More broadly, these results show that…
overfeed.news indexes and links. We publish a short excerpt — the full article stays at OpenAI News.
More from OpenAI News
Log in to follow this sourceOur decision on Cursor following its acquisition by SpaceX
Our decision to wind down our contract providing OpenAI models to Cursor following its acquisition by SpaceX.
Supporting Thailand’s next generation of AI startups
OpenAI and Thailand’s MHESI launch an eight-week accelerator helping 10 health, wellness, and education startups turn AI prototypes into trusted products.
Better answers, broader thinking: What students gain from ChatGPT and critical-thinking training
A randomized study of more than 1,000 students examines ChatGPT, critical thinking, originality, and student performance on a real-world university assignment.
Expanding OpenAI’s presence in Brazil
OpenAI is expanding its presence in Brazil, deepening engagement with developers, businesses, and communities to support AI adoption across the country.