Generative modeling with sparse transformers
7y
Age
- Published
- Collected
We’ve developed the Sparse Transformer, a deep neural network which sets new records at predicting what comes next in a sequence—whether text, images, or sound. It uses an algorithmic improvement of the attention mechanism to extract patterns from sequences 30x longer than possible previously.
overfeed.news indexes and links. We publish a short excerpt — the full article stays at OpenAI News.
More from OpenAI News
Log in to follow this sourceBetter answers, broader thinking: What students gain from ChatGPT and critical-thinking training
A randomized study of more than 1,000 students examines ChatGPT, critical thinking, originality, and student performance on a real-world university assignment.
Expanding OpenAI’s presence in Brazil
OpenAI is expanding its presence in Brazil, deepening engagement with developers, businesses, and communities to support AI adoption across the country.
Learning never stops: How AI makes learning continuous
OpenAI’s new report explores how students and educators use ChatGPT to make learning more continuous, with support that extends beyond the classroom.
Bringing ChatGPT for Teachers to more U.S. school districts
ChatGPT for Teachers is expanding to 55 U.S. school systems, bringing secure AI tools, training, and support to over 100,000 more educators and staff.