Improving language understanding with unsupervised learning
8y
- Published
- Collected
We’ve obtained state-of-the-art results on a suite of diverse language tasks with a scalable, task-agnostic system, which we’re also releasing. Our approach is a combination of two existing ideas: transformers and unsupervised pre-training. These results provide a convincing example that pairing supervised learning methods with unsupervised pre-training works very well; this is an idea that many have explored in the…
overfeed.news indexes and links. We publish a short excerpt — the full article stays at OpenAI News.
More from OpenAI News
Log in to follow this source
Franchisee reconciles cash in 94% less time with ChatGPT Work
An independent Schlotzsky’s and Cinnabon franchisee cut menu repricing time by 95% and built a workflow that could free up $100K in cash flow.
Terminal X matches senior investor judgment 88.6% with GPT-6.1
Terminal X’s Eval Agent grades outputs against a 54-criterion rubric, catching 25% more deviances from a fund’s mandate with GPT-6.1 Sol.
OpenAI acquires Orchestra Bio
OpenAI is acquiring Orchestra Bio to bring AI-powered operations to life sciences R&D, helping biopharma teams accelerate drug development and advance new treatments.
Advancing Coding in Low-Resource Programming Languages
Learn how OpenAI is improving ChatGPT across low-resource programming languages, with new evaluations for software engineering, SQLite, and COBOL migration.