What exactly does word2vec learn?
1y
- Published
- Collected
What exactly does word2vec learn, and how? Answering this question amounts to understanding representation learning in a minimal yet interesting language modeling task. Despite the fact that word2vec is a well-known precursor to modern language models, for many years, researchers lacked a quantitative and predictive theory describing its learning process. In our new paper , we finally provide such a…
overfeed.news indexes and links. We publish a short excerpt — the full article stays at Berkeley AI Research Blog.
More from Berkeley AI Research Blog
Log in to follow this source
From CUDA to MLX: How K-Search Brings Decades of Kernel Expertise to Apple Silicon
Figure 1: CUDA-to-MLX optimization translation map. CUDA optimization knowledge can be translated into architecture-native MLX strategies rather than copied instruction-for-instruction. We face a new epoch in computing. Hardware is changing rapidly — not just faster GPUs, but a growing range of chips from different vendors, each with its own architecture and often tailored to specific AI workloads. Software is changing…
Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction
Overview of ABBEL compared to traditional recursive summarization. Beliefs replace the full interaction history as the agent’s working context, and belief grading improves performance by supervising the contents of each belief state.. As task horizons grow, LLM contexts can’t scale forever. Self-summarization enables concise, interpretable contexts, but at a significant performance cost, especially for human assistance domains where high quality…
Intelligence is Free, Now What? <br> Data Systems for, of, and by Agents
... government of the people, by the people, for the people ... — Abraham Lincoln, Gettysburg Address (1863) The cost of AI is dropping rapidly. GPT-4-class capabilities cost roughly 30 per million tokens in early 2023; today the same runs under 1 , and some providers are pushing costs below $0.10 . Across benchmarks, inference prices have fallen between 9x…
2026 BAIR Graduate Showcase
Congratulations to the Berkeley Artificial Intelligence Research (BAIR) Lab class of 2026! This year, BAIR celebrates another remarkable group of Ph.D. graduates whose curiosity, creativity, and perseverance have pushed the frontiers of artificial intelligence and machine learning. Their work spans the breadth of modern AI — robotics and embodied intelligence, large language models and reasoning, computer vision, generative modeling, AI…