Why Artificial General Intelligence (AGI) Needs Context Enrichment
Back in 2017, I interned at AStar Laboratories, working on reinforcement learning agents that could play Atari games. It was fascinating to see these agents dominate highly repetitive games like Pong and Space Invaders, often surpassing even the best human players. But the performance dropped sharply when we introduced them to games like Montezuma’s Revenge or Private Eye. Despite their relatively simple controls, these games presented a challenge rooted in complexity and nuance—game mechanics intertwined with storytelling or real-world concepts.
From a human perspective, the agents excelled at recognising patterns in repetitive scenarios. Yet, they floundered when asked to navigate non-repetitive environments or interpret subtle clues—a requirement for success in exploratory games. Naively, I suggested pre-training these agents on physical models of the world, underestimating the monumental challenge of such a task at the time.

State of the art performance of RL agents on Atari games back in 2015. Source: Nature.com
Fast forward to today, and we’re witnessing a remarkable evolution in AI with large language models (LLMs) and Test-Time Compute systems like GPT-o1 and DeepSeek R1. Unlike the reinforcement learning agents I mentioned earlier, these models have the ability to reason, plan, and execute tasks by drawing on the extensive knowledge they’ve acquired during training. They can construct chains of thought to tackle complex problems and play even harder games like Minecraft by leveraging their internal understanding and representations of the world. This leap in conceptual understanding is transformative and prompts a profound question: What will AGI look like in the future?
The Anatomy of AGI
Yann LeCun, one of AI’s OGs, has outlined a conceptual architecture for an Artificial General Intelligence (AGI) like system that mirrors human-like learning and adaptability. Such a system generally consists of six core components:
Configurator: Handles executive control, preparing other modules for specific tasks.
Perception: Collects real-time signals from the environment, forming an understanding of the current world state.
World Model: Predicts missing information, estimates world states, and forecasts plausible future scenarios.
Cost Module: Quantifies discomfort or misalignment, combining intrinsic cost measures and a trainable critic.
Actor: Proposes actions that minimise future costs.
Short-Term Memory: Tracks and stores the current and predicted world states, along with associated costs.
Of these components, much attention has been given to the World Model, Configurator, and Actor which could be implemented with existing LLMs and test-time compute models. But there’s a quieter, unsung hero in this architecture: Perception.
Context Enrichment - The Critical Role of Perception
Perception is the bridge between an AI system and its environment. Without it, the system operates in a vacuum, relying solely on its internal knowledge. This is where the risks emerge—misjudgments or hallucinations caused by incomplete or outdated information can lead to incorrect predictions and actions.
Other than physical sensors, to empower perception, Context Enrichment is a promising approach. Context Enrichment (i.e. Retrieval-Augmented Generation, RAG) integrates external, contextually relevant data into an AI model’s input. By retrieving up-to-date information and feeding it into the model’s context window, RAG minimises hallucinations and enhances reasoning capabilities which directly improves a models output. Recent research has also shown its potential in bolstering test-time compute models’ performance.

A diagram demonstrating how Retrieval-Augmented Generation (RAG) works. It finds relevant documents to a users query using embedding representations of the data. Source: https://arxiv.org/abs/2005.11401
This capability isn’t just a “nice-to-have”; it’s critical as AI expands its reach across fields like medicine, law, and finance. Imagine a medical AI parsing vast amounts of research to identify groundbreaking findings or a legal AI assistant scanning case law to pinpoint relevant precedents. Without access to accurate and timely information, these systems risk making flawed recommendations—a risk we cannot afford in high-stakes industries.
The Road Ahead: Collaboration in Context
At Valyu, we’re tackling this challenge head-on. Our mission is to bridge the gap between AI systems and the data they need, whether for training or real-time context enrichment. High-quality data isn’t just a catalyst for advancing AI capabilities; it’s the foundation for deploying AI applications that users can truly rely on.
We also recognise the value of data held by rights owners. Their datasets are the fuel for the next generation of AI breakthroughs. That’s why we’re committed to creating a platform where rights holders can efficiently distribute their data to AI developers at scale.
If you’re building in AI and need access to high quality data—or if you’re a rightsholder/content platform looking to monetise your datasets—let’s collaborate. Together, we can shape the future of AI and ensure it’s grounded in the best possible context.
—-
Photo by Google DeepMind from Pexels.