The cognition-trajectory model conceptualizes cognitive processes as dynamic trajectories through a high-dimensional phase space, highlighting the motion and interconnectedness inherent in thinking.
I don’t think any of this is new. Indeed, I think almost all of it is probably a painfully obvious evolution of the Language of Thought Hypothesis, viewed through a contemporary connectionist lens. (But, as a professor of computer science, holding forth at length on painfully obvious things is my long suit!)
The Phase Space of Cognition
Central to this model is the idea of a high-dimensional cognitive phase space. In this context, the phase space can be seen as an abstract representation of possible cognitive states, each point representing a unique configuration of cognitive components - a thought or cognitive state. The dimensionality of this space is determined by the complexity of the system in which it is implemented/embedded.
While one could visualize this phase space as a semantic space where each dimension represents some aspect of meaning, the model remains open to other interpretations and possibilities.
Cognitive Trajectories as Thought Processes
In the cognition-trajectory model, the act of thinking is depicted as movement through this high-dimensional phase space. Each thought or cognitive process forms a trajectory linking different regions of the space, weaving together various cognitive elements in the formation of complex thoughts.
This idea of trajectories provides an intuitive way to consider cognitive transitions, evolution of ideas, and the flow of thought. It allows us to conceptualize how a thought can shift and evolve over time, potentially influenced by a multitude of factors that steer its course through this cognitive space.
Mechanisms of Cognition and Linguistic Projections
The trajectories through this cognitive phase space are generated and shaped by the mechanisms performing cognition. These mechanisms determine how trajectories are formed, guided, and potentially modified over time, reflecting the dynamic and responsive nature of cognitive processes.
An intriguing aspect of the cognition-trajectory model is the suggestion that these thought-trajectories can be projected into natural language. In other words, an utterance can be viewed as a mapping of a cognitive trajectory into natural language. This aspect of the model integrates the richness of language as a medium for expressing and communicating our internal cognitive states.
This model is not simply a theoretical abstraction. Large language models provide a mechanistic testbed to apply, and interrogate, these ideas.
A Perspective from Transformer Language Models
Transformer-architected large language models like GPT-4, have demonstrated remarkable capability to capture the complexities of natural language. They rely on multi-layered self-attention mechanisms and an intricate interplay of embeddings to transform input tokens into coherent, contextually informed, outputs. This process can be interpreted through the lens of the cognition-trajectory model, with the evolution of the transformer's residual stream serving as a trajectory through cognitive phase space.
Transformer Mechanisms as Cognitive Trajectories
The residual stream in a transformer model undergoes successive transformations as information is processed through the blocks of the network. Each of these transformations can be viewed as inducing a movement through the model's cognitive phase space (defined by the embedding space of the transformer). This phase space represents a comprehensive set of possible cognitive states the transformer can reach.
The initial input tokens pass through the embedding layer, marking the starting point of the trajectory. As these activations propagate through the self-attention layers, they are continually updated in an additive way, with the residual connections enabling the retention of earlier information. These updates represent the model's cognitive process, a thought evolving over time, and each intermediate state forms a point on a micro-trajectory through the cognitive phase space.
The trajectory's form and direction are shaped by the model's learned parameters and architecture, mirroring the role of cognitive mechanisms in guiding thought-trajectories in the human brain. They reflect the context of the given input and the cumulative learning from the vast amount of data the model was trained on.
Projection into Language Space
The cognition-trajectory model suggests that cognitive trajectories can be projected into natural language, translating internal cognitive states into verbal or written form. This is mirrored in the transformer's final output layer, which projects the terminal residual stream back into token space.
Following the completion of the self-attention layers, the final embedding states are transformed back into the vocabulary's dimension, resulting in a distribution over potential output tokens. This final step embodies the projection from the high-dimensional cognitive phase space into the discrete space of language. The selected output tokens are a linguistic expression of the model's "thoughts.”
Positioning the Cognition-Trajectory Model
The cognition-trajectory model aligns with many existing themes in cognitive science. Aspects of the model echo the applications of Dynamical Systems Theory in Cognitive Science, where cognitive processes are depicted as dynamic trajectories through state spaces. This model extends this perspective by incorporating elements of semantic understanding and suggesting that utterances, or projections of thought-trajectories, can be mapped into natural language.
This mapping aspect aligns with Vector Symbolic Architectures (VSAs), which portray symbols as high-dimensional vectors and apply algebraic operations to model cognitive processes. VSAs, however, often focus on computational cognition as the application of discrete algebraic operations rather than as continuous trajectories through a phase space.
There are also direct similarities with Eliasmith’s Semantic Pointer Architecture (SPA), particularly the manipulation of high-dimensional vectors to represent concepts and thoughts. However, the SPA tends to have a more mechanistic focus and is often tied to specific cognitive tasks, whereas we search for a model that leaves room for a broader and perhaps more flexible conceptualization of cognition.
The cognition-trajectory model builds on these foundations to offer a more comprehensive perspective by integrating semantic understanding, dynamical systems theory, and linguistic expression. And it offers modern LLMs as direct applied testbeds. This synthesis may invite opportunities for exploration, particularly in bridging computational models and human cognitive phenomena.
Summarizing the painfully obvious
The cognition-trajectory model offers a lens for understanding cognition, capturing the dynamism, complexity, and high-dimensionality of thought. By representing thoughts as trajectories in a high-dimensional phase space and considering how these can be translated into language, this model provides a bridge between internal cognitive states and external linguistic expression, and presents an approach to the study of mind and cognition.
Moving from the abstract to the applied, the model also offers an opinion on the functioning of transformer-based large language models. The evolution of LLM residual streams during processing can be viewed as a trajectory through a cognitive phase space, illustrating a form of thought process. Further, the final projection into token space aligns with the model's concept of thought-to-language mapping. This interpretation provides a bridge between the complex internal dynamics of these models and the tangible linguistic outputs we observe, perhaps enriching our understanding of both artificial and natural cognitive processes.