Introduction
The advent of large language models (LLMs) like GPT-4 has fuelled longstanding philosophical debates about the nature of knowledge, understanding, and representation. These AI systems, some of which operate purely in the domain of written language, contribute to the ongoing dialog about what it means to "know" and "understand" in the absence of embodied experience and interaction with the world.
Believe it or not, these questions find a striking parallel in ancient Babylonian1 epistemology, where the technology of writing was seen as the very foundation of knowledge and intellectual authority. By tracing the connections between these two "textual" conceptions of knowledge - separated by millennia but united in their elevation of the written word - we might gain new insights into the philosophical implications of language-based AI.
Babylonian Epistemology and the Primacy of Writing
In ancient Mesopotamia, the development of writing transformed not only the recording and transmission of information, but the very nature of knowledge itself. As historian Marc Van De Mieroop has compellingly argued, literacy became the basis for all intellectual pursuit in Babylonian culture. To be knowledgeable was to be literate, and the highest forms of understanding were those that could be rendered in cuneiform script on clay tablets.
In this context, knowledge was inextricably bound up with the material and social practices of literacy; knowledge was not merely represented by writing, but was constituted by it. To understand the world was, in a fundamental sense, to read and write about it. The Babylonian scribes grasped reality through its written form, engaging in intricate hermeneutic practices that treated the cuneiform script as a gateway to truth. Their approach to interpretation, exemplified, e.g., in the analysis of the divine names in the Enūma Eliš, relied on the polysemy and flexibility of the writing system itself. Every cuneiform sign contained a multiplicity of potential readings and meanings, allowing for complex explanations that aimed to reveal the deep structure of the universe.
This elevation of writing to the very foundation of knowledge finds an obvious echo in the architecture of modern LLMs, systems which construct true world models out of the patterns and regularities of written language. They are trained on enormous corpora of text data, and their "knowledge" of the world is entirely derived from models constructed from the associations and structures found within that textual domain. Like a Babylonian scribe sitting before a clay tablet in a state of flow, LLMs operate in a purely textual universe, consuming and producing knowledge solely in the form of written words.
Moreover, the "understanding" embodied by LLMs emerges from the complex interplay of the components that make up their training data, an interesting parallel to the Babylonian conception of meaning arising from the intricate webs of equivalences and associations between cuneiform signs. The knowledge of an LLM is not a set of explicit facts and rules, but a dense network of statistical relations extracted from the patterns of written language. In this sense, LLMs epitomize the notion of knowledge as constituted by the manipulation of linguistic symbols, an idea that was central to Babylonian epistemology.
The Babylonian elevation of writing to the pinnacle of epistemology anticipates, in a sense, the textual universe of the LLM. By reflecting on these parallels, we can gain a new perspective on the age-old question of the relationship between language, knowledge, and understanding; though, as we’ve discussed here before, there are also much more contemporary voices with opinions on this matter.
The Epistemology of Language Models and the "Language of Thought"
In a sense, LLMs embody a kind of "pure" textual epistemology, where knowledge is wholly constituted by the manipulation of linguistic symbols. Viewing cognition through this lens, as essentially a process of symbol manipulation, has deep roots in modern philosophy of mind. The "language of thought" hypothesis, championed by philosophers like Jerry Fodor, holds that thinking itself is a kind of computation over language-like representations. On this view, the "knowledge" embedded in an LLM is not so different in kind from the knowledge in a human mind - both are a matter of processing and generating symbolic structures.
This perspective finds an advocate in this blog’s favourite philosopher Daniel Dennett. In "What RoboMary Knows," Dennett argues that a complete textual description of the world would be sufficient for full understanding, even of seemingly ineffable qualities like the experience of colour. If Mary, a scientist who knows everything there is to know about colour perception but has never seen colour herself, were to finally step outside her black-and-white room, Dennett contends that she would not learn anything new. All the knowledge there is to have about colour is already contained in the scientific theories she has mastered - the textual knowledge.
Applying this reasoning to LLMs, we might argue that their purely textual "understanding" is not as limited as it might seem. If all knowledge can be reduced to symbolic representation and manipulation, then an LLM's model of language could in principle encompass all there is to know about the world. Just as Mary's theoretical knowledge exhausts the facts of colour perception, an LLM's textual "knowledge" could be said to exhaust the facts of the world as represented through language. In the view of Van De Meiroop, this stance would be wholly concordant with the epistemological expectations, and beliefs, of a Babylonian scribe.
The Limits of Textual Knowledge: Embodiment and Enaction
Of course, this is a controversial claim, and many philosophers have pushed back against the idea that textual knowledge alone could be sufficient for true understanding. Thinkers in the tradition of embodied and enactive cognition, like Francisco Varela and Alva Noë, argue that cognition is fundamentally grounded in an organism's sensorimotor engagement with its environment. On this view, the disembodied, purely textual "knowledge" of an LLM would be missing something crucial - the lived experience and practical know-how that comes from interacting with the world.
The "symbol grounding problem," articulated by philosopher Stevan Harnad, points out the primary difficulty of connecting abstract symbols to real-world referents without some form of embodied interaction; the Babylonian scribes, whatever their preferred intellectual frameworks, would certainly have also had plenty of embodied experience interacting with their world!
What did Arad-Nanna of Ur know?
But what if the parallels between the textual epistemology of ancient Babylonia and the architecture of modern language models are not merely an interesting historical footnote? The twin technologies of cuneiform writing and LLMs mutually support, across millennia, a convergent philosophical position: that knowledge, in its fullest and most essential form, is fundamentally a matter of symbols (and, perhaps as a corollary, cognition a matter of symbol manipulation). The Babylonian scribes, with their unwavering faith in the power of the written word, were not just pioneering a new technology of literacy - they were articulating a bold epistemological stance that resonates with surprising relevancy 5,000 years later.
The epistemological stance of the Babylonians invites us to view the written word as not just a tool for representing knowledge, but the very stuff of knowledge itself. And with the rise of large language models, we might be seeing this ancient insight realized in silico, in the most powerful knowledge technology yet devised by human ingenuity. Perhaps the Babylonian dream of a purely literate mind, once a fanciful philosophical notion, is finally becoming a manifest technological reality.
𒃻𒉡𒉽𒊏𒀀𒀭
I use the term ‘Babylonian’ here not to refer to a single language, place, or people, but rather as a collective reference to the broad cosmopolitan ANE culture characterized by intertwined adoption of the Sumerian and Akkadian languages, ranging from Early Dynastic Sumer to late Bronze Age Anatolia.