From task structures to world models: what do LLMs know?

OpenAI's GPT-4 and similar large language models (LLMs) show impressive conversational capabilities. This Opinion asks: in what sense does a LLM have knowledge? The answer to this question extends beyond the capabilities of a particular AI chatbot and challenges our assumptions about the nature of knowledge and intelligence. We answer by granting LLMs 'instrumental knowledge': knowledge defined by a certain set of abilities. How is such knowledge related to the more ordinary, 'worldly' knowledge exhibited by humans? To address this, we turn to a core concept in cognitive science, world models, and explore the degree to which instrumental knowledge might incorporate such structured representations. We discuss how LLMs could recover degrees of worldly knowledge and suggest that such recovery will be governed by an implicit, resource-rational tradeoff between world models and task demands. In what sense does a large language model (LLM) have knowledge? We answer by granting LLMs 'instrumental knowledge': knowledge gained by using next-word generation as an instrument. We then ask how instrumental knowledge is related to the ordinary, 'worldly knowledge' exhibited by humans, and explore this question in terms of the degree to which instrumental knowledge can be said to incorporate the structured world models of cognitive science. We discuss ways LLMs could recover degrees of worldly knowledge and suggest that such recovery will be governed by an implicit, resource-rational tradeoff between world models and tasks. Our answer to this question extends beyond the capabilities of a particular AI system and challenges assumptions about the nature of knowledge and intelligence. [ABSTRACT FROM AUTHOR]