Language vs. thought / non-linguistic cognition and perception
Only published papers are included; for preprints, see Papers.
Last Updated: May 2024
Language and thought in the human brain
Language is a relatively late cultural invention. By the time language emerged in our species, many perceptual, motor, and cognitive capacities were already in place. Besides, language bears similarities to many other human abilities. As a result, claims have been made over the years that in the human brain, language shares machinery with diverse non-linguistic capacities and that language is a prerequisite for complex thought and reasoning. However, it turns out that language processing mechanisms are actually exquisitely selective for language and distinct from the mechanisms that support our knowledge and reasoning abilities, as well as other non-linguistic perceptual and cognitive abilities.
The brain areas that support language processing are selective for language: The beginning
This paper demonstrates that language-responsive brain areas are highly selective for language processing relative to arithmetic processing, executive function tasks, and music perception.
This paper makes a similar point but zooms in on “Broca’s area” specifically and shows that language-selective areas lie adjacent to domain-general areas that belong to a distinct brain network.
This review summarizes the evidence for the dissociation between language and thought based on both brain imaging studies and investigations of patients with severe damage to the language system (which results in global aphasia).
The language network and the Multiple Demand (MD) network are robustly dissociated
The Multiple Demand (MD) network has been linked to general fluid intelligence, which encompasses executive functions (working memory, cognitive control, and attention), domain-general reasoning abilities, skill acquisition, and novel problem solving. Many have argued over the years that language is what made us smart, and that language mediates complex thought and reasoning.
These papers contest these claims: they focus on the relationship between language processing mechanisms and those that support general fluid intelligence (and are housed in the Multiple Demand network) and provide evidence—using diverse approaches—for a dissociation between the language network and the MD network.
This paper probes the correlates of gene expression in the language network and uses the Multiple Demand network as a control and also finds a dissociation between them.
This review that summarizes the evidence for a dissociation between the language network and the domain-general Multiple Demand / cognitive control network; it focuses on “Broca’s area”, but discusses the two networks more broadly as well.
The Multiple Demand (MD) network, which supports cognitive control abilities, plays a limited role in language processing
These papers build on the finding of the dissociation between the language network and the MD network and probe the possible contributions of the MD network, which supports cognitive control abilities, to language comprehension.
This review summarizes the above papers and argues for local implementation of linguistic computations. Although language draws on computations that are general in nature (like structure building and prediction) and likely used for information processing in other domains (e.g., music, visual processing), these computations are implemented in the brain areas that store domain-specific knowledge representations (cf. in hubs that multiple domains draw on).
This early position paper discusses the role of cognitive control in language processing (it contains some useful notions, but we have learned a lot since the time I wrote this).
This position paper argues for a limited role of domain-general executive resources in linguistic prediction.
This patient study suggests that the MD network is causally important per perceiving speech in noise, so the MD network may play some role in perceptual processes that support language comprehension even if it does not support any linguistic computations (i.e., computations that are related to lexical access or combinatorial processing).
The language network is selective for language relative to social perception
Language is typically used in social settings. Does language share machinery with our abilities to process other socially-relevant information, like facial expressions or body language? It doesn’t appear so.
This paper shows that language-responsive brain areas do not respond during the observation of dynamic faces, hands, or bodies.
This paper shows that language-responsive brain areas do not respond during the observation of communicative actions (co-speech gestures).
The language network is selective for language relative to social cognition (Theory of Mind)
We often use language to share our thoughts, beliefs, and desires and to understand those of others. Does language share machinery with mechanisms that support reasoning about others’ minds?
These papers show that the language network robustly dissociates from the Theory of Mind network, although the Paunov et al. (2019, J Neurophys) paper also shows that the two networks show some degree of functional correlation (which may be taken to suggest frequent interactions and information sharing).
The language network is selective for language relative to computer code comprehension
This piece outlines some reasons for why one might expect the processing of computer languages to draw on the language network.
These papers show that understanding computer code draws on the Multiple Demand network, and not the language network.
The language network is selective for language relative to the processing of non-linguistic meaning
We use language to talk about the world and a lot of what we know about the world we learn from language. However, we can also understand complex meanings from non-linguistic inputs, like a picture, a video, or a sequence of sounds. The relationship between language and abstract conceptual knowledge is a big open (and highly controversial) question in human cognition.
This paper shows that the language network shows some response to visual semantics (understanding pictures of events), although the response is i) much lower than the response to sentences, and apparently, ii) not functionally critical for visual semantic processing given that individuals without a functioning language system have no difficulties with visual event semantics.
This paper shows that the language network is not engaged during semantic object categorization, and individuals with severe aphasia are not impaired in object categorization.
The language network is selective for language relative to music processing
The hypothesis that structure processing in language and music draw on the same or overlapping resources remains popular. However, this hypothesis does not find empirical support. In Fedorenko et al. (2011, PNAS), we showed that language areas show little/no response when participants listen to music.
This paper dives more deeply into this question and shows—across several fMRI experiments—that music processing does not engage the language network; furthermore, individuals with severe aphasia are still able to process music structure.
This is an older paper that shows that the language areas are not sensitive to music structure, and showing sensitivity to music structure in several areas in the auditory cortex.
The neuroimaging and aphasia evidence supersede the earlier paper from Ev’s work based on a dual-task behavioral approach where participants listened to sung sentences in a self-paced listening paradigm, and linguistic vs. tonal-structure complexity were manipulated. In this paper, we had found no evidence of resource sharing during online language comprehension, but we found some evidence based on patterns of comprehension question accuracies. These offline effects are most likely due to the engagement of the domain-general Multiple Demand system by both linguistic task demands (e.g., see Diachek, Blank, Siegelman et al., 2020, J Neurosci for evidence and discussion) and the attention-grabbing violations of music structure, rather than the same brain areas supporting the processing of structure in language and music.
Although higher-level structural processing is not shared between language and music, lower-level perceptual processes, including related to pitch processing, are likely shared. Here is an old behavioral study that provides some support for shared pitch processing between language and music.
Language and thought in machines
The relationship between language and thought has recently come to the forefront of the field in the context of large language models (LLMs), like GPT-2. These models can produce text that is often hard to distinguish from human output. As a result, claims have emerged—both in the popular press and in the academic literature—that LLMs represent not only a major advance in language processing but, more broadly, in Artificial General Intelligence (AGI), i.e., a step towards a “thinking machine”. In this line of work, we are probing LLMs’ representations to understand a) what kinds of information about the world can be learned from distributional linguistic input alone and b) the limitations of LLMs’ knowledge and reasoning capacities.
“Semantic projection”: Recovering rich knowledge of object features from word embeddings
Language reflects our knowledge of the world. This paper devises a novel approach for representing different dimensions of objects in word embedding spaces by using adjective antonyms as anchor points (e.g., size can be represented as a line from ‘small’ to ‘big’) and shows that ‘projecting’ nouns onto these dimensions captures human similarity judgments quite well.
Pragmatic reasoning in large language models
This paper investigates the ability of large language models (LLMs) to perform pragmatic reasoning and finds that some of the larger models achieve quite high performance, which suggests either that distributional linguistic information may suffice for (at least some components of) pragmatic reasoning, or that LLMs implicitly acquire Theory of Mind reasoning capacities.