  Pretrained language models that have been trained to predict the next word
over billions of text documents have been shown to also significantly predict
brain recordings of people comprehending language. Understanding the reasons
behind the observed similarities between language in machines and language in
the brain can lead to more insight into both systems. Recent works suggest that
the prediction of the next word is a key mechanism that contributes to the
alignment between the two. What is not yet understood is whether prediction of
the next word is necessary for this observed alignment or simply sufficient,
and whether there are other shared mechanisms or information that is similarly
important. In this work, we take a first step towards a better understanding
via two simple perturbations in a popular pretrained language model. The first
perturbation is to improve the model's ability to predict the next word in the
specific naturalistic stimulus text that the brain recordings correspond to. We
show that this indeed improves the alignment with the brain recordings.
However, this improved alignment may also be due to any improved word-level or
multi-word level semantics for the specific world that is described by the
stimulus narrative. We aim to disentangle the contribution of next word
prediction and semantic knowledge via our second perturbation: scrambling the
word order at inference time, which reduces the ability to predict the next
word, but maintains any newly learned word-level semantics. By comparing the
alignment with brain recordings of these differently perturbed models, we show
that improvements in alignment with brain recordings are due to more than
improvements in next word prediction and word-level semantics.
