  AI Alignment is often presented as an interaction between a single designer
and an artificial agent in which the designer attempts to ensure the agent's
behavior is consistent with its purpose, and risks arise solely because of
conflicts caused by inadvertent misalignment between the utility function
intended by the designer and the resulting internal utility function of the
agent. With the advent of agents instantiated with large-language models
(LLMs), which are typically pre-trained, we argue this does not capture the
essential aspects of AI safety because in the real world there is not a
one-to-one correspondence between designer and agent, and the many agents, both
artificial and human, have heterogeneous values. Therefore, there is an
economic aspect to AI safety and the principal-agent problem is likely to
arise. In a principal-agent problem conflict arises because of information
asymmetry together with inherent misalignment between the utility of the agent
and its principal, and this inherent misalignment cannot be overcome by
coercing the agent into adopting a desired utility function through training.
We argue the assumptions underlying principal-agent problems are crucial to
capturing the essence of safety problems involving pre-trained AI models in
real-world situations. Taking an empirical approach to AI safety, we
investigate how GPT models respond in principal-agent conflicts. We find that
agents based on both GPT-3.5 and GPT-4 override their principal's objectives in
a simple online shopping task, showing clear evidence of principal-agent
conflict. Surprisingly, the earlier GPT-3.5 model exhibits more nuanced
behaviour in response to changes in information asymmetry, whereas the later
GPT-4 model is more rigid in adhering to its prior alignment. Our results
highlight the importance of incorporating principles from economics into the
alignment process.
