  With Artificial Intelligence systems increasingly applied in consequential
domains, researchers have begun to ask how these systems ought to act in
ethically charged situations where even humans lack consensus. In the Moral
Machine project, researchers crowdsourced answers to "Trolley Problems"
concerning autonomous vehicles. Subsequently, Noothigattu et al. (2018)
proposed inferring linear functions that approximate each individual's
preferences and aggregating these linear models by averaging parameters across
the population. In this paper, we examine this averaging mechanism, focusing on
fairness concerns in the presence of strategic effects. We investigate a simple
setting where the population consists of two groups, with the minority
constituting an {\alpha} < 0.5 share of the population. To simplify the
analysis, we consider the extreme case in which within-group preferences are
homogeneous. Focusing on the fraction of contested cases where the minority
group prevails, we make the following observations: (a) even when all parties
report their preferences truthfully, the fraction of disputes where the
minority prevails is less than proportionate in {\alpha}; (b) the degree of
sub-proportionality grows more severe as the level of disagreement between the
groups increases; (c) when parties report preferences strategically, pure
strategy equilibria do not always exist; and (d) whenever a pure strategy
equilibrium exists, the majority group prevails 100% of the time. These
findings raise concerns about stability and fairness of preference vector
averaging as a mechanism for aggregating diverging voices. Finally, we discuss
alternatives, including randomized dictatorship and median-based mechanisms.
