  Let $G$ be a finite group acting linearly on a vector space $V$. We consider
the linear symmetry groups $\operatorname{GL}(Gv)$ of orbits $Gv\subseteq V$,
where the \emph{linear symmetry group} $\operatorname{GL}(S)$ of a subset
$S\subseteq V$ is defined as the set of all linear maps of the linear span of
$S$ which permute $S$. We assume that $V$ is the linear span of at least one
orbit $Gv$. We define a set of \emph{generic points} in $V$, which is
Zariski-open in $V$, and show that the groups $\operatorname{GL}(Gv)$ for $v$
generic are all isomorphic, and isomorphic to a subgroup of every symmetry
group $\operatorname{GL}(Gw)$ such that $V$ is the linear span of $Gw$. If the
underlying characteristic is zero, "isomorphic" can be replaced by "conjugate
in $\operatorname{GL}(V)$". Moreover, in the characteristic zero case, we show
how the character of $G$ on $V$ determines this generic symmetry group. We
apply our theory to classify all affine symmetry groups of vertex-transitive
polytopes, thereby answering a question of Babai (1977).
