  We consider the structures formed by isogenies of abelian varieties with
polarizations that are not necessarily principal, specifically with the
$[\ell]$-polarizations we have previously defined. Our primary interest is in
superspecial abelian varieties, where the isogenies are related to quaternionic
hermitian forms. We first consider isogeny graphs. We show that these
$[\ell]$-isogeny graphs are a generalized Brandt graph and construct them
entirely in terms of definite quaternion algebras. We prove that they are
connected and give examples to show that the regular graphs obtained are
sometimes Ramanujan and sometimes not. Isogenies of $[\ell]$-polarized abelian
varieties can be closed under composition, with the consequence that such
isogenies naturally form semi-simplicial complexes as introduced by Eilenberg
and Zilber in 1950 (later also called $\Delta$-complexes) -- the
higher-dimensional analogues of multigraphs.
  We show that these isogeny complexes can be constructed from the arithmetic
of hermitian forms over definite quaternion algebras and that they are
quotients of the Bruhat-Tits building of the symplectic group by the action of
a quaternionic unitary group. Working with quaternions these isogeny graphs and
complexes are amenable to machine computation and we include many examples,
concluding with a detailed examination of the $[2]$-isogeny complexes of
superspecial abelian surfaces in characteristic $7$.
