  We describe the implications of permutation symmetry for the state space and
dynamics of quantum mechanical systems of matrices of general size $N$. We
solve the general 11- parameter permutation invariant quantum matrix harmonic
oscillator Hamiltonian and calculate the canonical partition function. The
permutation invariant sector of the Hilbert space, for general Hamiltonians,
can be described using partition algebra diagrams forming the bases of a tower
of partition algebras $P_k(N)$. The integer $k$ is interpreted as the degree of
matrix oscillator polynomials in the quantum mechanics. Families of interacting
Hamiltonians are described which are diagonalised by a representation theoretic
basis for the permutation invariant subspace which we construct for $ N \ge 2k
$. These include Hamiltonians for which the low-energy states are permutation
invariant and can give rise to large ground state degeneracies related to the
dimensions of partition algebras. A symmetry-based mechanism for quantum many
body scars discussed in the literature can be realised in these matrix systems
with permutation symmetry. A mapping of the matrix index values to lattice
sites allows a realisation of the mechanism in the context of modified
Bose-Hubbard models. Extremal correlators analogous to those studied in AdS/CFT
are shown to obey selection rules based on Clebsch-Gordan multiplicities
(Kronecker coefficients) of symmetric groups.
