  Black holes in binaries with other compact objects can provide natural venues
for indirect detection of axions or other ultralight fields. The superradiant
instability associated with a rapidly spinning black hole leads to the creation
of an axion cloud which carries energy and angular momentum from the black
hole. This cloud will then decay via gravitational wave emission. We show that
the energy lost as a result of this process tends toward an outspiraling of the
binary orbit. A given binary system is sensitive to a narrow range of axion
masses, determined by the mass of the black hole. Pulsar-black hole binaries,
once detected in the electromagnetic band, will allow high-precision
measurements of black hole mass loss via timing measurements of the companion
pulsar. This avenue of investigation is particularly promising in light of the
recent preliminary announcements of two candidate black hole-neutron star
mergers by LIGO/VIRGO (#S190814bv and #S190426c). We demonstrate that for such
a binary system with a typical millisecond pulsar and a 3-solar-mass black
hole, axions with masses between $2.7 \times 10^{-12}$ eV and $3.2 \times
10^{-12}$ eV are detectable. Recent gravitational wave observations by
LIGO/VIRGO of binary black hole mergers imply that, for these binaries,
gravitational radiation from the rotating quadrupole moment is a dominant
effect, causing an inspiraling orbit. With some reasonable assumptions about
the period of the binary when it formed and the spins of the black holes, these
observations rule out possible axion masses between $3 \times 10^{-13}$ eV and
$6 \times 10^{-13}$ eV. Future binary black hole observations, for example by
LISA, are expected to provide more robust bounds. In some circumstances,
neutron stars may also undergo superradiant instabilities, and binary pulsars
could be used to exclude axions with certain masses and matter couplings.
