  The advent of gravitational-wave (GW) astronomy has presented us with a
completely new means for observing the Universe, allowing us to probe its
structure and evolution like never before. In this thesis, we explore three
distinct but complementary avenues for using GW observations to gain new
insights into cosmology and fundamental physics.
  In chapter 1, we study the astrophysical GW background (AGWB): the cumulative
GW signal arising from a large number of compact binary coalescences (CBCs)
throughout the Universe. Since these compact binaries reside in galaxies, the
AGWB contains anisotropies that trace out the large-scale structure of the
cosmic matter distribution. We investigate the angular power spectrum of the
AGWB, with the goal of developing predictions that can be confronted with
directional AGWB searches.
  In chapter 2, we calculate the nonlinear GW memory emitted by cusps and kinks
on cosmic string loops, which are among the most promising cosmological sources
of GWs. We show that, surprisingly, the cusp memory signal diverges for
sufficiently large loops, indicating a breakdown in the validity of the
weak-field description of the cusp. We then present one tentative possible
solution to this divergence, in which the portion of the string surrounding the
cusp collapses to form a primordial black hole (PBH).
  Finally, in chapter 3 we develop a powerful new method for GW detection based
on precision measurements of the orbits of binary systems. In the presence of a
stochastic GW background (GWB) the trajectories of the binary's components are
perturbed, giving rise to a random walk in the system's orbital parameters over
time. We calculate the sensitivity of binary pulsars and lunar laser ranging to
the GWB through this effect, and show that present data are already sensitive
enough to place the strongest constraints to date in the $\mu$Hz frequency
band.
