  Motivated by approaches to the word problem for one-relation monoids arising
from work of Adian and Oganesian (1987), Guba (1997), and Ivanov, Margolis and
Meakin (2001), we study the submonoid and rational subset membership problems
in one-relation monoids and in positive one-relator groups. We give the first
known examples of positive one-relator groups with undecidable submonoid
membership problem, and apply this to give the first known examples of
one-relation monoids with undecidable submonoid membership problem. We
construct several infinite families of one-relation monoids with undecidable
submonoid membership problem, including examples that are defined by relations
of the form $w=1$ but which are not groups, and examples defined by relations
of the form $u=v$ where both of $u$ and $v$ are non-empty. As a consequence we
obtain a classification of the right-angled Artin groups that can arise as
subgroups of one-relation monoids. We also give examples of monoids with a
single defining relation of the form $aUb = a$, and examples of the form
$aUb=aVa$, with undecidable rational subset membership problem. We give a
one-relator group defined by a freely reduced word of the form $uv^{-1}$ with
$u, v$ positive words, in which the prefix membership problem is undecidable.
Finally, we prove the existence of a special two-relator inverse monoid with
undecidable word problem, and in which both the relators are positive words. As
a corollary, we also find a positive two-relator group with undecidable prefix
membership problem. In proving these results, we introduce new methods for
proving undecidability of the rational subset membership problem in monoids and
groups, including by finding suitable embeddings of certain trace monoids.
