  Fuchsian groups with a modular embedding have the richest arithmetic
properties among non-arithmetic Fuchsian groups. But they are very rare, all
known examples being related either to triangle groups or to Teichmueller
curves.
  In Part I of this paper we study the arithmetic properties of the modular
embedding and develop from scratch a theory of twisted modular forms for
Fuchsian groups with a modular embedding, proving dimension formulas,
coefficient growth estimates and differential equations.
  In Part II we provide a modular proof for an Apery-like integrality statement
for solutions of Picard-Fuchs equations. We illustrate the theory on a worked
example, giving explicit Fourier expansions of twisted modular forms and the
equation of a Teichmueller curve in a Hilbert modular surface.
  In Part III we show that genus two Teichmueller curves are cut out in Hilbert
modular surfaces by a product of theta derivatives. We rederive most of the
known properties of those Teichmueller curves from this viewpoint, without
using the theory of flat surfaces. As a consequence we give the modular
embeddings for all genus two Teichmueller curves and prove that the Fourier
developments of their twisted modular forms are algebraic up to one
transcendental scaling constant. Moreover, we prove that Bainbridge's
compactification of Hilbert modular surfaces is toroidal. The strategy to
compactify can be expressed using continued fractions and resembles
Hirzebruch's in form, but every detail is different.
