His first wife was Marie Dorothea Sophie Münster (1803–1827). Thoresen's second marriage was to Sara Margrethe Daae (April 6, 1806 – May 10, 1841), the daughter of the priest Johan Christopher Haar Daae. The two of them had five children, and their daughter Suzannah later married Henrik Ibsen. Sara Daae was also the aunt of the politician Ludvig Daae. Thoresen's third wife was the author Anna Magdalene Kragh (1819–1903). Magdalene accepted a position as a governess with Thoresen in 1842; in June 1843, she gave birth to a son, who was placed in foster care in Copenhagen, and she then married Thoresen in October 1843. According to Aagaat Daae, Magdalene Kragh came to Herøy in 1842. Magdalene became a stepmother to Thoresen's daughter Suzannah.
