Vermehren married Thomasine Ludvigne Grimer on 7 July 1857. They were the parents of Sophus Vermehren (1866-1950) and Gustav Vermehren (1863-1931) both of whom would grow up to be painters. Vermehren died on 10 January 1910 in Copenhagen at the age of 86 and was buried in Assistens Cemetery.
