He was born in Etnedal as a son of chief physician Jacob Dybwad Sømme (1866–1923) and Helene Sofie Sørensen (1870–1955). The family moved to Mesnalien in 1899, Stavanger in 1908 and Molde in 1917. He was a brother of Sven Sømme and first cousin of Axel Sømme. His sister Ingerid Sømme married Sigval Bergesen the Younger, a son of Sigval Bergesen. Iacob Dybwad Sømme married scientist Aslaug Sverdrup (1891–1955), a daughter of bishop and politician Jakob Sverdrup, in January 1930 in Oslo. The marriage lasted until 1942. They had one child together.
