Sangster was married and divorced three times. In May 1960 he married Christine Street, leaving her in 1976 (divorcing in 1978) for socialite Susan Peacock, the ex-wife of Australian Liberal Party politician Andrew Peacock. That marriage did not last, with Sangster having well-publicised affairs with Jerry Hall and Susan Lilley. Sangster married Lilley in 1985, divorcing in 2000. By his first wife he had three sons, Guy, Ben and Adam, and a daughter, Kate. With Susan Lilley he had two sons, Sam and Max.
