Stern owns an apartment in The Chatham, a building he designed in New York City. In 1966, he married photographer Lynn Gimbel Solinger, the daughter of David Solinger and the granddaughter of Bernard Gimbel, a marriage that ended by divorce in 1977. They had one son, Nicholas S. G. Stern, who manages the boutique construction and planning firm Stern Projects.
