They had four daughters: in order, Amaryllis, Henrietta, and the twins Nerissa and Frances; eventually the couple separated. Amaryllis was an actress who had a small part in Harold Pinter's film adaptation of The Go-Between (1970). She drowned in the Thames, aged 29. Henrietta (1945—2019) married Lytton Burgo Partridge, the nephew of her father's first wife Ray, but was left a widow with a newborn infant when she was 18;Adam Kuper, Incest and Influence: The Private Life of Bourgeois England, Harvard University Press, 2009, p. 242, . she oversaw the legacies of both David Garnett and Duncan Grant. Nerissa Garnett (1946–2004) was an artist, ceramicist, and photographer. Fanny (Frances) Garnett moved to France where she became a farmer.
