Fisher was the brother of H. A. L. Fisher, Edmund Fisher, Charles Fisher, Florence Henrietta Darwin and Adeline Vaughan Williams. His sister Cordelia Fisher married the author, critic and journalist Richard Curle and was the mother of the academic Adam Curle.
