Fortescue was the third son of Hugh Fortescue, 2nd Earl Fortescue, and Lady Susan, daughter of Dudley Ryder, 1st Earl of Harrowby. His paternal grandmother Hester Grenville was the daughter of George Grenville. Hugh Fortescue, 3rd Earl Fortescue and the Hon. John Fortescue were his elder brothers. He was educated at the University of Oxford.
