Fry was born to Peter Fry (b.c. 1768–1846) and Joanna Chapman in Compton Bishop, a village in northern Somerset, some time during 1795. He had two younger brothers and two sisters. His younger brother Bruges later lived at Hill House in Cheddar and Peter lived at Compton House in Compton Bishop. He trained as a solicitor and began his career in the legal profession but later he became one of the pioneers of photography in the country.
