Kirby was a grandson of the Suffolk topographer John Kirby (author of The Suffolk Traveller) and nephew of artist-topographer Joshua Kirby (a friend of Thomas Gainsborough's). He was also a cousin of the children's author Sarah Trimmer. His parents were William Kirby, a solicitor, and Lucy Meadows. He was born on 19 September 1759 at Witnesham, Suffolk, and studied at Ipswich School and Caius College, Cambridge, where he graduated in 1781. Taking holy orders in 1782, he spent his entire working life in the peaceful seclusion of an English country parsonage at Barham in Suffolk, working at the parish church of St Mary's for 68 years, first as curate, then as rector from 1797. He assisted in the publication of pamphlets against Thomas Paine during the 1790s.
