Flesher was born in Kingston-upon-Hull, England in 1825, the eldest child of Rev. John Flesher, a Methodist preacher and Jane (née Cawood) Flesher. He was educated and spent his formative years in England. After he married Jane Foster in 1847 at the age of 22, they emigrated to Canada West.
