He married his first wife, Ann Barlow in 1813. They had six children: Westley, Emma, Caroline, Edward Harding, Ellen, and Mary Anne. William and Ann divorced before 1823. He married his second wife, Harriett Seale (1793–1865) on 18 January 1823 at St Pancras Old Church, London. He had three children by his second marriage, Charles, George Seale, and William.
