In 1887, Chadwick married Emma Plummer, with whom he had four children. In 1915, Emma led a design committee formed by the Daughters of the American Revolution to design a new flag of Washington. Chadwick's son Stephen Fowler Chadwick ran for the United States House of Representatives in 1926 and the United States Senate in 1932 and 1940.
