The fifth born child, Williams lived with his parents, a brother and five sisters. His family eventually moved to Annapolis, Maryland. Shortly after when Williams was nine, his father died of tuberculosis. Williams' mother realized she could not manage the entire family and sent some of the children to live with relatives. Williams was apprenticed to a shoemaker in Baltimore, Maryland but ran away to join his mother, who had moved to Rockford, Illinois. He later moved to Edgerton, Wisconsin, where he joined his sister and opened his own barber shop. After moving to nearby Janesville, Wisconsin, Williams became fascinated by the work of a local physician and decided to follow his path.
