Hugh Bryan married three times. His first wife is unnamed and her date of death unknown. His second wife was Catherine Barnwell, who died in 1740. She was the daughter of John Barnwell. His third wife was Mary Prioleau, who survived Bryan, and married his friend William Hutson in 1758. Hugh Bryan had at least one daughter, who was a married adult in 1741, when he wrote a letter to her. He also had at least one son, to whom he addressed two letters in 1740 while the son was attending school in Charleston. In 1751 he wrote in his diary that he visited a daughter's grave.
