The house was built facing Deer Street sometime between 1749, when Samuel Hart mortgaged the land, and 1756, when it was inventoried for his estate. In the early 19th century it was owner by William Hart, a ship's captain, who made modest alterations to the interior trim. Rice was locally prominent as a privateer during the War of 1812.
