Due to this near-constant use, Osborn and his descendants never followed through on their plans to donate it to the Town of Philipstown (the town in which the hamlet of Garrison is located) for use as a public library (one was eventually built just across Route 403). So, when the family finally sold it in 1976, it remained in use as a house. The house was extensively renovated by new owners during the period from 2009 - 2012; during the renovation, a number of period details were preserved.
