The building was erected in 1848 to house a boarding school called the Alger Institute, which was one of a number of such schools established in the region after the Housatonic Railroad began service in 1842. The school only survived until 1851, and was converted for use as a boarding house for summer visitors in 1884. It was converted back to academic use in 1886, but none of its occupants were long-lived. In 1907 the Rumsey Hall School moved into the premises from Seneca Falls, New York, and operated here until 1949. It was purchased by a neighboring landowner in 1855, and bequested to the town in 1986, which made plans to adapt the building for use as its town hall. The building was demolished in 2010.
