The community was originally called Albade in the 1850s when it first began to be settled. Its post office was called Albade and was established in 1859. It was viewed as somewhat scattered. The nearby Lytton Springs caused the community to be called this in the early 1860s and was officially changed to that name in 1888. There was a cotton gin that was powered by steam, a Baptist church, and a general store in the mid-1880s. Most of the community's early settlers raised livestock. Ranching gave way to cotton in the 1900s, but it was revived in 1930 as an important economic value. Oil was discovered in the community in 1925, but only a few wells were producing it in the 1940s. The post office closed in 1958 and mail was sent to the community from Dale. On the 1989 county highway map, there were three churches, two cemeteries, and only one business in Lytton Springs. The community had 125 inhabitants in the mid-1880s. It then grew to 300 by the 1890s. 131 people were living in the community in 1900 and grew to 200 by the 1920s. It fell to 116 by the 1960s. It then had 76 inhabitants in the early 1970s and stayed at that number through 1990. The population then grew dramatically to 500 in 2000.
