Schäfer was born in a farmer's village in Obhausen. His father was extremely nationalistic. As a young adult, Schäfer got into the bricklayer's trade. After becoming unemployed, he got a job as a farmhand in his hometown. Schäfer joined the Nazi Party in 1932 and the SS in 1933. During this time, he got into street fights with political opponents.
