Brock was born in El Dorado, Arkansas, to a family of sharecroppers. His family moved to Collinston, Louisiana, when he was two years old. While his family did not have much money, he said that he never felt poor because, "If you don't have something, you don't miss it." Brock grew up as a fan of the Brooklyn Dodgers, the team that included Jackie Robinson, Don Newcombe, and Roy Campanella. Although he did not play in organized baseball until he reached the 11th grade, he learned much about the sport from listening to Cardinals radio broadcaster Harry Caray describe the way major league hitters stood at the plate. After attending high school in Mer Rouge, Louisiana, he received academic assistance to attend Southern University in Baton Rouge, but when a low grade in his first semester meant the possibility of losing his scholarship, he decided to try out for the school's baseball team in order to secure an athletic scholarship.
