Born in The Bronx, Carlos was raised in Harlem, New York. His mother, Vioris (née Lawrence, 1919–2016), was born in Jamaica West Indies to Black Cuban parents and grew up in Santiago, Cuba. His father, Earl V. Carlos, Sr. (1895–1969), was from South Carolina. Carlos participated in a 200-meter dash and as a member of the 4×400-meter relay helped lead ETSU to the 1967 Lone Star Conference Championship. After his first year, Carlos enrolled at San Jose State University where he was trained by future National Track & Field Hall of Fame coach Lloyd (Bud) Winter.
