Brooks was born as Reuben Sax to Hyman and Esther Sax, Russian Jewish immigrants. Married teenagers when they immigrated to the United States in 1908, they found employment in Philadelphia's textile and clothing industry. Their only child, Reuben Sax, was born in 1912 in Philadelphia. He attended public schools Joseph Leidy Elementary, Mayer Sulzberger Junior High School and West Philadelphia High School, graduating from the latter in 1929.
