Miller was born at his family's home, 450 East 85th Street, in the Yorkville section of Manhattan, New York City. He was the son of Lutheran German parents, Louise Marie (Neiting) and tailor Heinrich Miller.Dearborn, The Happiest Man Alive, pp. 20–22. As a child, he lived for nine years at 662 Driggs Avenue in Williamsburg, Brooklyn,Jake Mooney, "'Ideal Street' Seeks Eternal Life," The New York Times, May 1, 2009. known at that time (and referred to frequently in his works) as the Fourteenth Ward. In 1900, his family moved to 1063 Decatur Street in the Bushwick section of Brooklyn.Dearborn, The Happiest Man Alive, p. 36. After finishing elementary school, although his family remained in Bushwick, Miller attended Eastern District High School in Williamsburg.Dearborn, The Happiest Man Alive, p. 38. As a young man, he was active with the Socialist Party of America (his "quondam idol" was the black Socialist Hubert Harrison).Introduction from A Hubert Harrison Reader, University Press of New England He attended the City College of New York for one semester.Dearborn, The Happiest Man Alive, p. 42.
