John Rubin married Roberta Frances (née Torin) of St. Louis, Missouri, on August 19, 1972. They returned to the Atlanta area in 2018, after his retirement from the Kansas Legislature, residing since in Acworth, Paulding County, Georgia. They celebrated their 48th wedding anniversary on August 19, 2020. They have two children, Shana McNeil, of Denver, Colorado, and Sandy Rubin, of Marietta, Georgia, and three grandchildren. Rubin is a practicing Roman Catholic.
