Lazarus was married twice, first to Eileen Lazarus, which ended in divorce; then to Sally Mitchell, daughter of comic-strip gag writer Ed Mitchell. Lazarus lived in Los Angeles from the 1970s until his death on May 24, 2016. from complications from Alzheimer's. He had three daughters, Margie, Suesan and Cathie; six grandchildren and one great grandson.
