After the war, Ruebhausen returned to practice at Debevoise & Plimpton, where he remained until his retirement in 1987. From 1950 to 1951, he also served as counsel to the International Development Advisory Board, where he began a lifelong relationship with Nelson A. Rockefeller, the organization's chairman, as a friend and political adviser. When Rockefeller became Governor of New York, Ruebhausen served him in a number of capacities, including as chairman of a Task Force on Protection from Radioactive Fallout, as Special Adviser on Atomic Energy, and as chairman of a panel on Insurance Holding Companies.
