Following his tenure as governor general, he returned to political and diplomatic life, serving as Secretary of State for the Colonies between 1922 and 1924, before retiring to his estate in Derbyshire, where he died on 6 May 1938. He was the last Duke to ever hold a cabinet post.
