Hoping for a career in publishing, Withers took a job in a London bookshop, J. and E. Bumpus, then worked in the advertising section of a book publisher. During the Great Depression she lost her job and was unemployed for several months, during which she lived on an allowance of £2 a week from an uncle. This gave her a sympathy for the political left, and she was a Labour supporter for most of her life, until she joined the SDP in the 1980s. In 1931 she secured a new job as a sub-editor on Condé Nast's Vogue. The magazine had a small staff and she rose quickly, so that in 1940 Harry Yoxall appointed her as editor. She also joined the Condé Nast board.
