Swailes was born and raised in Leeds, England, and worked in the textile industry. His father was a coal miner, who later worked in a woollen mill. His mother was active in the Salvation Army and the suffragette movement, and served time in prison for smashing windows in a London department store during a demonstration. The younger Swailes played the trombone for the Salvation Army and took courses from the University of Leeds. During World War I, he worked as a musician at the Opera House in Cork, Ireland, and later for the Australian Navy. After the war, he toured New Zealand with a professional band.
