Handley began his football career in September 1931 with Hednesford Town. He soon came to the attention of West Bromwich Albion, for which he signed in June 1932. Despite playing many matches in the reserves and being capped at the amateur level for England, Handley never appeared for the first team at West Brom. Transferred to Crystal Palace in 1934, he played 5 matches in the Football League Third Division South before being signed by Brierley Hill Alliance the next season. Handley's next and final club was Darlaston, where he played until the outbreak of the Second World War in 1939.
