Stoker's first marriage was to Jacqueline (née Trelfer) in 1962. They were divorced in 1985. He married his second wife, Gill (née Watson), in 1986. He was a member of the Garrick Club for several years.
