Foussard was born in Nouméa, New Caledonia around 1882,[Public Records Office of Victoria, Central Register for Male Prisoners 29937 - 30336 (1902-1903)] and worked on ships as a young man. In 1899, he deserted his ship the St. Elizabeth while it was docked in Sydney, and spent the next few years as a vagrant, wandering around NSW and later to Melbourne and Adelaide, supporting himself by occasional labouring work and petty theft.
