On June 18, 1864, Primus married Amoretta Prime, of Norfolk, Connecticut. Their daughter Leila was born six months later, on December 22, 1864. In the spring of 1865, the family moved to Boston so that Primus could study art and find work as an artist. He first studied art with Edward Mitchell Bannister, a prominent African-American portrait painter and landscape artist. Primus stayed with Bannister only three months, as he felt the older artist was not interested in promoting the younger artist's work. He went on to study with lithographer, Charles Stetfield, who worked in the same building as Bannister. Stetfield was not as talented as Bannister, but he was a better teacher and mentor.
