He was born in Perast in 1680 (at the time in Venetian Albania, now Montenegro). His uncle Andrija Zmajević, was an archbishop of Roman Catholic Archdiocese of Bar and a poet. He was also a primate of Serbia. The family was in conflict with the Bujović, another family from Perast. After Duke Vicko Bujović was killed, Matija left Perast as he was accused of being involved in his death. He escaped to the Republic of Ragusa (Dubrovnik) and then to Istanbul, where he found refuge with Russian ambassador Peter Tolstoy. Zmajević had known Tolstoy since a 1698 training trip the latter took to Perast. In 1712, Tolstoy sent him with a recommendation to Peter I.
