On his return to Scotland, Rae began the seminal Scottish jazz group The John Rae Collective. As its leader, the ensemble went on to contest Smith's dominance of the Scottish jazz world in the late 1980s and the sextet was a key breeding ground for musicians who subsequently came to prominence in their own right, including pianist Brian Kellock, saxophonist Phil Bancroft, trumpeter Colin Steele, bassist Kenny Ellis and guitarist Kevin Mackenzie, all of whom are active in multiple projects both as leaders and sidemen. Another of Rae's early ventures alongside Dundee musician and composer Kevin Murray, was the Giant Stepping Stanes, who broke new ground in combining elements of modern jazz with Scottish folk music, an area of cross-fertilization that has since expanded steadily and fruitfully, not least through the early work of Rae.
