Fraser dropped to the Fourth Division to sign for Fulham's neighbours Brentford in July 1976 on a free transfer. He was a near ever-present during the 1976–77 season and was used in a utility role by manager Bill Dodgin Jr, playing both full back positions and on the wing. He was a regular until a groin injury ruined a successful 1977–78 season, in which he helped the Bees to automatic promotion to the Third Division by filling in at the midfield and at inside forward. He rejected a new contract and departed Griffin Park early in the 1980–81 season, despite having missed just four matches of the previous season. Fraser made 131 appearances and scored seven goals during three seasons with Brentford.
