Born in Liverpool, Rimmer became an apprentice with Everton, turning professional in April 1984. He made one first team appearance for Everton, as a substitute, before being released at the end of the 1984–85 season. In August 1985 he joined Ipswich Town, but made less than 30 first team appearances over the next three seasons. In July 1988 he joined Wigan Athletic. During the 1990–91 season he had a spell on loan with Mossley, playing three times. He returned to Wigan and went on to make over 250 first team appearances.
