Carroll first came to prominence on the inter-county scene as a member of the Offaly minor hurling team in 1973. He also played at this level in 1974 before later joining the county under-21 team. Carroll spent four years playing with the under-21 team, however, he had little success in this level either. He made his senior debut in a National Hurling League game against Antrim in 1975. It would be 1977 before Carroll made his debut in the championship.
