During Scott's first three years with Baltimore, he was a special teams standout. However, he saw little time in the defensive rotation. In his rookie season, he played in all 16 regular season games and recorded five tackles and one interception on defense while tying with another teammate to lead the team in special teams tackles at 17. He made his NFL debut at the Carolina Panthers on September 8, 2002. The following season, he again played in all 16 games and one post-season game, recording nine tackles and one fumble recovery on defense. His 19 special teams tackles, a career high, ranked second on the team. In the 2004 season, he played in 13 games, making 17 special teams tackles, adding five tackles on defense playing as a safety and linebacker.
