After college Shergill formed a band named Kaffir. The band played in some competitions and college festivals before parting ways. Shergill initially composed jingles for advertisement agencies such as Yamaha RX-T motorbikes and Times FM. He had an unsuccessful stint with Sony Music and Tehelka before finally releasing his debut album Rabbi in 2004 under Phat Phish Records. Relying on word-of-mouth publicity and a music video, he had a chart topper song "Bulla Ki Jaana". Most of the songs in the album were composed and written by Shergill himself except for "Bulla ki Jana" based on the Kafi poem written by the Punjabi Sufi saint Bulleh Shah. "Heer" from "Heer" by Waris Shah and "Ishtihar" by Shiv Kumar Batalvi. Shergill had one song "Dilli" in the Hindi movie, Delhii Heights.
