Celebrate the Bullet is a dark and tense album, featuring a "less lively sound" than previous releases, and is strongly political, with lyrics focusing on social and racial relations, "cold war paranoia and fear for the future." In the words of Martin C. Strong, the album was a "more lugubrious state-of-the-nation musing" than Too Much Pressure (1980). In the Rough Guide to Rock, Peter Buckley said, "with a title like Celebrate the Bullet, we knew we were in for a serious sit-down talking-to. Sure, it was still pop music, it was still Britska (a term that, thankfully, was never widely used), it was still Pauline and the lads dancing around with lots of brass and a beat you could lean against, but it wasn't happy, not at all." One interviewer, whilst interviewing Black, described it as "a dark, haunting, bluesy iteration of ska that to my knowledge has never been attempted before or since." He said that "at times the songs have a new wave feel via synthesized keyboard melodies that buzz over Neol Davies' blistering, bluesy and soulful guitar solos and riffs. Other times its almost undefinable as the songs are driven by a seamless melting pot of rock, reggae and new wave via memorable melodies that stick in your head."
