Karlsen was born in Sarpsborg. Inspired by the Nat King Cole jazz ensembles from the time of World War II, he started his own orchestra at the age of 16 in 1947, The Syncopators Band, in Sarpsborg, and he soon became a central figure in the local jazz scene. In the period 1953–59, he had a series of bands with local musicians, many to be influential on the Norwegian and some on the World jazz scenes, like the jazz singer Karin Krog (1955–56), the saxophone players Totti Bergh (1955–59), Bjørn Johansen (1956–58) and Harald Bergersen (1959), and drummer Ole Jacob Hansen (1959), most of whom later would play central roles in the renowned Kjell Karlsens Big Band that he started together with saxophone player Mikkel Flagstad (1959–64). In addition to the ones listed above, K.K.B.B. featured musicians like trombonist Frode Thingnæs, bassist Erik Amundsen and the singers Grynet Molvig, Laila Dalseth, Odd Børre and Kirsti Sparboe among others. Karlsen also worked as a pianist and was an accompanist for a number of Norwegian and International Jazz soloists. He attended the first International Jazz Festival in Molde 1961.
