M strike from around 40 years, but two goals in three minutes from 20-year-old Dembele saw the home side ahead.
Substitute keeper Craig Gordon replaced Dorus de Vries for the second half - the keeper was suffering from a chest strain - before goals from James Forrest, Leigh Griffiths, Scott Sinclair (penalty) and Tom Rogic stretched Celtic’s lead at the top of the table over Hearts to five points with a game in hand.
It was a perfect confidence-boost for the home side who will host Manchester City in their second Champions League Group C match on Wednesday, looking to make amends after losing 7-0 at Barcelona in their opener.
Celtic manager Brendan Rodgers made changes again as De Vries came back in for Gordon with veteran defender Kolo Toure replaced by Erik Sviatchenko.
Forward Sinclair came back into the side as did midfielder Nir Bitton, while Griffiths started on the bench after missing four games with hamstring and calf injuries.
Kilmarnock boss Lee Clark was without injured captain Steven Smith but Greg Taylor returned from suspension to take his place at left-back.
As early as the fifth minute Killie defenders were throwing themselves in front of Celtic shots on goal.
In addition, Dembele (twice), Sinclair, skipper Scott Brown and Forrest all cleared the bar with efforts.
Clark’s side could not get out of their own half.
However, just after the half-hour mark Coulibaly drew gasps of shock from the home fans and cheers from the travelling support when he took a pass from Gary Dicker, turned inside Brown and aimed a missile at the goal which beat De Vries and dipped under the bar.
That unlikely lead only lasted a couple of minutes, though, before Dembele swept a Mikael Lustig pass past keeper Jamie MacDonald from 10 yards.
The former Fulham player then did the same in the 38th minute, this time from a Tierney cut-back, to put Celtic ahead.
It was no more than the champions deserved.
Gordon replaced De Vries for the second half while moments later Kilmarnock substitute Charlee Adams came on for the injured Dean Hawkshaw.
Celtic increased their lead in the 52nd minute when Forrest combined with Rogic before stabbing the ball past MacDonald from 12 yards.
Griffiths came on for Dembele in the 65th minute and scored a couple of minutes later with a close-range header when Jozo Simunovic helped on a Sinclair corner.
Celtic were relentless. When Griffiths was fouled inside the box by Greg Taylor in the 72nd minute, the Scotland striker handed Sinclair the ball, allowing him to score in a sixth consecutive league game since the start of the season, by sending MacDonald the wrong way with the penalty.
Rodgers’ side missed several more chances before MacDonald gifted Rogic the sixth when he let the midfielder’s low effort from the edge of the box escape over his hands.
It was impressive stuff from the league leaders but Rodgers will know that Manchester City will undoubtedly provide a tougher test in midweek.