  The emergence of price comparison websites (PCWs) has presented insurers with
unique challenges in formulating effective pricing strategies. Operating on
PCWs requires insurers to strike a delicate balance between competitive
premiums and profitability, amidst obstacles such as low historical conversion
rates, limited visibility of competitors' actions, and a dynamic market
environment. In addition to this, the capital intensive nature of the business
means pricing below the risk levels of customers can result in solvency issues
for the insurer. To address these challenges, this paper introduces
reinforcement learning (RL) framework that learns the optimal pricing policy by
integrating model-based and model-free methods. The model-based component is
used to train agents in an offline setting, avoiding cold-start issues, while
model-free algorithms are then employed in a contextual bandit (CB) manner to
dynamically update the pricing policy to maximise the expected revenue. This
facilitates quick adaptation to evolving market dynamics and enhances algorithm
efficiency and decision interpretability. The paper also highlights the
importance of evaluating pricing policies using an offline dataset in a
consistent fashion and demonstrates the superiority of the proposed methodology
over existing off-the-shelf RL/CB approaches. We validate our methodology using
synthetic data, generated to reflect private commercially available data within
real-world insurers, and compare against 6 other benchmark approaches. Our
hybrid agent outperforms these benchmarks in terms of sample efficiency and
cumulative reward with the exception of an agent that has access to perfect
market information which would not be available in a real-world set-up.
