  Sustaining users' interest and keeping them engaged in the platform is very
important for the success of an e-commerce business. A session encompasses
different activities of a user between logging into the platform and logging
out or making a purchase. User activities in a session can be classified into
two groups: Known Intent and Unknown intent. Known intent activity pertains to
the session where the intent of a user to browse/purchase a specific product
can be easily captured. Whereas in unknown intent activity, the intent of the
user is not known. For example, consider the scenario where a user enters the
session to casually browse the products over the platform, similar to the
window shopping experience in the offline setting. While recommending similar
products is essential in the former, accurately understanding the intent and
recommending interesting products is essential in the latter setting in order
to retain a user. In this work, we focus primarily on the unknown intent
setting where our objective is to recommend a sequence of products to a user in
a session to sustain their interest, keep them engaged and possibly drive them
towards purchase. We formulate this problem in the framework of the Markov
Decision Process (MDP), a popular mathematical framework for sequential
decision making and solve it using Deep Reinforcement Learning (DRL)
techniques. However, training the next product recommendation is difficult in
the RL paradigm due to large variance in browse/purchase behavior of the users.
Therefore, we break the problem down into predicting various product
attributes, where a pattern/trend can be identified and exploited to build
accurate models. We show that the DRL agent provides better performance
compared to a greedy strategy.
