  In traditional (desktop) e-commerce search, a customer issues a specific
query and the system returns a ranked list of products in order of relevance to
the query. An increasingly popular alternative in e-commerce search is to issue
a voice-query to a smart speaker (e.g., Amazon Echo) powered by a voice
assistant (VA, e.g., Alexa). In this situation, the VA usually spells out the
details of only one product, an explanation citing the reason for its
selection, and a default action of adding the product to the customer's cart.
This reduced autonomy of the customer in the choice of a product during
voice-search makes it necessary for a VA to be far more responsible and
trustworthy in its explanation and default action.
  In this paper, we ask whether the explanation presented for a product
selection by the Alexa VA installed on an Amazon Echo device is consistent with
human understanding as well as with the observations on other traditional
mediums (e.g., desktop ecommerce search). Through a user survey, we find that
in 81% cases the interpretation of 'a top result' by the users is different
from that of Alexa. While investigating for the fairness of the default action,
we observe that over a set of as many as 1000 queries, in nearly 68% cases,
there exist one or more products which are more relevant (as per Amazon's own
desktop search results) than the product chosen by Alexa. Finally, we conducted
a survey over 30 queries for which the Alexa-selected product was different
from the top desktop search result, and observed that in nearly 73% cases, the
participants preferred the top desktop search result as opposed to the product
chosen by Alexa. Our results raise several concerns and necessitates more
discussions around the related fairness and interpretability issues of VAs for
e-commerce search.
