  Recently there has been significant progress in the field of dialogue system
thanks to the introduction of training paradigms such as fine-tune and prompt
learning. Persona can function as the prior knowledge for maintaining the
personality consistency of dialogue systems, which makes it perform well on
accuracy. Nonetheless, the conventional reference-based evaluation method falls
short in capturing the genuine text comprehension prowess of the model,
significantly relying on the quality of data annotation. In contrast, the
application of metamorphic testing offers a more profound insight into the
model's distinct capabilities without necessitating supplementary annotation
labels. This approach furnishes a more comprehensive portrayal of the model's
intricacies and exposes intricacies concealed within reference-based validation
techniques. Consequently, we introduce a persona-centric metamorphic relation
construction for metamorphic testing, aimed at evaluating both the persona
consistency and robustness of personalized dialogue models. For that reason,
this work evaluates several widely used training paradigms including learning
from scratch, pretrain + fine-tune and prompt learning in personalized dialogue
retrieval to know if they are more robust or if they have the same flaws as
their predecessor. Under three kinds of designed metamorphic relations with
consistent outputs, our experimental results reveal that prompt learning shows
stronger robustness compared to training from scratch and fine-tune. Although
tested retrieval models gain competitively high retrieval accuracy according to
the traditional reference-based validation, they are still fragile and
demonstrate various unexpected behaviors, thus there is still room for future
improvement in personalized dialogue retrieval.
