  Both in scientific literature and in industry,, Semantic and context-aware
Natural Language Processing-based solutions have been gaining importance in
recent years. The possibilities and performance shown by these models when
dealing with complex Language Understanding tasks is unquestionable, from
conversational agents to the fight against disinformation in social networks.
In addition, considerable attention is also being paid to developing
multilingual models to tackle the language bottleneck. The growing need to
provide more complex models implementing all these features has been
accompanied by an increase in their size, without being conservative in the
number of dimensions required. This paper aims to give a comprehensive account
of the impact of a wide variety of dimensional reduction techniques on the
performance of different state-of-the-art multilingual Siamese Transformers,
including unsupervised dimensional reduction techniques such as linear and
nonlinear feature extraction, feature selection, and manifold techniques. In
order to evaluate the effects of these techniques, we considered the
multilingual extended version of Semantic Textual Similarity Benchmark (mSTSb)
and two different baseline approaches, one using the pre-trained version of
several models and another using their fine-tuned STS version. The results
evidence that it is possible to achieve an average reduction in the number of
dimensions of $91.58\% \pm 2.59\%$ and $54.65\% \pm 32.20\%$, respectively.
This work has also considered the consequences of dimensionality reduction for
visualization purposes. The results of this study will significantly contribute
to the understanding of how different tuning approaches affect performance on
semantic-aware tasks and how dimensional reduction techniques deal with the
high-dimensional embeddings computed for the STS task and their potential for
highly demanding NLP tasks
