  Logical query answering over Knowledge Graphs (KGs) is a fundamental yet
complex task. A promising approach to achieve this is to embed queries and
entities jointly into the same embedding space. Research along this line
suggests that using multi-modal distribution to represent answer entities is
more suitable than uni-modal distribution, as a single query may contain
multiple disjoint answer subsets due to the compositional nature of multi-hop
queries and the varying latent semantics of relations. However, existing
methods based on multi-modal distribution roughly represent each subset without
capturing its accurate cardinality, or even degenerate into uni-modal
distribution learning during the reasoning process due to the lack of an
effective similarity measure. To better model queries with diversified answers,
we propose Query2GMM for answering logical queries over knowledge graphs. In
Query2GMM, we present the GMM embedding to represent each query using a
univariate Gaussian Mixture Model (GMM). Each subset of a query is encoded by
its cardinality, semantic center and dispersion degree, allowing for precise
representation of multiple subsets. Then we design specific neural networks for
each operator to handle the inherent complexity that comes with multi-modal
distribution while alleviating the cascading errors. Last, we define a new
similarity measure to assess the relationships between an entity and a query's
multi-answer subsets, enabling effective multi-modal distribution learning for
reasoning. Comprehensive experimental results show that Query2GMM outperforms
the best competitor by an absolute average of $5.5\%$. The source code is
available at \url{https://anonymous.4open.science/r/Query2GMM-C42F}.
