  With the widespread use of knowledge graphs (KG) in various automated AI
systems and applications, it is very important to ensure that information
retrieval algorithms leveraging them are free from societal biases. Previous
works have depicted biases that persist in KGs, as well as employed several
metrics for measuring the biases. However, such studies lack the systematic
exploration of the sensitivity of the bias measurements, through varying
sources of data, or the embedding algorithms used. To address this research
gap, in this work, we present a holistic analysis of bias measurement on the
knowledge graph. First, we attempt to reveal data biases that surface in
Wikidata for thirteen different demographics selected from seven continents.
Next, we attempt to unfold the variance in the detection of biases by two
different knowledge graph embedding algorithms - TransE and ComplEx. We conduct
our extensive experiments on a large number of occupations sampled from the
thirteen demographics with respect to the sensitive attribute, i.e., gender.
Our results show that the inherent data bias that persists in KG can be altered
by specific algorithm bias as incorporated by KG embedding learning algorithms.
Further, we show that the choice of the state-of-the-art KG embedding algorithm
has a strong impact on the ranking of biased occupations irrespective of
gender. We observe that the similarity of the biased occupations across
demographics is minimal which reflects the socio-cultural differences around
the globe. We believe that this full-scale audit of the bias measurement
pipeline will raise awareness among the community while deriving insights
related to design choices of data and algorithms both and refrain from the
popular dogma of ``one-size-fits-all''.
