text: This paper tackles the problem of endogenous link prediction for Knowledge Base completion. Knowledge Bases can be represented as directed graphs whose nodes correspond to entities and edges to relationships. Previous attempts either consist of powerful systems with high capacity to model complex connectivity patterns, which unfortunately usually end up overfitting on rare relationships, or in approaches that trade capacity for simplicity in order to fairly model all relationships, frequent or not. In this paper, we propose Tatec a happy medium obtained by complementing a high-capacity model with a simpler one, both pre-trained separately and then combined. We present several variants of this model with different kinds of regularization and combination strategies and show that this approach outperforms existing methods on different types of relationships by achieving state-of-the-art results on four benchmarks of the literature.
  @cite_18: 
  @cite_1: We consider the problem of learning probabilistic models for complex relational structures between various types of objects. A model can help us "understand" a dataset of relational facts in at least two ways, by finding interpretable structure in the data, and by supporting predictions, or inferences about whether particular unobserved relations are likely to be true. Often there is a tradeoff between these two aims: cluster-based models yield more easily interpretable representations, while factorization-based approaches have given better predictive performance on large data sets. We introduce the Bayesian Clustered Tensor Factorization (BCTF) model, which embeds a factorized representation of relations in a nonparametric Bayesian clustering framework. Inference is fully Bayesian but scales well to large data sets. The model simultaneously discovers interpretable clusters and yields predictive performance that matches or beats previous probabilistic models for relational data.
  @cite_11: Abstract Holland and Leinhardt (1981) proposed the p 1 model for the analysis of binary directed graph data in network studies. Such a model provides information about the “attractiveness” and “expansiveness” of the individual nodes in the network, as well as the tendency of a pair of nodes to reciprocate relational ties. When the nodes are a priori partitioned into subgroups based on attributes such as race and sex, the density of ties from one subgroup to another can differ considerably from that relating another pair of subgroups, thus creating a situation called blocking in social networks. The p 1 model completely ignores this extra piece of information and is, therefore, unable to explain the block structure. Blockmodels that are simple extensions of the p 1 model are proposed specifically for such data. An iterative scaling algorithm is presented for fitting the model parameters by maximum likelihood. The methodology is illustrated in detail on two empirical examples.
summary: While there has been a lot of focus recently on algorithms purely based on learning embeddings for entities and or relationships, many earlier alternatives had been proposed. We discuss works carried ou in the Bayesian clustering framework, as well as approaches that explicitly use the graph structure of the data. The Infinite Relational Model of @cite_18 , which is a nonparametric extension of the Stochastic Blockmodel @cite_11 , is a Bayesian clustering approach that learns clusters of entities of the same kind, i.e. groups of entities that can have similar relationships with other entities. This work was followed by @cite_1 , a 3-way tensor factorization model based on Bayesian clustering in which entities within a cluster share the same distribution of embeddings.