  We propose a novel way to incorporate expert knowledge into the training of
deep neural networks. Many approaches encode domain constraints directly into
the network architecture, requiring non-trivial or domain-specific engineering.
In contrast, our approach, called MultiplexNet, represents domain knowledge as
a logical formula in disjunctive normal form (DNF) which is easy to encode and
to elicit from human experts. It introduces a Categorical latent variable that
learns to choose which constraint term optimizes the error function of the
network and it compiles the constraints directly into the output of existing
learning algorithms. We demonstrate the efficacy of this approach empirically
on several classical deep learning tasks, such as density estimation and
classification in both supervised and unsupervised settings where prior
knowledge about the domains was expressed as logical constraints. Our results
show that the MultiplexNet approach learned to approximate unknown
distributions well, often requiring fewer data samples than the alternative
approaches. In some cases, MultiplexNet finds better solutions than the
baselines; or solutions that could not be achieved with the alternative
approaches. Our contribution is in encoding domain knowledge in a way that
facilitates inference that is shown to be both efficient and general; and
critically, our approach guarantees 100% constraint satisfaction in a network's
output.
