  We explore value decomposition solutions for multi-agent deep reinforcement
learning in the popular paradigm of centralized training with decentralized
execution(CTDE). As the recognized best solution to CTDE, Weighted QMIX is
cutting-edge on StarCraft Multi-agent Challenge (SMAC), with a weighting scheme
implemented on QMIX to place more emphasis on the optimal joint actions.
However, the fixed weight requires manual tuning according to the application
scenarios, which painfully prevents Weighted QMIX from being used in broader
engineering applications. In this paper, we first demonstrate the flaw of
Weighted QMIX using an ordinary One-Step Matrix Game (OMG), that no matter how
the weight is chosen, Weighted QMIX struggles to deal with non-monotonic value
decomposition problems with a large variance of reward distributions. Then we
characterize the problem of value decomposition as an Underfitting One-edged
Robust Regression problem and make the first attempt to give a solution to the
value decomposition problem from the perspective of information-theoretical
learning. We introduce the Maximum Correntropy Criterion (MCC) as a cost
function to dynamically adapt the weight to eliminate the effects of minimum in
reward distributions. We simplify the implementation and propose a new
algorithm called MCVD. A preliminary experiment conducted on OMG shows that
MCVD could deal with non-monotonic value decomposition problems with a large
tolerance of kernel bandwidth selection. Further experiments are carried out on
Cooperative-Navigation and multiple SMAC scenarios, where MCVD exhibits
unprecedented ease of implementation, broad applicability, and stability.
