text: With the emergence of diverse data collection techniques, objects in real applications can be represented as multi-modal features. What's more, objects may have multiple semantic meanings. Multi-modal and Multi-label [1] (MMML) problem becomes a universal phenomenon. The quality of data collected from different channels are inconsistent and some of them may not benefit for prediction. In real life, not all the modalities are needed for prediction. As a result, we propose a novel instance-oriented Multi-modal Classifier Chains (MCC) algorithm for MMML problem, which can make convince prediction with partial modalities. MCC extracts different modalities for different instances in the testing phase. Extensive experiments are performed on one real-world herbs dataset and two public datasets to validate our proposed algorithm, which reveals that it may be better to extract many instead of all of the modalities at hand.
  @cite_7: Traditional Chinese Medicine (TCM) is an influential form of medical treatment in China and surrounding areas. In this paper, we propose a TCM prescription generation task that aims to automatically generate a herbal medicine prescription based on textual symptom descriptions. Sequence-to-sequence (seq2seq) model has been successful in dealing with conditional sequence generation tasks like dialogue generation. We explore a potential end-to-end solution to the TCM prescription generation task using seq2seq models. However, experiments show that directly applying seq2seq model leads to unfruitful results due to the severe repetition problem. To solve the problem, we propose a novel architecture for the decoder with masking and coverage mechanism. The experimental results demonstrate that the proposed method is effective in diversifying the outputs, which significantly improves the F1 score by nearly 10 points (8.34 on test set 1 and 10.23 on test set 2).
summary: In this paper, taking both multi-label learning and feature extraction into consideration, we propose MCC model with an end-to-end approach @cite_7 for MMML problem, which is inspired by adaptive decision methods. Different from previous feature selection or dimensionality reduction methods, MCC extracts different modalities for different instances and different labels. Consequently, when presented with an unseen instance, we would extract the most informative and cost-effective modalities for it. Empirical study shows the efficiency and effectiveness of MCC, which can achieve better classification performance with less average modalities.