text: Multi-modal data is becoming more common in big data background. Finding the semantically similar objects from different modality is one of the heart problems of multi-modal learning. Most of the current methods try to learn the inter-modal correlation with extrinsic supervised information, while intrinsic structural information of each modality is neglected. The performance of these methods heavily depends on the richness of training samples. However, obtaining the multi-modal training samples is still a labor and cost intensive work. In this paper, we bring a extrinsic correlation between the space structures of each modalities in coreference resolution. With this correlation, a semi-supervised learning model for multi-modal coreference resolution is proposed. We firstly extract high-level features of images and text, then compute the distances of each object from some reference points to build the space structure of each modality. With a shared reference point set, the space structures of each modality are correlated. We employ the correlation to build a commonly shared space that the semantic distance between multi-modal objects can be computed directly. The experiments on two multi-modal datasets show that our model performs better than the existing methods with insufficient training data.
  @cite_24: Abstract We previously introduced a voxel-based, multi-modal application of the partial least square algorithm (MMPLS) to characterize the linkage between patterns in a person's complementary complex datasets without the need to correct for multiple regional comparisons. Here we used it to demonstrate a strong correlation between MMPLS scores to characterize the linkage between the covarying patterns of fluorodeoxyglucose positron emission tomography (FDG PET) measurements of regional glucose metabolism and magnetic resonance imaging (MRI) measurements of regional gray matter associated with apolipoprotein E (APOE) e4 gene dose (i.e., three levels of genetic risk for late-onset Alzheimer's disease (AD)) in cognitively normal, late-middle-aged persons. Coregistered and spatially normalized FDG PET and MRI images from 70 of the subjects (27 e4 homozygotes, 36 e4 heterozygotes and 67 e4 non-carriers) were used in a hypothesis-generating MMPLS analysis to characterize the covarying pattern of regional gray matter volume and cerebral glucose metabolism most strongly correlated with APOE-e4 gene dose. Coregistered and spatially normalized FDG PET and MRI images from the remaining 30 of the subjects were used in a hypothesis-testing MMPLS analysis to generate FDG PET-MRI gray matter MMPLS scores blind to their APOE genotype and characterize their relationship to APOE-e4 gene dose. The hypothesis-generating analysis revealed covarying regional gray matter volume and cerebral glucose metabolism patterns that resembled those in traditional univariate analyses of AD and APOE-e4 gene dose and PET-MRI scores that were strongly correlated with APOE-e4 gene dose (p
summary: Besides these methods above, there still are other models proposed for multi-modal problems. @cite_24 employed voxel-based multi-modal partial least square(PLS) to analyze the correlations between FDG PET glucose uptake-MRI gray matter volume scores and apolipoprotein E epsilon 4 gene dose in cognitively normal adults.