Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Fusion of medical imaging and electronic health records with attention
  and multi-head machanisms   Doctors often make diagonostic decisions based on patient's image scans, such
as magnetic resonance imaging (MRI), and patient's electronic health records
(EHR) such as age, gender, blood pressure and so on. Despite a lot of automatic
methods have been proposed for either image or text analysis in computer vision
or natural language research areas, much fewer studies have been developed for
the fusion of medical image and EHR data for medical problems. Among existing
early or intermediate fusion methods, concatenation of features from both
modalities is still a mainstream. For a better exploiting of image and EHR
data, we propose a multi-modal attention module which use EHR data to help the
selection of important regions during image feature extraction process
conducted by traditional CNN. Moreover, we propose to incorporate multi-head
machnism to gated multimodal unit (GMU) to make it able to parallelly fuse
image and EHR features in different subspaces. With the help of the two
modules, existing CNN architecture can be enhanced using both modalities.
Experiments on predicting Glasgow outcome scale (GOS) of intracerebral
hemorrhage patients and classifying Alzheimer's Disease showed the proposed
method can automatically focus on task-related areas and achieve better results
by making better use of image and EHR features.

36
The categories of this paper are: cmp-lg
3
The categories of this paper are: physics
0
The categories of this paper are: hep-ph
32
The categories of this paper are: atom-ph
35
The categories of this paper are: mtrl-th
5
The categories of this paper are: gr-qc
15
The categories of this paper are: nucl-ex
6
The categories of this paper are: astro-ph
21
The categories of this paper are: adap-org
30
The categories of this paper are: q-alg
34
The categories of this paper are: chem-ph
17
The categories of this paper are: q-fin
29
The categories of this paper are: funct-an
27
The categories of this paper are: comp-gas
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys