  At hybrid analog-digital (HAD) transceiver, an improved HAD rotational
invariance techniques (ESPRIT), called I-HAD-ESPRIT, is proposed to measure the
direction of arrival (DOA) of desired user, where the phase ambiguity due to
HAD structure is addressed successfully. Subsequently, a machine-learning (ML)
framework is proposed to improve the precision of measuring DOA. In the
training stage, the HAD transceiver works as a receiver and repeatedly measures
the values of DOA via I-HAD-ESPRIT to form a slightly large training data set
(TDS) of DOA. From TDS, we find that the probability density function (PDF) of
DOA measurement error (DOAME) is approximated as a Gaussian distribution by the
histogram method in ML. This TDS is used to learn the mean of DOA and the
variance of DOAME, which are utilized to infer their values and improve their
precisions in the real-time stage. The HAD transceiver rapidly measures the
real-time value of DOA some times to form a relatively small real-time set
(RTS), which is used to learn the real-time mean and variance of DOA/ DOAME.
Then, three weight combiners are proposed to combine
the-maximum-likelihood-learning outputs of TDS and RTS. Their weight factors
depend intimately on the numbers of elements in TDS and RTS, and
signal-to-noise ratios during the two stages. Using the mean and variance of
DOA/DOAME, their PDFs can be given directly, and we propose a robust beamformer
for directional modulation (DM) transmitter with HAD by fully exploiting the
PDF of DOA/DOAME, especially a robust analog beamformer on RF chain. Simulation
results show that: 1) The proposed I-HAD-ESPRIT can achieve the HAD CRLB; 2)
The proposed ML framework performs much better than the corresponding real-time
one without training stage, 3) The proposed robust DM transmitter can perform
better than the corresponding non-robust ones in terms of bit error rate and
secrecy rate.
