  Incorporating artificial intelligence and machine learning (AI/ML) methods
within the 5G wireless standard promises autonomous network behavior and
ultra-low-latency reconfiguration. However, the effort so far has purely
focused on learning from radio frequency (RF) signals. Future standards and
next-generation (nextG) networks beyond 5G will have two significant evolutions
over the state-of-the-art 5G implementations: (i) massive number of antenna
elements, scaling up to hundreds-to-thousands in number, and (ii) inclusion of
AI/ML in the critical path of the network reconfiguration process that can
access sensor feeds from a variety of RF and non-RF sources. While the former
allows unprecedented flexibility in 'beamforming', where signals combine
constructively at a target receiver, the latter enables the network with
enhanced situation awareness not captured by a single and isolated data
modality. This survey presents a thorough analysis of the different approaches
used for beamforming today, focusing on mmWave bands, and then proceeds to make
a compelling case for considering non-RF sensor data from multiple modalities,
such as LiDAR, Radar, GPS for increasing beamforming directional accuracy and
reducing processing time. This so called idea of multimodal beamforming will
require deep learning based fusion techniques, which will serve to augment the
current RF-only and classical signal processing methods that do not scale well
for massive antenna arrays. The survey describes relevant deep learning
architectures for multimodal beamforming, identifies computational challenges
and the role of edge computing in this process, dataset generation tools, and
finally, lists open challenges that the community should tackle to realize this
transformative vision of the future of beamforming.
