  Machine-to-Machine (M2M) communication is crucial in developing Internet of
Things (IoT). As it is well known that cellular networks have been considered
as the primary infrastructure for M2M communications, there are several key
issues to be addressed in order to deploy M2M communications over cellular
networks. Notably, the rapid growth of M2M traffic dramatically increases
energy consumption, as well as degrades the performance of existing
Human-to-Human (H2H) traffic. Sustainable operation technology and resource
management are efficacious ways for solving these issues. In this paper, we
investigate a resource management problem in cellular networks with H2H/M2M
coexistence. First, considering the energy-constrained nature of machine type
communication devices (MTCDs), we propose a novel network model enabled by
simultaneous wireless information and power transfer (SWIPT), which empowers
MTCDs with the ability to simultaneously perform energy harvesting (EH) and
information decoding. Given the diverse characteristics of IoT devices, we
subdivide MTCDs into critical and tolerable types, further formulating the
resource management problem as an energy efficiency (EE) maximization problem
under divers Quality-of-Service (QoS) constraints. Then, we develop a
multi-agent deep reinforcement learning (DRL) based scheme to solve this
problem. It provides optimal spectrum, transmit power and power splitting (PS)
ratio allocation policies, along with efficient model training under designed
behaviour-tracking based state space and common reward function. Finally, we
verify that with a reasonable training mechanism, multiple M2M agents
successfully work cooperatively in a distributed way, resulting in network
performance that outperforms other intelligence approaches in terms of
convergence speed and meeting the EE and QoS requirements.
