  By employing powerful edge servers for data processing, mobile edge computing
(MEC) has been recognized as a promising technology to support emerging
computation-intensive applications. Besides, non-orthogonal multiple access
(NOMA)-aided MEC system can further enhance the spectral-efficiency with
massive tasks offloading. However, with more dynamic devices brought online and
the uncontrollable stochastic channel environment, it is even desirable to
deploy appealing technique, i.e., intelligent reflecting surfaces (IRS), in the
MEC system to flexibly tune the communication environment and improve the
system energy efficiency. In this paper, we investigate the joint offloading,
communication and computation resource allocation for IRS-assisted NOMA MEC
system. We firstly formulate a mixed integer energy efficiency maximization
problem with system queue stability constraint. We then propose the
Lyapunov-function-based Mixed Integer Deep Deterministic Policy Gradient
(LMIDDPG) algorithm which is based on the centralized reinforcement learning
(RL) framework. To be specific, we design the mixed integer action space
mapping which contains both continuous mapping and integer mapping. Moreover,
the award function is defined as the upper-bound of the Lyapunov
drift-plus-penalty function. To enable end devices (EDs) to choose actions
independently at the execution stage, we further propose the Heterogeneous
Multi-agent LMIDDPG (HMA-LMIDDPG) algorithm based on distributed RL framework
with homogeneous EDs and heterogeneous base station (BS) as heterogeneous
multi-agent. Numerical results show that our proposed algorithms can achieve
superior energy efficiency performance to the benchmark algorithms while
maintaining the queue stability. Specially, the distributed structure
HMA-LMIDDPG can acquire more energy efficiency gain than centralized structure
LMIDDPG.
