  The difficulty of appropriately assigning credit is particularly heightened
in cooperative MARL with sparse reward, due to the concurrent time and
structural scales involved. Automatic subgoal generation (ASG) has recently
emerged as a viable MARL approach inspired by utilizing subgoals in
intrinsically motivated reinforcement learning. However, end-to-end learning of
complex task planning from sparse rewards without prior knowledge, undoubtedly
requires massive training samples. Moreover, the diversity-promoting nature of
existing ASG methods can lead to the "over-representation" of subgoals,
generating numerous spurious subgoals of limited relevance to the actual task
reward and thus decreasing the sample efficiency of the algorithm. To address
this problem and inspired by the disentangled representation learning, we
propose a novel "disentangled" decision-making method, Semantically Aligned
task decomposition in MARL (SAMA), that prompts pretrained language models with
chain-of-thought that can suggest potential goals, provide suitable goal
decomposition and subgoal allocation as well as self-reflection-based
replanning. Additionally, SAMA incorporates language-grounded RL to train each
agent's subgoal-conditioned policy. SAMA demonstrates considerable advantages
in sample efficiency compared to state-of-the-art ASG methods, as evidenced by
its performance on two challenging sparse-reward tasks, Overcooked and MiniRTS.
