text: Increased stress, fuel consumption, air pollution, accidents and delays are some of the consequences of traffic congestion usually incurring in tremendous economic impacts, which society aims to remedy in order to leverage a sustainable development. Recently, unconventional means for modeling and controlling such complex traffic systems relying on multi-agent systems have arisen. This paper contributes to the understanding of such complex and highly dynamic systems by proposing an open-source tool-chain to implement multi-agent-based solutions in traffic and transportation. The proposed approach relies on two very popular tools in both domains, with focus on traffic light control. This tool-chain consists in combining JADE (Java Agent DEvelopment Framework), for the implementation of multi-agent systems, with SUMO (Simulation of Urban MObility), for the microscopic simulation of traffic interactions. TraSMAPI (Traffic Simulation Manager Application Programming Interface) is used to combine JADE and SUMO allowing communication between them. A demonstration of the concept is presented to illustrate the main features of this tool-chain, using Q-Learning as the reinforcement learning method for each traffic light agent in a simulated network. Results demonstrate the feasibility of the proposed framework as a practical means to experiment with different agent-based designs of intelligent transportation solutions.
  @cite_14: 
  @cite_15: This research develops and evaluates a new multi-agent adaptive traffic signal control system based on swarm intelligence and the neural-fuzzy actor-critic reinforcement learning (NFACRL) method. The proposed method combines the better attributes of swarm intelligence and the NFACRL method. Two scenarios are used to evaluate the method and the new NFACRL-Swarm method is compared with its NFACRL counterpart. First, the proposed control model is applied to isolated intersection signal adaptive control to evaluate its learning performance. Then, the control system is implemented in signal control coordination in a typical arterial. In the isolated intersection, the proposed hybrid method outperforms its previous counterpart by improving the learning speed and is shown to be insensitive to reward function parameters. In the network, by introducing a coordination scheme inspired by swarm intelligence, the proposed method improves the performance by up to 12 and has a faster learning speed.
  @cite_16: The operation of large dynamic systems such as urban traffic networks remains a challenge in control engineering to a great extent due to their sheer size, intrinsic complexity, and nonlinear behavior. Recently, control engineers have looked for unconventional means for modeling and control of complex dynamic systems, in particular the technology of multi-agent systems whose appeal stems from their composite nature, flexibility, and scalability. This paper contributes to this evolving technology by proposing a framework for multi-agent control of linear dynamic systems, which decomposes a centralized model predictive control problem into a network of coupled, but small sub-problems that are solved by the distributed agents. Theoretical results ensure convergence of the distributed iterations to a globally optimal solution. The framework is applied to the signaling split control of traffic networks. Experiments conducted with simulation software indicate that the multi-agent framework attains performance comparable to conventional control. The main advantages of the multi-agent framework are its graceful extension and localized reconfiguration, which require adjustments only in the control strategies of the agents in the vicinity.
  @cite_9: Coping with dynamic changes in traffic volume has been the object of recent publications. Recently, a method was proposed, which is capable of learning in non-stationary scenarios via an approach to detect context changes. For particular scenarios such as the traffic control one, the performance of that method is better than a greedy strategy, as well as other reinforcement learning approaches, such as Q-learning and Prioritized Sweeping. The goal of the present paper is to assess the feasibility of applying the above mentioned approach in a more realistic scenario, implemented by means of a microscopic traffic simulator. We intend to show that to use of context detection is suitable to deal with noisy scenarios where non-stationarity occurs not only due to the changing volume of vehicles, but also because of the random behavior of drivers in what regards the operational task of driving (e.g. deceleration probability). The results confirm the tendencies already detected in the previous paper, although here the increase in noise makes the learning task much more difficult, and the correct separation of contexts harder.
summary: Regarding this simulation tools some examples of multi-agent system approaches for traffic lights control can be seen in @cite_16 , @cite_14 , @cite_15 and @cite_9 . Simulators used in these works were Aimsun, ITSUMO, VisSim and ITSUMO, respectively.