We need a formal definition of the concept of having private utilities be "aligned" with G. Constructing such a formalization is a subtle exercise. For example, consider systems where the world utility is the sum of the private utilities of the individual agents.  This might seem a reasonable candidate for an example of "aligned" utilities. However such systems are examples of the more general class of systems that are "weakly trivial". It is well-known that in weakly trivial systems each individual agent greedily trying to maximize its own utility can lead to the tragedy of the commons  (Hardin, 1968; Crowe, 1969) and actually minimize G. In particular, this can be the case when private utilities are independent of time and G = E n g n . Evidently, at a minimum, having G = E n g n is not sufficient to ensure that we have "aligned" utilities; some alternative formalization of the concept is needed. Note that in the simple network discussed in Section 2.1, the utilities are weakly trivial, since G ( ī, ī, ī, ) = gx ( ī, ) + gy ( ī, ). This provides another perspective on the suboptimality of ISPA in that network.

A more careful alternative formalization of the notion of aligned utilities is the concept of "factored" systems. A system is factored at time τ when the following holds for each agent η individually: A change at time τ to the state of η alone, when propagated across time, will result in an increased value of g η,τ (ζ) if and only if it results in an increase for G(() (Wolpert & Tumer, 1999).

For a factored system, the side-effects of any change to η 's t = τ state that increases its private utility cannot decrease world utility. There are no restrictions though on the effects of that ange on the private utilities of other agents and /or times. In particular, we don't preclude an agent's algorithm at two different times from "working at cross-purposes" to each other, so long as at both moments the agent is working to improve G . In game-theoretic terms, in factored systems optimal global behavior corresponds to the agents' always being at a private utility Nash equilibrium (Fudenberg & Tirole, 1991). In this sense, there can be no tragedy of the commons for a factored system. As a trivial example, a system is factored for g n, τ = G ∀η, a system conventionally called a 'team game'.

Furthermore, if our system is factored with respect to private utilities { g n, τ }, we want each agent to be in a state at time r that induces as high a value of the associated private utility as possible (given the initial states of the other agents). Assume η is ML-based and able to achieve fairly large values of most private utilities we are likely to set it for time r, i.e., assume that given that private utility gn,r, the rest of the components of C,, are set by n's algorithm in such a way so as to achieve a relatively high value of gn,,. So our problem becomes determining for what { g n, τ } the agents will best be able to achieve high g n (subject to each other's actions) while also causing dynamics that is factored for G and the {9n,r}
Define the effect set of the agent-time pair ( η, τ ) at ζ, C ( η, τ ) ( ζ ), as the set of all agents
  
respect to the state of agent n at t = r. Intuitively, (n,r)'s effect set is the set of the states of all agents L n ,t>, that would be affected by a change in the state of agent n at time r .

Next, for any set σ of agents (η′,t), define CL σ (ζ) as the "virtual" vector formed by clamping the components of the vector ζ delineated in σ to an arbitrary fixed value, which