taking a risky autonomous action. Informally, the agent reasons that it should eventually make a decision if the expected cost of continued waiting exceeds the difference between the user's decision quality and its own. More formally, the agent should eventually take back decision-making control iff, for some time t :

$$\int_{t}^{<t}P_{\top}(t^{\prime}){\mathcal W}(t^{\prime}).d t^{\prime}-{\mathcal W}(t)>E\,Q_{U}^{d}(t)-E\,Q_{A}^{d}(t)$$
$$\left(\,7\,\right)$$

where the left-hand side calculates the future expected wait costs and the right-hand side calculates the extra utility to be gained by getting a response from the user. This result leads to the following general conclusion about strategies that end with giving control back to the agent:
LEMMA 1:  If s E S is a strategy ending with e E , and s' is sA, then EU d > EU d iff Ve E E , Ðt < < such that fi < P + (t ′ ) W (t ′ ) .dt ′ − W (t) > E Q e (t) − E Q a (t)
Lemma 1 says that if, at any point in time, the expected cost of indefinitely leaving control in the hands of the user exceeds the difference in quality between the agent's and user's decisions, then strategies which ultimately give the agent control dominate those which do not. Thus, if the rate of wait cost accrual increases or the difference in the relative quality of the decision-making abilities decreases or the user's probability of response decreases, then strategies where the agent eventually takes back control will dominate. A
key consequence of the Lemma (in the opposite direction) is that, if the rate that costs accrue does not accelerate, and if the probability of response stays constant (i.e., Markovian), then the agent should indefinitely leave control with the user (if the user had originally been given control), since the expected wait cost will not change over time. Hence, even if the agent is faced with a situation with potentially high total wait costs, the optimal strategy may be a one-shot strategy of handing over control and waiting indefinitely, because the expected future wait costs at each point in time are relatively low. Thus, Lemma 1 isolates the condition under which we should consider appending an A transfer-of-control action to our strategy.

We can perform a similar analysis to identify the conditions under which we should include a D action in our strategy. The agent has incentive in changing coordination constraints via a D action due to the additional time made available for getting a highquality response from an entity. However, the overall value of a D action depends on a number of factors (e.g., the cost of taking the D action and the timing of subsequent transfers of control). We can calculate the expected value of a D by comparing the EU of a strategy with and without a D . The D is useful if and only if the increased expected value of the strategy with it is greater than its cost, Dcost, LEMMA 2: if s E S has no D and s is s with a D included at t then EU s > EU s iff ʃ P + (t')W(t).dt' - ʃ P + (t')W(t|D).dt' > Dco st We can illustrate the consequences of Lemma 2 by considering the specific problem model of Appendix I (i.e., P † (t) = ρ exp − ρ t , W(t) = ω exp ω t , E Q ξ (t) = c, and candidate strategies eA and eDA ). In this case, EU eDA > EU eA iff -(p-w)wpexp-(p-w)A(1 - exp-wP ealse ) > Dcost . Figure 4 plots the value of the D action as we vary the rate of wait cost accumulation, w, and the parameter of the Markovian response probability function, p. The graph shows