Decision Making with Sets of Probabilities

Suppose an agent maintains a loss function and a convex set of probability distributions. Because an act is a function from states to consequences, the expected loss of an act can be calculated with respect to any distribution in the set of distributions. And now the agent can compare acts by comparing expected loss.

If an act a₁ has smaller expected loss than another act a₂, no matter which distribution the agent picks from his beliefs, then a₁ has to be better than a₂.

But suppose that the agent picks the distributions and notices that a₁ has expected loss that is sometimes smaller, sometimes larger than the expected loss of a₂. The agent concludes that a₁ and a₂ are not comparable with respect to his beliefs: for all the agent knows, a₁ is not better than, worse than or equal to a₂. The agent is indeterminate with respect to a₁ and a₂.

A Bayesian agent can always say that one option is better than, worse than, or equal to another option. Our agent here may be in a different situation, in an indeterminate state with respect to some acts.

When the agent is represented by a partial order of preferences (and therefore a set of probability distributions), it is not clear how the agent will choose between alternatives that are incomparable. An example clarifies the problem.

An Example of Decision Making with Sets of Probabilities

Suppose the agent has three alternatives, a₁ (go to the park, a₂ (go to the movies), and a₃ (stay home. There are two states of nature, $\theta_1$ (sunny) and $\theta_2$ (cloudy).

The agent has a convex set of probability distributions:

$$p(\theta_1) = 0.3 \alpha + 0.7 (1-\alpha)$$

with $\alpha$ in the interval [0,1]. Remember that $p(\theta_2) = 1 - p(\theta_1)$.

Consider a utility function defined like this:

	sunny	cloudy
a1 ( park)	10	-10
a2 ( market)	-5	4
a3 ( home)	0	0

For a fixed value of $\alpha$, we have:

• Expected utility of a₁ is $E[a_1] = 10 p(\theta_1) - 10 p(\theta_2) = 4 - 8 \alpha$.
• Expected utility of a₂ is $E[a_1] = -5 p(\theta_1) + 4 p(\theta_2) = - 2.3 + 3.6 \alpha$.
• Expected utility of a₃ is $E[a_1] = 0 p(\theta_1) + 0 p(\theta_2) = 0$.

Figure 3 shows a picture of the expected utilities of the acts as $\alpha$ varies:

Figure 3: Expected losses as $\alpha$ varies

If the agent has no preference among the possible values of $\alpha$, then there is no clear ranking of decisions. There is an interval of possible expected loss for a₁, a₂ and a₃. As $\alpha$ varies, these intervals overlap! To see this, consider:

• If $\alpha = 0$, then a₂ is best, a₁ is the worst, and a₃ is better than a₁.
• If $\alpha = 0$, then a₁ is best, a₂ is the worst, and a₃ is better than a₂.

The rational agent has freedom to choose among decisions that are incomparable by expected utility. As far as the agent is concerned, a₁, a₂ and a₃ are incomparable; the agent needs some advice in order to choose a definite action.

Proposals for Decision Making with Sets of Probabilities

There is considerable debate about how an agent makes a decision using sets of probabilities. Some representative examples:

• Secondary measures have been proposed to help select distributions in the credal set [10].
• Good even proposes that a random choice of distributions is rational for a Quasi-Bayesian agent [12].
• Fertig and Breese, in their work with interval probabilities, simply report all admissible plans [2,6]. This leaves the actual actions unspecified.
• I. Levi argues that any admissible plan should be optimal with respect to a distribution in the credal set. He calls such a choice E-admissible [18]. Since there may be several E-admissible plans, Levi suggests secondary guidelines that enforce ``security''.
• Others have suggested the agent should choose minimize the maximum possible value of expected loss, an approach common in robust Statistics under the name of $\Gamma$-minimax [1,10,16].

Take the example above. For Levi, the agent excludes dominated alternatives (like a₃) and then looks at the worst situation in each decision, and picks the decision that presents the best worst case. Decision a₁ can lead to an expected utility of -4; decision a₂ can lead to an expected utility of -1.3; therefore a₂ is better. This approach is non-Bayesian, as can be seen by dropping a₁ from consideration; if this happens, a₃ is the recommended decision. The exclusion of a₁ leads to reversal of preferences (a₂ was preferred to a₃, but now a₃ is preferred to a₂), something inconceivable in Bayesian theory. Now, for the $\Gamma$-minimax approach, a₃ is the best option since it has the best worst case.