basic_decisions.qmd

# [Basic decision problems]{.lightblue} {#sec-basic-decisions}
{{< include macros.qmd >}}

Decision Theory analyses any decision-making problem in terms of nested or sequential *basic* or *minimal* decision problems. The assembly-line scenario of the [introduction @sec-intro] is an example.

## Graphical representation and elements

A basic decision problem can be represented by a diagram like this:

<!-- ::::: {.column-page-inset-right} -->
![](basic_decision_tree.png){width=100%}

It has one [**decision node**]{.blue}, usually represented by a square {{< fa regular square >}}, from which the available decisions depart as lines. Each decision leads to an [**inference node**]{.blue},^[also called *chance node* or *uncertainty node*] usually represented by a circle {{< fa regular circle >}}, from which the possible outcomes depart as lines. Each outcome leads to a particular gain or loss, depending on the decision. The uncertainty of each outcome is quantified by a probability.

A basic decision problem is analysed in terms of the following elements:


::::{.column-margin}
:::{.callout-tip}
##  
[{{< fa seedling >}} Remember: What matters is to be able to identify these elements in a concrete problem, understanding their role. Their technical names don't matter.]{.small}
:::
::::

- [{{< fa cube >}} **Agent**]{.blue}, and [**background**]{.blue} or  [**prior information**]{.blue}. The agent is the person or device that has to make the decision. An agent possesses (or has been programmed with) specific background information that is used and taken for granted in the decision-making process. This background information determines the probabilities, gains, and losses of the outcomes, together with other available data and information. Different agents typically have different background information.

:::{.column-margin}
*Agent* means "conductor", "mover", and similar (from Latin *ago* = *to move* or *drive* and similar meanings).

We'll use the neutral pronouns *it*/*its* when referring to an agent, since an agent could be a person or a machine.
:::

- [{{< fa cube >}} **Data**]{.blue} and other [**additional information**]{.blue}, sometimes called [**evidence**]{.blue}. They differ from the background information in that they can change with every decision instance made by the same agent, while the background information stays the same. In the assembly-line scenario, for example, the test results could be different for every new electric component.


- [{{< fa cube >}} **Decisions**]{.blue} available to the agent. They are assumed to be mutually exclusive and exhaustive; this can always be achieved by recombining them if necessary, as we'll discuss later.

:::{.column-margin}
*Decisions* are called *courses of action* in some literature.
:::

- [{{< fa cube >}} **Outcomes**]{.blue} of the possible decisions. Every decision can have a different set of outcomes, or some outcomes can appear for several or all decisions (in this case they are reported multiple times in the decision diagram). Note that even if an outcome can happen for two or more different decisions, its probabilities can still be different depending on the decision.

:::{.column-margin}
Many other terms instead of *outcome* are used in the literature, for instance *state* or *event*.
:::

- [{{< fa cube >}} **Probabilities**]{.blue} for each of the outcomes and for each decision. Their values typically depend also on the background information and the additional data.

- [{{< fa cube >}} **Utilities**]{.blue}: the gains or losses associated with each possible outcome and each decision. We shall mainly use the term [**utility**]{.blue}, instead of "gain", "loss", and similar, for several reasons:

	+ gain and losses may involve not money, but time, or energy, or health, or emotional value, or other kinds of commodities and things that are important to us; or even a combination of them. The term "utility" is useful as a neutral term that doesn't mean "money", but depends on the context
    + we can just use one term instead of two: for example, when the utility is positive it's a "gain"; when it's negative it's a "loss"

    The particular numerical values of the utilities are always context-dependent: they may depend on the background information, the decisions, the outcomes, and the additional data.

    [We shall sometimes use the generic currency sign\ \ [**¤**]{.blue}\ \ to denote utilities, to make clear that gains and losses do not necessarily involve money, and not to reference any country in particular.]{.blue}

\

The relation between the elements above can be depicted as follows -- but note that this is just an intuitive illustration:
![](basic_decision_tree_agent.png){width=75%}




:::{.callout-important}
## {{< fa exclamation-triangle >}} Don't over-interpret the decision diagram

- The diagram above *doesn't have any temporal meaning*, that is, it doesn't mean that the decisions happen before the outcomes, or vice versa.
    
	In some situations the outcome can be realized after the decision is made; for instance, someone bets on heads or tails, and then a coin is tossed.
	
	In other situations, the outcome can be realized before the decision is made; for instance, sometimes a coin is tossed and covered, then one is asked to bet on what the outcome was. Another example is some research decision made by a archaeologist, the unknown being some detail about a dinosaur from millions of years ago.
	
	In yet other situations the outcome may have a complex nature, and it may be realized partly before the decision is made, and partly after; for instance, someone can bet on the outcome of two coin tosses; one coin is tossed before the decision is made, and the other after.

- The diagram above is not something that an agent *must* use in making decisions. It is not part of the theory. It's just a very convenient way to visualize and operate with the mathematics underlying the theory.

- It not always the case that the *outcomes* are unknown and the *data* are known. As we'll discuss later, in some situations we reason in hypothetical or counterfactual ways, using hypothetical data and considering outcomes which have already occurred. In such situations we can still use diagrams like the one above, because the help us doing the calculation, although the actual outcome is already known.
:::



::: {.callout-warning}
## {{< fa book >}} Study reading
- §1.1.4 in [*Artificial Intelligence*](https://hvl.instructure.com/courses/28605/modules)

- *Skim through* Ch. 15 of [*Artificial Intelligence*](https://hvl.instructure.com/courses/28605/modules). No need to read thoroughly: just quickly glimpse whether there are ideas and notions that look familiar (a little like when you're in a large crowd and look quickly around to see if there are any familiar faces)
:::

:::{.callout-caution}
## {{< fa user-edit >}} Exercise
- Identify the elements above in the assembly-line decision problem of the [introduction @sec-intro].
- Sketch the decision diagram for the assembly-line decision problem.
:::

Some of the decision-problem elements listed above may need to be in turn analysed by a decision sub-problem. For instance, the utilities could depend on uncertain factors: thus we have a decision sub-problem to determine the optimal values to be used for the utilities of the main problem. This is an example of the modular character of decision theory.

We shall soon see how to mathematically represent these elements.

The elements above must be identified unambiguously in every decision problem. The analysis into these elements greatly helps in making the problem and its solution well-defined.

An advantage of decision theory is that its application *forces* us to make sense of an engineering problem. A useful procedure is to formulate the general problem in terms of the elements above, identifying them clearly. If the definition of any of the terms involves uncertainty of further decisions, then we analyse it in turn as a decision sub-problem, and so on.

> Suppose someone (probably a politician) says: "We must solve the energy crisis by reducing energy consumption or producing more energy". From a decision-making point of view, this person has effectively said *nothing whatsoever*. By definition the "energy crisis" is the problem that energy production doesn't meet demand. So this person has only said "we would like the problem to be solved", without specifying any solution. A decision-theory approach to this problem requires us to specify which concrete courses of action should be taken for reducing consumption or increasing productions, and what their probable outcomes, costs, and gains would be.

:::{.column-margin}
:::: {.callout-tip}
## {{< fa rocket >}} For the extra curious
See MacKay's options-vs-costs rational analysis in [Sustainable Energy -- without the hot air](https://www.withouthotair.com)
::::
:::


## Setting up a basic decision problem {#sec-decision-matrices}

A basic decision problem can be set up along the following steps (which we illustrate afterwards with a couple of examples):

:::{.callout-note}
## Setup of a basic decision problem

1. **List all available decisions**

2. **For each decision, list its possible outcomes**

3. **Pool together all outcomes of all decisions, counting the common ones only once**

4. **Prepare two tables: in each, display the decisions as rows, and the pooled outcomes as columns** (or you can do the opposite: decisions as columns and outcomes as rows)

5. **In one table, report the probabilities for all decision-outcome pairs. If an outcome is not available for that decision, give it a $0\%$ probability**

6. **In the other table, report the utilities for all decision-outcome pairs. If an outcome is not available for that decision, give it a $0$ utility**
:::



#### Example: the assembly-line problem

Let's apply the steps above in the assembly-line example of [ch. @sec-intro]:

[**1. List all available decisions**]{.midgrey .small}

Easy: they are "[accept]{.green} the electronic component" and "[discard]{.yellow} it".

\

[**2. For each decision, list its possible outcomes**]{.midgrey .small}

In general you will notice that some outcomes may be common to all decisions, while other outcomes can happen for some decisions only, or even for just one decision.

In the present example, the [accept]{.green} decision has two possible outcomes: "the component works with [no faults]{.blue} for at least a year" and "the component [fails]{.red} within a year".

The [discard]{.decision} cannot have those outcomes, because the component is discarded. It has indeed only one outcome: "component [discarded]{.purple}".

\

[**3. Pool together all outcomes of all decisions, counting the common ones only once**]{.midgrey .small}

In total we have *three* pooled outcomes:

- [**no faults**]{.blue} (from the [accept]{.green} decision)
- [**fails**]{.red} (from the [accept]{.green} decision)
- [**discarded**]{.purple} (from the [discard]{.yellow} decision)

\

[**4. Prepare two tables: in each, display the decisions as rows, and the pooled outcomes as columns** (or you can do the opposite: decisions as columns and outcomes as rows)]{.midgrey .small}

In the present example each table looks like this:

|                               | [no faults for a year]{.blue .small} | [fails within a year]{.red .small}   |       [discarded]{.purple .small}    |
|:------------------------------|:------------------------------------:|:------------------------------------:|:------------------------------------:|
| [**accept**]{.green .small}   |                                      |                                      |                                      |
| [**discard**]{.yellow .small} |                                      |                                      |                                      |
: {.sm}

\

[**5. In one table, report the probabilities for all decision-outcome pairs. If an outcome is not available for that decision, give it a $0\%$ probability**]{.midgrey .small}

|                               | [no faults for a year]{.blue .small} | [fails within a year]{.red .small}   |       [discarded]{.purple .small}    |
|:------------------------------|:------------------------------------:|:------------------------------------:|:------------------------------------:|
| [**accept**]{.green .small}   |       $90\%$                         |           $10\%$                     |   [$0\%$]{.grey}                     |
| [**discard**]{.yellow .small} |       [$0\%$]{.grey}                 |           [$0\%$]{.grey}             |                $100\%$               |
: [*Probability table*]{.midgrey} {.sm}

Note how the outcomes that do not exist for a particular decision have been given a [$0\%$]{.grey} probability (in [grey]{.grey}). This is just a way of saying "this outcome can't happen, if this decision is made".

\

[**6. In the other table, report the utilities for all decision-outcome pairs. If an outcome is not available for that decision, give it a $0$ utility**]{.midgrey .small}


|                               | [no faults for a year]{.blue .small} | [fails within a year]{.red .small}   |       [discarded]{.purple .small}    |
|:------------------------------|:------------------------------------:|:------------------------------------:|:------------------------------------:|
| [**accept**]{.green .small}   |            $+1\$$                    |          $-11\$$                     |           [$0\$$]{.grey}             |
| [**discard**]{.yellow .small} |           [ $0\$$]{.grey}            |          [$0\$$]{.grey}              |            $0\$$                     |
: [*Utility table*]{.midgrey} {.sm}

Note how the outcomes that do not exist for a particular decision have been given a [$0\$$]{.grey} utility (in [grey]{.grey}). We shall see later that it actually doesn't matter which utilities we give to these impossible outcomes.

\

:::{.callout-caution}
## {{< fa user-edit >}} Exercises
Apply the steps above to the following basic decision problems (you only need to set them up with their probability & utility tables, but feel free to solve them as well, if you like):

- The "heads-bet" vs "tails-bet" example of [§@sec-optimality]. Assume that the "small amount" of money is $10\$$, the "large amount" is $1000\$$, and the two outcomes' probabilities are $50\%$ each.

- Peter must reach a particular destination, and is undecided between two alternatives: *go by car*, or *ride a bus*, or *go on foot*. If he goes by car, he could *arrive without problems*, with a probability of $80\%$ and a utility of $10\,¤$, or he could *get stuck in a traffic jam* and arrive late, with a probability of $20\%$ and a utility of $-10\,¤$.  If he rides a bus, he could *arrive without problems*, with a probability of $95\%$ and a utility of $15\,¤$, or arrive in time but *travelling in a fully-packed bus*, with a probability of $5\%$ and a utility of $-10\,¤$. If he goes on foot, he could *arrive without problems*, with a probability of $20\%$ and a utility of $20\,¤$, or he could *get soaked from rain*, with a probability of $80\%$ and a utility of $-5\,¤$.<!-- 6, 13.75, 0 -->
    
	(We are using the symbol\ \ "$¤$"\ \ because Peter's utilities are a combination of money savings, time of arrival, and comfort.)
:::

\

<!-- :::{.column-page-right} -->
<!-- |                               | [arrive, no problems]{.blue .small} | [traffic jam]{.red .small}   |       [full-packed bus]{.lightblue .small}    |[soaked from rain]{.purple .small}    | -->
<!-- |:------------------------------|:------------------------------------:|:------------------------------------:|:------------------------------------:|:------------------------------------:| -->
<!-- | [**car**]{.green .small}   |                                      |                                      |                                      | -->
<!-- | [**bus**]{.midgrey .small} |                                      |                                      |                                      | -->
<!-- | [**foot**]{.yellow .small} |                                      |                                      |                                      | -->
<!-- : {.sm} -->
<!-- ::: -->

## How to make a basic decision? {#sec-make-decision}

Up to now we have seen what are the elements of a basic decision problem, and how to arrange them in a diagram and with tables. *But how do we determine what's the optimal decision?*

Decision Theory says that the optimal decision is determined by the "[**principle of maximal expected utility**]{.blue}".

We shall study this principle more in detail toward the end of the course, although you already know its basic idea, because you intuitively used this very principle in solving all decision problems we met so far, starting from the assembly-line one.

However, let's quickly describe already now the basic procedure for this principle:

:::{.callout-note}
## Principle of maximal expected utility

1. For each decision, multiply the probability and the utility of each of its outcomes, and then sum up these products. This way you obtain the *expected utility* of the decision.

2. Choose the decision that has the largest expected utility; if several decisions are maximal, choose any of them unsystematically.
:::

This procedure can also be described in terms of the probability and utility tables introduced in the previous section:

a. [Multiply element-by-element the probability table and the utility table, obtaining a new table with the same number of rows and columns]{.green}
b. [Sum up the elements of each row of the new table (this sum is the expected utility); remember that every row corresponds to a decision]{.green}
c. [Choose the decision corresponding to the largest of the sums above; if there are several maximal ones, choose among them unsystematically]{.green}

#### Example: the assembly-line problem

Multiplying the *Probability table* and the *Utility table* above, element-by-element, we obtain the following table, where we also indicate the sum of each row:

:::{.column-page-right}

|                               | [no faults for a year]{.blue .small} | [fails within a year]{.red .small}   |       [discarded]{.purple .small}    | *sum*                                |
|:------------------------------|:------------------------------------:|:------------------------------------:|:------------------------------------:|:------------------------------------:|
| [**accept**]{.green .small}   |            $+0.9\$$                    |          $-1.1\$$                     |           [$0\$$]{.grey}          | $\boldsymbol{-0.2\$}$                             |
| [**discard**]{.yellow .small} |           [ $0\$$]{.grey}            |          [$0\$$]{.grey}              |            $0\$$                     |$\boldsymbol{0\$}$                                 |
: [*Probability × Utility table*]{.midgrey} {.sm}

:::

and, as we already knew, [discard]{.yellow}ing the electronic component is the decision with the maximal expected utility.



:::{.callout-caution}
## {{< fa user-edit >}} Exercise
Feel free to sketch some code (in your preferred programming language) that chooses the optimal decision according to the principle above. The code should take two inputs: the table or matrix of probabilities, and the table or matrix of utilities; and should give one output: the row-number of the optimal decision.
:::

## Plan for the next chapters

The [**expected-utility maximization**]{.blue} above is intuitive and simple, and is the last stage in a basic decision problem.

But there are two stages which occur before, and which are the most difficult:

[{{< fa cube >}} **Inference**]{.blue}
: is the stage where the probabilities of the possible outcomes are calculated. Its rules are given by the [**Probability Calculus**]{.blue}. Inference is independent from decision: in some situations we may simply wish to assess whether some hypotheses, conjectures, or outcomes are more or less plausible than others, without making any decision. This kind of assessment can be very important in problems of communication and storage, and it is specially considered by [**Information Theory**]{.blue}.

The calculation of probabilities can be the part that demands most thinking, time, and computational resources in a decision problem. It is also the part that typically makes most use of data -- and where data can be most easily misused.

Roughly half of this course will be devoted in understanding the laws of inference, their applications, uses, and misuses.
\

[{{< fa cube >}} **Utility assesment**]{.blue}
: is the stage where the gains or losses of the possible outcomes are calculated. Often this stage requires further inferences and further decision-making sub-problems. The theory underlying utility assessment is still much underdeveloped, compared to probability theory.
\

<!-- [{{< fa cube >}} **Expected-utility maximization**]{.blue} is the final stage where the probabilities and gains or costs of the possible outcomes are combined, in order to determine the optimal decision. -->
\

We shall now explore each of these two stages. We take up inference first because it is the most demanding and probably the one that can be optimized the most by new technologies.