Conditional probability measures
One of the most important goals of modeling random phenomena is to account for partial information. We often discover something about the outcome of an experiment before we know the outcome exactly. For example, when we flip a fair coin twice, we see the result of the first flip before we see the result of the second flip, and we would like to define a new probability measure which reflects this intermediate knowledge. We call this a conditional probability measure.
Suppose we observe that the first of two flips is a tail. Then all of the 's which are incompatible with this observation should receive a probability of zero under our conditional probability measure. Since we have no new information about the remaining 's, it makes sense to keep their probabilities in the same proportions as in the original probability measure.
These two observations are sufficient to fully determine the conditional probability measure. In other words, to condition on an event , we set the masses at elements of to 0 and multiply the amount of mass at each point in by to get the total mass up to 1 without changing the proportions:
Given a probability space and an event whose probability is positive, the conditional probability mass function given , written as is defined by
The conditional probability measure given is the measure associated to : for all events , we have
Two objects are submerged in a deep and murky body of water. The objects are chosen to be both
After the objects are released, an observer sees one of them emerge at the water's surface. What is the conditional probability, given the observed information, that the second object will emerge?
Solution. Let's use the given sample space:
The emergence of the object tells us precisely that the event
occurs. The conditional probability of the event given is
One reason that conditional probabilities play such an important role in the study of probability is that in many scenarios they are more fundamental than the probability measure on .
Consider the following experiment: we roll a die, and if it shows 2 or less we select Urn A, and otherwise we select Urn B. Next, we draw a ball uniformly at random from the selected urn. Urn A contains one red and one blue ball, while urn B contains 3 blue balls and one red ball.
Find a probability space which models this experiment, find a pair of events and such that .
Solution. The four possible outcomes of this experiment are (A, blue), (A, red), (B, blue), and (B, red). So we let our probability space consist of those four outcomes.
The probability of the outcome (A, blue) is equal to the probability that Urn A is selected times the conditional probability of selecting a blue ball given that Urn A was selected. We interpret the information that Urn A contains an equal number of blue and red balls as a statement that this conditional probability should be . Therefore, we assign the probability to the event (A, blue).
Likewise, the probabilities we assign to the three other outcomes are , , and ,
With probabilities thus assigned to the outcomes in , we should have where is the event that we select a blue ball and is the event that Urn B was selected. Let us check that this is indeed the case:
We have arrived at an important insight: a probability space may alternatively by specified via a tree diagram showing conditional probabilities, or by the probability space consisting of the endpoints of the tree diagram. We can translate back and forth between these two representations by multiplying along branches to get from the tree's conditional probabilities to 's outcome probabilities or by calculating conditional probabilities to go from to the tree diagram.
Consider three random variables , , and , each of which is equal to with probability 0.6 and to 0 with probability 0.4. These random variables are not necessarily independent.
- Find the greatest possible value of the event .
- Find the least possible value of the event .
- By monotonicity, we have
We note that this maximum can be attained by setting
- The least possible value is zero. This minimum can be attained, for example, if we take , with probability mass , , and ,
respectively, and set , , , , , , and .
Bayes' theorem tells us how to update beliefs in light of new evidence. It relates the conditional probabilities and :
The last step follows from writing out as .
Bayes' theorem has many applications to everyday life, some intuitive and others counterintuitive.
Suppose you're 90% sure that your package was delivered today and 75% sure that if it was delivered it would be on your door step rather than tucked away in your mailbox. When you arrive at home and do not see your package right away, what is the conditional probability—given the observed information—that you'll find it in your mailbox?
Solution. The desired conditional probability is , which by Bayes' theorem is
Suppose a disease has 0.1% prevalence in the population and has a test with 90% reliability. A random selected person is tested for the disease and tests positive. What is the conditional probability that the person has the disease, given the positive test result?
Solution. Let be the event that a person has the disease and be the event that a person tests positive to the test. We would like to find given that , and . By Bayes' Theorem,
(Note: The fact that follows from the fact that the test is unreliable. In general, it is not the case that for any two events and .)