I recently stumbled over this tweet:

Suppose you think you’re 80% likely to have left your laptop power adapter somewhere inside a case with 4 otherwise-identical compartments. You check 3 compartments without finding your adapter. What’s the probability that the adapter is inside the remaining compartment?

Eliezer Yudkowski

I thought I had trained my mind to update whenever new information arrives, but it still didn’t. My first thought was still “80% because nothing has changed about the bag”. But by looking through 3 compartments of the bag, you have gained information and should update your estimates! So I thought it through and came up with a quite general and useful rule:

The odds of finding x are updated by the fraction of space you haven’t looked at yet.

In Eliezer’s example, the prior odds where 4:1 that your power adapter is somewhere inside the case. So far you have looked through 3/4 of the case, so the fraction of space you haven’t looked at is 1/4. The odds of finding your adapter in the remaining compartment are therefore 1:1 (or 50% probability).

This works whenever you make a prediction about finding something in a region of space (or time!) and look through some part of it without finding anything.

Why is that?

Let’s derive this rule. The odds of finding x in some non-empty set A, given that we haven’t found it in the subset S \subseteq A are

\frac{P(x \in A | x \notin S)}{P(x \notin A | x \notin S)} = \frac{P(x \notin S | x \in A)\,P(x \in A)\,P(x \notin S)}{P(x \notin S | x \notin A)\,P(x \notin A)\,P(x \notin S)}\;\;.

This is just Bayes theorem applied twice, once to the numerator and once to the denominator. The P(x \notin S) terms cancel out and the probability to not find x in the subset S, given that x isn’t in A is P(x \notin S | x \notin A) = 1 (you won’t find anything that isn’t there).

Now, the probability to not find x in S, given that we know that x is somewhere in A should be the probability to find x in the compliment A \backslash S (the part of A that we haven’t looked at yet), so

P(x \notin S | x \in A) = P(x \in (A \backslash S) | x \in A) = \frac{|A \backslash S|}{|A|}\;\;.

Thus, we arrive at the above-mentioned rule

\frac{P(x \in A | x \notin S)}{P(x \notin A | x \notin S)} = \frac{|A \backslash S|}{|A|}\,\frac{P(x \in A)}{P(x \notin A)}\;\;.

The left hand side are the posterior odds of 1:1 (how likely we think it is to find the adapter after we’ve looked through 3/4 of the case). The first fraction on the right hand side is the Bayes factor of 1/4, and the second fraction on the right hand side represents the prior odds of 4:1 (what we believed before looking into the bag).

A quick sanity check: If we don’t look in the bag, S is empty and therefore A\backslash S = A, so the Bayes factor is 1 — the odds don’t change. If we look through the entire bag and don’t find anything, S = A and the Bayes factor is 0 — it is impossible that we’d find it in the bag, since we’ve seen that it’s not there.

P.S.: In case you don’t trust my calculation or reasoning here, you can verify the rule with this Monte Carlo simulation.