In his “The Probability of Induction,” C.S. Peirce describes the formulas for adding and multiplying probabilities. The formulae are the same as I remember from college, but the exposition is perhaps interestingly skewed. That’s because Peirce attaches probability to specific inference rules that predict conclusions from premises, rather than just the recording of events that happen or don’t (with premises or causes left unmentioned).
This page is a companion to my discussion of the whole of Peirce’s essay.
Peirce is working from a frequentist interpretation of probability, in which the probability attached to this inference rule:
Observation of A means you will also observe B
… is calculated by dividing «how often both A and B are observed» by «how often A is observed».
Rather than express inference rules as «if A then B», which is only one of many possible inference rules, I’ll write «A suggests B». So «A suggests B» means that if you observe A, there’s an associated probability p that you’ll observe B.
Rule for the addition of probabilities
Suppose the rule «A suggests B» has a probability of 0.1, the rule «A suggests C» has a probability of 0.11, and B and C are mutually exclusive events. Then the probability of «A suggests B OR C» is «0.1+0.11 = 0.21».
Rule for the multiplication of probabilities
If you have «A suggests B» with probability 0.2 and «both A and B being observed suggests C» with probability 0.3, then the probability of «A suggests both B AND C» is «0.2×0.3 = 0.06».
Rule for the multiplication of independent probabilities
Consider two rules:
A suggests B» has a probability of 0.5.A suggests C» has a probability of 0.8.What if we know that the chance of observing C given A is independent of whether B was also observed? That means:
both A and B suggests C» must also have a probability of 0.8.A suggests both B AND C» is «0.5×0.8 = 0.4».The difference here is that independence allows us not to have to know the probability of «A and B suggests C». We need only know the probability of «A suggests C».
Example
Suppose the probability of throwing a die and getting • is 1/6. Because the first throw cannot affect the second, the probability of both throws being • is 1/6×1/6=1/36.
However, consider the question of throwing • and ••• in either order.
• and ••• is the same as throwing • and •, that is 1/36.••• and • is also 1/36.• then ••• is incompatible with ••• then •, the addition rule must be used, so the in-either-order probability is 1/36+1/36=1/18.