Probability - Conditional probability, multiplication theorem on probability

Conditional Probability: This concept refers to the probability of an event $A$ occurring given that another event $B$ has already occurred, denoted as $P(A|B)$. Visually, this process is equivalent to restricting the original sample space $S$ to a new, smaller sample space $B$. We then look for outcomes that belong to both $A$ and $B$ within this new boundary.

Multiplication Theorem: This theorem provides a way to calculate the probability of the simultaneous occurrence of two events $A$ and $B$. It states that $P(A \cap B)$ is obtained by multiplying the probability of the first event by the conditional probability of the second event, given that the first has occurred. This is often visualized using a tree diagram where each branch represents a conditional probability.

Independent Events: Two events $A$ and $B$ are said to be independent if the probability of occurrence of one is not affected by the occurrence of the other. Mathematically, $P(A|B) = P(A)$ and $P(B|A) = P(B)$. In a Venn diagram, the intersection $A \cap B$ for independent events represents exactly the product of their individual probabilities relative to the total area.

Property of Complements: For any two events $E$ and $F$ associated with a sample space $S$, the conditional probability of the complement of $E$ given $F$ is $P(E'|F) = 1 - P(E|F)$. This is helpful in solving problems where finding the probability of 'at least one' or 'not happening' is easier by subtracting the positive case from the total probability of $1$.

Conditional Probability of Union: The additive rule for probability also applies to conditional cases. $P(A \cup B | F) = P(A|F) + P(B|F) - P(A \cap B | F)$. Visually, if you look only inside the set $F$, the area covered by $A$ or $B$ is the sum of their individual areas within $F$ minus the area where they overlap within $F$.

Multiplication Rule for $n$ Events: The multiplication theorem can be extended to three or more events. For events $A, B,$ and $C$, $P(A \cap B \cap C) = P(A) \cdot P(B|A) \cdot P(C|A \cap B)$. This reflects a sequential dependency where each subsequent event's probability is conditioned on the intersection of all preceding events.

Sampling With vs. Without Replacement: The multiplication theorem highlights the difference between these two methods. Sampling with replacement typically results in independent events (the sample space stays the same), while sampling without replacement results in dependent events (the sample space and favorable outcomes decrease), requiring the use of conditional probabilities.