Say I have 2 two coins A and B. I do 5 different rounds of 10 coins flips.
For each round, I choose 1 coin from A and B and flip ten times. In other words, same coins is used for each round.
Also, we never know which coin is chosen for each round.
Now, we want to estimate probability of each coin showing head for each flip.
In this case, we can use EM algorithm, which iterates the expectation and maximization process until the convergence of expectation.
we first initialize the probability at random, say:
$$ P_A=0.6, P_B = 0.5 $$
Then, we do such a process as shown below:
| H | T | A | B | Sum | Norm A | Norm B | |
|---|---|---|---|---|---|---|---|
| Round 1 | 5 | 5 | 0.0008 | 0.00098 | 0.00177 | 0.45 | 0.55 |
| Round 2 | 9 | 1 | 0.00403 | 0.00098 | 0.00501 | 0.80 | 0.20 |
| Round 3 | 8 | 2 | 0.00269 | 0.00098 | 0.00366 | 0.73 | 0.27 |
| Round 4 | 4 | 6 | 0.00053 | 0.00098 | 0.00151 | 0.35 | 0.65 |
| Round 5 | 7 | 3 | 0.00179 | 0.00098 | 0.00277 | 0.65 | 0.35 |
Note that, the third and fourth column is compute as such (say round 2)
$$ (0.6)^90.4 = 0.00403 \newline (0.5)^90.5 = 0.0098 $$
Then, we do such a process as shown below:
| E[A,head] | E[A,tail] | E[B,head] | E[B,tail] | |
|---|---|---|---|---|
| Round 1 | 2.2 | 2.2 | 2.8 | 2.8 |
| Round 2 | 7.2 | 0.8 | 5.9 | 1.5 |
| Round 3 | 5.9 | 1.5 | 2.1 | 0.5 |
| Round 4 | 1.4 | 2.1 | 2.6 | 3.9 |
| Round 5 | 4.5 | 1.9 | 2.5 | 1.1 |
| sum | 21.3 | 8.6 | 11.7 | 8.4 |
Then, we can update the probability as such:
$$ P_A^{new}=\frac{21.3}{21.3+8.6}=0.71 \newline P_B^{new}=\frac{11.7}{11.7+6.4}=0.58 $$