Bayes' Rule

$$\mathbb{P}(A\vert B) = \frac{\mathbb{P}(B\vert A)\mathbb{P}(A)}{\mathbb{P}(B)}$$

• Often we observe a posterior outcome or measurement, say the event $B$, and wish to evaluate the probability of a prior condition, say the event $A$. That is, given some measurement or knowledge we wish to evaluate how likely is it that a prior condition occurred.

Was it a 0 or 1?

• Considering a communication channel involving a stream of transmitted bits (0's and 1's), where 70% of the bits are 1, and the rest are 0. The channel is imperfect dut to physical disturbances such as interfering radio signals, and furthermore the bits received are sometimes distorted. Hence there is a chance($\epsilon_{0}$) of interpreting a bit as 1 when it is actually 0, and similarly there is a chance ($\epsilon_{1}$) of interpreting a bit as 0 when it is actually 1.
• Now we received ($R_{x}$) a bit, and interpreted it as 1. This is the posterior outcome. What is the chance that it was in fact transmitted ($T_{x}$) as 1? $$\mathbb{P}(T_{x}\vert R_{x}) = \frac{\mathbb{P}(R_{x}\vert T_{x})\mathbb{P}(T_{x})}{\mathbb{P}(R_{x})} = \frac{(1-\epsilon_{1})0.7}{0.7(1-\epsilon_{1}) + 0.3\epsilon_{0}}$$
• If $\epsilon_{0} = 0.1$ and $\epsilon_{1}=0.05$, then $\mathbb{P}(T_{x}\vert R_{x})=0.9568$
• Solution by Julia

using Random
Random.seed!(1)

N = 10^5
prob1 = 0.7
eps0, eps1 = 0.1, 0.05

filpWithProb(bit, prob) = rand() < prob ? xor(bit,1) : bit

TxData = rand(N) .< prob1
RxData = [x == 0 ? filpWithProb(x, eps0) : filpWithProb(x, eps1) for x in TxData]

numTx1 = 0
totalRx1 = 0
for i in 1:N
    if RxData[i] == 1
        global totalRx1 += 1
        global numTx1 += TxData[i]
    end
end

monteCarlo = numTx1/totalRx1
analytic = ((1-eps1)*0.7)/(0.7*(1-eps1) + 0.3*eps0)

println("Monte Carlo: ", monteCarlo, "\t\tAnalytic: ", analytic)

The Monty Hall Problem

• Consider a contestant on television show, with three doors in front of her. One of doors contains a prize, while the other two are empty. The contestant is then asked to guess which door contains the prize, and she makes a random guess. Following this, the game show host (GSH) reveals an empty door form one of the two remaining doors not choosen.The conestant is then asked if she wishes to stay with their original choice, or if she wishes to switch to the remaining closed door.
• The two possible policies for the contestant are as follows:
  • Policy I: Stay with their original choice after the door is revealed.
  • Policy II: Switch after the door is revealed.
• Let $A_{i}$ be the event that the prize is behind door $i$. Let $B_{i}$ be the event that door $i$ is revealed by the GSH, if the player initially choose door 1 and then the GSH revealed door 2, then $$\begin{split} \mathbb{P}(A_{1}\vert B_{2}) = \frac{\mathbb{P}(B_{2}\vert A_{1})\mathbb{P}(A_{1})}{\mathbb{P}(B_{2})} = \frac{\frac{1}{2}\times \frac{1}{3}}{\frac{1}{2}} = \frac{1}{3}\quad & (\textbf{Policy I}) \\ \mathbb{P}(A_{3}\vert B_{2}) = \frac{\mathbb{P}(B_{2}\vert A_{3})\mathbb{P}(A_{3})}{\mathbb{P}(B_{2})} = \frac{1\times\frac{1}{3}}{\frac{1}{2}} = \frac{2}{3}\quad & (\textbf{Policy II}) \\ \end{split}$$
• Solution by Julia

using Random
Random.seed!(1)

function montyHall(switchPolicy)
    prize, choice = rand(1:3), rand(1:3)
    if prize == choice
        revealed = rand(setdiff(1:3, choice))
    else
        revealed = rand(setdiff(1:3, [choice, prize]))
    end

    if switchPolicy
        choice = setdiff(1:3, [revealed, choice])[1]
    end
    return choice == prize
end

N = 10^6
println("Success probability with policy I (stay): ", 
    sum([montyHall(false) for _ in 1:N])/N)
println("Success probability with policy II (switch): ",
    sum([montyHall(true) for _ in 1:N])/N)