Entry level, very easy

## Bayesian Basic

Difference in Frequentist(point estimate) and Bayesian(distribution):
Frequentist: $\hat{\theta}=argmax_{\theta}P(X|\theta)$
Bayesian: $P(\theta|X)$

The basic is

which is equal to

Some rules:
1.for discrete distributions, $p(B)=\sum_{A}p(A,B)=\sum_{A}p(B|A)p(A)$
2.for continuous distributions, $p(A|B)=\frac{p(B|A)p(A)}{\int {p(B|A)p(A)} \, {\rm d}A}$

Some related basic concepts:
1.Sensitivity: p(pred=1|disease=1)=p(pred=1,disease=1)/p(disease=1)=(TP/N)/((TP+FN)/N)=TP/(TP+FN)
2.Specificity: p(pred=0|disease=0)=p(pred=0,disease=0)/p(disease=0)=TN/(TN+FP)
3.Precision: TP/(TP+FP)=P(pred=1,disease=1)/p(pred=1)
4.Joint Probability density, assume gaussian distribution: $p(x_{1},x_{2},…,x_{N}=\prod_{i=1}^N \frac{1}{sqrt(2\pi (\sigma)^2)} exp(-1/2 \frac{(x_{i}-\mu)^2}{(\sigma)^2})$.

Example: You have a landing page where you get people to signup, but not everyone who visits your site will signup so that the conversion rate is the proportion of people who sign up. You want to compare your page with another new page. We know that the conversion rate(page1)=1/10 vs rate(page2)=2/10 is not as good as rate(page1)=10/100 vs rate(page2)=20/100.
Simple A/B Testing Recipe
t test statistic $t=\frac{\overline{X_{1}} - \overline{X_{2}}}{s_{pool}\sqrt{2/N}}$ where $s_{pool}=\sqrt{\frac{(s_1)^2+(s_2)^2}{2}}$.
If the size of each group is different, t test statistic $t=\frac{\overline{X_{1}} - \overline{X_{2}}}{s_{pool}\sqrt{1/n_{1}+1/n_{2}}}$.
If the standard deviation of each group is different, use “Welch’s t test”.

chi-square test statistic $(\chi)^2=\sum_{i} \frac{(observed_{i}-expected_{i})^2}{expected_{i}}$. Example in the course on udemy

practice in the course on udemy

## Bayesian A/B Testing

The Explore/Exploit problem: If you get 3/3 from bandit 1, 0/3 from bandit 2, should you exploit bandit 1 or explore more bandits? Or as for the advertisement A and advertisement B, if A has more clicks than B in 10 clicks, should you exploit advertisement A or explore more?
Epsilon-Greedy algorithm
Choose a small number of epsilon between 0 and 1, and generate a random probability, if p < epsilon, then explore, otherwise exploit.

UCBI
Define upper and lower limit to represent where we believe true CTR(click-through rate) is. Using a Chernoff-Hoeffding bound, $\epsilon > 0$, then $P(\hat{\mu} > \mu + \epsilon) \leq exp(-2(\epsilon)^2 N)$ and the opposite site $P(\hat{\mu} < \mu + \epsilon) \leq exp(-2(\epsilon)^2 N)$, which is equal to say $P(|\hat{\mu} - \mu| \leq \epsilon) > 1-2exp(-2(\epsilon)^2 N)$. Therefore the probability of $\mu$ in an interval is larger than a function of the number of games.
We choose the epsilon $\epsilon = \sqrt{\frac{2lnN}{N_j}}$, where N is the number of total games and $N_j$ is the number of games played in bandit j.

Bayesian A/B Testing

1.Conjugate Priors: If we choose beta distributions for $P(X|\theta)$ and $P(\theta)$, then we can make $P(\theta | X)$ have the same type of distribution as $P(\theta)$.
Example:

$P(\theta | X)=Beta(a^{'},b^{'})$ where $a^{‘}=a+\sum_{i=1}{N}x_{i}$ and $b^{‘}=b+N-\sum_{i=1}{N}x_{i}$. And the prior is beta(1,1), which is equal to uniform(0,1).

Thompson Sampling Convergence
Assuming the CTR $\theta$ of each bandit follows the distribution $beta((\alpha)_k, (\beta)_k)$. Everytime producing a random number from each bandit following the current distribution, and select the bandit with largest number. If it wins, then $\alpha = \alpha +1$, otherwise $\beta =\beta +1$. And interates until the CTR converges.