Question

A coin having probability p = 2 3 of coming up heads is flipped 6 times. Compute the entropy of the outcome of this experiment.

Short answer

The entropy of the outcome of this experiment is$H\left(\mathbb{X}\right)=5.5098$.

Step-by-step solution

Given Information

We have given the probability of the coin$p=\frac{2}{3}$.

Simplify

Considering random vector $\mathbb{X}=\left(X_1,\dots ,X_6\right)$where $X_i$are independent equally distributed variables with distribution which given as

$X_i~\text{Bern}\left(\frac{2}{3}\right)$

Calculating the entropy of $\mathbb{X}$.

localid="1648135034976" $H\left(\mathbb{X}\right)=-\sum _{x\in {\left(0,1\right)}^6}p\left(x\right)\text{log}p\left(x\right)$

This sum contains $2^6$summands. Then,

localid="1648135254966" $$\begin{gathered} \sum _{x\in {\left(0,1\right)}^6}=p\left(x\right)\text{log}p\left(x\right)=-\left(\begin{array}{c}6\\ 0\end{array}\right){\left(\frac{1}{3}\right)}^6\text{log}{\left(\frac{1}{3}\right)}^6 \\ =-\left(\begin{array}{c}6\\ 1\end{array}\right){\left(\frac{1}{3}\right)}^5\frac{2}{3}\text{log}{\left(\frac{1}{3}\right)}^5\frac{1}{6} \\ =-\left(\begin{array}{c}6\\ 6\end{array}\right){\left(\frac{2}{3}\right)}^6\text{log}{\left(\frac{2}{3}\right)}^6 \end{gathered}$$

Hence,

$H\left(\mathbb{X}\right)=5.5098$