Question

Consider a system consisting of rolling a six-sided die. What happens to the entropy of the system if an additional die is added? Does it double? What happens to the entropy if the number of dice is three?

Short answer

Answer: The entropy of the system increases as more six-sided dice are added, but it doesn't strictly double or triple. The reason for this change is that adding more dice increases the number of possible outcomes, and thus increases the disorder or randomness of the system. The entropy is a measure of this increasing disorder, and it grows at a rate that is dependent on the specific nature of the system.

Step-by-step solution

Calculate the Entropy for Rolling a Single Die

For a single die, the number of possible outcomes is 6. Using the formula for the entropy of a discrete probability distribution, we have: \(H(X_1) = \displaystyle\sum_{i=1}^{6} p(x_i) \log_2 \frac{1}{p(x_i)} = \sum_{i=1}^{6} \frac{1}{6} \log_2 6\), Since the probabilities are the same for each outcome, we can simplify the sum: \(H(X_1) = 6 \times \frac{1}{6} \times \log_2 6 = \log_2 6\).

Calculate the Entropy for Rolling Two Dice

For two dice, the number of possible outcomes is 36 (since each die has six sides, and the two dice are independent). To simplify the calculation, we note that the sum of the rolled numbers on the two dice can range from 2 to 12, and each of these sums has different probabilities of occurring. We can create a new probability distribution for these sums and calculate the entropy based on those probabilities. First, let's find the probabilities for each sum: \(p(2) = \frac{1}{36},\; p(3) = \frac{2}{36},\;\cdots,\;p(12) = \frac{1}{36}\). Now, we can calculate the entropy: \(H(X_2) = \displaystyle\sum_{i=2}^{12} p(x_i) \log_2 \frac{1}{p(x_i)}\). After calculating the entropy, we get: \(H(X_2) = 5.17\) (rounded to 2 decimal places).

Calculate the Entropy for Rolling Three Dice

For three dice, the number of possible outcomes is 216 (since each die has six sides, and the three dice are independent). Similar to Step 2, we find the probabilities for each sum of the rolled numbers on the three dice and create a new probability distribution accordingly. The sum can range from 3 to 18. After calculating the entropy, we get: \(H(X_3) = 6.58\) (rounded to 2 decimal places).

Analyze the Results

As we can see, the entropy increases as more dice are added to the system, but it doesn't strictly double or triple. The reason is that adding more dice increases the number of possible outcomes, and thus increases the disorder or randomness of the system. The entropy is a measure of this increasing disorder, and it grows at a rate that is dependent on the specific nature of the system.