Question

The authors of the paper "Do Physicians Know when Their Diagnoses Are Correct?" (Journal of General Internal Medicine [2005]: \(334-339\) ) presented detailed case studies to students and faculty at medical schools. Each participant was asked to provide a diagnosis in the case and also to indicate whether his or her confidence in the correctness of the diagnosis was high or low. Define the events \(C, I,\) and \(H\) as follows: \(C=\) event that diagnosis is correct \(I=\) event that diagnosis is incorrect \(H=\) event that confidence in the correctness of the diagnosis is high a. Data appearing in the paper were used to estimate the following probabilities for medical students: $$\begin{aligned} P(C) &=0.261 & & P(I)=0.739 \\ P(H \mid C) &=0.375 & & P(H \mid I)=0.073 \end{aligned} $$Use the given probabilities to construct a "hypothetical 1000 " table with rows corresponding to whether the diagnosis was correct or incorrect and columns corresponding to whether confidence was high or low. b. Use the table to calculate the probability of a correct diagnosis, given that the student's confidence level in the correctness of the diagnosis is high. c. Data from the paper were also used to estimate the following probabilities for medical school faculty: $$\begin{array}{cl} P(C)=0.495 & P(I)=0.505 \\ P(H \mid C)=0.537 & P(H \mid I)=0.252 \end{array}$$ Construct a "hypothetical \(1000 "\) ' table for medical school faculty and use it to calculate the probability of a correct diagnosis given that the faculty member's confidence level in the correctness of the diagnosis is high. How does the value of this probability compare to the value for students calculated in Part (b)?

Short answer

The probability of a correct diagnosis given a high confidence for students is 0.645 while for faculty, it is 0.677. So faculty are more likely to make a correct diagnosis when their confidence level is high, compared to students.

Step-by-step solution

Construct the 'hypothetical 1000' table for students

Start by multiplying each probability by 1000 to get the respective number of students. For instance, for \(P(C)\) multiply 0.261 by 1000 to get 261, do the same for \(P(I)\) and you would get 739. For the number of high confidence individuals in each group, multiply the number of people in each group by the respective conditional probability. So for instance for \(P(H|C)\) multiply 0.375 by 261 (the number of correct diagnoses) to get 98 (rounding to the nearest integer). Do the same with \(P(H|I)\) to get 54 (rounding to the nearest integer). This will provide a part of the table below: \[ \begin{array}{ccc} & \text{High Confidence} & \text{Low Confidence}\\ \hline \text{Correct} & 98 & – \\ \hline \text{Incorrect} & 54 & – \\ \hline \end{array} \] To fill in the values for low confidence, calculate the difference between the total number of correct and incorrect diagnoses and their respective high confidence values. So you should have 261-98=163 with low confidence who made correct diagnoses and 739-54=685 with low confidence who made incorrect diagnoses. Thus the completed table becomes: \[ \begin{array}{ccc} & \text{High Confidence} & \text{Low Confidence}\\ \hline \text{Correct} & 98 & 163\\ \hline \text{Incorrect} & 54 & 685\\ \hline \end{array} \]

Calculate the conditional probability of a correct diagnosis, given a high confidence level for students

This is calculated by dividing the number of correct diagnoses made with high confidence by the total number of diagnoses made with high confidence. This can be calculated by using the formula: \(P(C|H) = \frac{P(H \cap C)}{P(H)}\) where \(P(H \cap C)\) is 98 and \(P(H)\) is the sum of the high confidence totals which is 98+54=152. Thus the required probability is 98/152 = 0.64473684211 rounded to \(0.645\) when rounded to three decimal places

Construct the 'hypothetical 1000' table for faculty

This is done following the same procedure outlined in Step 1, but using the different probabilities given. Convert the given probabilities to their 'hypothetical 1000' equivalents, then calculate the high confidence numbers as before to obtain the table: \[ \begin{array}{ccc} & \text{High Confidence} & \text{Low Confidence}\\ \hline \text{Correct} & 266 & 229\\ \hline \text{Incorrect} & 127 & 378\\ \hline \end{array} \]

Calculate the conditional probability of a correct diagnosis, given a high confidence level for faculty

As explained in step 2, divide 266 (faculty with correct diagnoses and high confidence) by 393 (total number of all faculty with high confidence level). So the probability would be \(266/393\) which is equivalent to 0.67684887459807 rounded to \(0.677\) when rounded to three decimal places.

Compare the probabilities

From the results calculated in steps 2 and 4, faculty are more likely than students to make a correct diagnosis when their confidence level is high. This can be concluded from the fact that \(P(C|H)\) for faculty (0.677) is higher than for students (0.645)