7.4 Searching for Item Statistics - Maple T.A. 2016 Help
Maple T.A. Online Help

Instructor
Search Maple T.A. 2016 Help:

7.4 Searching for Item Statistics

To perform a gradebook search:

  1. From the Class Homepage, select Gradebook from the menu bar.
  1. Click Item Statistics.
  1. In the Select Assignments tab, select the assignments you want to include in the search.
  • To select multiple assignments, hold down the Ctrl key (Command, on Mac) to select more than one assignment.
  1. Specify additional search criteria.
  1. In the View Panel tab, select the data to include in the search results.
  1. Click Search to perform the search.

For each question, item statistics are displayed (number of correct, partially correct, and incorrect responses as well as success rate, p-value, d-value, p-biserial, and r-biserial).

Important: The system allows a variety of possible assignment applications for each question in the question repository, which means calculating traditional item analysis on questions may produce unexpected results. Remember that the analysis and statistics produced in the Item Statistics reports here are based on the limited use of question items in the specific context of the assignments you specify. If you specify assignments that are of more than one assignment mode or that do not have identical assignment settings, item statistics calculated here are unreliable. To produce reliable item analysis results, select only one assignment for Item Statistics or ensure that all selected assignments have identical rules and policies.

Table 7.3: Item Statistics

Success Rate

The Success Rate of an item is the average normalized score on that item. With each item graded between 0.0 and 1.0 (that is, normalized), the average score is computed over the set of all scores recorded for that item in the current assignment.

p-Value

The p-Value of an item measures the proportion of students who got the item correct. It is defined as the ratio of the number of fully correct responses to the total number of responses in the data set. That is,

where n is the number of responses and F is the number of fully correct responses.

d-Value

The d-Value measures the discrimination of an item. To calculate it, the dataset is divided into two groups based on the scores on the entire assignment. The sets consist of those that scored greater than (or equal to) the median grade on the assignment and those that scored less than the median grade on the assignment. The p-Value for each set is calculated, and the d-Value is the difference of the p-Value for the high-scoring group and the p-Value for the low-scoring group.

where

n1 = number of assignments with scores median grade for the assignment

n2 = number of assignments with scores median grade for the assignment

F1 = number of fully correct responses on this item from the top-scoring set

F2 = number of fully correct responses on this item from the bottom-scoring set

p-Biserial

The p-Biserial is a correlation coefficient relating performance on an item and performance on the total assignment. The p-Biserial correlation coefficient is an index of discrimination that measures the extent to which students who score high on the assignment tend to get the item correct and those who score low tend to get the item incorrect.

where

= mean grade of assignments on which this item was answered incorrectly.

= mean grade of assignments on which this item was answered fully correctly.

p = p-Value

q = 1-p

= standard deviation for assignment grades

r-Biserial

The r-Biserial is a correlation coefficient relating performance on an item and performance on the total assignment. The r-Biserial correlation coefficient is an index of discrimination that measures the extent to which students who score high on the assignment tend to get the item correct and those who score low tend to get the item incorrect.

where

= mean grade of assignments on which this item was answered incorrectly.

= mean grade of assignments on which this item was answered fully correctly.

p = p-Value

q = 1-p

= standard deviation for assignment grades

Y = height of the standardized normal distribution at the point z, where P(z'<z)=q and P(z'>z)=p

 

 

Note: Since the r-Biserial is scaled by a parameter that is based on a normal distribution, it can result in values outside of -1 to 1 if the scores are not normally distributed.