Feasibility of Using Prior Information about Predicted Item Difficulty in Increasing the Accuracy of Item Parameter Estimation and IRT Equating

For item banking or computerized adaptive testing to be successful, it is of vital importance to ensure the accuracy of item parameter estimation, especially when calibration needs to be conducted with the limited number of examinees for security reasons. This study investigated whether judgmental information about item difficulty would improve the accuracy of parameter estimation when used as prior information. Performance of using predictions of judges with various degrees of accuracy was evaluated in terms of item parameter invariance as well as effects on test equating, with reference to performances of other estimation methods under various simulation conditions. The findings of this study suggest that using priors based on judgmental information may increase the accuracy of b-parameter estimation and test equating in a considerable amount, unless predictions about item p-values are extremely inaccurate. The effects were even more obvious for the 1PL model and for smaller sample sizes. In estimating a-parameters and overall equating results for larger sample sizes, mixed results were found for the superiority of using judgmental information as priors.