일반화가능도 이론과 FACETS 모형을 적용한 채점모형 개발 연구 - 음악 실기평가를 중심으로 -

Performance-based assessment is an authentic evaluation which can grasp what and how well students' can do in real-like situation. Since it is an impending task to control reliability of performance-based assessment, however, we need more technical developments to improve it. The purpose of this study was to examine both Generalizability theory and Many-facet Rasch model which have been the most frequently applied psychometric methods lately in evaluating the reliability of performance assessment. It also intends to figure out a way to combine these two methods so that we get more reliable scores in performance assessment of Music.

The data used in this study consists of 15 examinees' subscores as well as total scores obtained in the performance-based assessment in the domains of piano and violin provided by the Royal Conservatory of Music Examinations of Canada.

For the first part of the analysis, the generalizability theory was applied. We estimated the variance component of instruments, raters, occasions, and evaluative factors to compare the relative influence of each facet, and determined the optimal number of grading conditions of each facet that maximizes the generalizability coefficient.

For the next part of the analysis, the many-facet Rasch model was applied. We detected the intensity of inconsistency in examinee scores caused by raters and provided examinee logit scores after adjusting the effects of a rater facet on raw scores.

The results show that both generalizability theory and many-facet Rasch model would be useful to produce more reliable examinee scores in performance assessment of music. However, it is also found out that they have the relative strengths and weaknesses in producing more reliable scores. G theory is useful in detecting various error components in performance assessment. On the other hand, Facets model makes it possible to produce examinee scores after controlling these error components.