Board Certified Behavior Analyst (BCBA) Practice Test

Which method is most effective for achieving inter-observer agreement when scoring a student’s written math problems?

1. Teachers scoring the work together with the student

2. Independent scoring of photocopies by both researcher and observer

3. Comparing scores verbally after each answer is completed

4. Using a shared rubric to score the work simultaneously

The correct answer is: Independent scoring of photocopies by both researcher and observer

The method involving independent scoring of photocopies by both the researcher and the observer is the most effective for achieving inter-observer agreement. This approach allows each scorer to evaluate the work independently without any influence from the other. By scoring photocopies instead of the original work, both observers can refer to the same material, ensuring consistency in what they are evaluating. Independent scoring minimizes potential bias or discrepancies that can occur when observers score in the same environment or have discussions about the scoring process, which could lead to conformity or altered judgments. This method is particularly valuable for establishing reliability in scores, as it provides a clear basis for comparison. Each observer can document their scores independently, and afterwards, the two sets of scores can be compared to quantify inter-observer agreement statistically. This methodological rigor helps ensure that the evaluations are objective and consistent, which is essential in a research or educational setting focused on accurate assessment. In contrast, scoring together with the student or comparing scores verbally can introduce biases or lead to variations in scoring practices. Using a shared rubric to score simultaneously may still result in differences in interpretation of the rubric itself between scoring individuals, thereby affecting the reliability of the scoring. Thus, independent scoring of photocopies stands out as the most effective way to achieve clear and