Question: What is the desired percentage of agreement for interobserver agreement?
Answer: 80% or more.
IOA refers to the degree to which two or more independent observers report the same thing after measuring the same behavior at the same time. IOA helps increase the believability of our data. It also helps determine the competence of new observers or ensure that the operational definition that we’ve written is clear enough to measure the behavior. More importantly, it helps us to see whether or not any behavior change is the product of the intervention, or any lack thereof, is the product of the intervention.
There are a few rules that you should know about IOA. First and foremost, all the observers have to use the same measurement system. Also, those same observers have to measure the same event. They also have to be independent of each other; they can’t be interacting while they’re that collecting data. It’s a little counterintuitive to the whole idea. IOA should be measured for a minimum of 20% of all behavioral sessions, and for the believability to be super believable, the closer to 100% agreement, the better. A minimum of 80%, but obviously, believability increases as you get closer to 100%.
