Dana Do’s: How to Define the Markers of Trustworthy Measurement

Section C-8 asks that you’re able to evaluate the validity and reliability of measurement procedures. That’s hard to do if you get those two terms mixed up, and even harder if we throw in a third term: accuracy– another marker of trustworthy measurement. Let’s discriminate between these terms once and for all.

Validity is the most important one. When we say a measurement or a measurement procedure is valid, we mean that it is meaningful. That can only be if we measure the right thing– behavior. Measuring the intended dimension of that behavior or, the intended timing, using a procedure that paints a meaningful picture of that behavior. If we use time sampling, is the data an over or under estimation? And if so, it’s not meaningful, and paints a misleading picture of the behavior. So first and foremost, you have to measure the right thing. Another hint about validity, it’s not a quantity, it’s a statement of fact: “I measured the right thing”.

Now, you have to prove that your measurement is reliable. To prove reliability, you’re going to need actual numbers. How do you do that? Well, you repeat the measurement in exactly the same way, at the exact same time. And if you yield the same results each time, you have reliable measurement. Think of it as measurement you can count on, like a reliable person who you can depend on to always be consistent.

And finally, accuracy. Another quantity-based item. And though not mentioned in the task list, it’s important and belongs here with validity and reliability. Accuracy means that, whatever you measured, you measured it right. You counted correctly and what you counted (that’s your observed value) matches the true value. You counted five fingers and sure enough, one-two-three-four-five– accurate.

So, the takeaways here: if you don’t have validity, then reliability and accuracy are moot. And if you are not able to yield an accuracy or a reliability measure, then you can focus on either the accuracy or reliability.

5th Edition Task List
  • C-8   Evaluate the validity and reliability of measurement procedures.
  • 6th Edition TCO
  • C.8. Evaluate the validity and reliability of measurement procedures.  
  • Accuracy
  • MiniBig C D F
  • Reliability
  • Section C
  • Section C-8
  • Trustworthy Measurement
  • Validity
  • Related Content

  • ABA terms you need to know: validity.Validity
    Measurement is trustworthy when the measurement that produces data is applicable and specific to the target behavior of interest and the relevant dimension of that behavior; in other words, you…
  • Reliability
    Measurement is trustworthy when the degree to which repeating a measurement procedure under the same conditions produces the same result.
  • Observer drift
    When observers collecting data have a shift/drift in how they interpret the operational definition of the target behavior that affects the validity of their data.
  • Observer reactivity
    When an observer’s data collection or performance is influenced by the awareness that they are being monitored and evaluated.
  • Measurement artifact
    Data that represents an unwarranted and misleading picture of behavior because of how it was measured, typically a product of time-sampling, limiting measurement scales, or poorly scheduled measurement periods.
  • Measurement bias
    When an observer’s measurement is influenced by an expectation or belief they hold rather than what actually occurred.
  • ABA terms you need to know: Interobserver agreement.Interobserver agreement (IOA)
    A measurement of the degree to which two or more observers report the same values when observing and collecting data for the same behaviors/events.
  • Dana Do's: All the Validities in ABA, Explained
    PTB founder Dana Meller helps connect each of the validities to the validity of this explanation.
  • PTB co-founder takes some of the fear out of the interobserver agreement math with simple tips that can help narrow your discrimination. Refer to 5th Edition Task List Section C-8: Evaluate the validity and reliability of measurement procedures.Dana Do's: Why IOA Math Should Not Intimidate You
    PTB co-founder takes some of the fear out of the interobserver agreement math with simple tips that can help narrow your discrimination.
  • It's particularly meaningful to be able to tell validity, social validity, internal and external validity apart.Dana Do's: How to Decipher the Many Forms of Validity
    It's particularly meaningful to be able to tell validity, social validity, internal and external validity apart. PTB co-founder Dana Meller has some tips to do just that.
  • Reactivity
    Refers to the changes in an individual’s behavior due to the presence of an observer.
  • Accuracy
    The degree to which what was measured (observed value) is representative of what actually occurred (true value); in other words, the data was measured correctly.
  • PTB co-founder Dana Meller discusses Interobserver Agreement (IOA) and its significance in behavior analysis. Discover the desired percentage of agreement for IOA and its role in increasing data credibility and evaluating intervention effectiveness.Questing for the Gold Standard of IOA Agreement
    PTB co-founder Dana Meller discusses Interobserver Agreement (IOA) and its significance in behavior analysis. Discover the desired percentage of agreement for IOA and its role in increasing data credibility and evaluating intervention effectiveness. Refer to BCBA® Task List (5th ed.) Section C-8: Evaluate the validity and reliability of measurement procedures.
  • ABA Terminology Break Down: Indicators of Trustworthy MeasurementABA Terminology Break Down ➠ Indicators of Trustworthy Measurement
    What are the three indicators of trustworthy measurement in ABA? PTB co-founder Dana Meller explains and why they ensure data accuracy and reliability. Refer to BCBA® Task List (5th ed.) Section C-8: Evaluate the validity and reliability of measurement procedures.
  • Mock BCBA® Exam Question Breakdown: Interobserver Agreement (IOA) Facts & Rules to Live ByBreak Down a Mock BCBA® Exam Question: Interobserver Agreement (IOA) Facts & Rules to Live By
    Test your Measurement, Data Display, and Interpretation skills with PTB co-founder Dana Meller as she reviews a BCBA® mock exam question about several #interobserveragreement (IOA) facts, with rules to live by. Here, Dana highlights a variety of important IOA benefits, as well as IOA percentage of agreement criteria details and the different reporting options. Refer to BCBA® Task List (5th ed.) Section C-8: Evaluate the validity and reliability of measurement procedures.
  • IOA Mini Manual
    Math can be daunting, but practice can help. In response to growing calls for more IOA practice opportunities, download the free PTB Interobserver Agreement Mini Manual. Designed to promote retention and reduce anxiety, the PTB IOA Mini Manual features 30 practice questions that cover the 9 IOA formulas you need to know to prep for the exam.
  • Test your ABA Terminology: Indicators of Trustworthy MeasurementTest your ABA Terminology ➠ Indicators of Trustworthy Measurement
    Test your knowledge with PTB co-founder Dana Meller as she explains the specific indicator of trustworthy measurement that corresponds with repeated measurement yielding the same result. Included is a description of all three indicators of trustworthy measurement validity, accuracy, and reliability. Refer to BCBA® Task List (5th ed.) Section C-8: Evaluate the validity and reliability of measurement procedures.
  • Test your ABA Terminology: Confounding Threats to Internal ValidityTest your ABA Terminology ➠ Confounding Threats to Internal Validity
    Test your knowledge of measurement confounds with PTB co-founder Dana Meller as she discusses one of the four types of threats to internal validity, with corresponding examples. Refer to BCBA® Task List (5th ed.) Section C-8: Evaluate the validity and reliability of measurement procedures.
  • BCBA® Mock Exam Question: IOA Battle Royal: Which Formula Gets Knocked Out?Breaking Down a BCBA® Mock Exam Question: It's the IOA Battle Royal. Which Formula Gets Knocked Out?
    Sharpen your measurement skills with PTB co-founder Dana Meller as she reviews a BCBA® mock exam question about IOA and helps explain the different formulas used for time sampling measurements. Dana covers four IOA formulas: Interval-by-Interval, Scored Interval, Trial-by-Tial, and Unscored Interval; and highlights key considerations and pitfalls associated with each. BCBA® Task List (5th ed.) Section C-8: Evaluate the validity and reliability of measurement procedures.
  • BCBA® Mock Exam Question: The #1 Threat to Accuracy and ReliabilityBreaking Down a BCBA® Mock Exam Question: The #1 Threat to Accuracy and Reliability
    Test your measurement knowledge with PTB co-founder Dana Meller as she reviews a BCBA® mock exam question about the key indicators of trustworthy measurement, revealing the most significant threat facing accuracy and reliability. Dana provides insight into the distinctions between reliability, validity, and accuracy and how they relate to measurement. BCBA® Task List (5th ed.) Section C-8: Evaluate the validity and reliability of measurement procedures.
  • Threats to Validity Exposed: When Misleading Data Plays Tricks on Your PerceptionThreats to Validity Exposed: When Misleading Data Plays Tricks on Your Perception
    Test your knowledge of measurement procedures with PTB co-founder Dana Meller as she explains, in detail, one of the threats to validity with several corresponding examples. Dana also shares a clever memorization study tip that will help you on the Big exam. Refer to BCBA® Task List (5th ed.) Section C-8: Evaluate the validity and reliability of measurement procedures.
  • The markers of trustworthy measurement.Dana Do's: How to Define the Markers of Trustworthy Measurement
    PTB co-founder goes into the three factors that define trustworthy measurement-- "validity", "reliability" and "accuracy". But what's the difference between each? Dana explains.
  • PTB co-founder Dana Meller dives into the concept of enhancing reinforcement schedules by adding an extra criterion to motivate quicker and more accurate responses. Refer to BCBA® Task List (5th ed.) Section G-8: Use chaining.Boosting Speed & Accuracy with Reinforcement Schedule Parameters
    PTB co-founder Dana Meller dives into the concept of enhancing reinforcement schedules by adding an extra criterion to motivate quicker and more accurate responses. Refer to BCBA® Task List (5th ed.) Section G-8: Use chaining.
  • It’s all too easy to mix up continuous and discontinuous measurement procedures. Need some help discriminating between the two? Get clarity with help from PTB founder Dana Meller.Dana Do's: What's the Difference Between Continuous and Discontinuous Measurement?
    It’s all too easy to mix up continuous and discontinuous measurement procedures. Need some help discriminating between the two? Get clarity with help from PTB founder Dana Meller.
  • Tripped up trying to make correct discriminations when it comes to functional analysis? PTB founder Dana Meller has some tips to help you answer those tricky exam questions.Dana Do's: Tips for Test Questions About Functional Analysis
    Tripped up trying to make correct discriminations when it comes to functional analysis? PTB founder Dana Meller has some tips to help you answer those tricky exam questions.
  • Looking to review direct assessment methods? PTB founder Dana Meller gets right to it with this quick explainer.Dana Do's: Six Direct Assessment Methods
    Looking to review direct assessment methods? PTB founder Dana Meller gets right to it with this quick explainer.
  • Dana Do's: Exploring the Four Types of Reinforcer Assessments.Dana Do's: Exploring the Four Types of Reinforcer Assessments
    PTB founder Dana Meller discusses reinforcer assessments, which should not be confused with stimulus preference assessments.
  • Can can a changing criterion design be used to evaluate shaping programs? Well, PTB founder Dana Meller says, it depends.Dana Do's: Can Changing Criterion Design be Used to Evaluate Shaping Programs?
    Can can a changing criterion design be used to evaluate shaping programs? Well, PTB founder Dana Meller says, it depends.
  • Dana Do's: Nine Considerations for Prioritizing Target Behaviors
    PTB founder breaks down each of the nine considerations for prioritizing target behaviors.
  • Experimental design tends to trip lots of people up. Here's an exam trick from Dana Meller that will help you breaking down questions on the exam.Dana Do's: A Trick to Help Dissect BCBA® Exam Questions on Experimental Designs
    Experimental design tends to trip lots of people up. Here's an exam trick from Dana Meller that will help you breaking down questions on the exam.
  • 10 Positive Effects From Prioritizing Self-Care
    Self-care. It’s a bit of a broad term, and a total buzzword these days. It’s often used in the context of bubble baths and “unplugging.” But, at its core, self-care is about more than just “me-time.” It’s about taking care of yourself in a way that works best for you and your lifestyle.
  • How do you know if your BCBA® study activities are effective?Dana Do's: How to Know if Your Study Activities are Effective
    How do you know if your BCBA® study activities are effective? PTB co-founder Dana Meller shares some key analysis that may help BCBA® exam candidates assess efficacy.
  • Caution Using BCBA Exam as a MockDana Do's: Careful About Getting Creative With Mock Exams
    PTB co-founder cautions candidates about using the actual BCBA® exam as a mock exam, listing all the reasons why you might want to reconsider getting creative with your mock exam prep experience.
  • Dana Do's. Warning. Watch out for extremes.Dana Do's: Warning. Be Careful of Extremes.
    PTB co-founder Dana Meller sends an important warning for test takers. Watch out for extremes in exam questions and answer options.
  • Dana Do's: Are mock exams an accurate predictor of test performance?Dana Do's: How to Use Mock Exams in Your Study Plan
    I was recently asked if mock exams are useful in predicting how someone will do on the actual big exam. And, is repeating the same mock exam an effective study…
  • Dana Do's: How to discriminate between dependent and independent variables.Dana Do's: How to Discriminate Dependent vs. Independent Variables If You're Not a Researcher
    Not a researcher? How about some help discriminating between dependent and independent variables. PTB co-founder Dana Meller offers a few tips to make that distinction on the BCBA® exam.
  • BCBA® Mock Exam Question: Winning Measurement System to Record Client Tolerance to LosingBreaking Down a BCBA® Mock Exam Question: Winning Measurement System to Record Client Tolerance to Losing
    Let's review and break down a BCBA® mock exam question about measurement techniques in ABA, specifically percentage of opportunity. PTB co-founder Dana Meller explains how to calculate the percentage of opportunities where the behavior is exhibited and why this method is superior to rate, count, and interval recording in certain situations. Refer to BCBA® Task List (5th ed.) Section C-3: Measure occurrence (e.g., count, frequency, rate, percentage).
  • MiniBig
     

    PTB MiniBig: Fast, Flexible, and Focused Exam Prep

    The PTB MiniBig enables BCBA® exam prep candidates to tailor their study needs by popping in on single sessions of the intensive PTB Big Exam Prep Workshop. Each 3.5-hour session, focused on specific sections of the Task List, includes a mock exam followed by real-time, live, interactive feedback and access to live session replays, without committing to the full workshop.
  • Test your ABA Terminology: Functional Assessment ProceduresTest your ABA Terminology ➠ Functional Assessment Procedures
    Focusing on ABC data collection, PTB co-founder Dana Meller discusses functional assessment procedures. Dana explains how to generate mathematical probabilities for target behaviors based on specific antecedents and consequences, providing a step-by-step guide to quantitatively analyzing the gathered information. Refer to BCBA® Task List (5th ed.) Sections F-7: Conduct a descriptive assessment of problem behavior and F-9: Interpret functional assessment data.
  • Test Your Descriptive Assessment Skills: ABC Recording MethodsTest Your Descriptive Assessment Skills ➠ ABC Recording Methods
    Get a comprehensive overview of ABC Recording Methods. PTB co-founder Dana Meller breaks down the two types, along with their differences, benefits, and practical applications. Refer to BCBA® Task List (5th ed.) Sections F-7: Conduct a descriptive assessment of problem behavior; and F-9: Interpret functional assessment data.
  • Test your ABA Terminology: Cumulative Record CalculationsTest your ABA Terminology ➠ Cumulative Record Calculations
    PTB co-founder Dana Meller reviews a question about using cumulative records and their significance in behavior analysis. Learn how to utilize cumulative graphs to track behavior over specific time periods and measure and calculate response rates for insightful data analysis. Refer to BCBA® Task List (5th ed.) Section C-11: Interpret graphed data.
  • 1-Minute Experimental Design Test HackDana Do's: 1-Minute Experimental Design Test Hack (you still have to study it though)
    PTB co-founder Dana Meller shares a huge secret to help break down experimental design questions. BCBA® Task List (5th ed.) Section D-3: Identify the defining features of single-subject experimental designs.
  • Breaking Down a Mock BCBA® Exam Question: Reversal Design Variations With Only ONE ReversalBreaking Down a Mock BCBA® Exam Question: Reversal Design Variations With Only One Reversal
    Test your Experimental Design skills with PTB co-founder Dana Meller as she reviews and breaks down a mock exam question about reversal design variations that display only one reversal. Included is a description of A-B-A, B-A-B, and A-B-A-B reversal designs. Refer to BCBA® Task List (5th ed.) Section D-5: Use single-subject experimental designs (e.g., reversal, multiple baseline, multielement, changing criterion).
  • PTB co-founder Dana Meller provides shortcuts to help eliminate incorrect answer choices and gain a better understanding of the basics of various experimental designs.Shortcuts to Understanding the Basics Behind Each Experimental Design
    Experimental design causes a lot of anxiety. PTB co-founder Dana Meller shares a few shortcuts to help on exam day, eliminating some answer options but also, more importantly, give you a head start on understanding the basics behind each design.
  • Test your ABA Terminology: Identify the Measurement System in This ScenarioTest your ABA Terminology ➠ Identify the Measurement System in This Scenario
    Test your measurement, data display, and interpretation knowledge with PTB co-founder Dana Meller as she discusses one specific measurement system that measures the effect of the behavior after its occurrence. Here, Dana details the two ways someone could implement this type of measurement system, provides examples of scenarios and a variety of corresponding measurement methods, and highlights some of the pros of utilizing this type of measurement system. Refer to BCBA® Task List (5th ed.) Section C-2: Distinguish among direct, indirect, and product measures of behavior.
  • Importance of Experimental DesignDana Do's: Importance of Experimental Design Fluency
    PTB co-founder Dana Meller talks about the importance of being fluent in experimental designs, not just to pass the exam, but to be a good clinician as well. Ref Section D-3: Identify the defining features of single-subject experimental designs (e.g., individuals serve as their own controls, repeated measures, prediction, verification, replication).
  • Breaking Down a Mock BCBA® Exam Question: Identify a 3-Phase Reversal Experimental DesignBreaking Down a Mock BCBA® Exam Question: Identify a 3-Phase Reversal Experimental Design
    Test your Experimental Design skills with a breakdown of a BCBA® mock exam question about the identification of a three-phase experimental design. Included is a detailed comparison between four different reversal designs: ABC reversal, ABA reversal, BAB reversal, and ABAB reversal. As a bonus, Dana also provides examples of instances in which each of the reversal designs would be implemented. Refer to BCBA® Task List (5th ed.) Section D-5: Use single-subject experimental designs (e.g., reversal, multiple baseline, multielement, changing criterion).
  • Test your ABA Terminology: Pivotal Behavior vs. Behavior CuspTest your ABA Terminology ➠ Pivotal Behavior vs. Behavior Cusp
    Test your behavior assessment knowledge with PTB co-founder Dana Meller as she compares and contrasts pivotal behavior vs. behavior cusp, providing several corresponding examples of each, as they relate to identifying the target behaviors that need prioritization for treatment. Refer to BCBA® Task List (5th ed.) Section F-3: Identify and prioritize socially significant behavior-change goals.
  • Test your ABA Terminology: Interpret Descriptive Assessment DataTest your ABA Terminology ➠ Interpret Descriptive Assessment Data
    Test your knowledge of interpreting descriptive assessment ABC data with PTB co-founder Dana Meller. Here she discusses conditional probability, including the rules that apply, and provides a step-by-step example using the calculation formula and how to report these data. Refer to BCBA® Task List (5th ed.) Sections F-7: Conduct a descriptive assessment of problem behavior; F-9: Interpret functional assessment data.
  • Test your ABA Terminology: Identify The Experimental Design Explained in This ScenarioTest your ABA Terminology ➠ Identify This Experimental Design Scenario
    Test your ABA terminology with PTB co-founder Dana Meller as she offers guidance to identify the specific experimental design relating to a scenario. Refer to BCBA® Task List (5th ed.) Section D-5: Use single-subject experimental designs (e.g., reversal, multiple baseline, multielement, changing criterion).
  • Test your ABA Terminology: Derivative Measures ExplainedTest your ABA Terminology ➠ Derivative Measures Explained
    Put your knowledge of occurence measurement to the test with PTB co-founder Dana Meller as she discusses the two types of derivative measures, and each of their defining features and applications. Dana focuses on the derivative measure used for comparing two or more procedures, teaching clients new skills, evaluating mastery over a class of concepts, and the various measurement methods. Refer to BCBA® Task List (5th ed.) Section C-6: Measure trials to criterion.
  • Experimental Validity: The Relationship between Behavior Change & Independent VariablesExperimental Validity: The Relationship between Behavior Change & Independent Variables
    PTB co-founder explains the different types of validity in ABA. Refer to BCBA® Task List (5th ed.) Section D-2: Distinguish between internal and external validity.
  • Breaking Down a BCBA® Mock Exam Question: Understanding Equal-Interval Graphs & Logarithmic ScalesBreaking Down a Mock BCBA® Exam Question: Understanding Equal-Interval Graphs & Logarithmic Scales
    Test your knowledge of ABA graphs with PTB co-founder Dana Meller as she reviews a BCBA® mock exam question about equal-interval graphs, exploring their unique properties. Included, Dana explains the concept of proportional values and highlights the graph(s) which operate on a logarithmic scale, offering a different perspective on changes in behavior. Refer to BCBA® Task List (5th ed.) Sections C-10: Graph data to communicate relevant quantitative relations (e.g., equal-interval graphs, bar graphs, cumulative records); C-11: Interpret graphed data.
  • Test your ABA Terminology: Fundamental Properties of Behavior ChangeTest your ABA Terminology ➠ Fundamental Properties of Behavior Change
    Test your ability to interpret graphed data with PTB co-founder Dana Meller. Here she explains the three fundamental properties of behavior change, focusing on one specific property akin to an average or median. Included are descriptions of level, trend, variability, and the y-axis. Refer to BCBA® Task List (5th ed.) Section C-11: Interpret graphed data.
  • Let's break down a BCBA® mock exam question.Let's Break Down a BCBA® Mock Exam Question: Broadening the Scope of Trials-to-Criterion Measurements
    PTB co-founder Dana Meller breaks down a BCBA® mock exam question about measurement used to derive trials to criterion data. Dana explains how this measurement can be a powerful tool in understanding and improving behavior performance across different dimensions. Refer to BCBA® Task List (5th ed.) Section C-6: Measure trials to criterion.
  • 3 Essentials for crafting operational definitions.ABA Terminology ➠ You Down With OCC? 3 Essentials For Crafting an Epic Operational Definition
    PTB co-founder Dana Meller breaks down the three defining features needed to craft a solid operational definition - Remember "OCC" for short. Refer to BCBA® Task List (5th ed.) Section C-1: Establish operational definitions of behavior.
  • Shopping Cart

    DAYS LEFT TO PASS

    Exam Changes to 6th Ed TCO on Jan. 1, 2025

    Days : Hours : Minutes : Seconds

    PASS IN 2024
    Scroll to Top