Single Scores in VDAC: The Key to Unlocking Reliable Attentional Capture Data?

Hey there, fellow science enthusiasts! Ever wonder why some things just grab your attention, even when you’re trying your best to ignore them? Especially if those things were once tied to something good, like a reward? That’s the fascinating world of value-driven attentional capture (VDAC) for you. It’s when our brains involuntarily snap our attention to neutral stuff just because it was previously linked to a payoff. Pretty neat, huh? And it’s not just a cool party trick for our brains; VDAC is super relevant for understanding things like addiction, ADHD, and even schizophrenia. So, getting a good, reliable handle on how to measure it is a pretty big deal.

What’s the Big Fuss About VDAC?

So, VDAC is basically our attention system getting a bit star-struck by things that used to mean “reward!” Imagine you’re in a training session where, say, red circles mean big bucks and green circles mean small bucks. You learn this association. Later, in a totally different task where colors don’t matter, if a red (previously high-reward) circle pops up as a distractor, your attention might just zoom over to it, even if you’re supposed to be looking for a specific shape. This was famously shown by Anderson and colleagues back in 2011. Their work highlighted that these past reward associations can mess with our reaction times (RTs), making us slower when these “valuable” but now irrelevant distractors are around. It’s like our brain can’t quite let go of that old “ooh, shiny reward!” feeling.

This isn’t just a quirky lab finding. Think about it: for someone struggling with addiction, cues related to their substance of choice can become incredibly potent, hijacking attention and derailing recovery. Understanding VDAC helps us figure out why these cues are so powerful and could even lead to better therapies. A recent meta-analysis by Rusz and colleagues confirmed that this reward-driven distraction is a robust effect across many studies. So, the phenomenon is real, but measuring it consistently? Ah, there’s the rub.

The Reliability Hiccup with Traditional VDAC Measures

For a while now, many of us in the field have noticed that VDAC measures, especially those based on reaction times, can be a bit… well, flaky. Studies looking into the reliability of VDAC have often reported pretty low numbers. For instance, Anderson and Kim (2019) found that while eye movement (EM) tracking of VDAC was quite reliable over time (a solid r = .80), the good old RT measures were disappointingly inconsistent (r = .12). Ouch. Freichel et al. (2020) also found low test-retest reliability (r = .09) for VDAC in reward contexts. It’s like trying to measure a cloud – it’s there, but it keeps changing shape!

Now, Garre-Frutos et al. (2023) did find some better reliability (up to 0.85 split-half) but it often depended on how they crunched the numbers, like including more trial blocks and using specific filters for RTs. This got us thinking: maybe the problem isn’t entirely with RTs themselves, but how we’re using them. Typically, VDAC is measured by a difference score – for example, RT when a high-reward distractor is present minus RT when no distractor is present. This makes sense experimentally because it tries to isolate the VDAC effect. However, statisticians will tell you that difference scores can be notoriously unreliable. Why? Because you’re essentially combining the measurement error from two scores, and you might be reducing the very thing that helps with reliability: between-participant variance (how much people naturally differ from each other).

Could Single Scores Be the Answer? Our Investigation

So, we decided to dive into this. What if, instead of using those tricky difference scores, we looked at the reliability of single RT scores – like just the average RT when a high-reward distractor is present? Our hypothesis was that these single scores might actually be more reliable because they retain more of that lovely between-participant variance and have less compounded measurement error.

We set up two experiments.

• Experiment 1: The classic RT-based VDAC paradigm, pretty much like Anderson’s original. Participants learned to associate colors with high or low monetary rewards in a training phase. Then, in a reward-free test phase, they did a visual search task where those colors could appear as irrelevant distractors. We measured their RTs.
• Experiment 2: We tried a data-limited accuracy-based VDAC paradigm. Here, stimuli are flashed very briefly and then masked, and we measure accuracy. The idea is that if your attention is snagged by a distractor, you’ll have less “data” from the target, leading to lower accuracy. We thought this might be similar to the reliable eye-movement measures.

Sixty-two college students joined us for the first experiment, and forty-one for the second. They all came back for a second visit about 4-5 weeks later so we could check test-retest reliability. We also looked at split-half reliability within each session.

What Did We Find? The RT-Version Shines (with Single Scores!)

Okay, drumroll, please! In Experiment 1 (the RT version), we definitely saw the VDAC effect. Participants were slower when high-reward and low-reward distractors were present compared to when there were no distractors. So far, so good – the paradigm was working as expected! We also saw a practice effect; folks got faster on their second visit, which is pretty normal.

Now for the juicy part: reliability.

• When we used the traditional RT cost (difference score), the reliability numbers were… well, not great. The correlation coefficients for both test-retest and split-half reliability were pretty low, basically not significantly different from zero. This matched what previous studies had found.
• But, when we used the single RT score (specifically, the RT from the pooled reward distractor conditions – averaging high and low reward distractors together), bingo! The correlation coefficients were strong. We’re talking high test-retest and split-half reliability. This was exciting! It suggested that RT-based VDAC can be reliable if we just look at the scores differently.

And the Data-Limited Accuracy Version? Not So Much This Time.

In Experiment 2 (the data-limited accuracy version), things didn’t quite pan out as we’d hoped. We didn’t find a significant VDAC effect. Participants’ accuracy wasn’t really different whether there were reward-associated distractors or not. Since there was no VDAC effect to begin with, we couldn’t really proceed to analyze its reliability – you can’t reliably measure something that isn’t there!

Why no effect? We suspect the task might have been a bit too easy. In our test phase, participants had to identify whether a line segment inside a shape was vertical or horizontal. It’s possible that even if their attention was briefly pulled away by a distractor, they could still snag enough info about the target line in that short presentation time (250 ms) to get it right. Maybe a tougher task, like identifying one of four similar letters, would be needed to really see VDAC in this kind_of setup. Food for future thought!

Why Single Scores Might Be the Heroes Here

So, why did single RT scores perform so much better in the reliability department? It comes down to that pesky thing called between-individual variance. When you subtract a baseline (like the no-distractor condition RT) to get a difference score, you’re often trying to get rid of individual differences that you might see as “noise” in an experimental context. But for reliability, these individual differences are golden! Reliability, in simple terms, is about how much of the score is “true score” versus “error.” If you shrink the overall variance by subtracting, but the error part stays relatively the same or even increases (due to combining errors from two measures), your reliability plummets.

In our study, the standard deviation (a measure of variance) for the single pooled RT was about three times larger than for the RT difference score. More variance for the “true” differences between people means better reliability. It’s like trying to hit a large target versus a tiny one – you’re more consistent with the larger target.

A Crucial Caveat: The Prerequisite for Meaningful Single Scores

Now, before we all rush off and only use single scores, there’s a super important point. A long RT in the reward distractor condition doesn’t automatically mean strong VDAC if the person’s baseline RT (in the no-distractor condition) is also super long. They might just be a generally slower responder!

So, here’s the catch: for a single RT score to be a meaningful indicator of VDAC, you first need to establish that there’s a VDAC effect at the group level. This means running your good old ANOVA and making sure there’s a significant main effect of distractor type (e.g., reward distractor RTs are significantly longer than no-distractor RTs). If that effect isn’t there, then talking about the reliability of any VDAC measure (single score or difference score) is a bit pointless.

Happily, in our RT study, the pooled reward RT was very strongly correlated with the baseline RT (correlations of 0.938 and 0.958 for the two visits!). This suggests that the pooled reward RT was indeed influenced by VDAC and could effectively represent it, especially since we’d already confirmed the overall VDAC effect with the ANOVA.

What Does This All Mean for VDAC Research?

We think this is pretty cool news! It suggests that the typical RT-version VDAC paradigm can indeed yield reliable data, as long as we consider using individual scores instead of just relying on difference scores. This could be a game-changer for researchers who want to study individual differences in VDAC, or track changes in VDAC over time, perhaps in response to an intervention.

VDAC isn’t just a lab curiosity; it’s linked to real-world issues like addictive behaviors and cognitive symptoms in disorders like ADHD and schizophrenia. If we can get highly reliable single scores for VDAC, these could potentially serve as valuable indicators. For example, could an addict’s RT in the presence of a reward-associated cue (a single score) predict their success rate in recovery? Longer RTs might mean they’re more easily distracted by such cues, possibly indicating a tougher road ahead. That’s speculative, of course, but it highlights the potential.

Our study focused on the original Anderson et al. paradigm. There are tons of variations out there exploring different facets of VDAC. We hope future researchers might take our approach – checking for the overall effect first, then examining the reliability of single scores – and apply it to these modified paradigms too.

Wrapping It Up: A Nod to Single Scores

To sum it all up, we found that the trusty RT-based VDAC paradigm can show the VDAC effect reliably. The real star, though, was using individual RT scores, which showed high test-retest and split-half reliability, unlike the traditionally used RT difference scores which, as in other studies, showed low reliability.

Our suggestion? Researchers should seriously consider using single scores when assessing VDAC reliability. They generally have lower measurement error and higher between-participant variance, which are key ingredients for good reliability. Just remember that crucial prerequisite: first, confirm that the VDAC effect is actually present at the group level. Without that, even a reliable single score might not mean what you think it means in terms of VDAC.

It’s an exciting step forward, and we’re keen to see how this might help refine our understanding and measurement of this captivating (pun intended!) aspect of attention.

Source: Springer