Unmasking Bias: What Policies Really Do in Biomedical Research

Hey there! Let’s chat about something super important, especially if you care about what goes into the medicines we take or the treatments we receive. We’re diving into the nitty-gritty of how money and connections might mess with the science behind it all – specifically, in biomedical research.

You see, it’s pretty well-known that when industry money or personal interests (what we call Conflicts of Interest, or COI) get involved, things can get a bit skewed. This can affect *what* research gets done, *how* it’s reported, and even lead to results that favor the sponsor. This isn’t just academic talk; it shakes public trust, could potentially harm patients, and frankly, puts a dent in the whole integrity of the research world.

So, tackling this head-on with smart policies is a big deal. But here’s the kicker: figuring out *which* policies actually work is tough. That’s why I’ve been looking into a cool piece of work – a *scoping review* – that rounded up all the available research on policies designed to keep funding bias and COI in check in biomedical research. Think of it as someone doing the legwork to see what we actually *know* about these policies.

The Problem: Bias is Real, and It Matters

Let’s be blunt. There’s a ton of evidence showing that industry funding and COI are linked to results that lean positive for the sponsor, less public trust, biases in deciding *what* research questions are even asked, and yes, potential harm to patients. Despite all this, meaningful policy changes to prevent and manage COI, especially in research, haven’t exactly been lightning fast.

While other areas like clinical practice have seen some policy tweaks, the research side is still a hot topic of debate. Some reviews have looked at existing policies in research groups, but we really needed a deep dive into the *empirical research* – the studies that actually looked at how these policies work (or don’t work).

What We Looked At: Diving into the Policy Landscape

This review I’m talking about aimed to do just that. It zeroed in on policies related to industry sponsorship, industry connections, and COI specifically within biomedical research, focusing on the pharmaceutical and medical device industries because, well, they’re major players and big funders.

The goal was threefold:

• Document the types of policies covered in the research.
• Map out the evidence we have in this area.
• Point the way for future research on these policies.

They built on previous work but narrowed the focus to just biomedical research and looked at published, peer-reviewed studies that *analyzed* policies, not just lists of existing ones.

They searched PubMed for studies published between 2009 (a key year because of a big report on COI) and August 2023. They were looking for empirical analyses of policies tackling industry ties and COI at *any* stage of the research process – from figuring out the research question to publishing the results.

They ended up screening over six thousand articles (wow!) and included 81 studies in the review. These studies came from all over the world, with international ones being the most common.

Digging into the Data: What the Research Shows

So, what did they find when they synthesized all this research? It turns out most of the studies (a substantial majority, actually) were focused on one thing: *disclosure policies*. You know, those statements where authors or researchers have to list their potential conflicts of interest.

The studies they found fell into five main types of analysis:

1. Surveys of COI policies (checking if policies exist and who they cover).
2. Disclosure compliance analyses (seeing if people actually disclose what’s required).
3. Disclosure concordance analyses (checking if what’s disclosed matches up with other records, like public payment databases).
4. COI policy effects analyses (trying to see if disclosure or other policies actually change anything).
5. Studies of policy perceptions and contexts (how people feel about policies and the challenges of implementing them).

Let’s break down what they learned from these types of studies.

Policy Prevalence: Are Policies Even There?

First off, are there even policies requiring disclosure? The review looked at 34 studies that surveyed COI policies, mostly at scientific journals. They found that policies covering authors are pretty common (median prevalence was 97%). But policies for editors and reviewers? Not so much, sometimes even non-existent in samples.

There’s a trend showing an increase in policy prevalence since that 2009 report I mentioned, which is good, I guess. But the studies surveyed different groups of journals, so it’s hard to make a definitive statement about overall trends. Policies exist, especially for authors, but coverage isn’t universal, particularly for those behind the scenes like editors and referees.

Disclosure Compliance: Are People Following the Rules?

Okay, policies exist, but are people actually disclosing? The review looked at 28 analyses of compliance. The short answer: compliance is far from perfect. Journal articles had the lowest median compliance rates.

Looking at trends over time, compliance rates seem to be holding steady or even declining, even in recent studies. Studies looking at meta-analyses showed higher compliance, maybe because they’re more recent or use templates that prompt disclosure early on.

It’s worth noting that most compliance studies just checked *if* a disclosure statement was there, not *what* it said. Plus, they only looked at what was *published*. Maybe some authors disclosed to editors but it didn’t make it into the final paper. We just don’t know from these studies.

Disclosure Concordance: Does What’s Disclosed Match Reality?

This is where things get really interesting, and sometimes, a bit worrying. Sixteen studies compared what authors disclosed in their papers to other sources of information, like public databases of payments from drug and device companies (like the U.S. Open Payments database).

The idea is, if you got paid by a company, you should disclose it, right? And it should match up with what the company reports paying you. Well, concordance rates – how often the disclosures matched the external data – varied *wildly*. We’re talking from a low of 1% in some studies (yikes!) to a high of 94% in others.

Why the huge range? It seems to depend on the medical specialty (lower in areas like oncology and ophthalmology), the specific products being researched, and whether the authors received really high payments. Studies looking at authors who received large sums often found much lower concordance.

This suggests that even when disclosure happens, it’s often incomplete or inconsistent. People aren’t always reporting all their industry relationships, or the details don’t match up with official records.

Policy Effects: Does Disclosure Actually Change Anything?

This is perhaps the most critical question: does all this focus on disclosure actually *do* what we hope it does – like reduce bias or help people evaluate research more critically? Based on this review, the evidence is… underwhelming.

Surveys suggest that professionals (like doctors reading research) *think* disclosure is useful. They say it’s important for evaluating results. But when researchers actually tested this in experiments, they found that disclosure often *didn’t* meaningfully affect how doctors evaluated research quality, made peer review decisions, or even changed their clinical actions.

For the general public (patients, research participants), the picture is mixed. Some studies found disclosure might affect whether patients participate in studies or how they perceive the quality of care or research. Others found no impact on trust. It seems like people aren’t quite sure how to interpret or use disclosure information.

Social science analyses add another layer, suggesting mandatory disclosure might not tackle the core problems and could even have negative side effects. It might make financial relationships seem more legitimate and create unpredictable reactions from the audience. Basically, just saying “here’s my conflict” might not be enough if people don’t understand the potential impact or if there’s no real mechanism to address it.

Policy Contexts e Challenges: The Real-World Hurdles

The review also looked at studies exploring how people perceive policies and the challenges of implementing them. Stakeholders generally agree that disclosure is needed. But actually making it work? That’s tough.

Challenges include:

• Creating systems where disclosure information is easy to access and understand.
• Lack of clear guidelines on how to identify and evaluate COI, especially for things that aren’t direct payments but could lead to future money.
• Policies that contradict each other – like institutions encouraging researchers to work with industry and commercialize their findings, while also trying to limit COI.
• A big one: lack of effective enforcement mechanisms. Often, the main way COI is managed is through voluntary recusal, which isn’t exactly a strong safeguard.

Studies looking at Institutional Review Boards (IRBs), which oversee research ethics, highlighted these issues, showing a lot of variation in how policies are applied and a perceived lack of clear guidance and enforcement.

The Big Picture e Where We Go Next

So, after sifting through all this research, what’s the main takeaway? The vast majority of studies focus on *disclosure policies*, often treating “COI policy” and “COI disclosure policy” as the same thing. And the evidence suggests that disclosure, while widespread, isn’t consistently implemented, isn’t always accurate, and isn’t particularly effective at mitigating bias or negative outcomes.

This tells us something important: the research agenda on COI policies needs a serious shake-up. Just surveying whether disclosure policies exist or checking if people tick a box isn’t cutting it. The evidence base is heavily skewed towards one type of intervention that doesn’t seem to be doing the heavy lifting we need.

Shifting Gears: The Need for a New Research Agenda

The review makes a strong case for moving beyond just disclosure. We need research that looks at *other* policy mechanisms. Think about things like:

• Management policies: How are COIs managed once disclosed?
• Prohibition policies: Are there certain relationships that should just be off-limits?
• Structural interventions: What about the bigger picture – the conditions that *create* COI in the first place?

We also need more *implementation research* – studies that look at *how* policies are put into practice and *why* there are differences in how they’re applied and enforced.

Furthermore, the research needs to get more granular. Not all industry relationships are the same. Some might carry more risk than others depending on the type of relationship, the amount of funding, or the level of industry control. Policies could potentially be more effective if they stratify based on these characteristics, but we need research to figure out which characteristics matter most.

Looking Ahead: Addressing the Roots of the Problem

Ultimately, the review points towards a need to look at the *systemic* issues. COI policies often focus on individual relationships, but industry influence is complex and multi-faceted. We need policies and research that address these broader influences and cultivate research independence.

This even ties into the larger research funding landscape. Some research suggests that COI might decrease if there were more opportunities for non-commercial funding. With federal funding declining and industry funding increasing in some places, there’s a risk of a “funding monoculture” that could exacerbate bias. Research into policies that encourage alternative funding models is crucial.

Wrapping Up

So, there you have it. This scoping review paints a clear picture: while we have lots of policies asking researchers to disclose their industry ties, the research on these policies tells us they’re often poorly implemented and not very effective on their own. We’ve barely scratched the surface when it comes to studying other ways to manage COI and address the systemic issues that create bias in biomedical research.

The call to action is clear: we need a significant shift in the research agenda to build a truly evidence-based policy landscape that can protect the integrity of science and the public trust it relies on. It’s a big job, but a necessary one!

Source: Springer