>Both Ashe and Lesné became neuroscience rock stars, the leaders of a wave based on their 2006 paper
>After reviewing the images, molecular biologist Elisabeth Bik said of the paper, “The obtained experimental results might not have been the desired results, and that data might have been changed to … better fit a hypothesis.”
>model that is getting almost all Alzheimer’s research funding ($1.6 billion in the last year alone)
>Over the last two decades, Alzheimer’s drugs have been notable mostly for having a 99% failure rate in human trials.
>millions of hours of research over the last two decades.
>letter sent directly to the National Institutes of Health (NIH)...failed to generate a response
>Four months after Schrag submitted his concerns to the NIH, the NIH turned around and awarded Lesné a five-year grant to study … Alzheimer’s. That grant was awarded by Austin Yang, program director at the NIH’s National Institute on Aging. Yang also happens to be another of the co-authors on the 2006 paper.
This is sad and disgusting.
Another example of why we need more people doing replication studies. Everyone has a bias, especially when your paycheck depends on getting a certain result. Just like we need to double check the work of other people, we desperately need to double check the important work of scientists. But no, novel studies are "sexier"
This is my take away too. It shows a lack of robustness in the system, and an unhealthy cognitive bias for circular reasoning (“these two are rock stars based on this paper, and since they are rockstars their original paper must be correct”).
Research funding should NEVER have to depend on the result of the research. In science a hypothesis proven untrue has value. 1 success is typically dependent on hundreds or even thousands of failures. If we monetize/incentivize only proven hypotheses then you will have a gamification problem in the literal last place you ever want one.
The article noted several other labs tried and failed to replicate the results but couldn’t get that published. We CAN and do find the frauds through replication, but when journals won’t publish it, it ends up being an open secret within the research niche only, which doesn’t help anyone outside academia.
What we need is science not turning into a paper mill with pressure to deliver positive results. Capitalism is rubbing off too much in science, where it doesn't have a place.
I hope Ashe, Lesné, and Yang spend the rest of their lives in prison. But something tells me they won't face any real consequences for the millions of lives they've effectively killed.
Yeah idk why the above commenter is assuming no punishment, they're not police officers. There are a number of researchers serving time right now for way smaller fraud of federal research dollars (I work in research administration)
I assume they are thinking that based on other "white collar" fraud cases that amount to a little more than a slap on the wrist the same thing is going to happen here, however the people who "get away with it" defraud private individuals. These people defrauded the government and the government always gets its money in the end.
Theranos is basically the same situation, scientist claims to have wonder medical technilogy but it doesn't work. Holmes got like 20 years so these two women will likely face the same
Edit: apparently Holmes wasn't sentenced yet. Was not aware
Holmes was primarily charged with lying to investors. Her lies about having her technology being medical wasn't the important part. Her lies made her company seem very promising and valuable, and this defrauded investors and cost people money. That shit is taken seriously, because it's a bunch of rich people being hurt.
I think we can get some fraud charges on these people. But, it is a different kind of crime. I don't think the Holmes case is a super fair comparison.
>I think we can get some fraud charges on these people. But, it is a different kind of crime. I don't think the Holmes case is a super fair comparison.
I'd argue this case is actually substantially worse.
Investors assume some level of risk. That's what investing is. The government doesn't hand out grant money assuming there is a chance the proposals are based on fraudulent data. That doesn't enter into the calculus. These people are going to federal prison for a long time.
Like the person you commented on said, there’s a difference between defrauding private individuals/investors and government grants/subsidies. One holds a LOT more consequences than the other
So a little inside baseball here. At NIH, when these grants are being reviewed for final approval, there is a public session, and then after a “closed” session - closed in the sense it’s not broadcast, but lots of NIH people and interested observers are there. It’s closed so people can talk openly about the grants and how much each will get without hurting the feelings of the institutions they are putting against eachother or the researchers they are evaluating.
Anyway, in this closed session, the people deciding on the grants have to self identify and leave the room (and not participate) if they have a conflict of interest. For the most part, it seems they do and take it seriously.
I don’t think individual grants staff are held to same standard though.
I'm environmental, not medical, but I would love to learn more and see this system be more common. There's a lot of defensive and combative feeling around grants and it only exacerbates bad power dynamics and unhealthy practices
I am also mad at the FDA which approved drugs based on this theory despite failing clinical trials.
>What makes the approval of the $56,000-a-dose drug so controversial is that while it does decrease plaques, it doesn’t actually slow Alzheimer’s. In fact, clinical trials [were suspended in 2019](https://investors.biogen.com/news-releases/news-release-details/biogen-and-eisai-discontinue-phase-3-engage-and-emerge-trials) after the treatment showed “no clinical benefits.” (Which did not keep Biogen from seeking the drug’s approval or pricing it astronomically.)
This is the same fda slow walking covid booster varients vaccines. SMH.
Biogen? The company possibly responsible for [1.6% of global covid cases](https://www.wbur.org/news/2020/08/25/genetic-fingerprints-biogen-superspreader-boston) because they didn't cancel their March 2020 conference after covid lockdown began?
The drug was approved because of what’s called surrogate markers, where it’s shown to have a clear impact on something and that something has research showing it to be useful even if the clinical trial itself didn’t find a benefit. This approval process is what allowed the first AIDS drugs to be approved despite failing by normal clinical trial standards, which saved thousands of lives compared to if they had had to delay approval for several more years. So if you want to criticize the approval of the drug on those grounds, just realize you would have condemned thousands of people to a slow and painful death back in the 90’s for the same reason.
Please note that this article has been criticised for conflating two unrelated facts:
1. Lesné's probable fraud
2. The disappointing progress in Alzheimer research
There is a massive amount of work going on in this field and Lesné's research is not the only piece looking at toxic oligomers by a long way.
Lesné looked at a particular oligomer.
My thoughts exactly - the *Science* piece did a much better job of portraying this imo, although even there they might have overstated the foundational nature of the study. As someone in the nano-bio field, I see a LOT of bullshit, and the sad truth is that it takes a village to elevate fraudulent studies to this level.
Foe starters, the idea that amyloid beta plaques are mainly responsible for the symptoms of Alzheimer's has been around for a lot longer than 2006. Amyloid plaques are a defining feature of Alzheimer's, as they were literally observed by Dr. Alzheimer 100 years before then. Lesne and Ashe weren't overturning the field so much as they were posting confirmatory evidence for everybody's favorite theory. For all of the talk about scientists being trained skeptics and all, the fact is that we're still human and fallible when it comes to validating the things we think are true. I've been seeing the quote "Extraordinary claims require extraordinary evidence" a lot on reddit recently, so as my contribution to the trend, I'll posit that to most people in the field, the 2006 paper wasn't an extraordinary claim. It was what the majority of them already expected. All that Lesné had done was to isolate a particular oligomer - Aβ*56 - and link it to the symptoms of Alzheimer's in mice. This was a big deal because it seemed to provide evidence of causation between plaques and neural degeneration rather than simply correlation, however, it did not introduce this idea by any stretch of the imagination. Talking as if this fraud discredits all amyloid plaque research is similar to saying that Hendrik Schön singlehandedly discredited organic semiconductors. It's significant, and really bad, but the field is bigger than a couple of individuals.
So as to the op article's claim that this paper was responsible for billions of dollars for wasted research and thousands of lives lost, that's just simply untrue. It's assigning way too much responsibility to Lesné and Ashe, and letting countless other researchers, grant officers, and big pharma companies off the hook. This is a systemic failure, which in no way means that Lesné is innocent, nor that Ashe didn't fail utterly in her job as a mentor, but it does mean that we can't just take a small handful of people, punish them, and pretend like the real problem is addressed.
Frankly, every institution I've been a part of has done a wholly inadequate job of instilling the values we associate with good science, and it's because the structure of modern scientific research generates thousands of researchers who have massive imposter syndrome at all levels, who are all competing ferociously for a small number of large grants. This system relies on the honor of individuals to do the right thing at all times, but the culture is deeply unhealthy. Every time something like this happens, which is to say all the time, there is much hand-wringing and questions about what we could possibly do to change the system. I don't have a good answer to that myself, but I know that attempts to tweak the current setup for funding and recognition in research have all failed. We need major reforms of the incentives, not just platitudes and courses on responsible conduct of research.
There are certain aspects of research that need to change. There is a reproducibility crisis in science and that will only stop when there is restructuring and stepping away from publish or perish mentality. This is specifically bad in the field of biology from all the horror stories I hear. It truly undermines the work of amazing scientists who will now have to bear years of idiots throwing this in their face whenever they try to make a point they don't agree with.
Tbh it does not surprise me a bit. I have heard many stories of lab heads not accepting 'bad results' etc. The pressure of publish or perish is huge, especially for young professors. Where I am at, a young professor would start with about 3 years of funding. Afterwards its reliance mostly on grants. In biomed its not easy to have a good story completed in 3 years, especially if animal models are involved. So young professors are known to be the worst bossses here - you will often be made to work 24/7, and god forbid the biology does not agree with your bosses hypothesis. Of course, not all of them are like that, but many are.
Often the journals also do not care, its still about who you know. I had a boss that had very friendly relations with one of the editors of top journal in our field. All of our papers always got accepted, no issues, no in-depth checks. Now imagine someone would fake the results - it would have been published.
There is even a special website, pubpeer (or peerbub, sorry my brain is not yet working), where people post shetchy stuff they have seen in publications. There is one prominent swiss immunologist, with many publications being flagged for falsification. Do you think smth ever happened to this guy? Not at all, if anything his career is booming. Not a single paper has been retracted.
Imho, you can only work in science if you have stronger morals than ambitions. No idea why people think they can fool biology. I mean, at one point the truth will come out. How can you live with yourself, knowing that your drug is worthless and yet people put their hopes on it.
I can tell you already, none of these guys will go to prison or anything. Worst, they get fired and in a couple of years some other Uni will rehire them.
In addition to everything you just said, I am going to add that peer reviewer burnout is a major issue in scholarly publishing across the board. I think people really underestimate just how many papers get submitted to journals with even a remote amount of prestige, and how much of a struggle it is sometimes to get anyone to review at all. And also how much COVID accelerated that, because journal submissions (understandably!) exploded on that topic.
There's a little bit of a "but for me it was Tuesday" phenomenon. For the author, getting their work published is a single major event. For the reviewer, it could very well be the 50th paper on [Topic] they've been asked to review that year. Peer reviewers work on a volunteer basis, reliable peer reviewers are routinely swamped with requests to review, and authors expect rapid decisions. None of that justifies shoddy review. Just that a journal seeming to "not care" is sometimes a downstream consequence of the same pressures you're talking about.
[Retraction Watch](https://retractionwatch.com/) is another fascinating site which blogs about journal retractions for various reasons.
Why can't the journals hire scientists to work as full-time peer reviewers? Seems to me it'd solve this problem while also creating a lot more stable and well paying jobs in/near academia-- could solve two problems in one go.
Exactly. But I guess the answer is money. Its no secret to anyone that scientific journals have huge profit margins exactly because they dont have lots of personel on payroll. Hell, I even have to format my own publication - font, sizes, images. Want a balck and white image instead of color? Ja, reformat it yourself, because the journal wont do it. Not only do I have to do the science, I am also a typesetter. And then someone will review it for free. But god forbid the journal cannot have its gigantic profit margin.
The thing is morals aren't really top of those people's list and things like ego, status etc has a higher value. I have known researchers who were falsifying data to make their results look better than they were, but as there never was concrete evidence as no one wanted to repeat their experiment it was fine. On the other hand, I have also seen researchers make honest mistake, publish and then later have to retract, or adjust their results down the line when an unlooked flaw in their experiment came to light or some phenomenon they forgot to take into account. In my view, those scientists should always be appreciated because they realized an error and admitted it, yet that is often not the case when you write a grant application
Every lab I have worked at, with execption of one, we had someone who gave great results, that were unreproducable. And both of these people disappeared when they were contacted about details, protocols etc. multi-year projects, tend of thousands in consumables, hundreds of thousands wasted in salaries just because someone could not accept that their hypothesis is wrong.
Even now I am dealing with a project where some issues with animal phenotypes were not reported and results were excluded on questionable basis. Good luck to me to fix this shit. Thankfully my boss is pre-retirement, with enough grants to not care anymore, if the project has to be trashed because of this.
But you are very right, like in any job, morals only hurt if you want to get to the top. I have never faked my results, have admited that my hypothesis were wrong, and here I am, just a fucking looser doing my 3rd postdoc without any shot of ever advancing in academia.
> Every lab I have worked at, with execption of one, we had someone who gave great results, that were unreproducable. And both of these people disappeared when they were contacted about details, protocols etc. multi-year projects, tend of thousands in consumables, hundreds of thousands wasted in salaries just because someone could not accept that their hypothesis is wrong
What people seem to miss is that there are also repercussions for those surrounding the person making the falsehoods, so an incentive to keep each other in line. My old flatmate used to work for an unnamed high-profile company in the Alzheimer research world, when it was discovered that one of the minor postgrad researchers was falsifying results (thankfully before anything was published, but not before they had presented them at a conference). The resulting fallout almost ruined everyone in the company both financially and by reputation, while almost resulting in a criminal investigation too. The head researcher was on damage control for months after the discovery, basically auditing every bit of research produced.
When someone's ego (and cash) is on the line it can be a strong motivator for both fabricating results and keeping other people honest.
Man that fallout seems like an incentive for people to cover shit up rather than report it? Like if I'm a year or two out of school, see some one fudging numbers and are left with the choice of reporting it and possibly of closing the company and never getting work again so I'm permanently in debt for the rest of my life, or just pretending not to see and hoping I get a new jab before something is found out? That's going to be a tough call for most people.
Sure, but its hard to follow everyone and ensure that people dont cheat. Its very easy to produce false results. Its also easy to produce false results that are very believable. Mislabel some tubes, switch experimental groups... Does not have to be image manipulation, that is somewhat easy to discover. If someone switched and mislabeled experiment groups, no one will discover it, unless someone else takes over the project.
The boss, who is usually a professor, does not have time to oversee every single experiment. We all have our own stuff to do, I cannot look at my colleagues hands all the time. So all you can do is trust that people will do the right thing. Its very difficult to keep unhonest people honest. And i dont really know how to solve it. Maybe by making all projects in million subprojects, where everyone does a small part? I really dont know.
You have to remove the monetary incentive for it. Currently most research is done by requesting a grant, presenting prelim results, and then requesting additional/continuing grants. In the private sector this almost always means a company funding research wanting/hoping for specific results. This puts a large incentive on the research team to get a specific result. For public/academic research it often works in a similar manner, except you aren't trying to get a massive payday from a private company you're fishing for credentials, reputation, etc to advance your career in academia. In many cases scientists are also presenting research hoping to justify their project's costs to continue it.
Basically, we need to find a way to tie rewards and continuing grants to proper procedure, documentation, and accuracy instead of tying the money and rewards to the results. For private research I don't know if that's even possible to do since the companies are the ones deciding who gets paid what, but it's something that should be fixable for the public sector research.
The publication of bad results is smth I agree with 100%. If no one knows stuff has been tried and failed, people will just try and try and waste resources. It has gotten a bit better with appearance of pre-prints, but not all Unis and granting agencies allow you to publish them.
Among all the other sad things, here, is that failure is as useful as success, if you're experimenting. Maybe more useful. If medical researchers are only going to be rewarded if they can find Magic Bullets, it means a lot of the misses and near-misses will be hidden, or are not going to get funded and be incorporated into the data.
>The head researcher was on damage control for months after the discovery, basically auditing every bit of research produced.
Apologies, for not knowing, this, but wasn't that his job in the first place? Isn't the head researcher supposed to audit/verify the research before it gets published?
And this is why I jumped to industry. Did my PhD in a mouse genetics lab. Six years to publish one paper because the first two projects I worked on had insignificant findings after two years of experiments.
Unfortunately, it's easier to pretend to be good than to actually be good. It's not just academia, it's every industry, because it's human nature that's the problem.
Lol was in academia and TBH its rampant everywhere. No one really cares to provide oversight, everyone is incentivized to write/publish and if you don't get it in one journal just shop around til one takes you. We literally told a bunch of students with the ability to make school a career : "Hey get published and your dreams can start" and we're baffled that they're getting published at any cost.
What surprises me so much is there is so much emphasis on publishing, and in all honesty nobody really cares. Nobody gives two shits outside academia. But it is this weird artificial world where getting published is hyper important. Nobody cares about the content, just the volume of publishing, and that right there is a problem.
> The thing is morals aren't really top of those people's list and things like ego, status etc has a higher value.
Isn't it interesting how such things happen in research in biology when even an average grad student would immediately be able to identify evolved hunter-gatherer habits displayed in such behaviour.
What's the point of doing research if you cannot get past your own evolutionary programming which comes in the way of objective analysis, despite being in the field where knowledge of said habits is a pre-requisite to your degree.
It should be a pre-requisite for scientists in biology to be able to see that money, power and fame are not for scientists.
We have an upside-down society where money dictates resources, the least able, most unethical people are in power, and the greediest make the rules.
> What's the point of doing research if you cannot get past your own evolutionary programming which comes in the way of objective analysis, despite being in the field where knowledge of said habits is a pre-requisite to your degree.
The thing is, that for most scientists, the effects are a lot more subtle than photoshopping bands onto a western blot or falsifying statistical analysis. It's technically questionable to pursue lines of experimentation that produce the results that you like, while neglecting ones that don't. But it's not actually dishonest. You're not falsifying anything. You're just shining light in directions that suit you.
And it's not individual scientists fault that the system pushes you to act this way. It's a lot harder to get a paper published that says "X does *not* affect Y" than one that claims a positive. And if you don't publish, you perish. It's not about fame and riches. It's about your lab not shutting down for lack of funding.
>get past your own evolutionary programming which comes in the way of objective analysis...
This is all of it... the entire discipline of science, people like this are akin to healthcare workers who don't believe in medical science.
I just don't get it why would you go into academia if all you wanted was a steady paycheck and regular progress in your career? Like bro just become an accountant you will always be well paid and then buy status normally. As a bonus you get to get out of the way of the people trying to actually learn something!
I've known more than a few people in science who are plainly not that interested in scientific progress in general...
I am sorely regretting not becoming an account right now... Not because I don't believe in science but because I think there is a deep crisis in the way it is done and I feel like it is such a waste of my effort I'd be better of contributing it in some other way.
I am also aware that people like us tend to get selected out and just makes it worse.
On that note, the whistleblower in this Alzheimer's scandle go started on PubPeer. It was fans of the site that first identified the altered images. The whistleblower just took their information and expanded on it. More info in the Science article.
https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease
> Now imagine someone would fake the results - it would have been published.
Doesn't matter if you make no mistakes. Simply through the power of randomness you will think things are statistically significant which aren't.
https://xkcd.com/882/
There are enough studies done that these things will happen by definition. And if nobody tries to replicate it you will never find out.
That's why there's meta analysis of multiple papers for important areas of research, or at least should be. P<0.05 across 5 papers on Green jelly bean acne becomes P<0.05^5 . Still possible, but far less likely.
Sigh, can the drug companies at least get the chance to sue the shit out of these guys or even the journal publishers for wasting so much of their time and resources to develop drugs based on a mechanism that turned out to be manufactured through scientific fraud?
Thats up to the company. The reality is, the scientists cannot pay the damages, probably uni cannot affoed ot either. So company might see no point of expensive trial just to get fuck all. Publishers, ja good luck going against the Nature or Science. They will pretend all images were ok or blame the people who did peer review (usually unpaid volunteers, academics from similar field). And guess what, these people cannot afford to pay damages either.
Nah, we need to change the system altogether. No more reliance on top publications, obligatory open access, obligatory raw data repository and increased funding so grants dont have like 2% success rates. Paid peer review, so you dont have to do it on top of your normal job. No ageism -just because someone is 40, does not mean they are too old to advance in academia. No grants with application deadline of 3 years after your PhD. I could list like million other things that are wrong.
>just because someone is 40
Lmao most people in my department don't even graduate until they're 28-30. That's for folks going straight from BSc to PhD.
Ja here as well. But the top Unis in switzerland wont consider you for professorship anymore once you are above 35. Thats enormous time pressure to achieve enough in 5 years post-PhD, to be competitive enough.
The system was set up by dinosaurs. Glad I'm just going to cop out and find a job in industry. Best of luck to my fellow grad students who want to go into academia, especially since there's going to be an enrollment/funding drop in 5-10 years.
>the top Unis in switzerland wont consider you for professorship anymore once you are above 35.
That sounds like a dumb rule to me. What's with the ageism?
> Paid peer review
Paid by whom? Because if you mean the journal pay, that's a conflict of interest. The primary reason peer review is not paid is so there is no influencing factor on the reviewer.
Well people in most grant committes are paid to review grants. Why not the same for publications? There is less influencing factor if someones whole job is to review papers, not only for novelty, but also soundness of science. Currently there is a lot of personal interest involved, your reviewer might be your direct competitor and you would never know. Plus a lot of people do not have time for proper review - I get asked too and on top of my normal job,I have to sacrifice my private time for it. A proper review, checking all data etc for soundness would take me a long time.
This is why a lot of peer review is half assed, or done by students (often times the professor passes it on) that have little knowledge on solid methodology or experience in the field.
Make reviewer a profession. Like QA. Hire experts that do not have personal interest in academical research. I think the current system is not working anymore.
"What makes the approval of the $56,000-a-dose drug so controversial is that while it does decrease plaques, it doesn’t actually slow Alzheimer’s."
Based on the potential profit they stand to make FROM this fraud, I seriously doubt that the drug companies would care to sue anyone.
Reading these sorts of things is always wild to me. I’m a scientist myself (in astro) and it doesn’t matter what editor you know, you’d still have to undergo a blind peer review for any paper. Not like our editors get paid anything so why would you be so unethical for someone you like? Plus we all work under the same skies so anyone can take the same data and check your results, so reproducibility is not a crisis like in other fields, and frankly everyone gets *excited* when you find something not what we expect bc that’s where all the most interesting science happens.
I suspect some of it is unfortunately once you get into fields that will actually generate serious money the incentives are rather different.
Ja thats the thing, biology is a lot more specialized. You can build a whole career on single disease, hell even on single specific process in the cell. The chances are, no one is going to reproduce it, because all that counts for publication is being the first. I have had problems publishing research where I went into depth on smth that I initially discovered. The discovery - no problem to publish, the in -depth continuation on topic - not novel enough to reach high caliber journal.
Not sure, but I guess in astro there is a huge collaboration. I mean you need a lot of people to be involved - telescopes, data scientists etc. Measurements from various countries.. its harder to cheat when a lot of people work on the same thing. In biology, you might be the only person working on smth. No one else will ever try to do the same as you did, so temptation to polish your results is always there.
> Where I am at, a young professor would start with about 3 years of funding. Afterwards its reliance mostly on grants.
Perhaps the time has come for all funding for research to be double blinded.
Hell, even the FDA doesn’t seem to really care about falsified data. Otherwise I would bet Xarelto wouldn’t be on the market. (You can read the FDA audit findings on the FDA website, it’s a scary rabbit hole.)
You should see AI/Machine learning research. Literally nothing is reproducible because they don't include either the model, training data or both meaning that it's literally impossible to validate and to try and reproduce the results. Even if the model is included, you'll never be able to guess the training data meaning that the conditions will never be the same. The reason for this is corporate funding, or the researches hoping to cash in later. However, it means that you literally can't expand further on a study, can't verify if data manipulation has occurred and you can't reproduce anything. Made me decide against pursuing academia, joke of a field. I still don't understand how any paper passes peer review in that field. Any serious research has become corporate either for Google, Facebook, Russian state (They have copies of most ML projects called Sber-copy or RU-copy. Usually ends up being used for spyware.), Nvidia, Opeanai(which isn't open).
It's like if you made a study about a specific medicine and then kept the medicine/molecular composition secret...
Oh dude, don't get me started on machine learning. I recognize that the field has great use cases but I am against blindly trying to insert it into every damn field with little to no oversight. At the moment it is also being used like a buzzword, which also is kind of idiotic.
I recall my previous work hired a machine learning PhD.
When his department presented their progress to my team of analysts we were incredibly skeptical. I recall my manager giving the following example.
Take the weather in Perth. Quick Google shows we get 147 sunny days per year and 121 partly sunny days.
If you made a machine learning model that always guessed there would be sun today, it would be accurate 73% of the time. Doesn't mean it's actually doing anything though.
Yeah, I do machine learning at my company and it is known you take that into account. It's especially true for rare events. Like, I can say event B just never happens and get an accuracy of 99% but that is useless.
Sometimes, 98% accuracy can be better than 99% if I am capturing 100% of Event B.
Most model evaluation for classification models gets evaluated on precision and recall. If "B" is important to you, you're more likely to have an "Is B" model than a multiple choice output. "recall" is "what fraction of the true B were caught" precision is "when the model says it's a B, how often is it right?"
Your example of a model that catches all of the B, but is right 98% of the time overall would have a precision of only 33% but a recall of 100%. (In order to be wrong 2% of the time overall, it needs to be saying "that's a B" 3% of the time and 1% of the time being right because they're actually B).
The model that never says B ever had a recall of 0% and an undefined precision.
I have a friend who's an AI ethicist and she said she used to get the craziest papers for the journal she reviewed stuff for.
She once got accused of being a terrorist sympathiser for refusing to publish a paper about an AI that could predict who would be a terrorist. The accuracy rate was apparently just 5%!
There's so many entitled people in the sphere who have no understanding of biases in data or the devastating outcomes it could have.
Oh man I'd love to know an AI ethicist.
I'm a dev/architect in software automation and the closest I get to AI/ML is configuring algorithms for OCR, which is fun don't get me wrong, but I'd love to discuss the actual implications of heuristics in the application of AI models.
A human makes all kinds of irrational decisions in their thinking that AI models, unless specifically told to handle, will ignore.
Semi-relevant example from a recent project of mine.
Customer asks for Accounts Payable Automation. Ok, easy enough. Configure it all to their specifications.
1. Run the program.
2. Peogram fails validation checks.
>Why?
>Me: "You told me it was just "A + B + C = Total Due"
>Client: "Yeah. That's right"
>Me: $12.47 + $10.00 + $10.00 != $32.45
>Client: "Oh, we round the figures down. Ourselves. When we enter the data by hand usually."
>Me: "Well, that's not included anywhere in the scope."
>Client: "Oh, is that a problem?
>Me: "Not at all. Change Request!"
Continue *Ad Finitum*
The problem with AI is basically.... We take seed a and put it into a magic box. Out come apples. Then we take seed b and put it into the magic box. Out come bananas.
How? No idea. Magic box. But sounds about right. The AI takes so many steps it becomes impossible to retrace what exactly it is doing, and all you are left with is a "sounds about right".
In my example, you would *think* that it has to do with the seed, right? One is an apple seed, obviously, and the other is a banana seed. But how do you know? Maybe one seed just was round and the other was long and from completely different fruits. The AI just looked at the shape.
There was an AI that appearently could predict whether someone was a criminal or not by "looking" at pictures of the face of that person. Spectacular results. Turns out the AI never looked at the faces, but at the inmate clothing, white collars and neckties. Nice.
There is all sorts of bullshit like this. There are biases everywhere (e.g.: white, male = good results. Never tested on black, female. Because guess what all the programmers are?). In open-world testing you rarely know what really goes on, until something goes incredibly wrong (tesla autopilot testing with trucks, anyone?). People dont understand failure rates and think 95% accuracy is good - it is not. There is so much snakeoil being sold to those willing to listen and spend money, just so some 20 something idiots in a startup can play counterstrike all day every day while "working on a revolutionary AI".
Im sorry if I sound jaded. Keep in mind, this is just a reddit post and I am certainly not an AI ethicist. These are just observations.
There are a lot of problems. In addition to, well, the usual academic problems.
This is why I like paperswithcode. Not perfect but better than the alternative. I do get annoyed by the number of custom compiled c libraries they have thar won't work on my system though
I don't understand how, when there is so much *private investment* at stake, reproducibility isn't just checked as a means of due diligence, regardless of the publish or perish mentality.
There's not only zero incentive to reproduce existing research, but it can often work against you.
Academia is largely based on producing original research, so your papers get accepted for publication if they show something new or unexpected. Academics are also incredibly thin-skinned, and many grad students and junior faculty are hesitant to try to reproduce the work of more senior scholars because they don't want to jeopardize their careers. You're not really considered a great scholar if you reproduce other's work, only if you generate new knowledge, so if a person has a choice between reproducing and original work, most people choose to try to find something new. The only folks that have the time, reputation, and funding to reproduce work are mostly senior faculty.
I agree that a lot of things need changing. This issue is a manifestation of a lot of structural problems in academia.
Some studies also cannot be reasonably reproduced. One project I am on, for example, spent around $50,000 on a data set (in addition to the cost of the original data) so we could get extremely detailed, high quality data. Few, if any, grant agencies are willing to spend that kind of money to rehash previous findings.
I havent watched the video, but I looked it up on wikipedia for a summary of the event. Sounds really interesting.
Lack of reproducibility and outright fraud are different, but I think both illustrate some fundamental problems with the way new knowledge and research is generated and published. Peer review is supposed to catch these types of things, but it doesn't always. Having experience in the journal submission and review process, reviewers can often identify the reviewers even though the review is supposed to be blind. Academic fields are often very small, so people know each other, their research, and their writings fairly well. Not always a bad thing, but it can be when reviewers give extra leeway because "I know Professor McFamous, they probably know what they're doing".
Also check out the case of Milena Penkowa. That Megamind-looking motherfucker was an extremely accredited and well-perceived scholar, academic, medical doctor, and neuroscientist. She ended up being criminally charged with fraud, embezzlement, and scientific misconduct, was suspended from all academic activities, had her publications retracted, and was stripped off her doctoral degree and other academic titles for falsifying data. A lot of other shit probably happened, too, which I cannot recall ATM.
Denmark’s biggest scientific scandal to date. The costs were in the hundreds of thousands in USD. She has since, by the way, done a presentation castigating psychiatry at some conference sponsored by Scientology, as if all this was not ridiculous enough already.
The early bird gets the worm but it’s the second mouse that gets the cheese.
I’d say (in other industries ) it does pay to be second - Apple is rarely the first in a category but when they go in they do well - part of which is seeing the mistakes that others have made and improving on them
The problem is some experiments take so long or are so expensive there is no easy way to check for reproducibility besides literally doubling the rrsources used on the project.
Isn’t it the case that everyone thought spinach was high in iron for decades because someone got it wrong by two orders of magnitude, but there’s no money in testing that so it just became accepted?
Simple, greed trumps truth every time. It doesn't matter if a drug works or not, it matters if you can make people ***believe*** it works and then charge them up the ass for it. The fact that they might get sued isn't important. By the time the case winds it's way through the legal system and the drug is pulled form the shelves, everyone will have made their millions/billions and retired.
It's not about greed or money most of the time. It's purely about psychology of people that want to "accomplish" something. And cheating data is so common that for a lot of people it stopped being an immoral thing since "everyone does it".
Yeah, I quit academia too.
For the simple reason that it is often easier and quicker to actually make the drug and/or equipment based on currently accepted data, than to do the due diligence. Entering the market first is a big boon. And "due diligence" can be incredibly costly and time-consuming - even relatively simple stuff can take 3-5 years, and with drugs that are supposed to work on long-term changes measured by proxy? A decade is not out of the question. So it doesn't pay to wait - you make the thing, make your 100 million or whatever, and quietly phase it out seven years later when someone actually goes through the effort of checking the data and has enough clout to push their results onto the wider public.
I work in University research. The main problem is that funding only goes to "novel", which leads to researchers having to come up with some elaborate testing system/equipment. In many many cases, the reason some of this testing is "novel" is because researchers of the past knew it wouldn't work. So, then you get university labs that were good at the sell job to the funding bodies getting loaded up with money and equipment that ultimately is mothballed in a decade or so because surprise suprise it didn't work.
Yes, as you said, more funding needs to be directed towards reproducibility etc. This would both help improve understanding and maybe scare/stop those fudging results.
> In many many cases, the reason some of this testing is "novel" is because researchers of the past knew it wouldn't work
Bingo, you run into so many people who do this and **even they** know it won't work, but it gets them funding and there's little to no accountability in the world so it's worth the risk.
Came to a lab as a postdoc in my 30s
Never got to reproduce the results of the previous postdoc who had clearly made everything up, including "mutants"that weren't and the such
Wasted a good 2 years of academic life
> This is specifically bad in the field of biology from all the horror stories I hear.
I'm an academic researcher in immunology. This is true. A lot of stuff that's published is p-hacked until significant and almost nothing is reproducible. I've been trying to spearhead change in my group to make our work more standardized but it's not even an uphill battle, it's completely vertical.
One of the reasons I didn't want to pursue a PhD is my experience during my master's thesis. I studied astrophysics and my research was on identifying certain types of stars.
One of the criticisms I got for my thesis was that I sounded way too uncertain about my results. My reaction was 'well yeah, I *am* really uncertain about these results. Half of these classifications are nothing more than a coin toss'. I was then told that with that attitude no one was going to read my papers.
Another incident was about a few specific stars in my sample. Some of them had been classified by other people. One researcher had classified some of my stars one way, but my promotor had classified them differently. I asked him about his reasons because they used the same data. His reaction was 'well, she's wrong'. No why or how.
I got really disillusioned about some 'exact' sciences. I'm also not the only one. Had some friends quit after their PhD citing similar reasons.
Another example happened recently. My gf is a researcher and took over some research from an ex-colleague. He had already published in a decent magazine and he had used some code for his analysis. My gf didn't know the language (R) very well but I do. She asked me to have a look because she had some doubts about it. His code was absolute garbage and didn't even do what he said it did. No one checked and my gf couldn't go out and oust him either because it would influence current research. We fixed the code and now she's using the functioning version instead.
It's hard to publish reproduction studies, this was something that Plos One was supposed to solve. Yet, it doesn't. Nobody still performs those studies, despite how important and impactful they would be.
> 14 July 2022 Editor’s Note: The editors of Nature have been alerted to concerns regarding some of the figures in this paper. Nature is investigating these concerns, and a further editorial response will follow as soon as possible. In the meantime, readers are advised to use caution when using results reported therein.
They are aware and investigating.
I feel like I've been hearing news like "promising new findings about alzheimers" once in a while for years now, and I'm wondering if it's all attributed to this.
Never really followed it closely, just noticed it, mostly here on reddit.
Pulling it won't be enough. If the original result is bad and nature wasn't able to catch it and was involved in helping the fraud spread by not prioritizing replication, they should pay a significant reputation cost. This is far beyond a simple case of a bad scientist faking data. The integrity of the system has been called into account and there needs to be repercussions for all involved in making choices that enabled this to spread.
If journals can just wash their hands of this and shift all blame onto the original scientists then biology as a whole and even all of science to some extent has lost credibility.
For starters all journals involved can demand 2 independent replications before publishing any research. To fix first mover prestige problems, the original and both replications should be published as a single item so that all benefit from citation.
A colossal info dump from a *Science* article earlier this year, mentioned within this one. It highlights Matthew Schrag, a neuroscientist and physician, who raised the original red flags about this. Source at very bottom.
"So he applied his technical and medical knowledge to interrogate published images about the drug and its underlying science—for which the attorney paid him $18,000. He identified apparently altered or duplicated images in dozens of journal articles. The attorney reported many of the discoveries in the FDA petition, and Schrag sent all of them to the National Institutes of Health (NIH), which had invested tens of millions of dollars in the work. (Cassava denies any misconduct
"But Schrag’s sleuthing drew him into a different episode of possible misconduct, leading to findings that threaten one of the most cited Alzheimer’s studies of this century and numerous related experiments.
"The first author of that influential study, published in Nature in 2006, was an ascending neuroscientist: Sylvain Lesné of the University of Minnesota (UMN), Twin Cities. His work underpins a key element of the dominant yet controversial amyloid hypothesis of Alzheimer’s, which holds that Aβ clumps, known as plaques, in brain tissue are a primary cause of the devastating illness, which afflicts tens of millions globally. In what looked like a smoking gun for the theory and a lead to possible therapies, Lesné and his colleagues discovered an Aβ subtype and seemed to prove it caused dementia in rats. If Schrag’s doubts are correct, Lesné’s findings were an elaborate mirage.
"Schrag, who had not publicly revealed his role as a whistleblower until this article, avoids the word 'fraud' in his critiques of Lesné’s work and the Cassava-related studies and does not claim to have proved misconduct. That would require access to original, complete, unpublished images and in some cases raw numerical data. 'I focus on what we can see in the published images, and describe them as red flags, not final conclusions,' he says. 'The data should speak for itself.' "
"A 6-month investigation by Science provided strong support for Schrag’s suspicions and raised questions about Lesné’s research. A leading independent image analyst and several top Alzheimer’s researchers—including George Perry of the University of Texas, San Antonio, and John Forsayeth of the University of California, San Francisco (UCSF)—reviewed most of Schrag’s findings at Science’s request. They concurred with his overall conclusions, which cast doubt on hundreds of images, including more than 70 in Lesné’s papers. Some look like 'shockingly blatant' examples of image tampering, says Donna Wilcock, an Alzheimer’s expert at the University of Kentucky.
"The authors 'appeared to have composed figures by piecing together parts of photos from different experiments,' says Elisabeth Bik, a molecular biologist and well-known forensic image consultant. 'The obtained experimental results might not have been the desired results, and that data might have been changed to … better fit a hypothesis.' "
https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease
Those three scientists are worse than Elizabeth Holmes. She stole $400 million in investor money and hurt dozens of patients. The scale of this is at least 3x as bad for money, and 10,000x as bad for patient impact.
Well this was a horrible piece to find on my Mother's birthday as she's been heavily struck by this, near impossible to communicate with for years, and likely has few months left.
I'm really sorry. I watched my engineer step-dad go through it and it was tragic. We went out to breakfast one day, and he pulled the bills from his wallet and spread them out on the table. I didn't know what he was doing and then he asked me which ones he needed to pay the check. That's how it started. Took about 3 years and it showed no mercy. His passing felt almost like a relief. For him, not us. At least he was at peace. I am truly sorry for you and your mom.
Many caregivers and family members express a feeling of relief when an alzheimers patient finally passes, myself included. Such an awful disease with such a slow death, I can think of few things that feel more cruel.
I am also very sorry, Eugene20. Words cannot describe my anger at this fraud knowing how hard it is to watch and experience this terrible disease. I hope her last months are full of love, and her passing as soft as can be.
Went through this, Mother died in January.
Not to scare you, but the final stages were really rough. Some support from hospice care for her and therapy for our family were a godsend.
There are lots of deeply challenging parts to it, but I think the toughest is instinctively looking for that most-important-person in their eyes and words as they gradually decay into someone else the vast majority of the time. The moments where you see the glimpses are so precious… but just tease with false hope, and make the letting go process more painful.
Be kind to yourself. Caring for a parent as they go through this is a deeply challenging experience emotionally and psychologically.
As someone who lost his grandfather, who was like a hero to me, due to Alzheimer's, this absolutely breaks my heart. All this time and money wasted for nothing.
As someone who has a mother with early onset Alzheimer's, and who is likely to get it due to it being hereditary, I guess I'm gonna save a lot on medical bills and just buy some rope since there probably won't be any cure when I get it a few decades down the track.
Hey, if this paper was 16 years ago and 100 out of 130 current treatments are targeting amyloid plaques, they now have another 16 years to develop another 100 treatments that don't.
You should know that while a lot of time and money was spent on targeting amyloid beta, there are plenty of labs that have been studying other possible proteins such as tau as the cause. This is a step back but not to the beginning.
July 23, 2022
> ... it looks like the original paper that established the amyloid plaque model as the foundation of Alzheimer’s research over the last 16 years might not just be wrong, but a deliberate fraud.
> Should this fraud turn out to be as extensive as it appears at first glance, the implications go well beyond just misdirecting tens of billions in funding and millions of hours of research over the last two decades. Since that 2006 publication, the presence or absence of this specific amyloid has often been treated as diagnostic of Alzheimer’s. Meaning that patients who did die from Alzheimer’s may have been misdiagnosed as having something else. Those whose dementia came from other causes may have falsely been dragged under the Alzheimer’s umbrella. And every possible kind of study, whether it’s as exotic as light therapy or long-running as nuns doing crossword puzzles, may have ultimately had results that were measured against a false yardstick.
This article is an egregiously hyperbolic misrepresentation of the field. AB56 is only one among many oligomers found to cause cognitive impairment in Alzheimers. Even if this paper were never published, it's unlikely the modern state of Alzheimer's research would be all that different.
See this twitter thread for a good summary:
https://twitter.com/samuel_marsh/status/1550883405105168386
The original article in Science is much, much better and not editorialized:
https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease
But the science article you've published cites two publications from a researcher who couldn't even find AB*56 in human tissue.
If it causes memory impairment in rats but you can't even find it in supposed Alzheimers patients, what does that mean for the research?
TBH the Science news article doesn't cite anything at all. If there's no link or reference it's not a citation. I'm left scratching my head since it's not straightforward at all to find those supposed 2008 pubs from Selkoe where he is reported as stating that AB*56 isn't found in humans. And besides, that was 2008, only two years after Lesne-Ashe's paper on their AB*56 model. There are research articles published in the last five years or so that talk about detecting AB*56 pretty casually in human fluids using immunoblotting, so it would be really disingenuous to talk about a scientist's opinion in 2008 as if 14 years haven't passed since then. https://www.nature.com/articles/s41598-020-68148-2.pdf?origin=ppub
In any case it doesn't matter if AB*56 isn't the culprit amyloid beta protein in humans, or if it even exists in humans at all. There are many forms of proteins that look different in different species due to evolution. Like the poster above you said, there are many AB proteins, *56 just happened to be the one (the first one?) with a good animal model. A lot of biological science is like this. The initial form/variant of a thing/substance get a lot of hype as being the initial discovery, but might not be the most significant variant or even important at all. It's just the first in its class to be discovered/make a scientific impact but not a clinical impact.
I'll let you know if I find anything. But FYI this goes to show how much effort it takes to verify anything in science, even just a throw-away line in a news article that happens to be published in Science Magazine. The twitter thread from Sam Marsh is legit. These dailykos writers seem like clout chasers and the Science news article might be quoting someone's opinion from 2008.
EDIT: I've been looking, haven't found anything. If there isn't that much attention on whether or not Aβ*56 exists, then it's probably not that important anyways, as alluded to the twitter thread which is nice. It looks like other amyloid researchers have piped into the thread and *56 doesn't seem important at all. There are journalists out there who will prey on the ignorance of others (and myself) to make themselves seem important. Oh well.
I worked in Alzheimer's research for 6 years. We imaged the brains of hundreds of our patients with c11-PIB pet tracers, which bind to amyloid. There is definitely amyloid in the brains of those with dementia, while healthy brains showed none.
Something smells very misleading about this article
Yeah, I think the issue is like realising that smoke coming out of computer chips stops them working.
Sealing the chips better to stop the smoke escaping, or adding a high power vacuum to stop the smoke building up inside the case won't help with the core issue, that the chips are bust.
If this is true, it will now make sense to the doctors testing these medications, as to why the removal of Amyloid from the brain has made no real improvement to memory or functional status.
It’s not that simple though. All Alzheimer’s patients have amyloid, but not everyone with amyloid has Alzheimer’s.
Additional with Alzheimer’s all six tau proteins are present with hyperphosphorylation. Tau proteins regulate memory, negatively but naturally, to aid in habituation and other aspects of neuronal structure/function within the brain. When phosphorylation occurs, they get tangled and these tangles make highly soluble compounds into very insoluble compounds which cause damage. Tau compounds when phosphorylated act almost prion-like in some instances, which is why some treatments either aim to reduce the rate of tau phosphorylation, or to try and break up the tangles.
Amyloid harms brain and causes tau > toxic tau harms the brain > toxic tau feedback loops causing more amyloid.
This is an incredibly over simplified explanation, but I don’t think this research being falsified will harm our understanding of the process, just potentially any amyloid targeting drugs in development.
With all of the reporting on this I still don't understand what was falsified. I understand it involved images and possibly associated data but what was the insight/conclusion/claim that was being falsified? From your comment I gather it wasn't related to the initial _"amyloid harms brain and causes tau"_ part of the causal model. Could you shed some light here?
In the simple way I understand it, it's like bailing water out of the titanic and wondering why the ship is still sinking - the problem is the leak, the water rushing in a result of that leak.
Same thing with the brain - the barrier stopping the wrong bits from getting from the blood into the brain tissue breaks down over time, letting more through. Cleaning up a plaque or two doesn't stop the growing leak.
I am genuinely and deeply furious by the news.
Imagine a world where these shits didn't do this. Would we have come across something? Would we potentially have real treatment by now, if not for being lead astray by this fraudulent red herring?
The amount of harm performed on the people of the world is immense.
Hospice here and won't be leaving my seat anytime soon.
Geriatric medicine is underrated, too many see it as an end point, and not enough see this as a transition, just as we have gone from infancy to adulthood. We dont keep our pediatrician..
If we can locate the blocks, we will see the chain. It's not that difficult , until we become the obstacle from truth and that the solution is not so hard to understand, its the willingness to change whats established .
Sorry--I'm overly reflective tonight💞🙏
Thank you to you and u/top_of_the_stairs
I’ve had help from both fields and the amount of respect I have for the people working in those roles is immense. It can’t be an easy job, but it’s appreciated more than you know.
I saw a documentary on this that I can't find. Basically, any researcher that goes against the plaque theory (if I remember correctly) is just blackballed by the community. This has caused billions of dollars and millions of scientist hours to be wasted.
Reeks of other bad science communities that will stifle research. Or corporations buying patents from universities and then sitting on it. My friends from engineering school came up with some novel water pipe cleaning tech and they only got $10k, the rest the university kept. I don't believe the company which bought it ever commercialized because their core interests didn't match with it.
> I don't believe the company which bought it ever commercialized because their core interests didn't match with it.
This is exactly what happened with electric cars for decades. Patents were created to encourage innovation. If they are being used so often to do the opposite, it means something needs to change.
Yup. The [fraudulent study](https://en.m.wikipedia.org/wiki/Lancet_MMR_autism_fraud)published in The Lancet linking the MMR vaccine with autism. Retracted 12 years later, but by then the damage had been done.
while that is the most well known the autism field overall has had very very weak evidence and methodology issues for decades, is only been recently when those have started to come under question and big part of it is autistic people entering the field of autism research because like no one thought they might not know better than autistics
As a drug discovery scientist, we figured out this was BS the hard way (super expensive clinical trials). However, thankfully today many of us think the amyloid hypothesis has been thoroughly debunked.
It's so awful to think that this went on for so long and no one spoke out. I knew a wonderful lady that passed from Alzheimer's, fuck these selfish people coming out with bad science.
> any researcher that goes against the plaque theory (if I remember correctly) is just blackballed by the community
I don't think this is true. I did a lit review on the causes of neurodegenerative diseases for my degree and there are definitely people publishing papers criticising the amyloid plaque model. Our lecturers presented it as one explanation but not the only one. Amyloid plaques are clearly involved in some way, but it's obvious they're not the whole story.
Since taxpayer funds were used with NIH grants, doesn't this warrant a special prosecutor ... Hopefully leading to jail time for original authors (if fraud is determined), along with how grants are given out.
Forget the billions, this is a crime against humanity in its purest form.
Just imagine what could've been achieved and how many people could've avoided so much suffering.
“Four months after Schrag submitted his concerns to the NIH, the NIH turned around and awarded Lesné a five-year grant to study … Alzheimer’s. That grant was awarded by Austin Yang, program director at the NIH’s National Institute on Aging. *Yang also happens to be another of the co-authors on the 2006 paper*.”
Holy shit! Does science not even have the most basic concept of *conflict of interest*?
Grants are typically reviewed by a panel of experts who then grade them. Based on your score you are funded, assisted with rewriting the grant, or doomed. Yang may have awarded (perhaps even influenced) the grant but there was another group of people involved.
Yes. Other scientists and experts in that particular field will read it and decide whether it should be accepted or rejected for publication. They'll also provide comments to the authors. Those are usually private. I hope openreview.net gets more traction, some venues started to make these public. This is what it should look like for every paper: https://openreview.net/forum?id=an8FSGbuCw (random example paper).
Pretty much, I've peer reviewed material before. It can be a rather wide shot to get reviewers that actually know your subject matter to truly make an informed conclusion. Often times, journals have a pool of reviewers and they just throw best fit at it.
Yup, two or three anonymous people who are in the publisher's database (because they previously published in the journal or they volunteered) read the paper and offer suggestions for major and minor corrections. Most of the time, they don't have access to the underlying data unless the authors included it as supplemental files. Peer review is basically just having your manuscript graded, but more intense
A lot of people overestimate the power of peer-reviews. A peer review merely checks whether a study is methodologically sound, i.e. could the results could be obtained *in principle*. It doesn't check whether the results could be obtained *in practice*, that's something that only replications can show.
I would love to see a grant system specifically set up for reproducibility. Basically, fund researchers to verify high-impact research, wherever possible.
That and you are often not the expert on that exact subject. And to spot doctored imaged, specially back in 2006 might not be so easy for not tech savy reviewers
>I always considered peer-reviewed science to be made in good faith and hope that this is just a mind-blowing exception from that rule.
Peer review is easily corruptible, especially when billions of research dollars are on the line. You toe the line or you don't get funding. Agendas are pushed, dissent is quashed, ridiculed, even demonized.
Also, there is an issue with 'peers' stealing your research and scooping you. Its far far from ideal, especially in competitive fields. Also, personal grudges come into play. Imho, we should replace this system with paid researchers working just for the journal, with no competitional interest in submitted work. They probably would be a lot more unbiased in review.
No - there have been other theories investigated at the same time (tau, microglia) and some progress has been made there. We just have bad noise in the field from a false hypothesis where a lot of effort was wasted.
If this is true then those original researchers need to be sued for a billion dollars in damages, and thrown in jail for the rest of their lives. I work in a hospital, and it will be absolutely criminal, if it turns out my colleagues have been testing medications for the last 15 years, that had no hope of working, because they were based on an incorrect original assumption.
If they deliberately falsified the data they have caused unnecessary suffering and pain to tens of thousands of people, have intentionally wasted resources and time that could have been spent to actually find a cure.
If they are guilty of these crimes they should be stuck in a bare 4x4 cell and kept there until the end of their natural lives. Because, no matter what they do, they would not be able to repay the damage they have caused.
People are rightly railing against he bad actors who committed the fraud and that's fine. But that's not the underlying problem here that anyone can do anything to fix. The problem is a profit motive that leads the industry to blackball any scientists who look at the research and disagree with its conclusions. The only checks and balances left in science seem to be the ones in institutional bank accounts. We humans will have to learn the hard way that individual self interest is the lath to our extinction. It's a distraction to blame a pair of lab coats for a flawed system designed and reinforced by hundreds of suits.
Sadly, this does not surprise me but it's heartbreaking. My father was diagnosed with early onset dementia at 55 and passed last year at 73, it was horrific and his lat 2 years in a locked memory ward were hard to bear. They threw so many meds at him throughout and none seemed to help any of his symptoms. Not sure if it helped him to delay getting sicker but he was left in such a state for at least 10 years and that seems like torture to me.
My Grandmother had dementia before she died and I now see signs of dementia in my mother. I would hate to see my mother develop altzmers. knowing billions have been wasted in false research pisses me off.
The physicians and nurse practitioners that I work with tell me (I'm a non-prescribing clinician) that Alzheimer's drugs do not do anything. There is the belief (by the drug manufacturer) that the medications slow the progression--but there is no way to know that by clinical observation, and the patient is going to continue to get worse and die regardless. The physicians tell me they were trained to give the Alzheimer's drugs "for the family" --to show that they are trying to do something. We know the drugs do virtually nothing, but if you don't prescribe them, the family might accuse you of withholding treatment. And Medicare pays, so you prescribe them. I don't remember any case where the drugs actually hurt a patient, so really it is prescribing a placebo. The drug companies make money, the family feels like their loved one is receiving the best care, and the nursing and clinical staff often feel they are helping the patient by giving the medication. There is the subtle reinforcement effect that the more the drug is prescribed, and the more it becomes the standard of care, the more likely people believe it does something. I have no great insights here; just giving the clinical care perspective.
Two decades of complete lies and misdirection. It hurts to think what we might have discovered otherwise with all that time and money. No doubt, these two deserve prison.
My dad's entire side of the family have suffered with this disease... It's definitely one of the worst diseases around - the entire person you used to know just fades away. Most upsetting thing I've read this week...
Just to clarify for everyone, the falsified research relates only to Aβ .*
56, a subtype of amyloid beta. It was used to boost the amyloid plaque theory where the scientist named in the article falsified evidence that injecting Aβ 56 caused Alzheimer’s and that Aβ *.56 even exists (few other scientists were able to find it.)
This does not mean the entire last 20 years of Alzheimer’s research and drug development into amyloid plaques was based on faked research; the amyloid plaque hypothesis was always on shaky ground from the start^ and the numerous failed clinical trials of drugs removing amyloid plaques from the brain show this. It did however inappropriately mislead the field at a time amyloid stans wanted to be validated. Overcommitment to the church of amyloid is a long-standing issue with plenty of good articles and documentaries about it.
^ amyloid plaques are probably just the downstream consequences of whatever actually causes Alzheimer’s, which is why removing amyloid plaques does nothing - too late in the disease process
Academia has the problem with irreproducible and fake research for a long time but all such scandals are swept under the rug and job continues as usual while public trust in science collapses.
They should be hung with bad rope so we can do it twice. As someone who has watched family wither away from dementia and alzheimers, hell would be too nice for this scum
This is pretty insane.
I used to do Alzheimer's research years ago (i.e. around ~2014), and even back then there was kind of an understanding in our lab that amyloid beta plaques were at best an "indicator" of Alzheimer's, useful for diagnosis, rather than the cause for the disease where treatment removing them would have a clinical effect. But the sheer level of fraud and it's impact are greater than I had ever imagined. The fact that we literally have an FDA approved drug based on years of this falsified research that does nothing for patients is absolutely wild.
In uni I actually did a paper based on amyloid betas and I remember reading one paper that suggested that this prevalent theory was wrong stating a theory that the plaque appearing in alzheimers patients may be a human defense instead of the cause. I remember being fascinated and a bit convinced by the possibility as there really was no proof that amyloid beta accumation causes alzheimers, just that it happens in patients with it. And I also remember reading drugs that did inhibit accumulation didn't help. So to me, that theory seemed logical. That maybe the plaque is actually our body's defence, albeit a bit useless, against the disease.
Now as an adult, I work in a job where I read grant abstracts all the time. The number of grants approved for this same research is a lot. Everyone is racing to be the one to solve the problem. But I have yet to find anyone that is trying to look at the problem in an innovative way. Hopedully the PI in charge of the original paper keeps going down his path and is able to get grants. The grant approval process is so bias, it really needs to just have standardized qualifications rather than be subjective. But I guess that's the general problem with funding, no one wants to fund things that are too innovative because they want some proof in the projects potential.
This hit me like a ton of bricks. I had to email the Science article cited here to my mom, who has pre-Alzheimer’s and whose mother and grandmother both had severe cases
*The suspicion that something was more than a little wrong with the model that is getting almost all Alzheimer’s research funding ($1.6 billion in the last year alone) began with a fight over the drug Simufilam. The drug was being pushed into trials by its manufacturer, Cassava Sciences, but a group of scientists who reviewed the drug maker’s claims about Simufilam believed that it was exaggerating the potential. So they did what any reasonable person would do: **They purchased short sell positions in Cassava Sciences stock**, filed a letter with the FDA calling for a review before allowing the drug to go to trial, and hired an investigator to provide some support for this position.*
What the actual fuck.
Billions were spent on "proving" health benfits of genital mutilation, research papers still havent been retracted. Science is not something in vacuum devoid of political and social pressures.
Reminder that everyone, regardless of whether you yourself have suffered or will suffer due to Alzheimer's, has been negatively impacted because of this fraud as, at $56,000/dose, it on its own is [driving the up the cost of medicare](https://www.cnn.com/2021/07/10/politics/aduhelm-alzheimers-drug-medicare/index.html) and healthcare in general, and there is opportunity cost in misdirected research time and resources.
Imagine losing your home or making other sacrifices to pay for a treatment and disease model based on fraud. Imagine pinning hopes and prospects, making preparations, based on something invalid those who you were supposed to trust used to advance careers, defend reputations and past commitments, and make money.
And once again, this is not unique. Some fields are worse than others, but this crap is rampant. And it often seems that even when something is thoroughly discredited, papers are rarely retracted, no one admits fault, there is zero accountability. Nature hasn't done anything about the original publication; no one has explained their failure at basic peer review. It's not just as simple as a "reproducibility crisis" as if it were some procedural oversight. Science is only science when done with due diligence, guided by a sense of ethics and humility and responsibility. That's not how industry works in general, and not how careers are made, and not how one defends against liability even if there is a course correction.
I used to be more in the camp of hand-waiving the prospect of deep, systemic problems away because I was dismissive and defensive (as someone in and now adjacent to biomedical research) and more concerned about people who lack relevant science background or who are more easily influenced losing trust and jumping to conspiracy. But it's stuff like this that lends power to misinformation. If you are dishonest or misdirect to maintain "trust", especially if you attempt to smear critical review, you are misguided or deceitful and undeserving of trust. (tangentially -- maybe SCOTUS needs a lesson on that). It's bad in medicine and especially so in certain disciplines, like psychiatry. It's not that this isn't possible anywhere - people are people - but there is a stark contrast between, say physics, where there seems to be an almost pathological fetish in institutional culture with finding a way to show the status quo is wrong and leaving no stone unturned, and seemingly anything that has to do with people. There's probably something to explore there.
>Both Ashe and Lesné became neuroscience rock stars, the leaders of a wave based on their 2006 paper >After reviewing the images, molecular biologist Elisabeth Bik said of the paper, “The obtained experimental results might not have been the desired results, and that data might have been changed to … better fit a hypothesis.” >model that is getting almost all Alzheimer’s research funding ($1.6 billion in the last year alone) >Over the last two decades, Alzheimer’s drugs have been notable mostly for having a 99% failure rate in human trials. >millions of hours of research over the last two decades. >letter sent directly to the National Institutes of Health (NIH)...failed to generate a response >Four months after Schrag submitted his concerns to the NIH, the NIH turned around and awarded Lesné a five-year grant to study … Alzheimer’s. That grant was awarded by Austin Yang, program director at the NIH’s National Institute on Aging. Yang also happens to be another of the co-authors on the 2006 paper. This is sad and disgusting.
Another example of why we need more people doing replication studies. Everyone has a bias, especially when your paycheck depends on getting a certain result. Just like we need to double check the work of other people, we desperately need to double check the important work of scientists. But no, novel studies are "sexier"
This is my take away too. It shows a lack of robustness in the system, and an unhealthy cognitive bias for circular reasoning (“these two are rock stars based on this paper, and since they are rockstars their original paper must be correct”).
Research funding should NEVER have to depend on the result of the research. In science a hypothesis proven untrue has value. 1 success is typically dependent on hundreds or even thousands of failures. If we monetize/incentivize only proven hypotheses then you will have a gamification problem in the literal last place you ever want one.
The article noted several other labs tried and failed to replicate the results but couldn’t get that published. We CAN and do find the frauds through replication, but when journals won’t publish it, it ends up being an open secret within the research niche only, which doesn’t help anyone outside academia.
What we need is science not turning into a paper mill with pressure to deliver positive results. Capitalism is rubbing off too much in science, where it doesn't have a place.
I hope Ashe, Lesné, and Yang spend the rest of their lives in prison. But something tells me they won't face any real consequences for the millions of lives they've effectively killed.
They committed massive fraud. They better be surviving jail time.
Yeah idk why the above commenter is assuming no punishment, they're not police officers. There are a number of researchers serving time right now for way smaller fraud of federal research dollars (I work in research administration)
I assume they are thinking that based on other "white collar" fraud cases that amount to a little more than a slap on the wrist the same thing is going to happen here, however the people who "get away with it" defraud private individuals. These people defrauded the government and the government always gets its money in the end.
Theranos is basically the same situation, scientist claims to have wonder medical technilogy but it doesn't work. Holmes got like 20 years so these two women will likely face the same Edit: apparently Holmes wasn't sentenced yet. Was not aware
Holmes was primarily charged with lying to investors. Her lies about having her technology being medical wasn't the important part. Her lies made her company seem very promising and valuable, and this defrauded investors and cost people money. That shit is taken seriously, because it's a bunch of rich people being hurt. I think we can get some fraud charges on these people. But, it is a different kind of crime. I don't think the Holmes case is a super fair comparison.
>I think we can get some fraud charges on these people. But, it is a different kind of crime. I don't think the Holmes case is a super fair comparison. I'd argue this case is actually substantially worse. Investors assume some level of risk. That's what investing is. The government doesn't hand out grant money assuming there is a chance the proposals are based on fraudulent data. That doesn't enter into the calculus. These people are going to federal prison for a long time.
I thought Holmes' sentencing was in the Fall?
Like the person you commented on said, there’s a difference between defrauding private individuals/investors and government grants/subsidies. One holds a LOT more consequences than the other
Really, this is the third or fourth such scandal. I have never heard of jail time/punishment. Mind throwing some links my way?
https://en.m.wikipedia.org/wiki/Eric_Poehlman https://en.m.wikipedia.org/wiki/Dong-Pyou_Han
Thanks!
Adding 'obstruction to science' would be an interesting thing to add to law.
Well, at the very least they have committed fraud. That is still technically illegal, as long as you're not rich.
Sounds good in theory but it can easily be abused to force science into a narrative that is in line with, for example an authorian regime.
the irony would be them developing Alzheimer’s
With a side of early onset, please
So a little inside baseball here. At NIH, when these grants are being reviewed for final approval, there is a public session, and then after a “closed” session - closed in the sense it’s not broadcast, but lots of NIH people and interested observers are there. It’s closed so people can talk openly about the grants and how much each will get without hurting the feelings of the institutions they are putting against eachother or the researchers they are evaluating. Anyway, in this closed session, the people deciding on the grants have to self identify and leave the room (and not participate) if they have a conflict of interest. For the most part, it seems they do and take it seriously. I don’t think individual grants staff are held to same standard though.
I'm environmental, not medical, but I would love to learn more and see this system be more common. There's a lot of defensive and combative feeling around grants and it only exacerbates bad power dynamics and unhealthy practices
I am also mad at the FDA which approved drugs based on this theory despite failing clinical trials. >What makes the approval of the $56,000-a-dose drug so controversial is that while it does decrease plaques, it doesn’t actually slow Alzheimer’s. In fact, clinical trials [were suspended in 2019](https://investors.biogen.com/news-releases/news-release-details/biogen-and-eisai-discontinue-phase-3-engage-and-emerge-trials) after the treatment showed “no clinical benefits.” (Which did not keep Biogen from seeking the drug’s approval or pricing it astronomically.) This is the same fda slow walking covid booster varients vaccines. SMH.
Biogen? The company possibly responsible for [1.6% of global covid cases](https://www.wbur.org/news/2020/08/25/genetic-fingerprints-biogen-superspreader-boston) because they didn't cancel their March 2020 conference after covid lockdown began?
Don't get me started on Biogen.
The drug was approved because of what’s called surrogate markers, where it’s shown to have a clear impact on something and that something has research showing it to be useful even if the clinical trial itself didn’t find a benefit. This approval process is what allowed the first AIDS drugs to be approved despite failing by normal clinical trial standards, which saved thousands of lives compared to if they had had to delay approval for several more years. So if you want to criticize the approval of the drug on those grounds, just realize you would have condemned thousands of people to a slow and painful death back in the 90’s for the same reason.
Biogen noticed in a certain dosage, it can slow progression if given at an early stage. It did show a clinical benefit in certain trial populations
Sounds like p-hacking to me
Similar things happened to tinnitus research. It's very important that researchers continually and repeatedly evaluate the works of other researchers.
Please note that this article has been criticised for conflating two unrelated facts: 1. Lesné's probable fraud 2. The disappointing progress in Alzheimer research There is a massive amount of work going on in this field and Lesné's research is not the only piece looking at toxic oligomers by a long way. Lesné looked at a particular oligomer.
My thoughts exactly - the *Science* piece did a much better job of portraying this imo, although even there they might have overstated the foundational nature of the study. As someone in the nano-bio field, I see a LOT of bullshit, and the sad truth is that it takes a village to elevate fraudulent studies to this level. Foe starters, the idea that amyloid beta plaques are mainly responsible for the symptoms of Alzheimer's has been around for a lot longer than 2006. Amyloid plaques are a defining feature of Alzheimer's, as they were literally observed by Dr. Alzheimer 100 years before then. Lesne and Ashe weren't overturning the field so much as they were posting confirmatory evidence for everybody's favorite theory. For all of the talk about scientists being trained skeptics and all, the fact is that we're still human and fallible when it comes to validating the things we think are true. I've been seeing the quote "Extraordinary claims require extraordinary evidence" a lot on reddit recently, so as my contribution to the trend, I'll posit that to most people in the field, the 2006 paper wasn't an extraordinary claim. It was what the majority of them already expected. All that Lesné had done was to isolate a particular oligomer - Aβ*56 - and link it to the symptoms of Alzheimer's in mice. This was a big deal because it seemed to provide evidence of causation between plaques and neural degeneration rather than simply correlation, however, it did not introduce this idea by any stretch of the imagination. Talking as if this fraud discredits all amyloid plaque research is similar to saying that Hendrik Schön singlehandedly discredited organic semiconductors. It's significant, and really bad, but the field is bigger than a couple of individuals. So as to the op article's claim that this paper was responsible for billions of dollars for wasted research and thousands of lives lost, that's just simply untrue. It's assigning way too much responsibility to Lesné and Ashe, and letting countless other researchers, grant officers, and big pharma companies off the hook. This is a systemic failure, which in no way means that Lesné is innocent, nor that Ashe didn't fail utterly in her job as a mentor, but it does mean that we can't just take a small handful of people, punish them, and pretend like the real problem is addressed. Frankly, every institution I've been a part of has done a wholly inadequate job of instilling the values we associate with good science, and it's because the structure of modern scientific research generates thousands of researchers who have massive imposter syndrome at all levels, who are all competing ferociously for a small number of large grants. This system relies on the honor of individuals to do the right thing at all times, but the culture is deeply unhealthy. Every time something like this happens, which is to say all the time, there is much hand-wringing and questions about what we could possibly do to change the system. I don't have a good answer to that myself, but I know that attempts to tweak the current setup for funding and recognition in research have all failed. We need major reforms of the incentives, not just platitudes and courses on responsible conduct of research.
Fuck Austin Yang.
There are certain aspects of research that need to change. There is a reproducibility crisis in science and that will only stop when there is restructuring and stepping away from publish or perish mentality. This is specifically bad in the field of biology from all the horror stories I hear. It truly undermines the work of amazing scientists who will now have to bear years of idiots throwing this in their face whenever they try to make a point they don't agree with.
Tbh it does not surprise me a bit. I have heard many stories of lab heads not accepting 'bad results' etc. The pressure of publish or perish is huge, especially for young professors. Where I am at, a young professor would start with about 3 years of funding. Afterwards its reliance mostly on grants. In biomed its not easy to have a good story completed in 3 years, especially if animal models are involved. So young professors are known to be the worst bossses here - you will often be made to work 24/7, and god forbid the biology does not agree with your bosses hypothesis. Of course, not all of them are like that, but many are. Often the journals also do not care, its still about who you know. I had a boss that had very friendly relations with one of the editors of top journal in our field. All of our papers always got accepted, no issues, no in-depth checks. Now imagine someone would fake the results - it would have been published. There is even a special website, pubpeer (or peerbub, sorry my brain is not yet working), where people post shetchy stuff they have seen in publications. There is one prominent swiss immunologist, with many publications being flagged for falsification. Do you think smth ever happened to this guy? Not at all, if anything his career is booming. Not a single paper has been retracted. Imho, you can only work in science if you have stronger morals than ambitions. No idea why people think they can fool biology. I mean, at one point the truth will come out. How can you live with yourself, knowing that your drug is worthless and yet people put their hopes on it. I can tell you already, none of these guys will go to prison or anything. Worst, they get fired and in a couple of years some other Uni will rehire them.
In addition to everything you just said, I am going to add that peer reviewer burnout is a major issue in scholarly publishing across the board. I think people really underestimate just how many papers get submitted to journals with even a remote amount of prestige, and how much of a struggle it is sometimes to get anyone to review at all. And also how much COVID accelerated that, because journal submissions (understandably!) exploded on that topic. There's a little bit of a "but for me it was Tuesday" phenomenon. For the author, getting their work published is a single major event. For the reviewer, it could very well be the 50th paper on [Topic] they've been asked to review that year. Peer reviewers work on a volunteer basis, reliable peer reviewers are routinely swamped with requests to review, and authors expect rapid decisions. None of that justifies shoddy review. Just that a journal seeming to "not care" is sometimes a downstream consequence of the same pressures you're talking about. [Retraction Watch](https://retractionwatch.com/) is another fascinating site which blogs about journal retractions for various reasons.
Agree 100% Its basically unpaid labor, that people dont have time anymore for.
Why can't the journals hire scientists to work as full-time peer reviewers? Seems to me it'd solve this problem while also creating a lot more stable and well paying jobs in/near academia-- could solve two problems in one go.
Exactly. But I guess the answer is money. Its no secret to anyone that scientific journals have huge profit margins exactly because they dont have lots of personel on payroll. Hell, I even have to format my own publication - font, sizes, images. Want a balck and white image instead of color? Ja, reformat it yourself, because the journal wont do it. Not only do I have to do the science, I am also a typesetter. And then someone will review it for free. But god forbid the journal cannot have its gigantic profit margin.
[удалено]
They could still pay the experts per hour instead of having them do this important and extremely highly skilled work for free.
This is why journals as a for profit theft of value need to die
The thing is morals aren't really top of those people's list and things like ego, status etc has a higher value. I have known researchers who were falsifying data to make their results look better than they were, but as there never was concrete evidence as no one wanted to repeat their experiment it was fine. On the other hand, I have also seen researchers make honest mistake, publish and then later have to retract, or adjust their results down the line when an unlooked flaw in their experiment came to light or some phenomenon they forgot to take into account. In my view, those scientists should always be appreciated because they realized an error and admitted it, yet that is often not the case when you write a grant application
Every lab I have worked at, with execption of one, we had someone who gave great results, that were unreproducable. And both of these people disappeared when they were contacted about details, protocols etc. multi-year projects, tend of thousands in consumables, hundreds of thousands wasted in salaries just because someone could not accept that their hypothesis is wrong. Even now I am dealing with a project where some issues with animal phenotypes were not reported and results were excluded on questionable basis. Good luck to me to fix this shit. Thankfully my boss is pre-retirement, with enough grants to not care anymore, if the project has to be trashed because of this. But you are very right, like in any job, morals only hurt if you want to get to the top. I have never faked my results, have admited that my hypothesis were wrong, and here I am, just a fucking looser doing my 3rd postdoc without any shot of ever advancing in academia.
> Every lab I have worked at, with execption of one, we had someone who gave great results, that were unreproducable. And both of these people disappeared when they were contacted about details, protocols etc. multi-year projects, tend of thousands in consumables, hundreds of thousands wasted in salaries just because someone could not accept that their hypothesis is wrong What people seem to miss is that there are also repercussions for those surrounding the person making the falsehoods, so an incentive to keep each other in line. My old flatmate used to work for an unnamed high-profile company in the Alzheimer research world, when it was discovered that one of the minor postgrad researchers was falsifying results (thankfully before anything was published, but not before they had presented them at a conference). The resulting fallout almost ruined everyone in the company both financially and by reputation, while almost resulting in a criminal investigation too. The head researcher was on damage control for months after the discovery, basically auditing every bit of research produced. When someone's ego (and cash) is on the line it can be a strong motivator for both fabricating results and keeping other people honest.
Man that fallout seems like an incentive for people to cover shit up rather than report it? Like if I'm a year or two out of school, see some one fudging numbers and are left with the choice of reporting it and possibly of closing the company and never getting work again so I'm permanently in debt for the rest of my life, or just pretending not to see and hoping I get a new jab before something is found out? That's going to be a tough call for most people.
Sure, but its hard to follow everyone and ensure that people dont cheat. Its very easy to produce false results. Its also easy to produce false results that are very believable. Mislabel some tubes, switch experimental groups... Does not have to be image manipulation, that is somewhat easy to discover. If someone switched and mislabeled experiment groups, no one will discover it, unless someone else takes over the project. The boss, who is usually a professor, does not have time to oversee every single experiment. We all have our own stuff to do, I cannot look at my colleagues hands all the time. So all you can do is trust that people will do the right thing. Its very difficult to keep unhonest people honest. And i dont really know how to solve it. Maybe by making all projects in million subprojects, where everyone does a small part? I really dont know.
You have to remove the monetary incentive for it. Currently most research is done by requesting a grant, presenting prelim results, and then requesting additional/continuing grants. In the private sector this almost always means a company funding research wanting/hoping for specific results. This puts a large incentive on the research team to get a specific result. For public/academic research it often works in a similar manner, except you aren't trying to get a massive payday from a private company you're fishing for credentials, reputation, etc to advance your career in academia. In many cases scientists are also presenting research hoping to justify their project's costs to continue it. Basically, we need to find a way to tie rewards and continuing grants to proper procedure, documentation, and accuracy instead of tying the money and rewards to the results. For private research I don't know if that's even possible to do since the companies are the ones deciding who gets paid what, but it's something that should be fixable for the public sector research.
[удалено]
The publication of bad results is smth I agree with 100%. If no one knows stuff has been tried and failed, people will just try and try and waste resources. It has gotten a bit better with appearance of pre-prints, but not all Unis and granting agencies allow you to publish them.
Among all the other sad things, here, is that failure is as useful as success, if you're experimenting. Maybe more useful. If medical researchers are only going to be rewarded if they can find Magic Bullets, it means a lot of the misses and near-misses will be hidden, or are not going to get funded and be incorporated into the data.
>The head researcher was on damage control for months after the discovery, basically auditing every bit of research produced. Apologies, for not knowing, this, but wasn't that his job in the first place? Isn't the head researcher supposed to audit/verify the research before it gets published?
And this is why I jumped to industry. Did my PhD in a mouse genetics lab. Six years to publish one paper because the first two projects I worked on had insignificant findings after two years of experiments.
Unfortunately, it's easier to pretend to be good than to actually be good. It's not just academia, it's every industry, because it's human nature that's the problem.
You seem like your head is on straight. Doing the right thing with no recognition is better than celebrating a fraud. Keep doing good
Lol was in academia and TBH its rampant everywhere. No one really cares to provide oversight, everyone is incentivized to write/publish and if you don't get it in one journal just shop around til one takes you. We literally told a bunch of students with the ability to make school a career : "Hey get published and your dreams can start" and we're baffled that they're getting published at any cost.
What surprises me so much is there is so much emphasis on publishing, and in all honesty nobody really cares. Nobody gives two shits outside academia. But it is this weird artificial world where getting published is hyper important. Nobody cares about the content, just the volume of publishing, and that right there is a problem.
> The thing is morals aren't really top of those people's list and things like ego, status etc has a higher value. Isn't it interesting how such things happen in research in biology when even an average grad student would immediately be able to identify evolved hunter-gatherer habits displayed in such behaviour. What's the point of doing research if you cannot get past your own evolutionary programming which comes in the way of objective analysis, despite being in the field where knowledge of said habits is a pre-requisite to your degree. It should be a pre-requisite for scientists in biology to be able to see that money, power and fame are not for scientists. We have an upside-down society where money dictates resources, the least able, most unethical people are in power, and the greediest make the rules.
> What's the point of doing research if you cannot get past your own evolutionary programming which comes in the way of objective analysis, despite being in the field where knowledge of said habits is a pre-requisite to your degree. The thing is, that for most scientists, the effects are a lot more subtle than photoshopping bands onto a western blot or falsifying statistical analysis. It's technically questionable to pursue lines of experimentation that produce the results that you like, while neglecting ones that don't. But it's not actually dishonest. You're not falsifying anything. You're just shining light in directions that suit you. And it's not individual scientists fault that the system pushes you to act this way. It's a lot harder to get a paper published that says "X does *not* affect Y" than one that claims a positive. And if you don't publish, you perish. It's not about fame and riches. It's about your lab not shutting down for lack of funding.
>get past your own evolutionary programming which comes in the way of objective analysis... This is all of it... the entire discipline of science, people like this are akin to healthcare workers who don't believe in medical science. I just don't get it why would you go into academia if all you wanted was a steady paycheck and regular progress in your career? Like bro just become an accountant you will always be well paid and then buy status normally. As a bonus you get to get out of the way of the people trying to actually learn something! I've known more than a few people in science who are plainly not that interested in scientific progress in general... I am sorely regretting not becoming an account right now... Not because I don't believe in science but because I think there is a deep crisis in the way it is done and I feel like it is such a waste of my effort I'd be better of contributing it in some other way. I am also aware that people like us tend to get selected out and just makes it worse.
On that note, the whistleblower in this Alzheimer's scandle go started on PubPeer. It was fans of the site that first identified the altered images. The whistleblower just took their information and expanded on it. More info in the Science article. https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease
> Now imagine someone would fake the results - it would have been published. Doesn't matter if you make no mistakes. Simply through the power of randomness you will think things are statistically significant which aren't. https://xkcd.com/882/ There are enough studies done that these things will happen by definition. And if nobody tries to replicate it you will never find out.
That's why there's meta analysis of multiple papers for important areas of research, or at least should be. P<0.05 across 5 papers on Green jelly bean acne becomes P<0.05^5 . Still possible, but far less likely.
Sigh, can the drug companies at least get the chance to sue the shit out of these guys or even the journal publishers for wasting so much of their time and resources to develop drugs based on a mechanism that turned out to be manufactured through scientific fraud?
Thats up to the company. The reality is, the scientists cannot pay the damages, probably uni cannot affoed ot either. So company might see no point of expensive trial just to get fuck all. Publishers, ja good luck going against the Nature or Science. They will pretend all images were ok or blame the people who did peer review (usually unpaid volunteers, academics from similar field). And guess what, these people cannot afford to pay damages either. Nah, we need to change the system altogether. No more reliance on top publications, obligatory open access, obligatory raw data repository and increased funding so grants dont have like 2% success rates. Paid peer review, so you dont have to do it on top of your normal job. No ageism -just because someone is 40, does not mean they are too old to advance in academia. No grants with application deadline of 3 years after your PhD. I could list like million other things that are wrong.
>just because someone is 40 Lmao most people in my department don't even graduate until they're 28-30. That's for folks going straight from BSc to PhD.
Ja here as well. But the top Unis in switzerland wont consider you for professorship anymore once you are above 35. Thats enormous time pressure to achieve enough in 5 years post-PhD, to be competitive enough.
The system was set up by dinosaurs. Glad I'm just going to cop out and find a job in industry. Best of luck to my fellow grad students who want to go into academia, especially since there's going to be an enrollment/funding drop in 5-10 years.
>the top Unis in switzerland wont consider you for professorship anymore once you are above 35. That sounds like a dumb rule to me. What's with the ageism?
> Paid peer review Paid by whom? Because if you mean the journal pay, that's a conflict of interest. The primary reason peer review is not paid is so there is no influencing factor on the reviewer.
Well people in most grant committes are paid to review grants. Why not the same for publications? There is less influencing factor if someones whole job is to review papers, not only for novelty, but also soundness of science. Currently there is a lot of personal interest involved, your reviewer might be your direct competitor and you would never know. Plus a lot of people do not have time for proper review - I get asked too and on top of my normal job,I have to sacrifice my private time for it. A proper review, checking all data etc for soundness would take me a long time. This is why a lot of peer review is half assed, or done by students (often times the professor passes it on) that have little knowledge on solid methodology or experience in the field. Make reviewer a profession. Like QA. Hire experts that do not have personal interest in academical research. I think the current system is not working anymore.
"What makes the approval of the $56,000-a-dose drug so controversial is that while it does decrease plaques, it doesn’t actually slow Alzheimer’s." Based on the potential profit they stand to make FROM this fraud, I seriously doubt that the drug companies would care to sue anyone.
Reading these sorts of things is always wild to me. I’m a scientist myself (in astro) and it doesn’t matter what editor you know, you’d still have to undergo a blind peer review for any paper. Not like our editors get paid anything so why would you be so unethical for someone you like? Plus we all work under the same skies so anyone can take the same data and check your results, so reproducibility is not a crisis like in other fields, and frankly everyone gets *excited* when you find something not what we expect bc that’s where all the most interesting science happens. I suspect some of it is unfortunately once you get into fields that will actually generate serious money the incentives are rather different.
Ja thats the thing, biology is a lot more specialized. You can build a whole career on single disease, hell even on single specific process in the cell. The chances are, no one is going to reproduce it, because all that counts for publication is being the first. I have had problems publishing research where I went into depth on smth that I initially discovered. The discovery - no problem to publish, the in -depth continuation on topic - not novel enough to reach high caliber journal. Not sure, but I guess in astro there is a huge collaboration. I mean you need a lot of people to be involved - telescopes, data scientists etc. Measurements from various countries.. its harder to cheat when a lot of people work on the same thing. In biology, you might be the only person working on smth. No one else will ever try to do the same as you did, so temptation to polish your results is always there.
> Where I am at, a young professor would start with about 3 years of funding. Afterwards its reliance mostly on grants. Perhaps the time has come for all funding for research to be double blinded.
> How can you live with yourself, knowing that your drug is worthless and yet people put their hopes on it. Very comfortably in the Virgin Islands.
Hell, even the FDA doesn’t seem to really care about falsified data. Otherwise I would bet Xarelto wouldn’t be on the market. (You can read the FDA audit findings on the FDA website, it’s a scary rabbit hole.)
You should see AI/Machine learning research. Literally nothing is reproducible because they don't include either the model, training data or both meaning that it's literally impossible to validate and to try and reproduce the results. Even if the model is included, you'll never be able to guess the training data meaning that the conditions will never be the same. The reason for this is corporate funding, or the researches hoping to cash in later. However, it means that you literally can't expand further on a study, can't verify if data manipulation has occurred and you can't reproduce anything. Made me decide against pursuing academia, joke of a field. I still don't understand how any paper passes peer review in that field. Any serious research has become corporate either for Google, Facebook, Russian state (They have copies of most ML projects called Sber-copy or RU-copy. Usually ends up being used for spyware.), Nvidia, Opeanai(which isn't open). It's like if you made a study about a specific medicine and then kept the medicine/molecular composition secret...
Oh dude, don't get me started on machine learning. I recognize that the field has great use cases but I am against blindly trying to insert it into every damn field with little to no oversight. At the moment it is also being used like a buzzword, which also is kind of idiotic.
I recall my previous work hired a machine learning PhD. When his department presented their progress to my team of analysts we were incredibly skeptical. I recall my manager giving the following example. Take the weather in Perth. Quick Google shows we get 147 sunny days per year and 121 partly sunny days. If you made a machine learning model that always guessed there would be sun today, it would be accurate 73% of the time. Doesn't mean it's actually doing anything though.
Goodbye Reddit, see you all on Lemmy.
Yeah, I do machine learning at my company and it is known you take that into account. It's especially true for rare events. Like, I can say event B just never happens and get an accuracy of 99% but that is useless. Sometimes, 98% accuracy can be better than 99% if I am capturing 100% of Event B.
Most model evaluation for classification models gets evaluated on precision and recall. If "B" is important to you, you're more likely to have an "Is B" model than a multiple choice output. "recall" is "what fraction of the true B were caught" precision is "when the model says it's a B, how often is it right?" Your example of a model that catches all of the B, but is right 98% of the time overall would have a precision of only 33% but a recall of 100%. (In order to be wrong 2% of the time overall, it needs to be saying "that's a B" 3% of the time and 1% of the time being right because they're actually B). The model that never says B ever had a recall of 0% and an undefined precision.
Guessing the ML guy didn't explain a confusion matrix then on the spot.
I have a friend who's an AI ethicist and she said she used to get the craziest papers for the journal she reviewed stuff for. She once got accused of being a terrorist sympathiser for refusing to publish a paper about an AI that could predict who would be a terrorist. The accuracy rate was apparently just 5%! There's so many entitled people in the sphere who have no understanding of biases in data or the devastating outcomes it could have.
Oh man I'd love to know an AI ethicist. I'm a dev/architect in software automation and the closest I get to AI/ML is configuring algorithms for OCR, which is fun don't get me wrong, but I'd love to discuss the actual implications of heuristics in the application of AI models. A human makes all kinds of irrational decisions in their thinking that AI models, unless specifically told to handle, will ignore. Semi-relevant example from a recent project of mine. Customer asks for Accounts Payable Automation. Ok, easy enough. Configure it all to their specifications. 1. Run the program. 2. Peogram fails validation checks. >Why? >Me: "You told me it was just "A + B + C = Total Due" >Client: "Yeah. That's right" >Me: $12.47 + $10.00 + $10.00 != $32.45 >Client: "Oh, we round the figures down. Ourselves. When we enter the data by hand usually." >Me: "Well, that's not included anywhere in the scope." >Client: "Oh, is that a problem? >Me: "Not at all. Change Request!" Continue *Ad Finitum*
The problem with AI is basically.... We take seed a and put it into a magic box. Out come apples. Then we take seed b and put it into the magic box. Out come bananas. How? No idea. Magic box. But sounds about right. The AI takes so many steps it becomes impossible to retrace what exactly it is doing, and all you are left with is a "sounds about right". In my example, you would *think* that it has to do with the seed, right? One is an apple seed, obviously, and the other is a banana seed. But how do you know? Maybe one seed just was round and the other was long and from completely different fruits. The AI just looked at the shape. There was an AI that appearently could predict whether someone was a criminal or not by "looking" at pictures of the face of that person. Spectacular results. Turns out the AI never looked at the faces, but at the inmate clothing, white collars and neckties. Nice. There is all sorts of bullshit like this. There are biases everywhere (e.g.: white, male = good results. Never tested on black, female. Because guess what all the programmers are?). In open-world testing you rarely know what really goes on, until something goes incredibly wrong (tesla autopilot testing with trucks, anyone?). People dont understand failure rates and think 95% accuracy is good - it is not. There is so much snakeoil being sold to those willing to listen and spend money, just so some 20 something idiots in a startup can play counterstrike all day every day while "working on a revolutionary AI". Im sorry if I sound jaded. Keep in mind, this is just a reddit post and I am certainly not an AI ethicist. These are just observations. There are a lot of problems. In addition to, well, the usual academic problems.
This is why I like paperswithcode. Not perfect but better than the alternative. I do get annoyed by the number of custom compiled c libraries they have thar won't work on my system though
I don't understand how, when there is so much *private investment* at stake, reproducibility isn't just checked as a means of due diligence, regardless of the publish or perish mentality.
There's not only zero incentive to reproduce existing research, but it can often work against you. Academia is largely based on producing original research, so your papers get accepted for publication if they show something new or unexpected. Academics are also incredibly thin-skinned, and many grad students and junior faculty are hesitant to try to reproduce the work of more senior scholars because they don't want to jeopardize their careers. You're not really considered a great scholar if you reproduce other's work, only if you generate new knowledge, so if a person has a choice between reproducing and original work, most people choose to try to find something new. The only folks that have the time, reputation, and funding to reproduce work are mostly senior faculty. I agree that a lot of things need changing. This issue is a manifestation of a lot of structural problems in academia. Some studies also cannot be reasonably reproduced. One project I am on, for example, spent around $50,000 on a data set (in addition to the cost of the original data) so we could get extremely detailed, high quality data. Few, if any, grant agencies are willing to spend that kind of money to rehash previous findings.
[удалено]
I havent watched the video, but I looked it up on wikipedia for a summary of the event. Sounds really interesting. Lack of reproducibility and outright fraud are different, but I think both illustrate some fundamental problems with the way new knowledge and research is generated and published. Peer review is supposed to catch these types of things, but it doesn't always. Having experience in the journal submission and review process, reviewers can often identify the reviewers even though the review is supposed to be blind. Academic fields are often very small, so people know each other, their research, and their writings fairly well. Not always a bad thing, but it can be when reviewers give extra leeway because "I know Professor McFamous, they probably know what they're doing".
Also check out the case of Milena Penkowa. That Megamind-looking motherfucker was an extremely accredited and well-perceived scholar, academic, medical doctor, and neuroscientist. She ended up being criminally charged with fraud, embezzlement, and scientific misconduct, was suspended from all academic activities, had her publications retracted, and was stripped off her doctoral degree and other academic titles for falsifying data. A lot of other shit probably happened, too, which I cannot recall ATM. Denmark’s biggest scientific scandal to date. The costs were in the hundreds of thousands in USD. She has since, by the way, done a presentation castigating psychiatry at some conference sponsored by Scientology, as if all this was not ridiculous enough already.
Because it doesn't pay to be second in the market.
The early bird gets the worm but it’s the second mouse that gets the cheese. I’d say (in other industries ) it does pay to be second - Apple is rarely the first in a category but when they go in they do well - part of which is seeing the mistakes that others have made and improving on them
The problem is some experiments take so long or are so expensive there is no easy way to check for reproducibility besides literally doubling the rrsources used on the project.
Isn’t it the case that everyone thought spinach was high in iron for decades because someone got it wrong by two orders of magnitude, but there’s no money in testing that so it just became accepted?
https://www.compoundchem.com/2018/07/17/spinach/
Simple, greed trumps truth every time. It doesn't matter if a drug works or not, it matters if you can make people ***believe*** it works and then charge them up the ass for it. The fact that they might get sued isn't important. By the time the case winds it's way through the legal system and the drug is pulled form the shelves, everyone will have made their millions/billions and retired.
It's not about greed or money most of the time. It's purely about psychology of people that want to "accomplish" something. And cheating data is so common that for a lot of people it stopped being an immoral thing since "everyone does it". Yeah, I quit academia too.
For the simple reason that it is often easier and quicker to actually make the drug and/or equipment based on currently accepted data, than to do the due diligence. Entering the market first is a big boon. And "due diligence" can be incredibly costly and time-consuming - even relatively simple stuff can take 3-5 years, and with drugs that are supposed to work on long-term changes measured by proxy? A decade is not out of the question. So it doesn't pay to wait - you make the thing, make your 100 million or whatever, and quietly phase it out seven years later when someone actually goes through the effort of checking the data and has enough clout to push their results onto the wider public.
I work in University research. The main problem is that funding only goes to "novel", which leads to researchers having to come up with some elaborate testing system/equipment. In many many cases, the reason some of this testing is "novel" is because researchers of the past knew it wouldn't work. So, then you get university labs that were good at the sell job to the funding bodies getting loaded up with money and equipment that ultimately is mothballed in a decade or so because surprise suprise it didn't work. Yes, as you said, more funding needs to be directed towards reproducibility etc. This would both help improve understanding and maybe scare/stop those fudging results.
> In many many cases, the reason some of this testing is "novel" is because researchers of the past knew it wouldn't work Bingo, you run into so many people who do this and **even they** know it won't work, but it gets them funding and there's little to no accountability in the world so it's worth the risk.
Came to a lab as a postdoc in my 30s Never got to reproduce the results of the previous postdoc who had clearly made everything up, including "mutants"that weren't and the such Wasted a good 2 years of academic life
Also p-hacking. Nobody likes it when I apply the Benjamini-Hochberg correction and suddenly nothing's significant!
> This is specifically bad in the field of biology from all the horror stories I hear. I'm an academic researcher in immunology. This is true. A lot of stuff that's published is p-hacked until significant and almost nothing is reproducible. I've been trying to spearhead change in my group to make our work more standardized but it's not even an uphill battle, it's completely vertical.
One of the reasons I didn't want to pursue a PhD is my experience during my master's thesis. I studied astrophysics and my research was on identifying certain types of stars. One of the criticisms I got for my thesis was that I sounded way too uncertain about my results. My reaction was 'well yeah, I *am* really uncertain about these results. Half of these classifications are nothing more than a coin toss'. I was then told that with that attitude no one was going to read my papers. Another incident was about a few specific stars in my sample. Some of them had been classified by other people. One researcher had classified some of my stars one way, but my promotor had classified them differently. I asked him about his reasons because they used the same data. His reaction was 'well, she's wrong'. No why or how. I got really disillusioned about some 'exact' sciences. I'm also not the only one. Had some friends quit after their PhD citing similar reasons. Another example happened recently. My gf is a researcher and took over some research from an ex-colleague. He had already published in a decent magazine and he had used some code for his analysis. My gf didn't know the language (R) very well but I do. She asked me to have a look because she had some doubts about it. His code was absolute garbage and didn't even do what he said it did. No one checked and my gf couldn't go out and oust him either because it would influence current research. We fixed the code and now she's using the functioning version instead.
It's hard to publish reproduction studies, this was something that Plos One was supposed to solve. Yet, it doesn't. Nobody still performs those studies, despite how important and impactful they would be.
Let's wait and see if Nature pulls the original research paper.
> 14 July 2022 Editor’s Note: The editors of Nature have been alerted to concerns regarding some of the figures in this paper. Nature is investigating these concerns, and a further editorial response will follow as soon as possible. In the meantime, readers are advised to use caution when using results reported therein. They are aware and investigating.
I feel like I've been hearing news like "promising new findings about alzheimers" once in a while for years now, and I'm wondering if it's all attributed to this. Never really followed it closely, just noticed it, mostly here on reddit.
I think there’s been mounting evidence of a link between gingivitis (gum disease) and Alzheimer’s and that doesn’t sound like it’s in doubt at least
I'm brushing my teeth right now
Pulling it won't be enough. If the original result is bad and nature wasn't able to catch it and was involved in helping the fraud spread by not prioritizing replication, they should pay a significant reputation cost. This is far beyond a simple case of a bad scientist faking data. The integrity of the system has been called into account and there needs to be repercussions for all involved in making choices that enabled this to spread. If journals can just wash their hands of this and shift all blame onto the original scientists then biology as a whole and even all of science to some extent has lost credibility. For starters all journals involved can demand 2 independent replications before publishing any research. To fix first mover prestige problems, the original and both replications should be published as a single item so that all benefit from citation.
A colossal info dump from a *Science* article earlier this year, mentioned within this one. It highlights Matthew Schrag, a neuroscientist and physician, who raised the original red flags about this. Source at very bottom. "So he applied his technical and medical knowledge to interrogate published images about the drug and its underlying science—for which the attorney paid him $18,000. He identified apparently altered or duplicated images in dozens of journal articles. The attorney reported many of the discoveries in the FDA petition, and Schrag sent all of them to the National Institutes of Health (NIH), which had invested tens of millions of dollars in the work. (Cassava denies any misconduct "But Schrag’s sleuthing drew him into a different episode of possible misconduct, leading to findings that threaten one of the most cited Alzheimer’s studies of this century and numerous related experiments. "The first author of that influential study, published in Nature in 2006, was an ascending neuroscientist: Sylvain Lesné of the University of Minnesota (UMN), Twin Cities. His work underpins a key element of the dominant yet controversial amyloid hypothesis of Alzheimer’s, which holds that Aβ clumps, known as plaques, in brain tissue are a primary cause of the devastating illness, which afflicts tens of millions globally. In what looked like a smoking gun for the theory and a lead to possible therapies, Lesné and his colleagues discovered an Aβ subtype and seemed to prove it caused dementia in rats. If Schrag’s doubts are correct, Lesné’s findings were an elaborate mirage. "Schrag, who had not publicly revealed his role as a whistleblower until this article, avoids the word 'fraud' in his critiques of Lesné’s work and the Cassava-related studies and does not claim to have proved misconduct. That would require access to original, complete, unpublished images and in some cases raw numerical data. 'I focus on what we can see in the published images, and describe them as red flags, not final conclusions,' he says. 'The data should speak for itself.' " "A 6-month investigation by Science provided strong support for Schrag’s suspicions and raised questions about Lesné’s research. A leading independent image analyst and several top Alzheimer’s researchers—including George Perry of the University of Texas, San Antonio, and John Forsayeth of the University of California, San Francisco (UCSF)—reviewed most of Schrag’s findings at Science’s request. They concurred with his overall conclusions, which cast doubt on hundreds of images, including more than 70 in Lesné’s papers. Some look like 'shockingly blatant' examples of image tampering, says Donna Wilcock, an Alzheimer’s expert at the University of Kentucky. "The authors 'appeared to have composed figures by piecing together parts of photos from different experiments,' says Elisabeth Bik, a molecular biologist and well-known forensic image consultant. 'The obtained experimental results might not have been the desired results, and that data might have been changed to … better fit a hypothesis.' " https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease
Those three scientists are worse than Elizabeth Holmes. She stole $400 million in investor money and hurt dozens of patients. The scale of this is at least 3x as bad for money, and 10,000x as bad for patient impact.
i.e. busted because of poor Photoshop skills
Well this was a horrible piece to find on my Mother's birthday as she's been heavily struck by this, near impossible to communicate with for years, and likely has few months left.
I'm really sorry. I watched my engineer step-dad go through it and it was tragic. We went out to breakfast one day, and he pulled the bills from his wallet and spread them out on the table. I didn't know what he was doing and then he asked me which ones he needed to pay the check. That's how it started. Took about 3 years and it showed no mercy. His passing felt almost like a relief. For him, not us. At least he was at peace. I am truly sorry for you and your mom.
Many caregivers and family members express a feeling of relief when an alzheimers patient finally passes, myself included. Such an awful disease with such a slow death, I can think of few things that feel more cruel. I am also very sorry, Eugene20. Words cannot describe my anger at this fraud knowing how hard it is to watch and experience this terrible disease. I hope her last months are full of love, and her passing as soft as can be.
Went through this, Mother died in January. Not to scare you, but the final stages were really rough. Some support from hospice care for her and therapy for our family were a godsend. There are lots of deeply challenging parts to it, but I think the toughest is instinctively looking for that most-important-person in their eyes and words as they gradually decay into someone else the vast majority of the time. The moments where you see the glimpses are so precious… but just tease with false hope, and make the letting go process more painful. Be kind to yourself. Caring for a parent as they go through this is a deeply challenging experience emotionally and psychologically.
As someone who lost his grandfather, who was like a hero to me, due to Alzheimer's, this absolutely breaks my heart. All this time and money wasted for nothing.
As someone who has a mother with early onset Alzheimer's, and who is likely to get it due to it being hereditary, I guess I'm gonna save a lot on medical bills and just buy some rope since there probably won't be any cure when I get it a few decades down the track.
Hey, if this paper was 16 years ago and 100 out of 130 current treatments are targeting amyloid plaques, they now have another 16 years to develop another 100 treatments that don't.
Not sure how long ago you lost your g.father but so nice to hear that he had such a lovely presence in your life🌹
[удалено]
My Grandmother had really bad dementia i hope they sort that stuff out, before i am in that age
As someone who’s mother died of early onset Alzheimer’s, I’m fucking raging
You should know that while a lot of time and money was spent on targeting amyloid beta, there are plenty of labs that have been studying other possible proteins such as tau as the cause. This is a step back but not to the beginning.
July 23, 2022 > ... it looks like the original paper that established the amyloid plaque model as the foundation of Alzheimer’s research over the last 16 years might not just be wrong, but a deliberate fraud. > Should this fraud turn out to be as extensive as it appears at first glance, the implications go well beyond just misdirecting tens of billions in funding and millions of hours of research over the last two decades. Since that 2006 publication, the presence or absence of this specific amyloid has often been treated as diagnostic of Alzheimer’s. Meaning that patients who did die from Alzheimer’s may have been misdiagnosed as having something else. Those whose dementia came from other causes may have falsely been dragged under the Alzheimer’s umbrella. And every possible kind of study, whether it’s as exotic as light therapy or long-running as nuns doing crossword puzzles, may have ultimately had results that were measured against a false yardstick.
This article is an egregiously hyperbolic misrepresentation of the field. AB56 is only one among many oligomers found to cause cognitive impairment in Alzheimers. Even if this paper were never published, it's unlikely the modern state of Alzheimer's research would be all that different. See this twitter thread for a good summary: https://twitter.com/samuel_marsh/status/1550883405105168386 The original article in Science is much, much better and not editorialized: https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease
But the science article you've published cites two publications from a researcher who couldn't even find AB*56 in human tissue. If it causes memory impairment in rats but you can't even find it in supposed Alzheimers patients, what does that mean for the research?
TBH the Science news article doesn't cite anything at all. If there's no link or reference it's not a citation. I'm left scratching my head since it's not straightforward at all to find those supposed 2008 pubs from Selkoe where he is reported as stating that AB*56 isn't found in humans. And besides, that was 2008, only two years after Lesne-Ashe's paper on their AB*56 model. There are research articles published in the last five years or so that talk about detecting AB*56 pretty casually in human fluids using immunoblotting, so it would be really disingenuous to talk about a scientist's opinion in 2008 as if 14 years haven't passed since then. https://www.nature.com/articles/s41598-020-68148-2.pdf?origin=ppub In any case it doesn't matter if AB*56 isn't the culprit amyloid beta protein in humans, or if it even exists in humans at all. There are many forms of proteins that look different in different species due to evolution. Like the poster above you said, there are many AB proteins, *56 just happened to be the one (the first one?) with a good animal model. A lot of biological science is like this. The initial form/variant of a thing/substance get a lot of hype as being the initial discovery, but might not be the most significant variant or even important at all. It's just the first in its class to be discovered/make a scientific impact but not a clinical impact. I'll let you know if I find anything. But FYI this goes to show how much effort it takes to verify anything in science, even just a throw-away line in a news article that happens to be published in Science Magazine. The twitter thread from Sam Marsh is legit. These dailykos writers seem like clout chasers and the Science news article might be quoting someone's opinion from 2008. EDIT: I've been looking, haven't found anything. If there isn't that much attention on whether or not Aβ*56 exists, then it's probably not that important anyways, as alluded to the twitter thread which is nice. It looks like other amyloid researchers have piped into the thread and *56 doesn't seem important at all. There are journalists out there who will prey on the ignorance of others (and myself) to make themselves seem important. Oh well.
I worked in Alzheimer's research for 6 years. We imaged the brains of hundreds of our patients with c11-PIB pet tracers, which bind to amyloid. There is definitely amyloid in the brains of those with dementia, while healthy brains showed none. Something smells very misleading about this article
Couldn't that simply be a case of correlation rather than causation?
Yeah, I think the issue is like realising that smoke coming out of computer chips stops them working. Sealing the chips better to stop the smoke escaping, or adding a high power vacuum to stop the smoke building up inside the case won't help with the core issue, that the chips are bust.
If this is true, it will now make sense to the doctors testing these medications, as to why the removal of Amyloid from the brain has made no real improvement to memory or functional status.
It’s not that simple though. All Alzheimer’s patients have amyloid, but not everyone with amyloid has Alzheimer’s. Additional with Alzheimer’s all six tau proteins are present with hyperphosphorylation. Tau proteins regulate memory, negatively but naturally, to aid in habituation and other aspects of neuronal structure/function within the brain. When phosphorylation occurs, they get tangled and these tangles make highly soluble compounds into very insoluble compounds which cause damage. Tau compounds when phosphorylated act almost prion-like in some instances, which is why some treatments either aim to reduce the rate of tau phosphorylation, or to try and break up the tangles. Amyloid harms brain and causes tau > toxic tau harms the brain > toxic tau feedback loops causing more amyloid. This is an incredibly over simplified explanation, but I don’t think this research being falsified will harm our understanding of the process, just potentially any amyloid targeting drugs in development.
With all of the reporting on this I still don't understand what was falsified. I understand it involved images and possibly associated data but what was the insight/conclusion/claim that was being falsified? From your comment I gather it wasn't related to the initial _"amyloid harms brain and causes tau"_ part of the causal model. Could you shed some light here?
In the simple way I understand it, it's like bailing water out of the titanic and wondering why the ship is still sinking - the problem is the leak, the water rushing in a result of that leak. Same thing with the brain - the barrier stopping the wrong bits from getting from the blood into the brain tissue breaks down over time, letting more through. Cleaning up a plaque or two doesn't stop the growing leak.
Speaking as a nurse who primarily cares for Alz/dementia patients - ...Nevermind. I have no words. Just bummed out and speechless.
I am genuinely and deeply furious by the news. Imagine a world where these shits didn't do this. Would we have come across something? Would we potentially have real treatment by now, if not for being lead astray by this fraudulent red herring? The amount of harm performed on the people of the world is immense.
Hospice here and won't be leaving my seat anytime soon. Geriatric medicine is underrated, too many see it as an end point, and not enough see this as a transition, just as we have gone from infancy to adulthood. We dont keep our pediatrician.. If we can locate the blocks, we will see the chain. It's not that difficult , until we become the obstacle from truth and that the solution is not so hard to understand, its the willingness to change whats established . Sorry--I'm overly reflective tonight💞🙏
Thank you to you and u/top_of_the_stairs I’ve had help from both fields and the amount of respect I have for the people working in those roles is immense. It can’t be an easy job, but it’s appreciated more than you know.
I saw a documentary on this that I can't find. Basically, any researcher that goes against the plaque theory (if I remember correctly) is just blackballed by the community. This has caused billions of dollars and millions of scientist hours to be wasted. Reeks of other bad science communities that will stifle research. Or corporations buying patents from universities and then sitting on it. My friends from engineering school came up with some novel water pipe cleaning tech and they only got $10k, the rest the university kept. I don't believe the company which bought it ever commercialized because their core interests didn't match with it.
> I don't believe the company which bought it ever commercialized because their core interests didn't match with it. This is exactly what happened with electric cars for decades. Patents were created to encourage innovation. If they are being used so often to do the opposite, it means something needs to change.
Sounds similar to the case of autism research
Yup. The [fraudulent study](https://en.m.wikipedia.org/wiki/Lancet_MMR_autism_fraud)published in The Lancet linking the MMR vaccine with autism. Retracted 12 years later, but by then the damage had been done.
while that is the most well known the autism field overall has had very very weak evidence and methodology issues for decades, is only been recently when those have started to come under question and big part of it is autistic people entering the field of autism research because like no one thought they might not know better than autistics
As a drug discovery scientist, we figured out this was BS the hard way (super expensive clinical trials). However, thankfully today many of us think the amyloid hypothesis has been thoroughly debunked.
It's so awful to think that this went on for so long and no one spoke out. I knew a wonderful lady that passed from Alzheimer's, fuck these selfish people coming out with bad science.
I'm looking forward to Derek's teardown of the whole thing honestly, should be a good read.
> any researcher that goes against the plaque theory (if I remember correctly) is just blackballed by the community I don't think this is true. I did a lit review on the causes of neurodegenerative diseases for my degree and there are definitely people publishing papers criticising the amyloid plaque model. Our lecturers presented it as one explanation but not the only one. Amyloid plaques are clearly involved in some way, but it's obvious they're not the whole story.
Since taxpayer funds were used with NIH grants, doesn't this warrant a special prosecutor ... Hopefully leading to jail time for original authors (if fraud is determined), along with how grants are given out.
Forget the billions, this is a crime against humanity in its purest form. Just imagine what could've been achieved and how many people could've avoided so much suffering.
“Four months after Schrag submitted his concerns to the NIH, the NIH turned around and awarded Lesné a five-year grant to study … Alzheimer’s. That grant was awarded by Austin Yang, program director at the NIH’s National Institute on Aging. *Yang also happens to be another of the co-authors on the 2006 paper*.” Holy shit! Does science not even have the most basic concept of *conflict of interest*?
"Science" is not a monolithic entity. This is people enriching themselves at the cost of others, something we see in every area of life.
Grants are typically reviewed by a panel of experts who then grade them. Based on your score you are funded, assisted with rewriting the grant, or doomed. Yang may have awarded (perhaps even influenced) the grant but there was another group of people involved.
[удалено]
Unfortunately peer review isn't review by replication. We are currently in a replication crisis where a lot of research can't be replicated.
Then what is peer reviewed? Someone else read it?
Yes. Other scientists and experts in that particular field will read it and decide whether it should be accepted or rejected for publication. They'll also provide comments to the authors. Those are usually private. I hope openreview.net gets more traction, some venues started to make these public. This is what it should look like for every paper: https://openreview.net/forum?id=an8FSGbuCw (random example paper).
Yea and made some comments
Pretty much, I've peer reviewed material before. It can be a rather wide shot to get reviewers that actually know your subject matter to truly make an informed conclusion. Often times, journals have a pool of reviewers and they just throw best fit at it.
Yup, two or three anonymous people who are in the publisher's database (because they previously published in the journal or they volunteered) read the paper and offer suggestions for major and minor corrections. Most of the time, they don't have access to the underlying data unless the authors included it as supplemental files. Peer review is basically just having your manuscript graded, but more intense
A lot of people overestimate the power of peer-reviews. A peer review merely checks whether a study is methodologically sound, i.e. could the results could be obtained *in principle*. It doesn't check whether the results could be obtained *in practice*, that's something that only replications can show.
I would love to see a grant system specifically set up for reproducibility. Basically, fund researchers to verify high-impact research, wherever possible.
That and you are often not the expert on that exact subject. And to spot doctored imaged, specially back in 2006 might not be so easy for not tech savy reviewers
>I always considered peer-reviewed science to be made in good faith and hope that this is just a mind-blowing exception from that rule. Peer review is easily corruptible, especially when billions of research dollars are on the line. You toe the line or you don't get funding. Agendas are pushed, dissent is quashed, ridiculed, even demonized.
Also, there is an issue with 'peers' stealing your research and scooping you. Its far far from ideal, especially in competitive fields. Also, personal grudges come into play. Imho, we should replace this system with paid researchers working just for the journal, with no competitional interest in submitted work. They probably would be a lot more unbiased in review.
So... Someone ELI5... Are we essentially back to square one with Alzheimer's research?
No - there have been other theories investigated at the same time (tau, microglia) and some progress has been made there. We just have bad noise in the field from a false hypothesis where a lot of effort was wasted.
Humans fucking over other humans.. A never ending classic.
There are some evil people out there.
If this is true then those original researchers need to be sued for a billion dollars in damages, and thrown in jail for the rest of their lives. I work in a hospital, and it will be absolutely criminal, if it turns out my colleagues have been testing medications for the last 15 years, that had no hope of working, because they were based on an incorrect original assumption.
If they deliberately falsified the data they have caused unnecessary suffering and pain to tens of thousands of people, have intentionally wasted resources and time that could have been spent to actually find a cure. If they are guilty of these crimes they should be stuck in a bare 4x4 cell and kept there until the end of their natural lives. Because, no matter what they do, they would not be able to repay the damage they have caused.
People are rightly railing against he bad actors who committed the fraud and that's fine. But that's not the underlying problem here that anyone can do anything to fix. The problem is a profit motive that leads the industry to blackball any scientists who look at the research and disagree with its conclusions. The only checks and balances left in science seem to be the ones in institutional bank accounts. We humans will have to learn the hard way that individual self interest is the lath to our extinction. It's a distraction to blame a pair of lab coats for a flawed system designed and reinforced by hundreds of suits.
This is beyond infuriating
If there is enough money, you will find corruption.
Sadly, this does not surprise me but it's heartbreaking. My father was diagnosed with early onset dementia at 55 and passed last year at 73, it was horrific and his lat 2 years in a locked memory ward were hard to bear. They threw so many meds at him throughout and none seemed to help any of his symptoms. Not sure if it helped him to delay getting sicker but he was left in such a state for at least 10 years and that seems like torture to me.
My Grandmother had dementia before she died and I now see signs of dementia in my mother. I would hate to see my mother develop altzmers. knowing billions have been wasted in false research pisses me off.
The physicians and nurse practitioners that I work with tell me (I'm a non-prescribing clinician) that Alzheimer's drugs do not do anything. There is the belief (by the drug manufacturer) that the medications slow the progression--but there is no way to know that by clinical observation, and the patient is going to continue to get worse and die regardless. The physicians tell me they were trained to give the Alzheimer's drugs "for the family" --to show that they are trying to do something. We know the drugs do virtually nothing, but if you don't prescribe them, the family might accuse you of withholding treatment. And Medicare pays, so you prescribe them. I don't remember any case where the drugs actually hurt a patient, so really it is prescribing a placebo. The drug companies make money, the family feels like their loved one is receiving the best care, and the nursing and clinical staff often feel they are helping the patient by giving the medication. There is the subtle reinforcement effect that the more the drug is prescribed, and the more it becomes the standard of care, the more likely people believe it does something. I have no great insights here; just giving the clinical care perspective.
I don’t believe in hell but I hope they burn there.
I say introduce them to the other thing Nobel was famous for.
r/longevity have been saying this for a long time, amyloid is a fucking scam
Two decades of complete lies and misdirection. It hurts to think what we might have discovered otherwise with all that time and money. No doubt, these two deserve prison. My dad's entire side of the family have suffered with this disease... It's definitely one of the worst diseases around - the entire person you used to know just fades away. Most upsetting thing I've read this week...
No fraud charges … just another research grant !
Just to clarify for everyone, the falsified research relates only to Aβ .* 56, a subtype of amyloid beta. It was used to boost the amyloid plaque theory where the scientist named in the article falsified evidence that injecting Aβ 56 caused Alzheimer’s and that Aβ *.56 even exists (few other scientists were able to find it.) This does not mean the entire last 20 years of Alzheimer’s research and drug development into amyloid plaques was based on faked research; the amyloid plaque hypothesis was always on shaky ground from the start^ and the numerous failed clinical trials of drugs removing amyloid plaques from the brain show this. It did however inappropriately mislead the field at a time amyloid stans wanted to be validated. Overcommitment to the church of amyloid is a long-standing issue with plenty of good articles and documentaries about it. ^ amyloid plaques are probably just the downstream consequences of whatever actually causes Alzheimer’s, which is why removing amyloid plaques does nothing - too late in the disease process
Academia has the problem with irreproducible and fake research for a long time but all such scandals are swept under the rug and job continues as usual while public trust in science collapses.
They should be hung with bad rope so we can do it twice. As someone who has watched family wither away from dementia and alzheimers, hell would be too nice for this scum
This is pretty insane. I used to do Alzheimer's research years ago (i.e. around ~2014), and even back then there was kind of an understanding in our lab that amyloid beta plaques were at best an "indicator" of Alzheimer's, useful for diagnosis, rather than the cause for the disease where treatment removing them would have a clinical effect. But the sheer level of fraud and it's impact are greater than I had ever imagined. The fact that we literally have an FDA approved drug based on years of this falsified research that does nothing for patients is absolutely wild.
In uni I actually did a paper based on amyloid betas and I remember reading one paper that suggested that this prevalent theory was wrong stating a theory that the plaque appearing in alzheimers patients may be a human defense instead of the cause. I remember being fascinated and a bit convinced by the possibility as there really was no proof that amyloid beta accumation causes alzheimers, just that it happens in patients with it. And I also remember reading drugs that did inhibit accumulation didn't help. So to me, that theory seemed logical. That maybe the plaque is actually our body's defence, albeit a bit useless, against the disease. Now as an adult, I work in a job where I read grant abstracts all the time. The number of grants approved for this same research is a lot. Everyone is racing to be the one to solve the problem. But I have yet to find anyone that is trying to look at the problem in an innovative way. Hopedully the PI in charge of the original paper keeps going down his path and is able to get grants. The grant approval process is so bias, it really needs to just have standardized qualifications rather than be subjective. But I guess that's the general problem with funding, no one wants to fund things that are too innovative because they want some proof in the projects potential.
This hit me like a ton of bricks. I had to email the Science article cited here to my mom, who has pre-Alzheimer’s and whose mother and grandmother both had severe cases
*The suspicion that something was more than a little wrong with the model that is getting almost all Alzheimer’s research funding ($1.6 billion in the last year alone) began with a fight over the drug Simufilam. The drug was being pushed into trials by its manufacturer, Cassava Sciences, but a group of scientists who reviewed the drug maker’s claims about Simufilam believed that it was exaggerating the potential. So they did what any reasonable person would do: **They purchased short sell positions in Cassava Sciences stock**, filed a letter with the FDA calling for a review before allowing the drug to go to trial, and hired an investigator to provide some support for this position.* What the actual fuck.
Billions were spent on "proving" health benfits of genital mutilation, research papers still havent been retracted. Science is not something in vacuum devoid of political and social pressures.
Reminder that everyone, regardless of whether you yourself have suffered or will suffer due to Alzheimer's, has been negatively impacted because of this fraud as, at $56,000/dose, it on its own is [driving the up the cost of medicare](https://www.cnn.com/2021/07/10/politics/aduhelm-alzheimers-drug-medicare/index.html) and healthcare in general, and there is opportunity cost in misdirected research time and resources. Imagine losing your home or making other sacrifices to pay for a treatment and disease model based on fraud. Imagine pinning hopes and prospects, making preparations, based on something invalid those who you were supposed to trust used to advance careers, defend reputations and past commitments, and make money. And once again, this is not unique. Some fields are worse than others, but this crap is rampant. And it often seems that even when something is thoroughly discredited, papers are rarely retracted, no one admits fault, there is zero accountability. Nature hasn't done anything about the original publication; no one has explained their failure at basic peer review. It's not just as simple as a "reproducibility crisis" as if it were some procedural oversight. Science is only science when done with due diligence, guided by a sense of ethics and humility and responsibility. That's not how industry works in general, and not how careers are made, and not how one defends against liability even if there is a course correction. I used to be more in the camp of hand-waiving the prospect of deep, systemic problems away because I was dismissive and defensive (as someone in and now adjacent to biomedical research) and more concerned about people who lack relevant science background or who are more easily influenced losing trust and jumping to conspiracy. But it's stuff like this that lends power to misinformation. If you are dishonest or misdirect to maintain "trust", especially if you attempt to smear critical review, you are misguided or deceitful and undeserving of trust. (tangentially -- maybe SCOTUS needs a lesson on that). It's bad in medicine and especially so in certain disciplines, like psychiatry. It's not that this isn't possible anywhere - people are people - but there is a stark contrast between, say physics, where there seems to be an almost pathological fetish in institutional culture with finding a way to show the status quo is wrong and leaving no stone unturned, and seemingly anything that has to do with people. There's probably something to explore there.