Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Could an enterprising person find such papers, cleverly select a dataset such that X is way worse, and finally refute the original paper?

This seems like something an amateur could do. Wouldn't that tear down a bunch of this junk?



Sure. But for approximately the same level of effort, you could generate a new paper in the same template. It would likely be easier to get that "novel result" published than the refutation.


And that's generally the truth in every scientific field.

Right now you have a "reproduction renaissance" in psychology and cancer, where you can get published, and get acknowledged, if you tear down a high level paper. But in general, unless you aim at refuting the highest level, it is hard to get published and advances your career nowhere, so people have no incentive to do it.


A non-replication does not "tear down" a paper... It is not any kind of "attack", your attitude is one of the sources of the problem.

Edit:

When I did medical research I wished people would try to replicate my findings so I could see how consistent the results were and how well my model fit future data... In fact it was really annoying that it was unlikely anyone would ever try to do a direct replication.


My attitude? What attitude is that?

A single non-replication does not. Consistent failure to replicate, after communicating with the original author, does.

See e.g.

http://www.slate.com/articles/health_and_science/cover_story...

http://www.nature.com/nature/journal/v483/n7391/full/483531a...

Or just google "reproducibility crisis", the two most common subjects that come up are cancer research and psychology research.


That is the problem: Why wouldn't try to do a replication?

Because the incentives are to provide "new research", not to confirm or deny a known result.

It seem to me that most probably no-one has gotten tenure by reproducing, disproving results.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: