Category: Research

Who will subscribe to journals of negative results?

published date
September 17th, 2006 by
Who will subscribe to journals of negative results?Thanks to SimulConsult CEO, Mickey Segal, MD, PhD for this post.

There is clear value to journals of negative results because they correct a bias towards publishing correlations:

A handful of journals that publish only negative results are gaining traction, and new ones are on the drawing boards.

“You hear stories about negative studies getting stuck in a file drawer, but rigorous analyses also support the suspicion that journals are biased in favor of positive studies,” says David Lehrer of the University of Helsinki, who is spearheading the new Journal of Spurious Correlations.

After a slow start in 2002, that journal is receiving more and better papers, says Dr. Olsen. One found that, contrary to other reports, the relative length of the bones of a woman’s index finger and ring finger may not be related to her exposure to testosterone in utero. Another found that a molecule called PYY doesn’t have a big influence on body weight; another, that variations in a gene that earlier studies had associated with obesity in mice and in American and Spanish women isn’t linked to obesity in French men or women.

Hopefully, each of these reports kept researchers, including those at drug companies, from wasting time looking for ways to repair the consequences of the supposed genetic association. But it isn’t clear that any would have been published without the new journal.

These reports of negative results are very important, but will people subscribe to journals of negative results? This is a good example of papers that should be published in free online journals.

Questionable correlations between a gene and cancer are the bread-and-butter of NOGO, the Journal of Negative Observations in Genetic Oncology, which Dr. Kern edits. “Fully half [of discoveries] of novel mutations in tumors, we found, were not confirmed in the subsequent literature,” he says. “You expect to see follow-ups if the claims held up, so the fact that we didn’t casts doubt on the original claim. But that wasn’t explicitly reported.”

Why are scientists coy about publishing negative data? In some cases, says Dr. Kern, withholding them keeps rivals doing studies that rest on an erroneous premise, thus clearing the field for the team that knows that, say, gene A doesn’t really cause disease B.

I once spent months trying to reproduce, without success, a paper in a top journal claiming that lipids had huge effects on the nervous system. I even flew overseas to visit the lab of the guy making the claim and made liposomes in his lab and brought them back. In the end I concluded that the effects were due to the solvents used to make the liposomes. I ran this hypothesis by the author of the original paper and he told me that solvents aren’t interesting, but lipids were interesting. I didn’t publish my negative results, and subsequently heard that others before and after me had failed to reproduce the results.

Having a forum to report negative results is important, but it is not clear that journals of negative results are a better place to publish than online free journals.

Nature opens the peer review door a crack. Will anyone step through?

published date
September 14th, 2006 by

Nature opens the peer review door a crack. Will anyone step through?

An article in today’s Wall Street Journal (Journal Nature Opens Peer-Review Process to Comments Online) describes and experiment by the journal Nature:

Nature, one of the world’s most prestigious scientific research journals, has embarked on an experiment of its own.

In addition to having articles submitted for publication subjected to peer reviews by a handful of experts in the field, the 136-year-old journal is trying out a new system for authors who agree to participate: posting the paper online and inviting scientists in the field to submit comments praising — or poking holes — in it.

Lay readers can see the submitted articles as well, but the site says postings are only for scientists in the discipline, who must list their names and institutional email addresses. Nature says its editors screen out those they find irrelevant, intemperate or otherwise inappropriate.

Meanwhile, the papers also make their way through the journal’s traditional peer-review gauntlet. Nature says it’s taking both sets of comments into account when deciding whether to publish.

Sounds like a potentially promising development that could spread to medical journals. That could be useful, for example, if reviewers used the opportunity to more heavily scrutinize industry-sponsored submissions. I asked Mickey for his view:

It is interesting that it makes the paper available more quickly. This may be a crucial element to attract people to view the papers and possibly comment. Such a system is used in economics, where the crucial release of a paper is into pre-publication review that is viewed by many people. However, I don’t see a single comment on the 10 pages that are listed on the Nature site.
What I like about this approach is that it associates comments to the article text. An alternate model is bloggers talking about papers, but the problem there is that the comments are scattered in many places and hard to find.
This is an interesting move by a prominent journal such as Nature, which can hope to get reader participation because of its prominence the same way the Wall Street Journal’s “Best of the Web Today” column gets submissions in part because of the prominence of the Wall Street Journal. Less prominent journals would be better served by the independent blogger approach.
It will be interesting to see which approach wins. The continued prominence of journals such as Nature depends on it. In the past such journals had advantages that have been reduced by the ability to measure impact of papers not just by the prominence of the journal but by counting citations of the paper. If the blogging model beats the group review model as illustrated by this effort by Nature the prominent journals will have lost much of their prominence.