Wednesday, April 8, 2015

Trying to fix the symptoms

In response to many complaints that the peer review process was so slow that it was impeding scientific progress, Nature announced that they would allow authors to pay to jump the queue:
As of 24th March 2015, a selection of authors submitting a biology manuscript to Scientific Reports will be able to opt-in to a fast-track peer-review service at an additional cost. Authors who opt-in to fast-track will receive an editorial decision (accept, reject or revise) with peer-review comments within three weeks of their manuscript passing initial quality checks. 
It is true that the review process is irritatingly slow, but this is a bad idea on many levels. Such a bad idea that an editorial board   member resigned in protest. Below the fold I discuss some of the levels.

First, it is an example of the innovative business model that Yves Smith has dubbed "crapification". A business that can reduce customer choice sufficiently then has a profit opportunity; it can make its product so awful that customers will pay for a slightly less awful version. Examples include
  • Cities that provide inadequate public transit so that their freeways get congested can make them even more congested and profit by designating toll lanes.
  • The US Customs & Border Patrol can reduce staffing at passport checks so that the lines get so long that people will pay the $100 to join Global Entry and use automated kiosks (which in the UK are free).
  • Airlines can make economy seats so awful that people will pay a higher economy fare just for the faint chance that they might get upgraded to business.
  • Google messing up organic search links to force businesses to pay for ads.
Ian Welsh, in Making the Rich and Powerful Work for Everyone, sums up the effect of crapification:
No part of society will continue to work properly if the powerful and rich have no interest in its doing so. There are three parts to this:
  1. If there is a public system, there cannot also be a private system which can be used to opt out of the public system.
  2. If there are limited resources, whether those are airplane flight slots at airports or medical care, then no one can be allowed to use either wealth or power to jump the queue, nor must they be allowed to use more resources than those without power or money.
  3. Any part of the economy where there is a monopoly or an oligopoly must either be publicly run or must be heavily regulated for quality, level of profits, and reinvestment.
Second, the fast-track system looks like extortion. An author, offered fast-track reviewing and afraid of getting scooped, is under enormous pressure to pay the additional fee. This reminds me of the notorious .sucks domain registration scheme, which has been described as predatory, exploitative and coercive.

Third, the problem with peer review is not that it is too slow, but that it doesn't perform its assigned role of sorting the sheep from the goats. Providing a fast lane is not going to make it better at sorting the sheep from the goats. The authors who are already paying services to create fake papers, and supply fake reviewers for them, will clearly not be deterred by an additional charge for fast-track review. Nature claims:
fast-track decisions will be made by in-house editors according to Scientific Reports standard publishing criteria
The implication being that the in-house editors are so good at distinguishing sheep from goats that only sheep will be fast-tracked. But if the editors and reviewers were so good at separating the sheep from the goats presumably there would be very few goats published, instead of the deluge of goats that keeps Retraction Watch busy. Thus it is likely that the fast-tracked papers will have the usual proportion of goats, and the time-pressed reviewers will be as unlikely as now to spot them.

In an excellent piece entitled Science's Big Scandal, Charles Seife lays out the problem highlighted by journals deploying software to detect software-generated papers:
Which is why Springer’s deployment of a SCIgen-detecting algorithm is so damning. A SCIgen paper, remember, is so transparently fake that even the most cursory review—like, say, reading the first four sentences of the article—should be enough for a computer science novice to know that it’s totally bogus. To spend any time, effort, or money to come up with a technological solution to screening out those papers is a tacit admission that even at the most reputable publishing houses, some peer-reviewed journals are incapable of providing even the most minimally competent peer review. It’s an open declaration that even the people in charge of the mint are now churning out counterfeit coins, publishing “peer-reviewed” journals that have no functional review process. An automated screen for SCIgen papers is simply an attempt to counteract an incredibly effective mechanism for exposing publishers whose journals are faking it. 
His colleague, Dr. Arthur Caplan of the Division of Medical Ethics at NYU's Langone Medical Center points out the result in a short but pertinent article The Problem of Publication-Pollution Denialism:
Publication pollution is corroding the reliability of science and medicine and yet neither the leadership nor those who rely on the truth of science and medicine are sounding the alarm loudly or moving to fix the problem with appropriate energy.

The currency of science is fragile, and allowing counterfeiters, fraudsters, bunko artists, scammers, and cheats to continue to operate with abandon in the publishing realm is unacceptable. Talk of free speech and the power of the marketplace of ideas to sort out the wheat from the chaff is naive. When the marketplace is full of dangerous and defective goods, there is no free market because the trust requisite to support a market has evaporated. The time for a serious, sustained international effort to halt publication pollution is now. Otherwise scientists and physicians will not have to argue about any issue - no one will believe them anyway.
John Michael Greer has a must-read post with a broader view of the implications:
institutional science only has the authority and prestige it possesses today because enough of those outside the scientific community accept its claim to speak the truth about nature. Not that many years ago, all things considered, scientists didn’t have the authority or the prestige, and no law of nature or of society guarantees that they’ll keep either one indefinitely. Every doctor who would rather medicate than cure, every researcher who treats conflicts of interest as just another detail of business as usual, every scientist who insists in angry tones that nobody without a Ph.D. in this or that discipline is entitled to ask why this week’s pronouncement should be taken any more seriously than the one it just disproved—and let’s not even talk about the increasing, and increasingly public, problem of overt scientific fraud in the pharmaceutical field among others—is hastening the day when modern science is taken no more seriously by the general public than, say, academic philosophy is today.
 My previous posts on scholarly communication.

1 comment:

  1. The TSA has discovered crapification! They want people to pay to be PreCheck, with the alternative of standing in line for three hours and missing their flight. But people aren't paying, so they are making the lines even longer! Tempting targets for the terrorists, as we saw in Brussels.

    Of course, even paying for PreCheck doesn't mean a lot:

    "The Precheck lines in Newark can take more than an hour to clear."

    That's when PreCheck is open. My recent experience was that it was open only 2/4 checkins.

    ReplyDelete