tag:blogger.com,1999:blog-4503292949532760618.post4832386631143424671..comments2024-03-16T18:42:21.178-07:00Comments on DSHR's Blog: What Is Peer Review For?David.http://www.blogger.com/profile/14498131502038331594noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-4503292949532760618.post-22406216275851993192012-03-24T17:13:59.787-07:002012-03-24T17:13:59.787-07:00There seems to be some misunderstandings and conce...There seems to be some misunderstandings and concern about the statement that “about 40% of rejected manuscripts have been accepted for publication in another journal”. The worry, of course, is that papers have been rejected from PLoS ONE as being unpublishable but end up getting published anyway. As the publisher of PLoS ONE, I would like to clarify.<br /><br />The data referred to originated from our annual survey of PLoS ONE authors. Specifically, in June 2011 we asked those authors whom we had rejected throughout 2010 what the status was of their rejected paper. We had 411 respondents to this question. 183 (44%) of them told us that their paper had been accepted by another journal (and this was a similar percentage to prior years). In that survey we did not go on to ask further questions on this topic. We did not, for example, ask whether it was the identical paper which went on to be published; or whether the authors felt that the initial rejection was fairly made against our criteria.<br /><br />Thus, of the approximately 31% of authors who submitted to PLoS ONE in 2010 but who had been rejected by the time of the survey, approximately 44% went on to be published elsewhere. i.e. approximately 14% total of our total submissions.<br /><br />Submissions might be rejected from PLoS ONE for a number of reasons, only one of which might be that they are deemed unpublishable. <br /><br />For example, they may be rejected because they are not article types that we not publish (e.g. they might be opinion pieces, or review articles); or they might simply be out of scope (e.g. they might be in the humanities or the social sciences). It is entirely reasonable that articles in these 2 categories might end up being published elsewhere.<br /><br />Another category of rejection includes papers which might need more work to become publishable. There are several reasons this might happen – for example the English language might have been too poor; or additional data analysis might have been required; or additional controls might have been needed to make the conclusions robust etc. It is entirely possible that authors who received a rejection on those grounds chose to do the work required to make the article publishable and then submitted the paper to another title (this would be the natural behaviour for those people as, after all, they had been rejected by PLoS ONE and we did not invite them to revise the work and resubmit it to us).<br /><br />Another category of papers includes papers which are inappropriately rejected from PLoS ONE. No system is perfect, and it is entirely possible that some of our rejection decisions were not correct. In these situations, again, it is reasonable for that class of paper to then go on to be accepted elsewhere.<br /><br />Finally, there is of course the category of paper which PLoS ONE deemed unpublishable due to some serious concern about the quality of the study, which was not then revised (or corrected) by the authors, and which went on to be published in another journal. We do not know the proportion of the 44% which this category represents, however we are in the middle of a project to find this out on a representative sample.Binfieldhttps://www.blogger.com/profile/08979896656262392443noreply@blogger.comtag:blogger.com,1999:blog-4503292949532760618.post-2695323782632561632012-03-23T13:00:51.951-07:002012-03-23T13:00:51.951-07:00Irene, thank you. Blogging time is scarce for me r...Irene, thank you. Blogging time is scarce for me right now but I will try to get a clarification from my contacts at PLoS.<br /><br />While the details may be open to clarification, it seems that the big picture, that the real question is not <i>whether</i> an article will get published, but <i>where</i> it will be published, is correct. And if it is, then the problem of effective research communication is a <a rel="nofollow">search engine optimization</a> problem. In other words, the problem is not as in the past to prevent bad research being published, but rather to help both humans and more importantly search engines understand the quality of the published research they are reading.David.https://www.blogger.com/profile/14498131502038331594noreply@blogger.comtag:blogger.com,1999:blog-4503292949532760618.post-24816302038421267742012-03-23T09:51:56.444-07:002012-03-23T09:51:56.444-07:00"Here I should acknowledge a debt to the fine..."Here I should acknowledge a debt to the fine work of the UK House of Commons Science and Technology Committee, on which some of the following is based."<br /><br />Hi, glad that you found the report useful (declaration: I was the specialist adviser to the inquiry/report). Can I just clarify the following from your post <br /><br />"... PLoS ONE publishes every submission that meets its criteria of technical soundness. Nevertheless, over 40% of the papers it rejects as unsound are published in other peer reviewed journals."<br /><br />My understanding is that the 40% rejected referred to aren’t rejected by PLoS ONE because they are all unsound. They don't end up being published in PLoS ONE for a number of reasons. I've copied below the relevant two questions (164 and 165, 23 May 2011 session) from the UK House of Commons Science and Technology Committee report ‘Peer Review in Scientific Publications’ (link in your post above) and the responses from Mark Patterson, who was at that time the Director of Publishing at PLoS.<br /><br />"Q164 Roger Williams: Thank you very much. These questions are directed very much to Dr Patterson but not solely to him if others want to come in. I think your journal [PLoS ONE] publishes 69% of all submitted articles. Does that mean the other 31% are technically unsound? <br />Dr Patterson: You are correct that it is about 69%, but that doesn't really mean we reject the other 31%. Some of them are "lost" in the sense that they may be sent back for revision—maybe 5% to 10% are sent back for revision[1]—and the others are rejected, as they should be, on the grounds that they don't satisfy technical requirements. We have done some work to look at the fate of those manuscripts. We did some author research in the last couple of years and we have seen that, in both cases, according to the authors' responses, about 40% of rejected manuscripts have been accepted for publication in another journal. There are probably several reasons for that. One is that some of them will have been rejected by PLoS ONE because they are hypotheses or perspectives and are out of scope, or something like that. We publish original research in PLoS ONE, so that is fair enough. They end up being published somewhere because there are appropriate venues. Other authors may have gone away and chanced their arm at another journal and got through their peer review process.<br /><br />Q165 Roger Williams: Is that without being refined? <br />Dr Patterson: That we don't know. They may have been revised. As the PLoS ONE process isn't perfect, another chunk will have been rejected inappropriately. We know there are some such articles. The academic world reviewers tend to get in the mode of peer review but we are doing something different and we have to try to get that message across. So there will be a small batch that is rejected inappropriately.<br />1 Note by witness: ... and are not resubmitted. This is an important clarification because the vast majority of articles are sent for revision and are ultimately resubmitted."<br /><br />The evidence as given is slightly ambiguous. Maybe someone from PLoS ONE can step in and provide clarification or updated stats?Irene Hameshttps://www.blogger.com/profile/12425767180084509699noreply@blogger.comtag:blogger.com,1999:blog-4503292949532760618.post-68051563968187953842012-03-22T11:20:53.350-07:002012-03-22T11:20:53.350-07:00Sorry, ignore that question. I found it on page 1...Sorry, ignore that question. I found it on page 138 of the PDF you referenced.Mike Taylorhttps://www.blogger.com/profile/06039663158335543317noreply@blogger.comtag:blogger.com,1999:blog-4503292949532760618.post-36218149462607953632012-03-22T11:16:11.863-07:002012-03-22T11:16:11.863-07:00"PLoS ONE publishes every submission that mee..."PLoS ONE publishes every submission that meets its criteria of technical soundness. Nevertheless, over 40% of the papers it rejects as unsound are published in other peer reviewed journals."<br /><br />That is a fascinating statistic! What is its source?Mike Taylorhttps://www.blogger.com/profile/06039663158335543317noreply@blogger.comtag:blogger.com,1999:blog-4503292949532760618.post-48178567841069236382012-03-22T05:14:30.416-07:002012-03-22T05:14:30.416-07:00In your last two paragraphs, you describe how the ...In your last two paragraphs, you describe how the published content is getting more and more dynamic, which also means that there will be some sort of (or at least option for) review of the intermediate steps.<br /><br />If the publishing of new research findings were then to be embedded into an encyclopedic context, a publication would effectively be just an update to a set of interrelated encyclopedic articles. As demonstrated in <a href="http://www.youtube.com/watch?v=kYKIGnF7-DM#t=13m20s" rel="nofollow">this talk</a> (and later in its discussion), that would actually be more easy to review than the 10-page items that we now routinely inject into the publishing system.<br /><br />I am also working on a platform that would provide such an encyclopedic context to (BOAI-compatible) publishing - a brief sketch of the idea is <a href="http://www.rockethub.com/projects/3755-transforming-the-way-we-publish-research" rel="nofollow">here</a>.Daniel Mietchenhttps://www.blogger.com/profile/05040094362028176756noreply@blogger.comtag:blogger.com,1999:blog-4503292949532760618.post-41688616617751102762012-03-22T02:58:58.920-07:002012-03-22T02:58:58.920-07:00I'd be interested to know what evidence suppor...I'd be interested to know what evidence supports the claim that "over 40% of the papers [PLoS ONE] rejects as unsound are published in other peer reviewed journals"caseybergmanhttps://www.blogger.com/profile/03950336432385222376noreply@blogger.comtag:blogger.com,1999:blog-4503292949532760618.post-15895997715664551102012-03-21T18:43:26.624-07:002012-03-21T18:43:26.624-07:00"PLoS ONE publishes every submission that mee..."PLoS ONE publishes every submission that meets its criteria of technical soundness. Nevertheless, over 40% of the papers it rejects as unsound are published in other peer reviewed journals". Where does that 40% number come from? thanksAlexhttps://www.blogger.com/profile/03302812592407176747noreply@blogger.comtag:blogger.com,1999:blog-4503292949532760618.post-64322105847392936442012-03-15T16:39:13.192-07:002012-03-15T16:39:13.192-07:00Why limit ourselves to pre/post publication? Many ...Why limit ourselves to pre/post publication? Many recent scientific frauds could have been caught if scientists and companies that funded the research were required to submit their hypotheses and methods before they even conducted the experiments. It is only after seeing the results that bad scientists jump through hoops to adjust interpretations, leave-out or change results to make their failed experiments look better. Obviously, in the face of explicit fraud this won't solve the problem, but it's certainly another tool for suspicious reviewers.Bjorn Rochehttps://www.blogger.com/profile/17072425815152893296noreply@blogger.comtag:blogger.com,1999:blog-4503292949532760618.post-9077953754506134772012-03-14T17:27:58.743-07:002012-03-14T17:27:58.743-07:00The best online discussion forums I follow don'...The best online discussion forums I follow don't bother with any sort of complex, automated reputation system. They simply have good moderators, who set the tone and ground rules, remove disruptive or unproductive comments (or deprecate them in various ways), and help direct the conversation in useful ways. They often are also the ones who write or accept the original postings that are then made available for comments.<br /><br />There's some overhead involved with this, but nothing that should be overwhelming, Generally speaking, it takes less effort to review papers than to write them, and in turn less effort to screen reviews/comments than to write them. Basically, the moderators correspond to the "editors" of the traditional system, just as commenters correspond to the "reviewers" of the traditional system.John Mark Ockerbloomhttps://www.blogger.com/profile/00798089462412042209noreply@blogger.com