Characteristics of Quorum review
Characteristics of Quorum Review
So far, the current peer-review process has been frequently criticized for a lack of clear criteria and systematic approaches in its review. The best way to achieve a quality peer review is that emphasis should be placed on the most updated evidence review. The evidence-based, impartial and systematic approach is called as evidence-based peer review, which is similar to a concept of evidence-based decision making process. Thus, the use of a standardized protocol specific to the purposes of the study has been recommended from reporting guidelines and checklists of most academic papers, as noted in the following examples: (a) Consolidated Standards of Reporting Trials for Randomized Controlled Trials; (b) Transparent Reporting of Evaluations with Non-randomized Designs for Non-randomized Clinical Trials; (c) Strengthening the Reporting of Observational Studies in Epidemiology for Cross Sectional, Case–control and Cohort Studies; and (d) Preferred Reporting Items Systematic Reviews and Meta-Analyses. For example, a journal asks authors to submit a standardized checklist indicating what items are contained in a manuscript before it is to be published, and this approach helps reviewers to determine the quality of the manuscript. Herst et al. investigated the wider usage of online reviewer instructions designated by 116 academic journals. They observed that the reporting guidelines were mentioned in a half of the reviewer instructions, but their real value to improve the manuscript quality has yet to be fully perceived Quorum’s review establishes its standardized protocol in the form of a checklist that is incorporated into the review report that is specific to each study design. This way, evidence-based screening criteria may be provided to reviewers, and authors may have better awareness of the most updated guideline so that any omitted items in the manuscript can be easily identified.
Studies have revealed that a great number of researchers (90%) cite the improvement of manuscript quality as the main element of effective peer review. Some authors said the journal’s peer review fails to add the value to the manuscript quality, whereas others responded that peer-review process brought highly positive changes to the manuscript before publication. Most authors want to receive more favorable assessment of the manuscript from peer reviewers, but resist the latter’s superficial review at the same time. There is wide consensus that receiving vague or negative comments from the reviewers can be very discouraging to all authors. In the event that peer reviewers’ comments are nothing but ‘seal of approval’ or center around ‘snapshot verdict’ persistently, these comments will put authors to be in confusion and consequently, authors cannot find any clues for significant improvement in their own manuscripts.
It is imperative that peer reviewers’ comments be actually helpful to the authors who may in return contribute to the manuscript quality. Most authors are often remindful of the fact that they were greatly impressed by reviewers’ constructive comments. Peer reviewers should serve as laboratory colleagues, project partners or even as an adviser. Peer reviewers’ primary mission is to help fill the gaps and fix errors that authors might have previously overlooked, and reviewers’ ongoing review for more scientific rigor and constructive feedback on insufficient or feared points of the proposed study would be of much help to the authors. Peer reviewers’ comprehensive comments about the strengths and weaknesses of a paper, which can be clearly perceived by the authors, would benefit them to proceed further towards publishing. In this context, Quorum’s informative review refers to peer reviewers’ objective, insightful and detailed comments that will help authors to improve the manuscript quality.
Peer reviewers justify their own opinion based on a constructive assessment of the value of the manuscript. Preferably, peer reviewers should be specific in their comments by providing evidence with appropriate references to substantiate the key statements. If this is the case, authors’ emotion could be settled down without triggering uncomfortable friction with peer reviewers. Evidences should be provided clearly and concisely, coupled with possible solutions when problems arise. If the problems are associated with specific references, it is feasible to mention the corresponding manuscript. This process may directly lead to better outcome of the study by inducing the authors’ motivation.
Quorum’ review has introduced an approach to reflect principal points in peer-review process in a systematic manner. As a guideline to review a manuscript, review template is used to reinforce the detailed elements in the review process in a stepwise manner. The template is also a form of checklist that deals with important issues in the manuscript and can give reviewers an idea about review opinion. The template provides a basic format to help enable reviewers to conduct their systematic review of the manuscript and answer important questions. This way, possible benefits include the followings: 1) reviewers ensure better transparency in their peer-review process, 2) a higher level of manuscript quality can be expected, 3) authors are informed of what items or contents are assessed by reviewers, and 4) authors may expect the improvement of the manuscript quality in the process of preparing new manuscripts.
This form of approach is also effective in error detection. The fact that peer reviewers have failed to reduce errors has been demonstrated in a number of previous studies. In a study conducted by Godlee and colleagues, test papers with 8 deliberate errors inserted were sent to a total of 420 reviewers who were asked to identify the errors. Reviewers found an average of two of the eight errors, but anyone did not identify more than 5 errors with 16% failing to identify any. Even in the effective use of peer review, errors are commonly found in the majority of published journals. For this reason, Quorum’s review assures that any errors in a manuscript can be identified by reviewers using the template, followed by cross identification of monitoring editors through re-review process.
The Public Library of Science (PLoS) describes the function of peer review by separating technical assessment from impact assessment. In general, technical assessment is objective and gives a reliability to post-publication outcomes, whereas impact assessment is subjective and its role has a tendency as being less distinct. Of these, the value of technical assessment is hardly problematic. Dr. Michaela Torkar, editorial director at BioMed Central, says, “To think of scientific soundness is straightforward, because this constitutes a basic aim in peer-review process. Further, there is evidence that many authors feel that peer review plays a role to improve the quality of their manuscripts published in the journal” [8:40].
Problems are often found in impact assessment. The assessment of any impacts or perceived importance of the study before the manuscript is published in the journal always requires subjective judgment and mistakes ensue inevitably. When the value of impact assessment is evaluated, a number of questions are raised, that is, “How can editors or reviewers of the journal assess the outcomes of a specific study?” There is a great deal of variability across reviewers in the issue addressed, and academic achievements are not immediately known to the public. Therefore, academic circles have advocated a new approach in peer-review process, to this effect that reviewers should focus on technical assessment of the manuscript before publication, and its impact assessment should be conducted after publication. With this in mind, Quorum’s review is conducting its peer review based on technical assessment that is objective and ensure the quality of the manuscript instead of subjective impact assessment.
Many studies suggest the presence of systematic bias in peer review, such as author’s opinion bias, prestige bias and geographical information bias [8:74]. The transparency in peer-review process is likely to bring a decisive impact to the systematic bias. Three principal forms exist to induce transparency in the review process: single-blind, double-blind, and open. In double-blind review, the identity of authors and reviewer is not known to each other. In the case of single-blind review, reviewers are unknown to authors, but the identity of authors is known to reviewers. In addition, under open peer review, the identity of authors and reviewers is known to each other. Double-blind review is a form of review to reduce bias. According to a survey, the majority of respondents (71%) cited double-blind review as the most effective form of review to induce transparency in peer-review process, followed by single-blind review (52%), post-publication review (37%) and open peer review (27%) [PRC]. Although double-blind review is the most preferred, the real problem is that author’s identity can be easily identified from references contained in the manuscript or other clues [13]. Notwithstanding this, Quorum’s review adopts double-blind review to ensure significant reduction of biases that may occur in peer-review process.