Performing a Quality Review
Peer review is an important extension of the scientific process. Peer review allows manuscripts submitted to a journal to be evaluated and commented upon by independent experts within the same field of research. The two goals are a fair evaluation of the merit of the work, and to provide a critique that improves the manuscript. Authors expect reviews to contain an honest and constructive appraisal, which is completed in a timely manner and provides feedback that is both clear and concise. The evaluation and constructive feedback generated from peer review helps authors improve their work and, importantly, allows the editor to assess the paper’s suitability for publication in the journal. Despite receiving much criticism and having acknowledged limitations, the peer-review process remains a widely recognized standard in terms of journal quality.
ASHA journals support the National Institutes of Health’s focus on rigor, reliability, and transparency in peer review. Reviewers are asked to assess the technical soundness of submitted papers, instead of evaluating them by more subjective measures, and the criteria used by reviewers to rate submissions are readily available to authors. ASHA journals strives to increase transparency and to evaluate content consistently by relying on clear criteria and widely adopted reporting frameworks, as described in this section.
Reviewing a Manuscript
Note
Editors and reviewers should ensure that the author has selected the proper article type for the submitted manuscript, and that the review template provided is appropriate for the manuscript. A reviewer should contact their editor if they believe the incorrect article type has been selected, or if they have any questions regarding the review template provided.
Once reviewers have been chosen and they accept their review assignment, they receive the reviewer guidelines and a form outlining the review criteria based on the article type. Reviewers use these criteria in determining merit. While the criteria may vary somewhat according to the type of article, in general reviewers will be checking that the work is original or new, that the study design and methodology are appropriate and described so that others could replicate what has been done, that the results are presented clearly and appropriately, that the conclusions are reliable and significant, and that the work is of a high enough standard to be published in the journal.
As a reviewer for the ASHA Journals, you will use the Editorial Manager peer review system to read and review the manuscript assigned to you. The bulk of your review will be framed by the score sheet provided in the online system for the journal. Please keep your comments as succinct as possible, listing strengths and weaknesses for the categories in the review template. Please do not provide line-by-line comments. Any accepted manuscript will go through a production process that includes copyediting.
In addition to the review criteria, reviewers are asked to consider the reporting frameworks appropriate for the manuscript content (see EQUATOR Network section below). ASHA journals strives to evaluate content consistently based on clear criteria and widely adopted reporting frameworks, such as those listed on the Equator Network.
In Editorial Manager, when you access the Review Assignments, you will see Action Links on the left-hand side. You will have the option to View Submission, Submit Recommendation, and Send Email. The Submit Recommendation tab contains the reviewer template for the journal. Please carefully read and follow the “Instructions for Reviewers,” paying close attention to your comments to the authors. When you are ready, click the Proceed link at the bottom of the template to enter your review.
EQUATOR Network
Reporting guidelines are tools for health researchers to use while writing manuscripts. They provide minimum lists of information needed to ensure a manuscript can be properly understood by a reader, replicated by a researcher, and included in a systematic review. They are also helpful for reviewers: if the information required by a reporting guideline is not included in a manuscript, then you cannot properly judge the quality of that study.
The ASHA journals endorse the use of reporting guidelines by both authors and peer reviewers. We encourage you to identify the correct reporting guideline for this manuscript. We encourage you to use this guideline to check whether the minimum information is included in the report, before commenting on the scientific merit and methodological quality of the work. The reporting guideline cannot be used to judge the quality of the methodology used in the study. However, if crucial information is not reported, you cannot judge the methodological quality.
In some cases, this process will be expedited by the submission of a completed checklist by the authors. This checklist will indicate the page numbers on which information on each guideline item is included.
If you find a manuscript does not include enough information to allow you to judge its methodological quality, you are welcome to end the review early and to simply indicate what additional information must be reported to allow a full review.
Once you have determined that the manuscript includes all of the information needed, use the reporting guideline to aid in your assessment of the manuscript. A reporting guideline cannot be used to directly judge the quality of the methodology used in the study, but does suggest the kinds of questions that should be considered when designing a study.
Common Study Types and Appropriate Guidelines: Some common study types and the appropriate guidelines are listed under Common Study Types and Appropriate Guidelines in the Authors Resource Center. If you cannot find an appropriate guideline there, search the full EQUATOR database.
Review Criteria and Reporting Guidelines
When conducting a review, editors and reviewers should evaluate a manuscript for
- Adequacy of the rationale for the study or paper
- Accuracy and scope of the literature review
- Appropriateness of research design, data analysis, and interpretation of results for research articles
- Organization
- Clarity
- Overall clinical or theoretical significance of the work
Clinical studies appearing in ASHA journals must meet recognized standards for reporting. Articles reporting randomized clinical trials must follow the Consolidated Standards for Reporting Trials (CONSORT), nonrandomized clinical evaluations must follow the Transparency of Reporting Evaluations of Nonrandomized Designs (TREND), and studies of diagnostic accuracy must meet the Standards for Reporting of Diagnostic Accuracy (STARD). Authors should find these standards useful as guides in designing and implementing their studies; however, the standards apply directly to the reporting of studies rather than to their implementation. Therefore, reviewers are encouraged to consider the reporting frameworks appropriate for the manuscript content when evaluating a paper as they provide a checklist for essential elements and information. Please see the EQUATOR Network section of the Author Resource Center for more information.
Reviewing Supplementary Material
When reviewing supplementary material provided with a manuscript, Editors and reviewers should consider the material’s significance, relevance, and clarity. The material should enhance the published article in a meaningful way, without being necessary for the article to be understood. Supplementary materials should not be used as a way to provide extraneous or unnecessary details about participants and methods.
When reviewing an example provided on video or audio, check whether a statement has been provided about the representativeness of the example (e.g., “in this ideal case….” or “in this case selected because it represents the mean performance of the group…”). Just as there are standards for published figures, there should be some level of quality expected for video examples.
If you cannot open a supplementary material file to review it, mention this fact in the confidential comments to the Editor upon returning manuscript feedback.
Comments to the Editor
You will have the opportunity to submit confidential comments to the editor. In the Comments for the Editor box, you may include a statement about your recommendation, or something which would not be constructive for the author, such as “I was really leaning towards reject, but I feel like there’s something valuable there. If the author were to significantly rewrite the paper I think it could be publishable at some point.” Some reviewers include a brief note for the editor in this box. Others leave it blank. Either one is fine, but please do not copy your comments to authors into this box.
Constructive Criticism
Constructive criticism and suggestions for changing the paper to improve the manuscript are welcomed. A review is more than a suggestion to revise, reject or accept. It should be meaningful. It should guide the author on what is good and what is not so good as you see it. As you complete your review, please keep the following considerations in mind:
- An opinion on the strengths and weaknesses of the manuscript is useful.
- Please make any recommendation for acceptance or rejection in the Comments to the Editor area, not in your comments for the author.
- Please maintain a tone of professional respect in your review. Criticisms should be put forth in positive ways, accompanied by specific suggestions for improvement whenever possible.
- Be objective, constructive, and specific. Provide feedback that improves the scientific merit of the manuscript and the communication of that science.
- Personal or derogatory language is to be avoided.
Revised Papers
When authors make revisions to their article in response to reviewer comments, they are asked to submit a list of changes and any comments for transmission to the reviewers. In the case of major revisions, the revised version is usually returned to the original reviewer if possible, who is then asked to affirm whether the revisions have been carried out satisfactorily. Otherwise, the editor may have enough information to render a decision without additional feedback from a particular reviewer.
Recommendation Types
Accept
Accept paper in its present form. Some minor copy-editing may still be required, but it will be caught at the copyediting stage so the authors need not submit a revision.
Minor Revision
The manuscript is generally good but requires minor content and/or editorial changes before its suitable for publication. Revised manuscripts do not require another review by the reviewers.
Major Revision
The paper contains one or more serious problems in substance or form, whose resolution might result in a generally acceptable manuscript. Resubmitted manuscripts typically are reviewed again by the Editor and reviewers. This category applies only to manuscripts that contain important information to begin with, whose flaws might be correctable.
Reject
The content, style, and/or preparation of the manuscript are flawed to the extent that it is unlikely that revisions can render the manuscript suitable for publication. The content of the manuscript might also be unsuitable or inappropriate for the journal.
Standards and Practices
Reviewer Anonymity
ASHA does not release reviewers’ identities to authors or to other reviewers. We ask reviewers not to identify themselves to authors without the editor’s knowledge. If they wish to reveal their identities, then they should consult with the editor and editorial administrator for the journal.
Editors and editorial administrators of ASHA journals must keep the identities of all reviewers of individual manuscripts hidden from authors, other reviewers, and the public. Identities of reviewers may be revealed to members of a publication’s editorial board or to ASHA staff as needed to solicit expert advice in special circumstances. In such cases, identities of a reviewer may also be made known to other reviewers of the same manuscript, provided that the consent of all affected reviewers is obtained. Reviewers must also maintain the confidentiality of other reviewers’ identities, as well as the reviews themselves, that are shared with them at any time.
An exception to the anonymity policy is made in the case of an open review in which the names of the authors and reviewers are disclosed to one another, provided that all reviewers (both editorial board member and, if applicable, ad hoc reviewers) are notified in advance of this practice.
This policy does not prevent the simple listing of all reviewers of a particular journal without reference to particular manuscripts for the purpose of acknowledgement, or the appearance of reviewers’ names in a composite database for use by the editors.
ASHA journals disapproves of any attempt by authors to confront reviewers or determine their identities. ASHA will neither confirm nor deny any speculation about reviewers’ identities and encourages reviewers to adopt a similar policy.
Confidentiality
Manuscripts submitted to ASHA journals are privileged information. They are confidential and must not be discussed with anyone other than the editor-in-chief and the assigned editor. Occasionally, a reviewer must consult with colleagues on some aspect of a paper, such as the statistical analysis. Such consultations should occur only with the editor’s permission and without providing the author’s identity or details of the manuscript’s content. After the final publication decision has been made, reviewers should destroy their copies of the manuscript.
For the purpose of training PhD students in the peer review process, a manuscript reviewer/mentor may engage a PhD student in the review process under the mentor’s guidance. The mentor bears full responsibility for the review. The PhD student will be bound by the same principles of confidentiality that govern the review process as a whole. It is the responsibility of the mentor to inform the journal’s editorial administrator and the editor of the mentee’s identity. The editorial administrator will enter the information into the manuscript administrative record.
If a doctoral student conducts a written review, the mentor must append the review to his or her formal review and mark the appended review as having been done by a doctoral student. Please review the full policy on confidentiality and privileged information.
What if You're Unable to Review
Sometimes you will be asked to review a paper when you do not have sufficient time available. In this situation, you should make the editorial office aware that you are unavailable as soon as possible. It is very helpful if you are able to recommend an alternative expert or someone whose opinion you trust.
If you are unable to complete your report on a paper in the agreed time-frame required by the journal, please inform the editorial office as soon as possible so that the refereeing procedure is not delayed.
Make the editors aware of any potential conflicts of interest that may affect the paper under review as soon as you recognize the conflict of interest. Please review the Ethical Considerations for Reviewers to determine if you have a conflict of interest.
What Will Happen to Your Review Comments?
Once you submit feedback, the editor will read your review along with any other reviews submitted for that particular manuscript. The editor may reference your comments in his/her comments to the author. The editor has the final decision on the manuscript as well as whether your comments will be forwarded to the corresponding author.
If an editor has concerns about your review comments, the review might be moved back to your Review Center for revision and resubmission.
As mentioned above, you will receive a copy of the decision letter for a manuscript on which you served. However, if revisions are requested by the editor, and you do not participate in the review of that revised manuscript, you will not receive a copy of that (final) decision letter.
Editors are encouraged to rate reviews on timeliness and quality. The timeliness rating is based on whether a review was delayed or not returned after a reviewer agreed to participate. The quality assessment rates whether the review was highly relevant, sufficient, below average, or not returned. Ratings in either area that are below our expectations may factor into whether a reviewer receives future invitations to participate in peer review, since it becomes a part of the reviewer’s peer review account. However, review ratings are neither communicated to the author nor stored in the reviewer’s ASHA account, if applicable. Only editors-in-chiefs, editors, National Office staff, and Editorial Manager employees can access reviews and rating results.
