BACK: Criteria 9 < Criteria 10 > NEXT: Criteria 11
When reviewing manuscripts, consider whether the authors include the limitations of their study or their theoretical analysis. The descriptions of the limitations may be concise, but they should be accurate and reflect the most important limitations of the study or analysis. Reporting the limitations of a study is a key aspect of honesty and integrity. Reporting the limitations of one’s study or theory does not invalidate the whole manuscript; it simply lets readers know whether and how they can make use of the research
findings or theories presented in the paper.
When discussing the limitations of an empirical study, authors should identify factors that could limit the study’s validity, reliability, dependability, or generalizability. The specifics of how authors report limitations of their studies depend on the type of research that they are reporting (for instance, quantitative, qualitative, or theoretical research). It is important to assess research according to the expectations for the particular type of research. A small qualitative study, for instance, should be evaluated according to the standards expected for a qualitative study, and not as if it were a large, quantitative study.
For quantitative research, authors should consider factors such as sample size, sampling bias, inability to complete data collection as originally planned, shortcomings of the particular form of statistical analysis, and the existence of variables that could bias the findings. If they are using an instrument or survey for data collection, they may comment on the validity and reliability of the research tool, including whether it has been normed for specific ethnic or cultural populations. Some research tools depend on self-report, so authors could identify limitations related to self-reporting (e.g., people who may not accurately share certain information due to embarrassment or stigma).
Although qualitative research does not require large, random samples, it is important to identify how research participants were included and how the nature of the sample may affect the dependability and the transferability of the findings to populations that are different from the participants in another context that the reader is considering. You may also comment on whether the authors did not follow certain qualitative research protocols for ensuring the dependability of the research (for instance, having a second person review transcripts of interviews to verify the accuracy of the themes identified by the initial researcher). To consider the transferability of the study, consider whether the author provides a rich (or detailed) description of the research sample. This allows the reader to consider the source of the data and how the findings may or may not apply to different contexts.
When an author describes a paper as an “exploratory study” or a “pilot study,” consider whether the research findings merit publication. Authors should provide their rationale for publishing the exploratory or pilot study, rather than waiting to publish their main study (e.g., a pilot study to validate a research instrument might be used by other researchers even before the original researchers complete their main study).
For theory-based articles, authors should describe any limitations of the theory that they are presenting or applying. When discussing deontology, teleology, virtue ethics, or another approach to analyzing ethical issues, authors should discuss any potential problems, gaps, or biases in the theory or approach to ethical analysis and decision making. For instance, some theories or approaches may apply well with one cultural group, but they may not apply as well with another cultural group. The author’s discussion of ethical issues should be balanced and honest.
Although authors may highlight the strengths of their research or theoretical analysis, it is also important for them to describe the limitations. Some authors may use the limitations section to inform their conclusions, including options for future research or analysis that could address the limitations of the current study.
Thank you for your dedication as a reviewer for this publication.