By Author
  By Title
  By Keywords

May 2014, Volume 64, Issue 5

Original Article

Trend over time; A structural review of articles published in Journal of Pakistan Medical Association from 1953-2009

Sina Aziz  ( Associate Editor, Journal of Pakistan Medical Association, Karachi, Pakistan. )
Haris Riaz  ( Intern, Journal of Pakistan Medical Association, Karachi, Pakistan. )
Syed Wajahat Kamal  ( Intern, Journal of Pakistan Medical Association, Karachi, Pakistan. )
Mohammad Kashif Riaz  ( Member Editorial Board, Journal of Pakistan Medical Association, Karachi, Pakistan. )
Kiran Ejaz  ( Intern, Journal of Pakistan Medical Association, Karachi, Pakistan. )
Hiba Tauheed  ( Intern, Journal of Pakistan Medical Association, Karachi, Pakistan. )

Abstract

Objective: To review the articles published in the Journal of Pakistan Medical Association from 1953 to 2009 and to assess the components of each article by a pretested proforma containing a checklist of items based on modified \'Strengthening the Reporting of Observational Studies in Epidemiology\'statement.
Methods: The retrospective study was conducted at the office of the Journal of Pakistan Medical Association, Karachi, from February to November 2010. A checklist of items in the modified \'Strengthening the Reporting of Observational Studies in Epidemiology\'statement was made after discussion among the authors of the study to finally include a revised checklist of 45 items instead of the checklist of 22 items of the statement. A total sample size of 370 was calculated. Simple randomisation was done for selection of articles from each year. For each article, major and minor items were documented. Data was fed into SPSS version 15.
Results: The scientific quality of reporting of most of the components of Introduction, Results and Discussion sections have improved progressively with time (p<0.001) whereas most components of Methodology have remained consistent.
Conclusion: A change in trend over time was observed over the study period in major and minor items of the articles in the Journal, showing improved reporting of various sub-components of articles. The modified \'Strengthening the Reporting of Observational Studies in Epidemiology\'statement provides a checklist that may be used to improve the quality of articles.
Keywords: Medical journals, Scientific writing, STROBE statement, Quality, Medical articles. (JPMA 64: 524; 2014).

Introduction

A tremendous amount of scientific literature is being published in biomedical journals annually, necessitating improvement in the quality of the manuscripts being published. This can be appreciated from the fact that the number of biomedical journals increased from approximately 1,000 at the beginning of the 20th century to over 20,000 in 1982, publishing over 2 million articles every year. The number of journals increases by two-fold every 10-15 years and 10-fold in 35-50 years.1
Writing skills are crucial in getting the scientific work published, without which even the most exhilarating data may not be accepted for publication by a peer-reviewed journal.2 Amongst premier medical journals, the Journal of American Medical Association (JAMA) emphasised the significance of improving the quality of paper-writing in early 1960s.2 Attempts have continuously been made to address any possible shortcomings in the medical literature and at present less than half of abstracts presented in scientific meetings are published as full text articles.3
The Journal of Pakistan Medical Association (JPMA) was the first Pakistani journal to be indexed in 1975 and since its very initiation has played a vital role in promoting the publication of high-quality research papers of both local and international authors. It is regarded as the finest Pakistani medical journal in terms of total number of citations.4 The Strengthening the Reporting of Observational studies in Epidemiology (STROBE) statement5 is a validated instrument comprising a checklist of 22 variables that ought to be described in any observational original article. The aim of the crrent study was to review a representative sample of original articles published in JPMA in the last 6 decades in terms of the components of a manuscript by the modified STROBE method.The ultimate objective was to determine a change in the trend over time, if any, and a possible detrimental effect on the quality of the manuscripts submitted over the years to JPMA. We also sought the applicability of the modified STROBE statement.

Material and Methods

The retrospective descriptive study was conducted from February to November 2010 at the JPMA office, Karachi, and comprised review of original articles published from January 1953 to December 2009. Starting from 1953, articles from every 5th year were included in the study. Since record of the year 1958 was not entirely available at the JPMA office, we omitted that year and included the next year instead.Thus, a total of 12 years (1953-2009) were included in the study.
Using convenience (non-probability) sampling we randomly selected half of the total number of articles published in the selected years. As such, of the 740 Original Articles published in the 10 selected years, 370 were included.
Only Original Articles related to observational studies in which the study design was cross-sectional, case-control or cohort were included as the original STROBE statement was developed only for observational studies. Original Articles related to blinded clinical trials, diagnostic accuracies, narrative and systematic reviews, case reports and correspondences were excluded. The selection procedure involved utilising the print versions of annual volumes of the selected years. The printed form was randomly opened, if the article was in accordance with the inclusion criteria, it was included. If not, then the next article fitting the inclusion criteria was selected.
A modified STROBE statement (proforma) was made after discussion among the authors in the JPMA office based on their local experience to finally include a revised checklist of 45 (major and minor) items (Figure-1).

This experience was based on review of the articles published in JPMA over past decades.
An Original Article was sub-divided into essential components comprising Abstract, Introduction/ Background, Methodology, Results, Discussion and References. These fundamental entities were assessed for the presence of different variables. Questions pertaining to the assessment of Abstract included the title (whether concise or not), use of abbreviations in title, whether Abstract was structured or un-structured and the use of key words. The Introduction was considered in terms of presence or absence of background of study, objectives and references of previous published work in that regard.
The pertinent variables related to Methodology inculcated in the questionnaire were study design, setting, participants, inclusion and exclusion criteria, description of variables assessed in the study, sources of data, efforts undertaken to reduce any potential bias, calculation of sample size, the sampling technique employed, use of qualitative and quantitative variables, description of statistical methods and programmes used for data entry and analysis and whether ethical approval from the relevant institute was sought or not.
Results were elaborated in terms of participants included, use of their demographic data, and description of study participants in terms of clinical and social profiles, numbers in terms of outcome measures, main results (unadjusted and confounder adjusted), other analysis in form of groups and sub-groups, use of relevant units and whether the data described in text has been re-elaborated in tables. Figure legends whether they were present or not were also checked.
The sub-components of Discussion included were concept of the study whether the findings in the form of key results had been mentioned or not, comparison with other similar published articles in terms of the content matter of the article, interpretation, generalisation, conclusion, recommendations and acknowledgements.
The data of the above-mentioned items was entered as Y (i.e. present, short form of 'yes'), N (not present, short form of 'no') and NA (not applicable). For each article, major and minor items were documented. Data was fed into SPSS version 15. Descriptive analysis was ascertained and the outcome measures were based on whether the trend of reporting a particular variable in the study had increased over time. The results were tabulated separately for the components in the Original STROBE statement versus the ones included only in the Modified STROBE statement.A p value less than 0.001 (p<0.001) was considered significant. The trend was also analysed graphically by demonstrating the frequency of reporting of different variables with duration.

Results

A total of 370 Original Articles were reviewed and their results were categorised as those related to components of Original STROBE statement (Table-1)

and those that were not a component of STROBE (Table-2).


Majority of articles bore a concise title and no statistical difference was noted among different year groups. None of the articles from 1953 to 1989 contained structured abstracts as the journal did not require a structured abstract prior to 1989. Whereas, 19/53 (35.8%) articles from 1990 to1999 and all 164 articles from 2000 to 2009 contained structured abstracts (p<0.001).
In articles selected from the 1950s, references from pertinent studies in the Introduction were described in just 15% of papers increasing to 97.5% in the last decade (Figure-2).


We assessed Methodology in 14 sub-components. The frequency of reporting significantly improved in just 5 of the components over the last 6 decades. These included mentioning the duration, setting, data sources, qualitative variables and statistical methods in the study.
The Results included details pertaining to the participants and their pertinent demographic details, descriptive data and the main results (p<0.01) (Figure-3).


We assessed Discussion in terms of 7 sub-components of the modified STROBE statement and found that the concept of the study being described, key results, comparison with other local and international studies published on a similar topic, interpretation, generalisability and limitations had shown an improvement in the frequency of reporting (Figure-4).


Among other variables assessed, mentioning of references in Vancouver style and using numbers in manuscript to indicate relevant references improved significantly with time.

Discussion

There has been a considerable effort in promoting uniformity amongst published papers. International Committee of Medical Journal Editors (ICMJE) in 1978 proposed guidelines for uniform submission of manuscripts which were adapted by many biomedical journals across the world. These guidelines were revised in 1997 and again in December 2013. Several guidelines have also been formulated based on a particular study design. These include the CONSORT (Consolidated statement for reporting of trials)5 for clinical trials, STARD statement (Standards for the reporting of diagnostic accuracy studies)6 and the STROBE statement.7 Several articles have shown an improvement in the quality of published articles after the formulation of these statement.5-10 Despite the improvements, it is emphasised that the studies should focus on accurate and improved reporting to overcome the shortcomings in published health researches.7
Besides international efforts, the need to improve the standard of reporting studies and providing uniformity of manuscripts in terms of components at local level has also been increasingly emphasised.11-13 Specially, in the last few years, awareness has been increased by seminars, webinars and workshops conducted by universities and the College of Physicians and Surgeons.
Publications in peer-reviewed journals are one of the primary criteria of promotions and success in academic institutions. STROBE statement was developed by the authors as a reporting guideline in observational studies to enhance the quality of reporting studies and to ensure that the study\'s design, conduct and findings are clear and complete. It has been cited in a large number of studies. We used a modified STROBE statement based on our experience and examined the Journal of Pakistan Medical Association(JPMA) articles from 1953 to 2009.
To the best of our knowledge, this is the first study analysing the quality of Original Articles in terms of the content of articles published in any Pakistani journal over a time period using a checklist of items. This is based on literature search from PubMed (Medline) and Pakmedinet.com. Unfortunately, despite its importance, research is not given the deserved place in Pakistan, and hence the country accounts for only 0.04% of the total amount of research papers published in the world.12 There are a myriad of factors accounting for these low numbers.However, one way to increase the quality of publications is by improving the knowledge of the components of a manuscript and skills of manuscript writing. This is especially important in view of the fact that recent years have witnessed an increased output of research amongst various universities in Pakistan.14
Our study showed that the trend to report most of the components have increased progressively with time, suggesting improved reporting means, whereas other variables have remained relatively consistent over the past few decades. JPMA changed the instructions to the authors to organise Abstracts as structured from non-structured in 1989. Evidence suggests that structured Abstracts are more likely to contain the relevant findings of the studies and should be adapted by the biomedical journals.2,15,16 After the introduction of structured Abstracts in mid 1980s most of the biomedical journals have conformed to their usage.17
Most of the sub-components in Introduction, Results and Discussion have shown a significant increase in the frequency of reporting. The frequency of reporting the 3 sub-components of Introduction in the modified STROBE statement i.e. background, objectives and references, all increased with the passage of time. It is relevant to state that the changes in trends observed over time are not due to the STROBE statement as most of the articles included in the review were published well before the STROBE statement was first formulated. Also, to date JPMA does not endorse the STROBE statement, but the modified STROBE statement is likely to be included in the instructions to the authors.
For instance, objectives were mentioned in the Introduction of just 15% articles in the 1950s compared to 87.5% after year 2000. Indeed, in terms of importance, Introduction has been compared to a door into the house.18 Similarly, reporting descriptive data in Results increased from 10% to 83.5% in the same period, while description of key results in Discussion increased from 15% to 95.7%.
The study depicts that the trend in reporting several components in the Methodology has not changed significantly with the passage of time. Methodology of any paper is vital in justifying the relevance of a protocol designed and ensuring reproducibility of the results. This is supported by the International Committee pf Medical Journal Editors (ICMJE) guidelines which clearly state the necessity of clear description of inclusion and exclusion criteria, study design and appropriate details of participants.19 Lack of evidence of increased reporting does not necessarily signify that a component should not be retained in the modified STROBE statement, for instance whether or not an ethical approval was obtained to conduct a study from the Ethical (or Institutional) Review Board prior to the study has not been reported with significantly increased frequency for articles published in JPMA, but it is relevant to state that in the manuscript.
The modified STROBE statement can be adapted by authors to promote uniformity and consistency and to ensure that imperative components are not missed while the papers are being written and published. The results depict that the trend of reporting most of the components have increased progressively with time. However, this statement is basically aimed at enhancing the writing skills only and is not meant to design an observational study protocol in accordance with the original purpose for which the authors conceived the statement.20 The study depicts the trend of observational articles published in JPMA only and cannot be generalised for other local and international journals. Of the over 50 peer-reviewed biomedical journals, JPMA is one of the 4 journals indexed by Medline, hence it may not be representative of the quality and trend in general of the medical manuscripts published from Pakistan.

Conclusions

Reporting of most of the components and sub-components of the Modified STROBE statement in articles published in JPMA increased progressively with time, suggesting an improvement in the quality of reporting of structured components of a manuscript based on the statement. However, most aspects of Methodology need to be better described while writing scientific papers. The Modified STROBE statement might be used by the authors to ensure that the imperative aspects of the scientific work are not missed while reporting their work in papers intended for publication.By developing the statement, the authors believe that the checklist could be included in the instructions meant for authors to help them improve consistency in reporting observational articles for JPMA and possibly in other peer-reviewed journals.

References

1. Lock S. Information overload: Solution by Quality? BMJ 1982; 284: 1289-90.
2. Taddio A, Pain T, Fassos F F, Boon H, Ilersich AL, Einarson TA. Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association. CMAJ 1994; 150: 1611-5.
3. Wenzel V, D√ľnser MW, Lindner KH. [How do I write an original article? An introduction for beginners]. Anaesthesist 2007; 56: 828-36.
4. Ullah M, Butt IF. Rating Pakistani medical journals using the principles of citation analysis. Health Info Libr J 2008; 25: 50-4.
5. Moher D, Jones A, Lepage L; CONSORT Group (Consolitdated Standards for Reporting of Trials). Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA 2001; 285: 1992-5.
6. Smidt N, Rutjes AW, Van der Windt DA, Ostelo RW, Bossuyt PM, Reitsma JB, et al. The quality of diagnostic accuracy studies since the STARD statement: has it improved? Neurology 2006; 67: 792-7.
7. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 2007; 335: 806-8.
8. Plint AC, Moher D, Morrison A. Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust 2006; 185: 263-7.
9. Prady SL, Richmond SJ, Morton VM, MacPherson H. A systematic evaluation of the impact of STRICTA and CONSORT recommendations on quality of reporting for acupuncture trials. PLoS One 2008; 3: e1577.
10. Moher D, Ocampo M, Altman Det al. Citing the CONSORT Statement and Explanation and Elaboration Paper: What\'s it all About? Vancouver, Canada: 6th International Congress on Peer Review and Biomedical Publication, 2009.
11. Riaz H, Rehman A. Inception of Journal of Pakistan Medical Students (JPMS): From Dream to a Reality. J Pak Med Sci 2011; 1: 1-2.
12. Mufti SA. Capacity Building in Bio-Medical Research in Pakistan. Commission on Science and Technology for Sustainable Development in the South. 1st ed. Islamabad: M/S Kamran Printers, 2003; pp 57-63.
13. Rohra D. Let\'s build our own International Journal. J Pak Med Sci 2011; 2: 65-6
14. Higher Education Commission. Islamabad: Expert Systems. Research Output by Universities of Pakistan. (Online) (Cited 2011 April 2). Available from URL: http://www.hec.gov.pk/ PublishingImages/HEC%20final%20(3).jpg.
15. Hartley J. Current findings from research on structured abstracts. J Med Libr Assoc 2004; 92: 368-71.
16. Hartley J. Is it appropriate to use structured abstracts in social science journals? Learned Publishing 1997; 10: 313-7.
17. Ad Hoc Working Group for Critical Appraisal of the Medical Literature. A proposal for more informative abstracts of clinical articles. Ann Intern Med 1987; 106: 598-604.
18. Setiati S, HarimurtiK. Writing for scientific medical manuscript: a guide for preparing manuscript submitted to biomedical journals. Acta Med Indones 2007; 39: 50-5.
19. Provide title as given link was not responding. Available at URL: http://www.icmje.org/manuscript_1prepare.html (Accessed on 10th March 2013).
20. Costa BR, Cevallos M, Altman DG, Rutjes AW, Egger M. Uses and misuses of the STROBE statement: bibliographic study. BMJ Open 2011; 1: e000048.

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: