CLINICIANS’ ACCESS TO PEER-REVIEWED PROSTHETICS RESEARCH ARTICLES
BACKGROUND: Evidence-based practice (EBP) is an important cornerstone of responsible clinical decision-making, and by extension, of high quality care provision in prosthetics and orthotics. However, many clinicians have been reluctant to embrace EBP, citing barriers such as high costs and time demands that are associated with obtaining pertinent published evidence for individual care scenarios.
OBJECTIVES: The purpose of this study was to determine how accessible peer-reviewed research articles are to prosthetists who seek to implement EBP techniques into their clinical work without expending unreasonable amounts of time and money.
METHODOLOGY: Two approaches were utilized. An academic approach entailed a search through five peer-reviewed research journals, including the Journal of Prosthetics and Orthotics and Prosthetics and Orthotics International. A practical approach simulated a typical evidence search as it might occur in the field, using a number of different clinical questions to inform search terms in Google Scholar. The ratio of freely accessible articles was computed and compared for both approaches.
FINDINGS: Out of a total of 796 prosthetics-relevant articles published in the analysed journals over the last years, 600 (75.4%) were found to be accessible to the public without any cost incurred. The practical approach showed that, among the top twenty search results for each search query, on average 40% to 75% of articles were freely available.
CONCLUSIONS: A majority of pertinent research papers is already publicly available to anybody with internet access. Prosthetists would not be required to invest in journal subscriptions or have to spend time at an academic library to obtain these articles. However, it is a concern that evidence-based decision making may be flawed if not all literature on a topic is considered. There is still a substantial fraction of articles that are not freely available to practitioners, motivating a continued expansion of open-access policies in the field.
Medical care, including the provision of prostheses to people with limb loss, needs to be informed by scientific evidence. In order to reduce the risk of making decisions that are not ideal for a given patient and scenario, clinicians have to find out what the latest research on the topic suggests. While finding, reading, and appraising the published literature is an important aspect of Evidence Based Practice (EBP), it can be a time consuming task that interferes with the other duties of busy clinicians. Searching for literature over the internet can reduce the associated burden, as no physical copies of research articles have to be retrieved and stored anymore. However, not all research is freely available online, and accessing the full text versions of some papers can incur substantial fees.
We wanted to find out how much of the scientific literature in the area of limb prosthetics is accessible free of charge on the internet. To that end, we first used the official websites of five established journals to search for prosthetics related articles and to check how many of them were free to read. Next, we used a search engine that is specialized in research articles (Google Scholar) to see how many of the offered search results point to full papers and how many are merely offering abstracts or otherwise abbreviated version of a full paper. Overall, we found that about 75% of research is freely accessible, which we believe to be sufficient for EBP in most cases.
ARTICLE PDF LINK: https://jps.library.utoronto.ca/index.php/cpoj/article/view/30009/22881
Accessibility of peer-reviewed research articles has been cited as a barrier that prosthetists and orthotists must overcome in order to use evidence-based practice (EBP) in their clinical practices (1-4). Short of frequenting a local university library that may or may not hold the publications of interest and/or allow some access by the general public, there are four basic ways by which practitioners may acquire peer-reviewed research articles: open-access or free-access articles (these are free to the public, usually online), paying for online versions of individual articles in peer-reviewed journals, subscribing to peer-reviewed journals that contain articles of interest, or joining an organization that includes journal access as one of the membership benefits.
One example of an open access journal relevant to the field of Prosthetics and Orthotics (P&O) was the Journal of Rehabilitation Research and Development (JRRD), which focused on veteran-centric rehabilitation research, including articles related to prosthetics, orthotics and other assistive technologies (5). JRRD was funded by the US Department of Veterans Affairs to cover operation costs, but ceased publishing within the Rehabilitation Research and Development Service in March of 2017, referring authors to Public Library of Science (PLOS) instead (6).
The recently introduced Canadian Prosthetics & Orthotics Journal (CPOJ) is utilizing a more commonly found open-access model based on article-processing charges. Select open-access articles can also be found in the field’s specialized journals that utilize the classic membership/subscription model for revenue, including the Journal of Prosthetics and Orthotics (JPO) and Prosthetics and Orthotics International (POI). JPO offers open-access to articles that are two years or older, and POI offers open-access to all articles three years and older as well as select recent articles through Sage Publishing. Several heavily cited papers focusing on prosthetic research have also been published in journals that are not specialized in P&O (7). These journals include Gait and Posture (GP) or the Archives of Physical Medicine and Rehabilitation (APMR).
In studies that focused on potential barriers P&O practitioners face when incorporating EBP into their own clinical practices, lacking accessibility has often been defined as any kind of cost incurred when attempting to access the articles desired. This cost includes membership requirements, individual article fees, and/or subscription fees that may be encountered when a clinician attempts to research a clinical question.
However, it has not yet been quantified how prohibitive these costs actually are, and whether the growing open-access movement in the past decades has helped mitigate the problems associated with costs of EBP. The purpose of this study was therefore to determine how much of the published evidence in a sub-field of prosthetics research is freely accessible to practitioners. This information is useful in discussing the actual effect that cost-barriers have on prosthetists’ ability to perform EBP in their clinical practice.
We assumed that having access to half of the published body of knowledge would enable to perform EBP in most cases, especially when there is a general consensus within the literature. Based on this deliberation, we hypothesized that the amount of currently available open-access articles in the field of limb prosthetics exceeds this 50% threshold. We further hypothesized that the share of relevant open-access articles increases over time.
While our protocol entailed the search and review of a large number of literature sources, it was not with the intent to evaluate or summarize the scientific contents of those papers. Rather, all eligible publications were tallied by whether they were freely accessible online or not (8).
This study utilized two separate approaches to investigate the hypothesis. An academic approach had the aim to determine how many of the relevant (prosthetics) articles in a given selection of journals were open-access articles. The second was a practical approach aimed to simulate what a prosthetist may utilize in daily practice by searching for evidence on a specific clinical topic in a scholarly literature database. In order to also estimate the tendency of changes over time, this practical approach was repeated after a one-year interval. The previously determined criterion of an accessibility percentage of 50% was used for interpretation of the findings across approaches.
Of all the journals with P&O content, five of the most commonly read were used as sample for this study: JPO, POI, JRRD, APMR and GP. The number of prosthetics articles available in each journal was determined using the search terms “prosthetic,” “prosthetics,” “prosthetist,” “prosthesis,” and “prostheses” as keywords in Scopus, one of the largest abstract and citation databases. The five journal names were entered under the filter category “Source title”. The search was conducted in late March of 2017 and was narrowed to publication years from 2007 to 2016 (i.e., only articles and reviews published between January 1, 2007 and December 31, 2016 were included in the initial selection). Search results were scrutinized to determine whether a limb prosthesis was part of the study design, and only articles were included that either focused on the design or fabrication of a prosthetic device or that otherwise utilized the device in a significant manner in the research. For example, if a prosthesis was merely mentioned but was not substantial for the study protocol at all, the respective paper was excluded from analysis. Articles that focused on prosthetic implants (other than osseointegrated limb prostheses) or on neuroprostheses were also excluded from this study, as it is unlikely that papers on those topics have great relevance in the realm of EBP in limb prosthetics at this time. For the same reason, articles that focused exclusively on the surgical implantation of osseointegrated prosthesis and not any of the follow up care were also excluded.
For the first round of filtering, article abstracts and titles were scanned for relevance to the study. During the second round of review any articles that were questionable for relevance were reviewed in full to make sure they met the inclusion requirements. The remaining articles were then looked up on each journal’s publishing website to determine which articles were accessible to the public as of March 27, 2017. The ratio of freely accessible articles to the total number of articles found was then computed separately for each journal.
In order to simulate the typical process a prosthetist goes through when gathering evidence to address a clinical problem, three random clinical questions were formulated and appropriate search terms (Table 1) were entered in Google Scholar, a search engine that searches scholarly literature and academic resources (9). Formulation of the question was intended to reflect both recommended approaches (e.g., using a PICO – Population, Intervention, Comparison, Outcome – format) and less structured questions that may be posed to clinicians by their patients. Search terms were derived from the main terms contained within each question. To keep congruency between the two approaches, filters were used to limit the search results to only articles from peer-reviewed journals that were published between January 1, 2007 and March 27, 2017.
Table 1: Research questions and search terms used in Google Scholar. (Please see the PDF file)
The accessibility of the first 20 search results for each question was determined by clicking on the original hyperlinks that are provided by the Google Scholar website. In the event that no such link was listed or that the listed link did not lead to a full-text version of the article in question, the article was recorded as “not accessible”. While there may have been other avenues to tracking down such articles online through a variety of different websites (e.g., an article of which only the abstract is available on the website ScienceDirect may be – unbeknownst to Google Scholar – posted in full on the private website of the article author), it is arguably least complicated to just click on the links provided by Google Scholar. A busy clinician with limited time for EBP will in many cases prefer this expedited approach to evidence gathering.
The initial search in Scopus turned up 1,042 articles. After title, abstract, and full-text screening, 796 total articles were included in this analysis. With nearly 300 articles, POI had the most prosthetics related articles, followed by JRRD, JPO, GP and APMR respectively. The amount of total free articles per journal followed a similar trend, with POI containing the largest number of accessible articles, closely followed by JRRD and JPO, then APMR and lastly GP. The ratio of accessible papers was greatest in JRRD, followed by APMR, JPO, POI and GP. Overall, about three out of every four articles in these journals were freely accessible by the public (Table 2).
Table 2: Number of reviewed journal articles (from 2007 through 2016). (Please see the PDF file)
The total number of freely accessible articles found through the three searches in 2017 was 24 out of a total of 60 obtained articles, with an average of eight accessible articles per search, with a standard deviation of one. Out of the articles collected, 16 (27%) were published in one of the five journals used in the academic approach.
Repeating the same searches (using the same filters) one year later showed that the number of freely accessible articles among the top search results had increased to 45 out of 60 (Figure 1).
Having free access to just a small part of relevant published research is a significant barrier toincorporating EBP into clinical practice. Based on our initial assumption that having access to at least 50% of papers would reasonably enable prosthetists to conduct EBP effectively and efficiently, our findings suggest that a sufficient ratio of research articles in prosthetics are indeed freely available to practitioners. Both our approaches agreed very well with each other that approximately three out of every four articles relevant to the field were accessible by an individual at no cost.
Figure 1: Changes in the number of freely accessible articles among the top 20 search results for Google Scholar searches between March 2017 and March 2018 (10). (Please see the PDF file)
Open access articles, especially in the specialized journals POI and JPO, can arguably be a valuable resource for prosthetists to utilize, given that these journals contain more clinically relevant research than other journals, like APMR or GP, that publish prosthetics research among research in many other areas. Approximately 57% of the accessible articles identified in our academic approach, were gathered from POI and JPO. Judged by this, the conditions for EBP in the fields of prosthetics and orthotics appear to be favourable compared to other health sciences.
While open accessibility of peer-reviewed articles through Google Scholar (the practical approach) was eventually found to be at around 75% as well, the same ratio was much lower when first investigated a year prior. In early 2017, only two out of every five peer-reviewed articles on average were accessible to an individual at no cost.
That open access to articles on Google Scholar almost doubled within one year has different possible explanations, including a change in composition of the top 20 of the search rankings. Higher-impact articles (i.e., articles that are cited more often) are ranked higher in the search result listings, and it is conceivable that articles that were referenced frequently over the past year have moved up and displaced less popular articles. This would be supported by the circumstance that articles that are freely accessible are by trend read (and potentially cited) by more people than comparable less easily accessible articles. The volume of full-paper articles that are being shared by their authors in online repositories, where they are freely accessible by the public, may have increased over time as well. This would have been encouraged if the copyright rules that impose restrictions on that practice have been loosened by some publishing journals. Irrespective of that, many dated materials become more freely available over time by trend (as detailed above), which may have played a role in the year-on-year differences as well.
Improvements in Google Scholar over the period between analyses appear to have led to a better selection of links to full-text papers. It was noted during the first data collection in 2017 that some of the Google Scholar links pointed to articles, already known to be open-access that could, however, not be accessed through the link provided by Google Scholar without a login or associated fee to the article. For example, a preview of a JRRD article, which is open-access, was found through a Proquest link that was given as a result in the Google Scholar websearch (11). Proquest is a search engine used to access journals, databases and ebook resources, that requires a login for access. As membership requirements were included in what was considered inaccessible, this known open-access article was marked as inaccessible based on the study methods.
It is possible that our results were affected by some limitations of this study. Only three clinical questions were formulated to inform the search term selection, resulting in a small and specific sample of data. A larger selection of search terms may bring about different results, especially if a topic is concerned that has only recently been widely investigated (i.e., has most relevant papers still subject to access restrictions) or has conversely not yielded much new research in many years (i.e., most papers fallen out of copyright protection). The top twenty articles that were included in our analysis were not filtered for relevance to the searched questions. This was done to increase replicability of the study, but may have led to some of the resulting papers not being responsive to the original question.
In the same sense, it may have been possible to identify more articles by including more than five search terms in the academic approach. Likewise, the exact phrasing of the clinical questions in the practical approach, which depends on the practitioner’s professional judgment and on the peculiarities of the individual case, may influence the selection of search terms and thereby the eventual search results. It should also be noted that the covered periods of reviewed articles were slightly different between academic and practical approach (ending with Dec 31, 2016 and with March 27, 2017, respectively). We believe that the resulting differences in the included sample of research articles would not have substantially altered our findings.
The search engine that is used will as well have an effect on the results. Not only do search engines employ different methods to determine which articles to display first, but not all search engines index all journals. For example, PubMed, a search engine used to retrieve data from MEDLINE, the National Library of Medicine journal citation database (12), has are not yet indexed JPO articles (13).
Accordingly, there are a few recommendations that prosthetists should consider when going through the process of implementing EBP into their clinical practices. Practitioners in our field should select scholarly search engines/literature databases that work best for the questions they wish to answer and be aware of the journals indexed in those databases. It is also worth considering that while an article may be inaccessible on one website, it may be freely accessible on another. Consulting the publishing journals’ websites can be helpful to determine a research article’s accessibility.
While strategies exist that can help prosthetists better search for articles and overcome accessibility issues, the effort to learn and employ those strategies is still part of the initially mentioned barrier that needs to be overcome to properly implement EBP. Nonetheless, compared with the pre-internet necessity of frequenting a university library to sift through physical copies of countless journals for a literature review, this small barrier should not discourage prosthetists or dissuade them from conducting a proper evidence search. It is possible that other barriers that are commonly cited as an impediment to EBP are not as substantive as they are perceived either. Future research may be recommendable that investigates the true effects of time constraints, clinical relevance of scientific publications, and lacking incentives have on P&O practitioners’ attitude toward implementation of EBP in their daily work.
We found that about 75% of research articles in the field of prosthetics are freely available online. Whether this is indeed a big enough ratio to facilitate EBP in most cases should be investigated in future research, utilizing a more accurate simulation of recommended practical approaches to EBP and analyzing the outcomes across a larger sample of cases and practitioners.
• Julie Burke: Conceptualization, study design, initial literature search and analysis, manuscript draft.
• Goeran Fiedler: Study oversight, secondary literature search and analysis, manuscript editing and revision.
DECLARATION OF CONFLICTING INTERESTS
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
1.Geil MD. Assessing the state of clinically applicable research for evidence-based practice in prosthetics and orthotics. Journal of Rehabilitation Research and Development. 2009; 46: 305-13. DOI:10.1682/JRRD.2008.02.0019.
2. Ramstrand N. Translating research into prosthetic and orthotic practice. Prosthetics and orthotics international. 2013; 37: 108-12. DOI: 10.1177/0309364612451268.
3. Ramstrand N, Brodtkorb TH. Considerations for developing an evidenced-based practice in orthotics and prosthetics. Prosthetics and orthotics international. 2008; 32: 93-102. DOI: 10.1080/03093640701838190.
4. Stevens PM. Barriers to the implementation of evidence-based practice in orthotics and prosthetics. Journal of Prosthetics and Orthotics. 2011; 23: 34-9. DOI: 10.1097/JPO.0b013e3182064d29.
5. Walters HN, Custis DL, Boren HG, Giannini MJ, Todd, Jr SP. Five Editorials. The Journal of Rehabilitation Research and Development. 1983; 20(1): 1-2.
6. JRRD to cease publication. The O&P EDGE. Northglenn, CO 80234: Western Media LLC, 2016. [Available at: https://opedge.com/Articles/ViewArticle/NEWS_2016-07-07_03] Accessed August 4, 2018.
7. Eshraghi A, Osman NAA, Gholizadeh H, Ali S, Shadgan B. 100 top-cited scientific papers in limb prosthetics. Biomedical engineering online. 2013; 12: 119. DOI:10.1186/1475-925X-12-119.
8. Burke J, Fiedler G. P&O Professionals’ access to peer-reviewed research articles. 44th Annual AAOP Meeting and Scientific Symposium. New Orleans, LA, 2018.
9. Baldwin VA. Using Google Scholar to search for online availability of a cited article in engineering disciplines. 2009. [Available at: https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1192&context=libraryscience] Accessed August 4, 2018.
10. Burke J, Fiedler G. Online Access to Research Papers - Changes over Time. American Orthotic and Prosthetic Association (AOPA) Assembly. Vancouver, BC, 2018.
11. Kahle JT, Highsmith MJ, Hubbard SL. Comparison of nonmicroprocessor knee mechanism versus C-Leg on Prosthesis Evaluation Questionnaire, stumbles, falls, walking tests, stair descent, and knee preference. Journal of Rehabilitation Research and Development. 2008; 45: 1. DOI: 10.1682/JRRD.2007.04.0054.
12. Factsheet. MEDLINE, PubMed, and PMC (PubMed Central): How are they different. 2018. [Available at: https://www.nlm.nih.gov/bsd/difference.html] Accessed August 4. 2018.
13. Miro RM, Lewandowski AL, Kahle JT, Mengelkoch LJ, Boone DA, Highsmith MJ. Bibliometric Analysis of Articles Published from 2009 through 2011 in the Journal of Prosthetics and Orthotics, Journal of the American Academy of Orthotists and Prosthetists. Journal of Prosthetics and Orthotics. 2013; 25: 201-8. DOI: 10.1097/JPO.0000000000000007.
ARTICLE PDF LINK: https://jps.library.utoronto.ca/index.php/cpoj/article/view/30009/22881
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles are permanently available online to the public without restrictions or subscription fees. All articles are free to be used, cited, and distributed, on condition that appropriate acknowledgment is included. Authors are the copyright holders of their original contributions and grant the Canadian Prosthetics & Orthotics Journal (CPOJ) a license to publish the article and identify itself as the original publisher. CPOJ articles are licensed under the Creative Commons Attribution 4.0 International License.