MEASUREMENT OF THE CONSISTENCY OF PATELLA-TENDON-BEARING MODIFICATION USING CAD
Study design: Pilot study
Background: Computer aided design (CAD) is now commonly used in prosthetic clinical practice. To create a patellar tendon bearing (PTB) socket, further modification of the transtibial shape is required.
Objectives: To investigate the consistency of transtibial shape modification for a PTB socket design using CAD.
Methods: 13 transtibial models with marked anatomical landmarks were made, each linked to a fictitious patient history. Three clinicians were asked to complete modification for a PTB socket with suspension sleeve at weekly intervals over the course of three weeks. Measurements were recorded at landmarks and compared for intra and inter reliability.
Results: Clinicians showed high intraclass and interclass correlation (ICC) values with narrow confidence intervals for the tibial tubercle, medial and lateral flares and distal end of the tibia. One clinician demonstrated moderate intra rater reliability for modification over the patellar tendon. All other ICC values for the patellar tendon and fibular head modification were low. Inter rater reliability was not calculated for fibular head and patellar tendon as intra ICC values should be above 0.6.
Conclusions: All clinicians showed good consistency at tibial tubercle, distal tibia, medial and lateral flares. Patellar tendon (0.345< ICC < 0.641) and fibular head (0.165< ICC < 0.513) showed poorer consistency and require improvement.
Computer aided design (CAD) is now commonly used to create artificial limbs. However, the shape of the amputated limb is captured when the patient is sitting down and therefore requires further adjustment. Modification of the shape is carried out by clinicians using a range on on-screen tools to remove and add material to the virtual model.
This study aims to investigate the consistency of clinicians when making these modifications. A range of 13 below the knee amputation models were made, each linked to a made-up patient history. Three clinicians were asked to randomly complete modification three times on each model at weekly intervals over the course of three weeks. Measurements were recorded at landmarks and compared.
Clinicians showed high reliability values for most landmark positions. However, modification was less reliable over important areas such as the patellar tendon and fibular head. Errors in such areas could potentially cause discomfort to the artificial limb wearer and greater consistency is required. This is only an initial study and further work is required to confirm results.
ARTICLE PDF LINK: https://jps.library.utoronto.ca/index.php/cpoj/article/view/30006/22878
Computer aided design (CAD) is now commonly used in prosthetic clinical practice (1) and current scanners have been shown to have a high level of accuracy during the shape capture process (2). Saunders et al implied that shape capture using CAD can save considerable time and make the process more quantifiable. They also acknowledged that models may be stored electronically and easily replicated, unlike plaster where modified plaster models are usually destroyed during socket fabrication which makes socket duplication more difficult (3). The most common level of amputation is transtibial therefore this is the focus of this study (4).
Two fundamentally different designs of transtibial prosthetic socket are currently used clinically: The patellar tendon bearing (PTB) and the total surface bearing socket (TSB).
The PTB socket is one in which pressure tolerant areas (mainly the patellar tendon) are loaded and pressure sensitive areas (bony prominences) are relieved (5). Alteration to the shape captured is carried out by the clinician, who removes material from pressure tolerant areas and adds material to pressure sensitive areas, either by using plaster or on CAD.
The TSB socket, first described by Murdoch (6), used water casting to load all of the surface area of the residual limb including pressure sensitive areas. The TSB design is based on the hydrostatic principle for load transfer (7). With a TSB socket minimal modification is required meaning that the final socket is likely to be more consistent as less clinical judgement is involved (8). The TSB socket was described as long ago as 1968, however, PTB sockets are still commonly prescribed in clinical practice. PTB sockets have shown to have higher variation in interface pressures in comparison to TSB sockets and a recent systematic review has demonstrated higher satisfaction with TSB sockets (9). PTB sockets however have a lower cost associated, and due to budget constraints this may be an important factor (10). It is also important to consider that TSB sockets may not be suitable for all users as they are generally prescribed with a liner which provides the suspension. It could be argued that both socket styles have an application and the clinician should use appropriate prescription criteria based on individuals’ requirements.
Modification, however, may lead to less consistency as more personal judgement and human error is involved. A previous study by Convery et al (11) looked into the consistency of PTB cast rectification with plaster. It was found that a clinician varied by up to 4.3mm. Although the clinical significance of a variation this size has not been tested one may assume that 4mm removed over a bony prominence might cause discomfort.
Shape capture for PTB sockets is carried out whilst the patient is sitting and without loading of the residual limb soft tissue. Modification is therefore required to allow forces to be transferred to the residual limb when the patient is statically and dynamically loading the prosthesis. Other shape capture methods have been developed to simulate soft tissue loading that occurs during stance; such as pressure casting and water casting. Such loading facilitates total surface bearing and therefore minimises the modification process.
Research in prosthetic shape capture is relatively limited. The majority of studies on the topic were prior to 1990 and considering that CAD has made a huge technological advancement in the recent years this was unanticipated. Only a few studies (10,12) exist relating to the consistency of PTB modification one of which was conducted in 2003 using a small sample and using plaster (10).
This study aims to evaluate the consistency of PTB modification using CAD, which has not been investigated previously.
Thirteen transtibial models were prepared by the researcher from a generic computer model on a leading prosthetic CAD system (WillowWood TM Tracer v12.2). A brief fictitious clinical note was compiled to accompany each model, which indicated the soft tissue consistency, time since amputation, gender, areas of sensitivity, adherent scars, and a brief social history.
Three clinicians were recruited from a single prosthetic centre using a poster inviting them to participate. Protocols for the investigation were approved by University of Strathclyde ethical committee. The poster was placed in the prosthetics office for a week and interested clinicians asked to contact the chief investigator, after which time they received a participant information sheet and consent form. Three clinicians with appropriate availability within the confines of the project timescale responded.
Clinicians were asked to randomly select an identifier from a hat that numbered clinicians A-C. No one knew the identity of the clinician apart from the clinicians themselves. Clinicians were asked to write their allocated letter within an envelope and write their name on the outside in case they forgot their identifier.
Clinicians were provided with a computer with WillowWood TM Tracer software v12.2 installed. Three clinicians (A, B and C), with minimum three months’ experience using CAD, were given 13 on screen transtibial models to modify. To achieve a power calculation of 80%, 13 models were used. All clinicians were familiar with the Tracer software and the scanner used for shape capture. A total of six clinically important landmarks were identified on each computer model by the researcher: patellar tendon, tibial tubercle, fibular head, distal end of tibia, medial flare and lateral flare.
To ensure safe transfer of data, clinicians were provided with instructions at the start of each week on how to download relevant files from a secure storage platform StrathCloud and how to upload the modified models when complete. They were asked to read the accompanying clinical note and modify as they normally would for a PTB socket without supracondylar suspension. The order of the models was randomised using a random number generator per clinician, per week.
Modification of models was based on clinician interpretation using the clinical note and on screen presentation. Modified models were then saved securely on the computer and uploaded to StrathCloud for researcher access. Clinicians operated in isolation and were not given access to the other participants’ work. This process was repeated for all 13 models. This was then repeated after a one-week interval and then again after two weeks.
Circumference, medio-lateral (ML) and antero-posterior (AP) measurements were recorded for all landmarks, for all models. Only those measurements deemed clinically relevant were statistically analysed (Table 1). This decision was made following discussion between the chief investigator, the researcher and a leading CAD expert. However, all raw measurement data exists for all landmarks to facilitate future evaluation.
When outlining the medial and lateral flares more than one marker was used in order to show the bony landmark. The middle point of both the medial and lateral flare was used in analysis to simplify the results.
Table 1: Selection of measurements at landmarks (Please see the PDF file).
The peak difference was recorded for patellar tendon and fibular head. This was carried out by overlaying the modified model over the original in Tracer, the software calculated the distance between the two landmarks. The results were then statistically analysed using IBM SPSS v21 to estimate the intraclass correlation coefficient (ICC) Model (2,1)., the confidence interval (CI) and statistical significance. For those landmarks with an ICC of above 0.6 the interclass was calculated (13).
To evaluate the homogeneity of the data, the standard deviation was calculated for each landmark. For results and statistical analysis, Clinicians A, B and C were renumbered 1-3 using a random number generator so the clinicians were unable to identify their own results.
To simplify study results, only those measurements deemed clinically relevant will be discussed. The ICC value indicates the level of reliability of modification between the weeks (intra) and between the clinicians (inter). A value of 1 is perfect reliability whereas 0 indicates no reliability. As seen in Table 2 the intra ICC values for the tibial tubercle, medial flare, lateral flare and distal tibia were high. This suggests that the clinicians are able to perform these modifications relatively consistently between weeks, with little variation. The confidence intervals for all were narrow indicating 95% probability that true reliability was indeed close to these values.
As shown in Table 2 the interclass ICC values for tibial tubercle, medial flare, lateral flare and distal tibia are high (ICC>0.7), suggesting that the process of modification is also consistent across the clinicians. The medial and lateral flare modifications are the most reliable as they have very high ICC values with narrow confidence intervals.
The standard deviations for all landmarks were also calculated (Table 3). The values for medial and lateral flares were the largest indicating heterogeneous data. The standard deviation values for tibial tubercle, medial flare, lateral flare, and distal tibia were also relatively large.
Table 2 shows that the ICC values for the patellar tendon were low (ICC<0.7), which suggest poor intra rater reliability of modification at this landmark. Although one clinician achieved a moderate ICC value at the patellar tendon, this value would ideally require further improvement to demonstrate good reliability in a clinical setting.
All ICC values at fibular head were low although one clinician performed better in comparison to the other clinicians (Table 2). The confidence intervals at the fibular head and patellar tendon are wide indicating less certainty in results.
The standard deviations, as highlighted by Table 3, for the patellar tendon and fibular head were low (SD 0.58-1.5). This may indicate relatively homogenous data for these landmarks; which could mean that a small variation may have had a disproportionate effect on the ICC value.
Table 2: ICC, CI and significance values for each landmark. Poor ICC<0.6, Moderate ICC 0.6-0.7, Good ICC>0.7 (11) (Please see the PDF File)
Table 3: Standard deviation (SD) for each landmark.
The variation in the results between the clinicians suggests that experience, skill and interpretation may have an impact on the consistency of modification. As shown in Table 2, one clinician (clinician 2) demonstrated moderate intra rater reliability (ICC=0.641) of modification of the patellar tendon, two other clinicians showed poor reliability (ICC<0.6). It was therefore not possible to determine inter rater reliability (between clinicians) as clinicians failed to demonstrate sufficient intra rater reliability. Variation between clinicians was also evident at the fibular head but to a lesser degree. Clinician 2 was able to achieve a higher ICC value (ICC=0.513) compared to the other clinicians, however, results still demonstrated poor reliability.
Reliability may be poor since the patellar tendon and fibular head required more targeted plaster removal/addition and this may have led to more variation in modification at these points. Measurements examined for patellar tendon and fibular head were peak measures whereas other landmarks used AP or circumference measures, this may have had an effect on the results. Whilst the ICC values suggest that the modification at the patellar tendon was not reliable, actual maximum variation was 3mm. It is debatable as to whether or not such a difference in modification would be clinically significant as very little scientific evidence appears to exist which suggests optimal modification in relation to socket fit. Future research is required to determine the clinical impact of modification variation on the resulting socket fit.
Low standard deviation values for the patellar tendon and fibular head (Table 3) may indicate that clinicians did not vary modification between the patients, and therefore were not fine tuning the modification depending on patient shape and their needs. The relatively large standard deviation values for tibial tubercle, medial flare, lateral flare and distal tibia suggest that clinicians varied modification most based on the patient residuum shape and clinical notes at these landmarks.
A minimum of 3 months’ experience was required in order to participate in this study. In hindsight, it may have been more appropriate for the frequency with which clinicians use software to be in the inclusion criteria. For example, a clinician could have been trained in using CAD for years but only use it once a month compared to a clinician who was trained two months ago but uses it five times a day. In future, it would be interesting to evaluate the effect of clinician experience and training on reliability of modification, particularly in the areas that showed less reliability (patellar tendon and fibular head). Data on experience was not gathered in this experiment as it would have identified the clinicians to the researcher and therefore had associated ethical issues.
The tibial tubercle, medial flare, lateral flare and distal tibia expressed high ICC values and were highly statistically significant, showing that in general the modification procedure in these areas was consistent. All six landmarks were considered clinically important, as they are key weight bearing and weight relieving areas. Modification of the patellar tendon and fibular head was inconsistent, however, improvements in these areas could be achieved and it may be possible to improve reliability of the overall process. Therefore, in order to improve consistency, it is important to focus on these two landmarks. Clinician 2 achieved a moderate ICC at the patellar tendon, unlike the other clinicians, which suggested that there might be techniques that can be used to increase reliability.
Using the ‘blend’ tool after carrying out modifications may have caused inconsistency as it works by smoothing the build up to the surrounding area. Therefore, this makes the modification less precise and more variation is likely. If there was a limit on the blend tool so it could not alter the modification by more than 1mm this may cause less variation. If there was a method in which a standardised procedure for modification was developed this could lead to more consistent results. Research in Southampton attempted to achieve this by creating a library of shapes to apply as a standard modification. However this was abandoned due to the large amount of variables involved (2).
When gaining measures at each landmark on Tracer, the mouse cursor was placed over the point of the landmark. However, as the system works to 0 decimal place there was a 1mm radius in which the cursor could be placed and the system stated it was directly on the landmark. Within this 1mm radius the circumference/peak/AP/ML measurement sometimes varied by up to 2mm. There were also some associated errors when gaining the peak measurements. The modified model had to be overlaid on the original model, and aligned by eye. One clinician extended the model proximally by 50mm each time and in order to align models this extension had to be removed. Due to the system rounding to 0 decimal places this may have introduced further error. In future research, errors could be reduced by requesting that clinicians do not reduce the ply and also requesting that one landmark be left unmodified in order to assist with alignment.
As clinicians did not mark on the landmarks themselves they may have interpreted them to represent different sites of bony anatomy. For example, one clinician may have interpreted the marker to be the border of the bone, whilst another may have interpreted the marker it to be the area that should have been modified. If the clinicians watched the marker placement on a residual limb prior to be scanned and were given an opportunity to palpate the anatomy themselves this may potentially have an effect on reliability. Similarly, clinicians may have interpreted clinical notes differently. Soft tissue was described but this is not quantitative. If clinicians were given an opportunity to assess patients this may also have an effect on reliability.
It could be argued that a shape capture technique where no modification is required could eliminate the problem of modification consistency. Although pressure casting appears to be less reliant on the clinicians’ skill there may still be an aspect of variation due to clinical judgement. Ossur’s Icecast technical manual (14) recommends inflating the bladder between 40mmHg-120mmHg dependent on activity level and shape of the residual limb. Therefore, it would be interesting to carry out the same experiment but using the ICECAST method of shape capture to evaluate the variation due to changes in pressure. As each residuum is different there is always going to be an aspect of tailoring to an individual and therefore a certain amount of variation.
It may be more clinically relevant to carry out this research on real patient residual limbs linked to feedback on the socket comfort when fitted (15). A socket comfort score could be used along with pressure analysis of the inside of the socket. Although consistency leads to a more scientific process it is important to appreciate that consistency does not necessarily lead to comfort.
As the clinicians were aware that models were not real patients it may have affected performance. Clinicians carried out the 13 modifications in one session, which may have led to participant fatigue. In a few cases it appeared that the clinicians might have omitted to carry out the modification at a landmark, which would have affected results. A clinician in error did not carry out modification of one model, for one week. This was therefore not included in the results. As this was for one out of the 13 models it will not have had much of an impact on the final results but it is important to note that the sample size will be smaller for this clinician. Although three clinicians and 13 models give a power calculation of 80% this may not be a large enough subject group to generalise the results clinically. This study took place in a single prosthetic centre using the software of one CAD system, where the clinicians have a very high CAD usage in comparison to plaster and therefore may not be representative of all clinicians. Future research should aim to use clinicians from multiple centres and analyse the inter reliability between different sites.
This study illustrated the reliability of modification by three clinicians at six important clinical landmarks. Four clinically relevant landmarks exhibited good consistency (tibial tubercle, distal tibia, medial and lateral flares), and two landmarks required further improvement (patellar tendon and fibular head) which required more targeted modification, which may have led to inconsistency. Further research should be conducted in multiple centres to assess the clinical relevance of these results by determining the effect of varying modification on socket fit.
All authors contributed equally in the preparation of this manuscript.
We would like to thank the three clinicians who participated in this study, and Ms Sally Bell for her assistance with the literature research.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Protocols for the investigation were approved by University of Strathclyde ethical committee.
- McGarry T, McHugh B, Buis A, McKay G. Evaluation of the effect of shape on a contemporary CAD system. Prosthetics and orthotics international. 2008;32(2):145-54. DOI: 10.1080/03093640802015920.
- Dickinson AS, Steer JW, Woods CJ, Worsley P. Registering methodology for imaging and analysis of residual-limb shape after transtibial amputation. Journal of Rehabilitation Research and Development.2016;53(2):207-18. DOI: 10.1682/JRRD. 2014.10.0272.
- Saunders CG, Foort J, Bannon M, Lean D, Panych L. Computer aided design of prosthetic sockets for below-knee amputees. Prosthetics and orthotics international. 1985;9(1):17-22. DOI: 10.3109/ 03093648509164819.
- Scott H, Patel R, Hebenton J. A Survey of the Lower Limb Amputee Population in Scotland 2010. Publisher: Glasgow [Scotland]: National Centre for Training and Education in Prosthetics and Orthotics, 2012.
- Radcliffe CW, Foort J. The patellarr-tendon-bearing below-knee prosthesis. Eberhart HD, Inman VT, Foort J, editors. Berkeley: Berkeley, Biomechanics Laboratory, Dept. of Engineering, University of California; 1961.
- Murdoch G. The "Dundee" socket for the below-knee amputation. Prosthetics and orthotics international. 1968;5(3):15-21.
- Kristinsson O. The ICEROSS concept: a discussion of a philosophy. Prosthetics and orthotics international. 1993;17(1):49-55. DOI:10.3109/03093649309164354.
- Dumbleton T. Trans-tibial prosthetic system design and benefits for the amputee, service providers and society: an evidence based clinical study. University of Strathclyde; 2007.
- Safari MR, Meier MR. Systematic review of effects of current transtibial prosthetic socket designs—Part 1: Qualitative outcomes. Journal of Rehabilitation Research and Development. 2015;52(5):491-508. DOI: 10.1682/JRRD.2014.08.0183.10. Selles RW, Janssens PJ, Jongenengel CD, Bussmann JB. A randomized controlled trial comparing functional outcome and cost efficiency of a total surface- bearing socket versus a conventional patellarr tendon- bearing socket in transtibial amputees. Archives of Physical Medicine and Rehabilitation. 2005;86(1):154-61.
- Convery P, Buis AWP, Wilkie R, Sockalingam S, Blair A, McHugh B. Measurement of the consistency of patellarr-tendon- bearing cast rectification. Prosthetics and orthotics international.2003;27(3): 207-213. DOI: 10.1080/03093640308726683.
- Mohammad Reza Safari, Philip Rowe, Angus McFadyen, and Arjan Buis. Hands-Off and Hands-On Casting Consistency of Amputee below Knee Sockets Using Magnetic Resonance Imaging. The Scientific World Journal. 2013, Article ID 486146, DOI: 10.1155/2013/486146.
- Evers A. The revised Dutch rating system for test quality. International Journal of Testing. 2001; 1(2):155-82.
- Plaster of Paris Casting with Iceross Compact. Reykjavik: Ossur; 2010. [Available from: http://assets.ossur.com/lisalib/getfile.aspx?itemid=7926.].
- Hanspal RS, Fisher K, Nieveen R. Prosthetic socket fit comfort score. Disability and rehabilitation. 2003;25:1278-1280. DOI:10.1080/09638280310001603983.
ARTICLE PDF LINK: https://jps.library.utoronto.ca/index.php/cpoj/article/view/30006/22878
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles are permanently available online to the public without restrictions or subscription fees. All articles are free to be used, cited, and distributed, on condition that appropriate acknowledgment is included. Authors are the copyright holders of their original contributions and grant the Canadian Prosthetics & Orthotics Journal (CPOJ) a license to publish the article and identify itself as the original publisher. CPOJ articles are licensed under the Creative Commons Attribution 4.0 International License.