Intratest Reliability in Determining the Subtalar Joint Axis Using the Palpation Technique Described by K. Kirby
J. De Schepper, K. Van Alsenoy, J. Rijckaert, S. De Mits, T. Lootens, P. Roosen
JAPMA, March/April 2012 Vol 2 #2, 122-129
This article concludes that "the low kappa values for the classification of the palpated axis location prevent the authors from defining the outcome of this technique as reliable and clinically useful"
"the low kappa values for the classification of the palpated axis location prevent the authors from defining the outcome of this technique as reliable and clinically useful"
Did the study conclude the Kirby test not reliable and clinically useful or not?
IMHO, it did.
If I'm understanding further what the authors research determines, it declared that you needed greater than 10 degrees of axis difference to have diagnostic and clinical import and where there is overlap or small degrees of difference the test becomes unreliable and clinically weak.
As one gets closer and closer to 10 degrees of variation (which is a huge amount) the clinical usefulness and applicability lacks power.
So if the STJ Axis of three patients, one zero deviated, the next 5 degrees deviated medially and the next 5 degrees laterally were presented, this paper concludes the Kirby Axis Measurement would not be sophisticated enough to pick up the deviations with accuracy or reliability.
I wish the authors could comment.
To use your five finger analogy, it's like saying that my fingertips cannot appreciate temperature gradients closer than 8 degrees.
This means that for my fingers to discern one spot to be hotter or colder than another they would need to be separated by 9 degrees +.
That's why we have thermometers.
Robert:
Why don't we stop diverting about my fingers or straw men or some other psychobabble and start with the words that these authors wrote that I quoted verbatim.
Hi drSha I don't think we've met in person or on the arena, but it looks about a good time for me to step in and try to explain - even though from the reactions above, i'm not sure any further explanations will help the situation.
There are levels of evidence, and up to now, no official research was performed on the clinical test to locate the STJA. This means that the highest level before the last edition of JAPMA was 'clinical expertise'. Doing a reliability study was a first step we took in trying to determine if it was sound to use a test, such as the one mentioned above.
Indeed, for non trained testers, the smallest detectable difference was about 10° for the angle and about 7 mm for the translation. This result had much to do with a couple of points discussed later on in the article. We only palpated under the calcaneum over a distance of about 5 to 8 cm. Also the thumb has a surface of about 1cm². these result in a low sensitivity of untrained testers.
We picked this one up in a second study, using a punchpin. Non trained testers went from 11° to 5° without any further training!
Trained testers had a SDD of about 3° to 5° not having a lot of influence by using the punchpin.
When we’re talking about clinical relevant, maybe you should try to draw an angle of 5° or 10° over a distance of 5 to 8 cm and place your thumb on that. Even on the widest point, your thumb is still going to cover the 10 degrees. Furthermore the population tested here had no real ‘extreme’ deviations. In my opinion it is not so important to discuss a difference of 5 degrees, but it is important to be able to see a difference in larger deviations as this could influence your treatment protocol
This would depend on the person performing the test… so start practicing – the more you do it, the better you get at it. Or use a punch pin…
here I have to explain that kappa statistics are done on a nominal data level and means that this is about being able to 'Label' the axis as medial, lateral or normal... but this would be an easy one to solve I think... if you can define a range, the kappa's will go through the roof!!!
In light of its power and your obvious skills as a researcher, I would hope that the editor (s) will be able to eradicate this thread ASAP so that my untrained conclusions can be eliminated from the internet.
This remains continued growth on my part as I am educated on EBM.
My overall opinion on the use and applicability of The STJ Axis remains in question as probably yours does for Functional Foot Typing but that's what happens when you are driving a Mercedes and someone pulls up to you in a BMW.
I hope your future research continues to show STJ Axis mearurement to be valid and clinically relevant as I think there are many ways to skin the biomechanical cat.
Well, we've heard from the author. That should do.
I don't want a scrap with you Dennis. I'm listening to bel's link, and looking forward to watching Episode 1 of series 2 of game of thrones later. I've had a shocking day but am now feeling mellow and well disposed to the world. I was merely pointing out the the difference between lack of evidence and evidence of lack. Thats hardly psychobabble. You got a bit excited and overstated your case, thats all.
My point exactly, and this is what my job as a lecturer and "researcher" is about. Only we have to accept that having 'an opinion' or 'an idea' isn't always good enough. A profound evidence based framework is what 's going to help us all move forward... how small the steps may be at any given time... So if I have to choose one way to skin that cat above another, I know what to choose.
Robert I have great respect for you and 99% of the people contributing on PA. Mike made the point that most of the threads lately have degenerated into "barefoot this and that and Dennis being Dennis". I have to agree with him, it is getting tiresome and I for one am frustrated. I'm sensing many others are as well?
I feel that at some point a decision needs to be made as to the value that these diversions add to the overall quality of the forum. I have purposely avoided the barefoot posts until very recently and then on only one post. Dennis is a whole other matter.
I for one will attempt to avoid Dennis and his ramblings. I recommend that Dennis be held accountable for a rational explanation of Eric's (and our) questions and a polite discussion and that he not be allowed to continue to try to persuade through attrition and straw man argument but by facts and proof. If he cannot provide such then there is really little else to say on the subject is there?
Yeah, I share the frustration. I'm just suffering a little battle fatigue! The barefoot loonies are merely trolls and should be booted. Dennis, Whilst I agree his postings most often start the fight and often add little, is a slightly different matter. The whole dig / counter dig thing will just continue. Forever.
Ignore me mate ;). I'll probably agree with you in the morning.
Now that I have been able to review the slides you were so kind to furnish, I can comment more intelligently on your paper.
We all know that evidence has a level and potentially contains bias upon the authors.
It must be reviewed as to its level, validity and applicability.
Your study has three subjects, the examiners.
We can assume that a greater number of examiners would be more valuable when their results are tabulated as to the reliability factor of their findings as well as to their reproducibility.
It is unicentered and not blinded so I would consider it a Level 4 study.
As far as I can tell (see attachment), one examiner found the STJ Axis of the foot involved to be "normal", the next found the STJ Axis to be "somewhat medial' and the third examiner found the STJ Axis to be "more medial".
Your examiners had a ten degree deviation in their readings.
The known studies for STJ Axis ROM shows a transverse plane range of between 30 and 40 degrees*
*Tomaro J, Burdett R, Chadran A: Subtalar Joint Motion and The Relationship to Lower Extremity Overuse Injuries; JAPMA, Vol 86, #9, 1996, p 427-32
This means your deviation of 10 degrees among examiners reflected an error rate of between 25 and 33% which is quite high.
In a treatment sense, these three examiners, using the same foot or model would apply different care.
The first would apply no medial skive, the second a low medial skive, lets say 2 degrees and the third would apply a higher medial skive, lets say 4 degrees.
In summary, your three examiners came up with a different STJ Axis diagnosis that would lead to three different treatments.
I would consider that, IMHO, unreliable and poorly reproducible yet you conclude that your study finds the STJ Axis Test of Kirby reliable.
I would conclude "this low level study seems to reveal that with further refinement and education and practice of the examiners, The STJ Axis Test of Kirby may prove to be of clinical import but that it currently lacks reproducibility and clinical validity".
I'm not sure what gave you the confidence to draw the conclusion that you did?
Your continued research may harvest higher level, more valid and more applicable evidence but IMHO, your case is not proven from the nul as required.
I was sitting at our family easter brunch, when I got the e-mail alert...
I'm happy this is getting so much attention!
Have to point out that the slides and the article are 2 different projects (but this just to make sure we're not mixing things up)
To talk about the bias upon the authors, what do you think the influence would be and how we could take this 'bias' out of the equasion?
Very correct.
It could be better if we could have a 50 podiatrists with a different level of expertise and perform a similar study.
this would indeed give us a more valuable results.
It was indeed unicentered, but is was was blinded.
We found it difficult to send 52 participant from one centre to another to be tested by different raters.
The choice here was rather practical.
More correctly said 'could lead' to different treatment protocols - but this would be a complete different research project.
Yes, this is possible and our conclusion is indeed that the labeling of the axis isn't reliable.
You could look at this in another way.
If you have a range of 30 to 40 degrees and you would have 3 to 4 labels with a range of 10°, your treatment protocol wouldn't be so different.
Every research project I've read, gives again more questions in need of an answer.
One could look at the results of this project and draw conclusions from it, deciding that the test isn't of any value, and that's OK, I think.
I learned, teaching podiatry (biomechanics and OT), that we can get further with an critical 'open mind' then not.
Discussing the use of bisections and the 'neutral' paradigm is one of my favourite topics.
To me, it's all about the 4th position of the Perry Model:Complex Dualism and Advanced Multiplicity.
If I may take a quote here from Wankat & Oreovicz in their chapter on teaching engeneering
So again, personally, this is one step closed to a more EBM approach in the biomechanical understanding of the lower limb and foot.
This doesn't mean that it all should stop just here.
This doesn't mean that there aren't other, maybe better clinical tests out there with a higher reliability and validity.
The difficulty is to get the results out there.
Personally, I didn't do this project because I wanted the STJA palpation technique to be bang on so I could stop teaching all the other stuff.
I did it to be able to tell my students how this test stands next to other clinical tests in helping them to decide how to treat.
Can I ask a silly question - why do you Dennis keep returning here to discuss your theory & argue against Kevin's? I'm genuinely interested in why you keep coming back here when in the main, people disagree with your theory. I know disagreement is not a reason to limit discussion but this seems a waste of your time & energy. Why not go to another site where your theories will be better accepted?
Markleigh (one of my daughters name is Leigh so I'm automatically fond of you):
If you look at this thread, I posted the results of an article published about the STJ Axis Test of Kirby.
Nothing to do with foot typing in any way shape or form.
In the very first posting, Mark Russell converted that to foot typing and me personally.
Robert Isaacs responded to the thread and I did likewise to him in kind.
Then Kenya stepped in and gave his own perspective of his paper and on his sayso, I immediately apologized for what at the time, seemed to be him educating me that his study did not overstep error in drawing the conclusions it did.
I hope Kenya was not feeling a personal attack in my response.
Robert chimed in that the author spoke and I apologized with "that should do it".
Then Kevin, Wedemeyer, Mark Russell and Mike Weber (one of The Moderators) went upon a character assasination on yours truly.
I responded with quotes around learning by your mistakes and sticking up for what you believe in until PROVEN wrong.
No personal attacks on my part.
Then, on another thread, Kenya was kind enough to share his slides and I got to see the one that related to the error rate between his three examiners.
I added my STJ Axis wording and the arrows and posted that in light of this new evidence, I once again had to claim that Dr Kirby's Test (Kenya's words in his article, not mine) seemed unreliable.
Then and this is my take, Kenya basically agreed with my presentation by stating "our conclusion is indeed that the labeling of the axis isn't reliable".
In Robert's words, once again "the author has spoken, we're done".
and I stopped posting.
Next you posted:
"Can I ask a silly question - why do you Dennis keep returning here to discuss your theory & argue against Kevin's? I'm genuinely interested in why you keep coming back here when in the main, people disagree with your theory. I know disagreement is not a reason to limit discussion but this seems a waste of your time & energy. Why not go to another site where your theories will be better accepted"?
I never discussed foot typing or foot centering, I never made a personal comment and tolerated that I was compared to the s*** on the sole of a shoe.
Please confirm your accusations or consider telling those who are displaying Chad behavior on The Arena to accept the message and not try to destroy the messenger and let's all live in peace but in answer to your request to leave The Arena, the reason I will never leave is that I love it here as it remains my main teaching tool for biomechanics.
drsha, I just have the feeling i'm saying the same thing over and over again, and you're changing my words.
You may find that this is what you mean with me being biassed, but I would rather call it being correct...
LABELING the subtalar joint shows not to be reliable... and you say:
converting your conclusion into cars:
Its not because an Alpha Romeo has a bad reputation with it's electric components, that it's a bad car to drive in. but you would conclude:
don't drive those Alpha's, they are not reliable...
Just to make sure you get the grip
I said
You wrote:
So the title of the tread should actually be:
Study Finds labeling the Kirby's Subtalar Joint Axis Measurement "Not Reliable or Clinically Useful"
I think 'Kenya' has enough of this thread, have fun guys!
Kenva
Please don't go away, defend your work or we will take it as a sign that you left the podium because you are not willing to soften your conclusions.
We are not discussing cars.
We are discussing your words and your results as I am actually trying to use them and yet you act as if I'm not keyboarding in English or believe that you can spin the facts to justify your conclusion.
Correct me once again.
According to your study (this study and not some future one that you are presupposing will go through the roof to correct the problem findings with this study....that would only reflect a bias to your future outcomes which you claim doesn't exist:
Your examiners labeled the same foot differently as to the location of the subtalar axis.
Summarily, they could not determine RELIABLY whether or not the STJ Axis was medial, lateral or normal.
Then you concluded that this was a reliable test.
Are we on the same page yet?
I was taught right here by Dr Spooner (I think?) that researchers are supposed to work from the nil.
They test an hypothesis as if it was incorrect.
In that way, bias is eliminated consciously and subliminally.
To justify the problems of this paper with statements that "This would be an easy one to solve, I THINK" and "the Kappas will go through the roof" reflect a new unproven hypothesis and nothing more.
OK drsha, once again,
The study published in JAPMA shows results of of an intratester reliability.
The labeling of the axis, and thus the kappa statistics were done on a test, retest protocol.
The results look at what a tester labeled the first, second and third time.
The intrarater reliability for the actual location of the STJA expressed with a Cronbach Alpha gives values around 0.9 - In my opinion, these are strong results underlining the fact that locating the axis for a tester in a test/retest setup is reliable.
When a tester has to label an axis, the results show less consistency with low kappa results, as discussed before...
damn, didn't I just say the same thing again as the postings before???
Answer a question for me...
Is the STJA location technique the technique where you look for the points of no rotation on the plantar surface of the foot, Marking those points and connecting them?
The possible added mechanical value behind this concept is that perpendicular pressure on one side or the other of the STJA creates pronatory or supinatory moments.
The fact that one would name/label the location of the axis 'medial, lateral or normal' doesn't influence the mechanical effect.
translating this to the results of the study, the test itself can be determines as reliable on an intrarater level, the labeling is not.
Drsha, The fact that you've been naming me Kenya instead of Kenva didn't mean you weren't referring to me right?
I really still have the feeling that we're going in circles here.
I've really enjoyed reading your comments and findings from the research.
I hope your weekend with the family was not too disrupted or irritated by this discussion.
Looking forward to reading more of you and your colleagues research in the future.
thanks for your postings and your explination of your research! Fantastic.
Dennis,
You are such a pain in the neck. I have never ever personally met anyone on this forum face to face, but i have learned a lot and would love to meet these people if i had the chance.
You, however, are annoying and I wish you would go away. Your egotistical ramblings are such a frustrating diversion from many interesting discussions.
Sorry to be so negative, but I just had to let you know how you make me feel.
Kenva:
So sorry.
Kenva is not a typical name for me and I got lost in my passion for responding to your great posts.
I think I finally understand what you are stating about your article and am more capable of summing up your finding as to the presence of the intrapersonal reliability of The Kirby test and a lack of an Interpersonal Reliability of The Kirby Test.
As per your findings, we can draw the following conclusions:
1.
Kirby's Subtalar Joint Axis Test Has Intrapersonal Reliability
2.
Kirby's Subtalar Joint Axis Test Lacks Interpersonal Reliability
Here I have to refer to the presentation I gave at the Belgian Podiatry Conference on March 23rd. Cadaver specimens were used to locate the STJA location and again 3 examiners performed a similar setup as the first research project.
We also looked at the interrater reliability.
Cronbach alpha's above 0.8 and 0.9 for the slope and intersection using the thumb would be considered as good to excellent.
However, these results are not published in a peer reviewed journal yet.
Your statement above, if you would rely on a conference presentation, would be wrong.
it doesn't lack of reliability.
Not inter nor intra.
Furthermore, a vicon was used to locate the STJ axis of rotation and there wasn't a significant difference between the vicon results and the testers...
so it's even a valid technique...
Ken, I should like to summarise your research findings for those struggling with their interpretation and to ensure we are all understanding correctly:
1) Both intra and inter-rater quantification of STJ axial position were found to have good-excellent reliability for the palpation technique.
2) When the palpation technique was compared to the "gold standard" using 3D kinematics and bone pin markers, there was no significant difference between the palpation technique and the gold standard. Thus, the palpation technique is valid.
3) When several raters try to assign a qualitative label of axial position: medial, normal, or lateral, then the inter-observer reliability is poor as indicated by the low Kappa values.
I just got back from Austria for a ski vacation with my family. As I had no internet available (on orders of my dear wife …
)
So, I was not able to participate and was only able to read this thread today.
I must admit, for once I was happy to have no internet access on vacation …
@ Ken,
Thanks for taking up this discussion in my absence !!
I guess indeed, almost everything is said on this one….