brevis
>so i ask?......are they a true reflection on the clinical capabilities of students?
OSCEs need to be constructed intelligently with all parties familiar with the mode of testing and its implication. The answer to your question brevis is ...conditons apply.
Charles Caleb Cotton said " Exams are formidable enough , for the biggest fool can ask the wisest man a question they cannot answer."
The sum of the parts is never the same as the whole (praxis) and testing (no matter how sophisticated) is a litmus at best and hopefully has a predictor value which would indicate under modifying circumstances the examinee will perfom at least at
same or higher level.
Objective Structured Clinical Examinations
For netizens unfamiliar with the concept of OSCEs,
this assessment is designed to evaluate intellectual activites which underpin clinical process. It is not designed to test clinical procedure and should not be considered as a substitute for in situ clinical exams. Objective structure relates to examination organisers who decide from the suite of
pre-published learning objectives considered
critical to level of student, a planned format of valid and reliable tests which measure aptitude. The activities must all be able to be completed within the same time frame at each station. The time allocated should be equivolent to a competent practioner completing the same activity. Unlike the standard classroom/clinical exam, the OSCE is usually held in a large room with a circle of tables (stations), candidates are allocated to one station to start the exam and after a desinated time (usually 3-4 minutes) a bell rings and the candidate moves to the next station (clockwise). A latent but major objective of the exam is time managment (a critical skill to the busy practitoner). Each station will usually have a silent observer ( this is the role of the senior student in the original paper), their role is to mark the candidates perfomance. The marking schedule is pre-agreed by experts (at the planning styage) and tested for interrater reliability.
Putting together OSCEs is complex and there are many pitfalls along the way. The advantages are each candidate should be given exactly the same tasks in the same time frame. These tasks should be directly associated with pre published objectives and the testing of performance levels should be valid and reliably marked. To make any valid statistical analysis on performance the more stations in the OSCE the better. Medical OSCE will last up to 3 hours long but this is not common in podiatry.
From an operation perspective
the number of students in the class ie. 20
multiplied by the time for each station e.g. 3 minutes would give an OSCE of one hour in duration.
For bigger classes the examination can be repeated but
obviously creates problems with cueing. Health and safety in the workplace can ad complications as can university policy, such as keeping students attetnion for no more than 45 minutes etc. Any event which stops the exam such as illness, can add considerably to the organisation. Sometimes patients may be used at stations but in longer examinations there needs to be substitutes and time to make these factored into the exam timetable. I am more convinced computer based OSCEs offer a better option with greater flexibility to the students and when based on sufficiently large bank of questions reduce cueing.
The OSCE is more suited to measure cognitive domain and affective domain (attitudes), although some
psychomotor activites can incorporated. Detailed procedures involing praxis
are better done either in OSPEs or in situ.
The more practice students
you have with OSCEs the higher they will
score (like IQ tests) so in the reported study the cohort of students were probably well schooled in OSCEs. OSCE are based upon a bank of questions which are not infinite and practice in the examination format is usually limited in podiatry schools (my experience)
so many students find the format a little intimidating because they are not always familiar with the mode presentation. This results for many in a testing of the candidates intelligence or ability to follow
syntax rather than the matter at hand (objective in OSCE). Some examiners might say the thought provoking (obscure - I would say) station is a good test for a competent candidate. As an educationalist, I would argue this is inappropriate in a balanced OSCE and that station should be removed.
One downside, well practised students
demonstrate "exam lock" by constantly practising the labours of mock stations. On the day of examination these candidates
fails to follow the task (stem) clearly, because they are anticipating the right behaviour/answer, and not interpreting the question as asked. Cueing is also a problem especially when candidates view others doing practical stations. So the locality of the OSCE becomes an issue.
Stations which link also present challenges because if the candidate makes
the wrong response to the first station there is little chance for them being able to redem
themselves on the link station. Some examiners are quite happy about this situation but it is not in the true spirit of the OSCE format.
toeslayer
Last edited by a moderator: Oct 28, 2007