Is the CSA racially biased?

THE facts are not in dispute. Failure rates in the clinical skills assessment (CSA) component of the MRCGP exam are disproportionately higher among black and minority ethnic (BME) medical graduates educated either in the UK or abroad.

Adjusting for age, gender and performance against the objective (machine-marked) applied knowledge test (AKT), BME international graduates are almost 15 times more likely to fail the CSA on their first attempt than white UK graduates. Non-white UK medical graduates are 3.5 times as likely to fail on their first attempt as compared to their white colleagues.

It is in the interpretation of these facts that the controversy lies – and which in April will be the subject of a judicial review brought by the British Association of Physicians of Indian Origin (BAPIO). Both the RCGP and the GMC will be defendants in the High Court – the college facing charges that the CSA is discriminatory against non-white exam candidates and the regulator being questioned over its failure to conduct an equality impact assessment of the requirements for RCGP membership.

The GMC might feel understandably put out having to answer to the court in connection to this dispute (though it publicly accepted the decision) as back in March 2013 it commissioned an independent report aimed at getting to the bottom of the growing controversy. Professor Aneez Esmail of the University of Manchester was appointed to conduct a data review of pass rates for different groups of medical graduates undertaking the MRCGP exam. But rather than provide a definitive answer the review only sparked further controversy.

The GMC published the report in September 2013 announcing that Professor Esmail had found that “while there are significant differences in pass rates between different groups of doctors, the way they are assessed in the CSA is not the cause of those differences”. In other words there was no racial bias in the conduct and marking of the CSA.

But on the same day the GMC report appeared, the BMJ published a paper based on the same review in which Professor Esmail and co-author Chris Roberts stated: “We cannot exclude subjective bias owing to racial discrimination in the marking of the clinical skills assessment as a reason for these differential outcomes.”

They added: “Previous training experience and cultural factors (which include physician-patient relationships, and communication and proficiency in spoken English) could help explain these differences between UK candidates and international medical graduates. However, these cultural factors cannot explain differences between white candidates and BME candidates who have trained in the UK, and who would have had similar training experiences and language proficiency.”

Esmail also stated in the BMJ paper: “The clinical skills assessment and its marking is based on a well-established pedagogy that is internationally recognised and used widely in postgraduate examinations. However, like any clinical examination, it is subject to bias. We cannot ascertain if the standardised patients (played by actors) behaved differently in front of candidates from non-white ethnic groups. Nor can we confidently exclude bias from the examiners in the way that they assessed non-white candidates.”

The RCGP responded with outrage. The then Chair, Professor Clare Gerada, said: “We are shocked and bemused that on the very day that Professor Esmail’s official and independent GMC investigation report finds no evidence of discrimination, the same author is publishing a contradictory paper that misleadingly suggests we may be guilty of bias.”

BAPIO felt vindicated by the BMJ paper and emboldened to press forward with the case on behalf of some of its members whose career ambitions have been thwarted by multiple CSA failures.

Dr Ramesh Mehta, BAPIO President said: “It is damning that in this day and age the sole reason that UK educated and trained doctors are four times as likely to fail this exam seems to be solely on the basis of their skin colour. A 15-times worse result imposed on international medical graduates, who have passed every assessment set to them before reaching the MRCGP CSA, by using culture and a variety of usual well-rehearsed excuses and platitudes should not wash any more”.

The RCGP is not the only college reporting differential pass rates in postgraduate exams. Both the Royal College of Psychiatrists and Royal College of Physicians have published data highlighting an increased failure rate among international medical graduates and British BME doctors. Similar disparities have been documented in Australia and the US. It has also been pointed out that differences in CSA failure rates for all non-UK educated candidates - whether white or BME – were no longer significant after controlling for scores in the AKT, IELTS, and PLAB examinations.

Understanding the barriers to exam success experienced by non-white candidates completing specialist medical training is essential. The issue is certainly not a trivial one for those involved nor for NHS workforce planning. In 2012, non-UK qualifiers accounted for almost a quarter of the GP workforce in the UK.

The judicial review will begin on 8 April and is expected to run for three days.

Jim Killgore is an associate editor of GPST