Universities have an obligation to ensure that their assessments are sound and defendable when they confer degrees for professional qualifications, such as in medicine. Can institutions confidently defend these results and what are the implications if they pass a student who is not competent?
These were some of the questions Prof Hanneke Brits, a family medicine specialist at the Free State Department of Health, as well as the Department of Family Medicine, at the University of the Free State (UFS), addressed during her inaugural lecture on Tuesday (12 August). The UFS, she concluded at the end of her lecture, titled To pass or not to pass: Can we confidently defend the outcome of our assessments? can defend its clinical assessments with the implementation of effective workplace-based assessment and trained examiners.
The implications of passing incompetent students
According to Prof Brits, who has supervised numerous undergraduate and postgraduate student research projects, she chose this topic because decisions have consequences. She gave an overview of the assessments in the clinical years of the undergraduate medical programme. In so doing, she also answered other questions including what may happen when universities pass students who are not competent and what may happen if they fail competent students. When the university passed a candidate, she said, that candidate may register with a professional body like the Health Professions Council of South Africa to work as a doctor.
“What are the implications if we fail to fail a student who is not competent? The implications are that patients may suffer if they are treated by an incompetent doctor, which may lead to the doctor running into trouble if it is found that their work is not up to standard. This may further lead the faculty being labelled as poor for training substandard doctors.
“The throughput rate of the university may go down and the university may not get subsidy for the students. The student must repeat his module with a lot of emotional and financial burden. They public may suffer because there are not enough healthcare professionals to treat them. Therefore, we must get this right,” she said.
When assessing students, assessors should start at the bottom: students should know, then they should know how, then they should show how and then they must do. All assessments should meet the basic requirements of validity, reliability, fairness, educational impact and feasibility, explains Prof Brits.
Workplace-based training and assessment
During her PhD study, she looked specifically at assessments in the clinical years of the undergraduate medical programme. “It is quite complicated,” said Prof Brits, “to do assessment for professional qualifications as you need to obey to the rules and regulations of the Department of Education, the Department of Health, the Health Professions Council of South Africa, the Colleges of Medicine of South Africa because they are our examining body, as well as our own university rules and international assessment guidelines and best practices.”
She compiled a framework to measure what they do at the UFS and found that the decision reliability was excellent – meaning the students that passed during the year passed at the end of the year and those that failed, failed. The reliability of some of the methods used for the final assessment was not good, however, if more assessments with supplementary exams were included, it was better.
The conclusion of her study was that the UFS mostly complied with the regulations of the regulatory bodies. The recommendation from this study was to implement workplace-based assessment (WBA) to improve both the validity and reliability of assessments and to make it more defendable. Prof Brits explained that WBA is where students get regular assessment and feedback while they work and receive training in hospitals or clinics. “For example, the student is seeing a patient in the emergency department who was stabbed with a knife on his hand. Is the student able to assess the severity, can the student manage the wound and what about follow-up?
“The advantage of WBA is that we train in real life situations and manage conditions that occur commonly. In real life situations, students use many senses while learning, e.g., seeing, hearing, touching, smelling, which all enhance knowledge retention. It is important that students receive feedback and that we document these encounters. To ensure a holistic approach to the management of patients we use Entrustable Professional Activities or EPAs – something that I can trust a person to do. It is a combination of knowledge, skills and attitudes.”