Why surgeon's skills need testing during training

At long last, surgeons are starting to realize that while they all have similar letters behind their names, some do a better job than others at the operating table.

A coalition of medical experts in Canada, Ireland, Scotland, England, Holland, Denmark, Hong Kong, Singapore, and Australia has been formed in the last year to determine how to measure surgeon skill in a way that is helpful to surgeons and patients.

Such moves are long overdue. As a surgical trainee, I can vividly remember assisting in my first few operations, and quickly realizing that every surgeon operates at a remarkably different skill level. Some surgeons are more precise with their dissections, cause less bleeding, and stitch faster. Some move effortlessly through operations while others not so much.

A study by the Michigan Bariatric Surgery Collaborative, published in the New England Journal of Medicine in October, found that a surgeons dexterity matters a lot. Researchers videotaped surgeons performing a common weight loss procedure, laparoscopic gastric bypass, and rated them on a scale of varying skill levels.

"The results were obvious," says Dr. John Birkmeyer, a surgeon from the University of Michigan and lead author of the study. Patients receiving operations from surgeons with better skill had a lower chance of death, and fewer complications, reoperations and readmissions to hospital.

I got my certification in 1981, and can tell you that it is entirely possible that I could have spent my entire career operating alone, without any assessment of my skills.- Dr. Kenneth Harris

Despite drawbacks of the study, such as examining only one type of operation, the study is the first to link surgeon skill directly to patient outcomes after surgery, says Birkmeyer.

In Canada, there are no standardized assessments of a surgeons skill level during training or after being hired. In fact, objective assessment of skill is not used anywhere in the world for surgeon training or licensing, says Dr. Teodor Grantcharov, a surgeon and education researcher at St Michaels Hospital in Toronto.

Surgeons in Canada graduate once they have mastered their trade as determined by their training programs, according to Dr. Kenneth Harris, the Director of Education at the Royal College of Physicians and Surgeons of Canada, the body which certifies doctors for practice.

Trainees operate with senior surgeons and are evaluated on their ability to perform surgeries, such as the ability to complete dissections in a stepwise approach, stop bleeding, stitch and cut with precision, and tie knots without tearing tissues. But, these evaluations are done through observation without formal skill testing.

Standards in each training program ensure that surgeons meet the requirements to safely practice surgery. But, "Final examinations for certification after training are based on one summative exam that has a multiple choice and oral exam component, with no assessment of skill level," Harris says.

Once surgeons are employed, they renew their licences yearly, but there is no requirement for skill assessment, Harris adds. According to Harris, hospitals assess many other general measures of surgeon quality, such as death and complication rates, but not skill level.

"I got my certification in 1981, and can tell you that it is entirely possible that I could have spent my entire career operating alone, without any assessment of my skills," Harris says.

Most patients in Canada rely on their family doctors or other specialists to refer them to surgeons. While patients receive excellent surgical care in many cases, they only have the hospital reputation and a surgeons academic credentials for reassurance.

No objective measures of skill are available, says Harris. "As a patient, wouldnt you want to know that your surgeons skill is being assessed and on par with others in the hospital?" he adds.

Many experts agree that measuring skill should become part of surgical training and the ongoing assessment of working surgeons.

"As with musicians and athletes, some surgeons will be more talented then others," says Birkmeyer. But, practice and coaching are mainstays in music and sports, he adds. The same can be done in surgery, by encouraging less skilled surgeons to learn from surgeons with superior skills - something that rarely occurs for practising surgeons, he says.

Surgical blackbox

Some surgeons may feel threatened and uncomfortable by having their skills assessed, according to Birkmeyer.

"Speaking as a surgeon myself, assessments of skill should be viewed as a learning opportunity, and not as a threat to a surgeons career or reputation," says Harris.

Before any skill testing is implemented, more research is required to develop tools that can adequately assess surgeon skill, says Grantcharov. "A simple rating scale cannot take into account the complexity of each operation and provide useful feedback," he adds.

Grantcharovs research team is developing tools that evaluate surgeons based on skill and error, while also providing "usable" feedback that can help surgeons improve their skills where needed. These assessment tools will be useful in the training of surgeons and among those already working, he adds.

Examples of measures of skill that may be used include a surgeons ability to move organs delicately enough to prevent tearing of structures and the ability to consistently complete all the steps required for a safe operation.

Grantcharov has already developed a surgical "blackbox," which records a surgeons errors in the operating room. As a result, surgeons can learn from their mistakes, says Grantcharov.

The Royal College of Physicians and Surgeons of Canada is moving in the same direction. "Its surprising how many trainees advance from year to year being told that they will improve, instead of having their skills assessed early on so they can improve in specific areas," says Harris.

The college is planning on introducing yearly "milestones" within training programs in 2015. Surgical trainees will be required to have a certain skill level before advancing to the next year of training, Harris says.

"The system needs to adapt to recognize that some surgeons may require more help than others," says Harris.
After all, "Not everyone can become an NFL quarterback, coaching and practice need to be tailored to each person," says Birkmeyer.