  Purpose: Previous research has shown that obtaining non-expert crowd
evaluations of surgical performances concords with the gold standard of expert
surgeon review, and that faster playback speed increases ratings for videos of
higher-skilled surgeons in laparoscopic simulation. The aim of this research is
to extend this investigation to real surgeries that use non-expert crowd
evaluations. We address two questions (1) whether crowds award more favorable
ratings to videos shown at increased playback speeds, and (2) if crowd
evaluations of the first minute of a surgical procedure differ from crowd
evaluations of the entire performance. Methods: A set of 56 videos of
practicing surgeons were used to evaluate the technical skill of the surgeons
at each video playback speed used for the first minute of the previously rated
performance, using the GEARS assessment criteria. Results: Crowds on average
did rate videos higher as playback speed was increased. This effect was
observed for both proficient and expert surgeons. Each increase in the playback
speed by 0.8x was associated with, on average, a 0.16-point increase in the
GEARS score for expert surgeons and a 0.27-point increase in GEARS score for
proficient surgeons, with both groups being perceived as obtaining relatively
equal skill at the fastest playback speed. It was also found that 22 out of the
56 surgeons were perceived to be significantly different in skill when just
viewing the first minute of performance. Conclusion: The observed increase in
skill ratings with video playback speed replicates findings for laparoscopy in
[2], and extends to real robotic surgeries. Furthermore, the large differences
in skill labels when comparing the first minute of surgery to the entire 15
minute video warrants further investigation into how much perceived skill
ratings vary in time (sub-task level) vs. summative metrics (task level).
