Tag Archive for: educational models

Posts

Article of the week: Moving up a GEAR: evaluating robotic dry lab exercises

Every week the Editor-in-Chief selects the Article of the Week from the current issue of BJUI. The abstract is reproduced below and you can click on the button to read the full article, which is freely available to all readers for at least 30 days from the time of this post.

In addition to the article itself, there is an accompanying editorial written by prominent members of the urological community. This blog is intended to provoke comment and discussion and we invite you to use the comment tools at the bottom of each post to join the conversation.

If you only have time to read one article this week, it should be this one

Face, content, construct and concurrent validity of dry laboratory exercises for robotic training using a global assessment tool

Patrick Ramos, Jeremy Montez, Adrian Tripp, Casey K. Ng, Inderbir S. Gill and Andrew J. Hung

USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA

OBJECTIVES

• To evaluate robotic dry laboratory (dry lab) exercises in terms of their face, content, construct and concurrent validities.

• To evaluate the applicability of the Global Evaluative Assessment of Robotic Skills (GEARS) tool to assess dry lab performance.

MATERIALS AND METHODS

• Participants were prospectively categorized into two groups: robotic novice (no cases as primary surgeon) and robotic expert (≥30 cases).

• Participants completed three virtual reality (VR) exercises using the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA, USA), as well as corresponding dry lab versions of each exercise (Mimic Technologies, Seattle, WA, USA) on the da Vinci Surgical System.

• Simulator performance was assessed by metrics measured on the simulator. Dry lab performance was blindly video-evaluated by expert review using the six-metric GEARS tool.

• Participants completed a post-study questionnaire (to evaluate face and content validity).

• A Wilcoxon non-parametric test was used to compare performance between groups (construct validity) and Spearman’s correlation coefficient was used to assess simulation to dry lab performance (concurrent validity).

RESULTS

• The mean number of robotic cases experienced for novices was 0 and for experts the mean (range) was 200 (30–2000) cases.

• Expert surgeons found the dry lab exercises both ‘realistic’ (median [range] score 8 [4–10] out of 10) and ‘very useful’ for training of residents (median [range] score 9 [5–10] out of 10).

• Overall, expert surgeons completed all dry lab tasks more efficiently (P < 0.001) and effectively (GEARS total score P < 0.001) than novices. In addition, experts outperformed novices in each individual GEARS metric (P < 0.001).

• Finally, in comparing dry lab with simulator performance, there was a moderate correlation overall (r = 0.54, P < 0.001). Most simulator metrics correlated moderately to strongly with corresponding GEARS metrics (r = 0.54, P < 0.001).

CONCLUSIONS

• The robotic dry lab exercises in the present study have face, content, construct and concurrent validity with the corresponding VR tasks.

• Until now, the assessment of dry lab exercises has been limited to basic metrics (i.e. time to completion and error avoidance). For the first time, we have shown it is feasibile to apply a global assessment tool (GEARS) to dry lab training.

 

Editorial: Validating dry lab exercises for robotic surgical skills training

Standardising and structuring of robotic surgery curricula: validation and integration of non-technical skills is required

Kamran Ahmed and Oliver Brunckhorst

Surgical simulation has advanced tremendously over the last two decades with the development of laparoscopic and robotic surgery. Because these procedures have a steep learning curve and because of the reduced training times experienced by trainees, safe adjuncts to operating room training are required [1]. Simulation training is a novel approach to surgical training and has been validated as a training and assessment tool and has been shown to improve a surgeon’s performance in the operating room.

In the present paper, Ramos et al. [2] evaluate the face, content, construct and concurrent validity of robotic dry laboratory (dry lab) exercises. They developed similar tasks to those included in the validated virtual reality da Vinci Skills Simulator using the da Vinci Surgical System in a dry lab environment. They also explored the applicability of Global Evaluative Assessment of Robotic Skills (GEARS) to assess dry lab performance. Good responses from the expert cohort with regard to realism and usefulness as a training tool confirmed the face and content validity of the dry lab exercises, whilst concurrent validity was also established, with experts outperforming novices in all but one of the individual metrics. Finally the simulator composite score achieved from the virtual reality simulator and GEARS scores from the dry lab exercises moderately correlated, thereby also establishing concurrent validity.

The present study raises an interesting question about the definitions of what constitutes an expert in robotic surgery. There are no consistent definitions of a ‘novice’ or an ‘expert’. A recent review has shown that the learning curve in the literature varies from 80 to 250 cases in robot-assisted laparoscopic prostatectomies, depending on previous open or laparoscopic experience and the outcome measures used [1]. With this in mind, Ramos et al. classified ‘experts’ as those performing >30 cases as the primary surgeon. This should be taken into consideration as the experts were so vital in establishing the face, content and construct validity of the dry lab curriculum. Additionally, although it has been shown that simulation models are valid and reliable for the initial phase of training and assessment in urological procedures, this is not the case for advanced and specialist level skill learning.

The present study provides validated dry lab exercises which could be incorporated into a robotic surgery training curriculum. With several simulators now having been validated for robotic surgery, integration of these within a structured robotic surgery curriculum is required. Steps have been taken to set out an effective curriculum similar to that which exists for laparoscopic surgery. The Fundamental Skills of Robotic Surgery is a simulation-based curriculum which has been validated and has a proven educational impact [3]. The Fundamentals of Robotic Surgery curriculum has also recently been laid out, but this is still in the process of validation [4]. One of the biggest initiatives in urology, however, was recently announced by the European Association of Robotic Urology Section Congress 2013 [5]. A multinational push to implement a standardized curriculum set out a multi-step curriculum which uses various techniques, including online theoretical training, simulation and observation and finally fellowship prior to certification, then allowing the surgeon to perform independent surgery. The next step for these curricula is for them to be fully validated and implemented, with the patient outcomes then being analysed.

Another aspect worth considering is the integration of non-technical skills within these curricula which has occurred in few of them. Full-immersion simulation (Fig. 1) offers the opportunity for training surgeons to develop critical skills such as communication, coordination and leadership. For the more experienced surgeon, at the later stages of a curriculum, full crisis simulation can help develop decision-making and team-working skills in difficult situations. It has been shown that integration of full-immersion simulation in simulation training is feasible and effective [6]. Surgeons performing robotic surgery need to develop the non-technical aspects alongside their technical skills, and full-immersion simulation certainly has a role to play in this and needs to be integrated within the curricula. The development of the Distributed Simulator offers a validated and low-cost method [7] of introducing full-immersion simulation into robotic surgery curricula and is an option that requires further investigation for its effectiveness in urology.

Figure 1. Aspects of non-technical skills developed through full-immersion simulation [8, 9].

In conclusion, the present study offers further examples of tasks that could be incorporated into robotic surgery training curricula. Whilst curricula exist, implementation and analysis of their effect on patient outcomes are the next steps. The integration of non-technical skills within these is important, and full-immersion simulation has an important role to play within robotic training curricula.

Kamran Ahmed* and Oliver Brunckhorst*

*MRC Centre for Transplantation, King’s College London, King’s Health Partners, and Department of Urology, Guy’s Hospital, London, UK

References

  1. Abboudi H, Khan MS, Guru KA et al. Learning curves for urological procedures: a systematic review. BJU Int 2013; doi: 10.1111/bju.12315. [Epub ahead of print]
  2. Ramos P, Montez J, Tripp A, Ng CK, Gill IS, Hung AJ. Face, content, construct and concurrent validity of dry laboratory exercises for robotic training using a global assessment tool. BJU Int 2014; 113: 836–842
  3. Stegemann AP, Ahmed K, Syed JR et al. Fundamental skills of robotic surgery: a multi-institutional randomized controlled trial for validation of a simulation-based curriculum. Urology 2013; 81: 767–774
  4. Smith R, Patel V, Satava R. Fundamentals of robotic surgery: a course of basic robotic surgery skills based upon a 14-society consensus template of outcomes measures and curriculum development. Int J Med Robot 2013; doi: 10.1002/rcs.1559 [Epub ahead of print]
  5. Khan R, Ahmed K, Mottrie A et al. Towards a standardised training curriculum for robotic surgery: a consensus of an international multidisciplinary group of experts. Poster presented at the EAU Robotic Urology Section Congress Stockholm, Sep 3–5 2013
  6. Shamim Khan M, Ahmed K, Gavazzi A et al. Development and implementation of centralized simulation training: evaluation of feasibility, acceptability and construct validity. BJU Int 2013; 111: 518–523
  7. Kassab E, Tun JK, Arora S et al. ‘Blowing up the barriers’ in surgical training: exploring and validating the concept of distributed simulation. Ann Surg 2011; 254: 1059–1065
  8. Flin R, Yule S, Paterson-Brown S, Maran N, Rowley D, Youngson G. Teaching surgeons about non-technical skills. Surgeon 2007; 5: 86–89
  9. Undre S, Sevdalis N, Healey AN, Darzi A, Vincent CA. Observational teamwork assessment for surgery (OTAS): refinement and application in urological surgery. World J Surg 2007; 31: 1373–1381
© 2024 BJU International. All Rights Reserved.