Case Study: Vicarious Surgical VR

This VR experience intends to convince surgeons and investors that the Vicarious Surgical system, a remotely operated robot featuring a novel anthropomorphic design, is a superior proposition to other contemporary laparoscopic systems. My team worked hard to bring a compact, faithful reproduction of complicated capital equipment to conference rooms and offices around the United States.

My Roles

Interaction Designer
UI Designer
Animation Director
Product Owner

My Deliverables

User personas
User journey
Instructional design
UI/UX and 3D assets

My Tools

Figma
Illustrator
Blender
Unity

The pre-FDA Vicarious Surgical system was undergoing continual development behind closed doors, yet the company needed continual buy-in from investors and influencers who could only be convinced by hands-on demos of its unique approach. Such prospects could only be reached remotely, but the large capital equipment of the system prohibited this.

A supplemental problem was that Vicarious had more than a single type of user to convince, so the experience had to be accommodating of two quite opposing use-cases: on the one hand, the affordances of the real-life system were many and varied, and would take a comprehensive user journey to impart to skeptical and interrogative subject matter experts; on the other, impressing time-poor conference-goers had to be a case of captivating them with a pared-down, accessible experience that could convince them within the space of five precious minutes.

The Solution

My team worked closely with Vicarious Surgical Inc. to produce a digital twin with the utmost fidelity to the complex physical design of the platform. Users could put it through its paces within differing contexts based on windows of time. We knew that any demos would be led by a Vicarious representative, so we left it to them to select the presentation mode: either an easy, showroom-style introduction to the product, or a thorough hands-on with all product features unlocked.

The Process

With realism being one of our goals and our users well-defined, I was adamant that we meet surgeons “where they live” to support the aim of evangelising the platform. We needed the Vicarious equivalent of a Fundamentals of Laparoscopic Surgery (FLS) trainer to validate the robotic system in the surgical context: adaptations of familiar exercises to the affordances of Vicarious’s unique platform would encourage users to make the leap more easily.

The user journey was inspired by the experience of buying a car. Beginning with a walk-around of the capital equipment to build intrigue and anticipation, it takes a little while for the user to be sat in the ‘driver’s seat’ to operate the robot.

At that point, a rundown of the controls is delivered piecemeal, up from elementary actions like moving the arms, to more complicated ones like clutching and moving the robot’s head.

I learned how laparoscopic training is delivered to surgeons in the real-world, trying many myself, and watching our internal medical consultants complete exercises with assuredness. I developed four ‘Fundamentals of Vicarious Surgery’ (FVS) trainers: Peg and Hoop, Shape Matching, Bubble Pop and Suture Threading exercises.

Not all FVS concepts were suitable however, due only partly to technical constraints; more critical would have been the demand placed on users whose cognitive load would quickly be topped with learning to operating the robot. Hence, Shape Matching and Suture Threading were out. The latter omission was well-justified: suture threading would be a prominent feature in the latter part of the product walkthrough in any case (soft-body and rope physics would be employed in an anatomically faithful suturing step) and this had the benefit of supporting one of my chief objectives of providing a gradual learning curve to reduce frustration as far as possible.

The user sits at the surgeon console to engage in a peg and hoop exercise.

Our experience in ‘hyper-training’ and gamification allowed us to have some fun reimagining conventional laparoscopic trainers to better suit an interactive digital medium.

I created the UI and the geometric ‘laparoscopic arena’ encapsulating the robot using Illustrator, Figma and Blender.

The gradual learning curve was a keystone principle in the design process. I intended the ‘narrative arc’ of the experience to support this, applying a usability gradient to its overall presentation (from passive, cinematic reveal of the robot in vivo, to embodiment of the confident observer/actor in a complex OR) and interaction & UX design (from piecemeal tutorial of isolated functions, to FVS trainer, to the culminating hernia repair exercise).

I leveraged the principles of the Zone of Proximal Development (ZPD) to foster engagement and learning throughout: their journeys were scaffolded with visual guidance and timely feedback, but with restraint, given that users were likely to have Vicarious reps on hand to offer their own additional guidance.

The gap between new users’ abilities and their potential aptitude was likely to be vast, even for those with experience operating surgical robots like Intuitive’s da Vinci.

**The robot from Vicarious Surgical Inc.** I drew the storyboard and directed the animation for the robot’s reveal in an opening cinematic segment, using visual storytelling techniques to stage the moment for dramatic impact.

The Challenges

This project was a significant challenge. I had to coordinate many technically complex tasks and manage a web of dependencies as Product Owner, alongside the significant contributions I made as Design Lead. For FundamentalVR, many innovations would be developed for the first time in this simulation, not least the use of soft-body and rope physics to simulate suturing tasks that the robot could perform in the real-world.

We integrated both first- and third-party proprietary software from many sources to build the experience. The soft-body physics used for the deformable abdominal cavity were so cutting-edge that my development team worked in parallel with the R&D department to synchronise our roadmaps and share resources.

Development of the robot continued in parallel with the VR experience production. To keep our digital twin faithful to the physical product, we initially set out to integrate Vicarious’ robotic control software into the Fundamental Surgery engine. However, this was quickly rendered a non-viable path for Vicarious, so our internal R&D team had to develop a unique IK system from scratch, based on extensive consultation with their technical group.

Navigating the Experience

**Performance optimisation (even on high-spec target hardware) was a constant concern** given the confluence of multiple physics engines, hi-fidelity graphics and multiple cameras.

The Result

There was ultimately the greatest engagement with the simplified showcase mode. Not only was this due to users being time poor, as expected and designed for, but they were invariably naive of VR as a platform, and were distracted by the novelty of that new experience.

Back to Projects