Future Colossal partnered with DDB Health to bring empathy and care to the treatment of Chronic Myeloid Leukemia (CML). At the American Society of Clinical Oncology's yearly conference in Chicago, Novartis deployed an innovative Augmented Reality (AR) mirror to allow attending doctors to put themselves in their patients' shoes and develop empathy for the way the treatment affects their lives.
DDB Health
BrandNovartis
UX Design, Content Design, Software Development, Creative Direction
The core component of this interactive AR mirror is what we called the "Green Screen Without A Green Screen", which consisted in segmenting the visitor engaging with the experience from their background and overlaying them on a background of our choice without the aid of a green screen.
We had several specific and challenging requirements for this effect. It had to be the cleanest cut possible. The goal was to be able to cut out individual fingers and hair strands while tracking a physical rock that was being handled and moved by the guest. Lastly, the experience needed to identify the person actively participating, and only show them over the background even when others were standing next to them.
To achieve this, our expert team of developers leveraged multiple sensors and combined their data with a several computer vision layers. The result surpasses any other background subtraction software currently available on the market and is so clean and sharp that it left some users questioning if they were actually just looking at a mirror and not truly being transported to a different scenario.
This innovative AR mirror took any attendee in front of it out of their environment and into a regular doctor's office, as if they were a patient. They were then asked to lift an innocuous rock that triggered the beginning of their journey. Throughout the experience, the rock in the mirror grows with every negative testimonial from a patient; symbolizing the weight that patients increasingly feel as they go through CML treatment, which on average can take in excess of 12 months.
The experience ran throughout the entire conference and impacted an average of 792 spectators that now have a new perspective on CML treatment.
