At Groove Jones, we believe technology has the power to move hearts, minds, and boundaries. When Pfizer challenged us to help communicate their story of resilience and groundbreaking cancer treatments, we leveraged the transformative capabilities of Apple Vision Pro and spatial computing to create an unforgettable, immersive experience.
Pfizer’s bold mission is clear: Outdo cancer through relentless innovation and hope. To vividly illustrate this commitment for patients, healthcare providers, and internal stakeholders, Pfizer collaborated with Groove Jones to craft an immersive mixed reality experience that inspires as much as it informs. Apple Vision Pro’s ability to seamlessly blend digital and real worlds provided the perfect canvas to emotionally connect audiences and showcase Pfizer’s oncology leadership.
Emotional Storytelling: Creating Deep Connections
The experience opens with poignant, real-life narratives from patients, families, and scientists—each sharing their personal struggles, hopes, and victories against cancer. Cinematic visuals, spatial audio, and immersive storytelling techniques create an empathetic bond, underscored by the sobering statistic that 1 in 3 people in the U.S. will face cancer in their lifetime. This storytelling approach ensures the experience deeply resonates and moves audiences.
Immersive Journey: Enter the Lab
Through smooth transitions guided by spatial sound cues, users enter an authentic Pfizer laboratory environment surrounded by scientists and equipment. Apple Vision Pro’s gesture controls empower users to interact naturally with the lab’s contents—accessing information, exploring advanced technologies, and engaging directly with scientific innovations.
At the microscope, users enter the ADC Builder, an interactive experience where they design their own Antibody-Drug Conjugate (ADC), Pfizer’s targeted cancer therapy.
Users can look closely and see the work in progress.
Learn by Doing: Hands-On ADC Creation
In the experience, participants can design and build their own unique ADC. An ADC is a guided delivery system that has three components: an antibody, a linker, and a cancer-killing drug, also called a payload. The antibody targets the cancer cell, while the linker is designed to hold the payload on the antibody until it reaches the cancer cell. The goal is to kill the cancer cell while limiting damage to normal tissue.
- Antibody Selection: Users choose an antibody, learning how antibodies specifically target cancer cells.
- Linker Integration: Selecting a linker teaches users about its critical role in binding antibodies and drug payloads.
- Payload Attachment: Users finalize their ADC by selecting a potent drug payload, understanding how ADCs precisely destroy cancer cells while sparing healthy tissue.
Spatial audio and real-time 3D visualization make complex scientific concepts engaging and clear.
Interactive Exploration: Inside the Bloodstream
Users are transported inside the human bloodstream, where their newly created ADC embarks on its mission. Interactive gameplay lets participants directly deploy ADCs, visually demonstrating how targeted treatments eliminate cancer cells.
Users release their personalized ADCs by targeting cancer cells through intuitive gesture controls. When enough ADCs are attached to the cancer cell, it is destroyed.
An ADC Tracker reinforces the experience, visually tracking participant progress in real time.
The Journey Behind Every Breakthrough
Bringing a new medicine to patients is a long and demanding journey, often taking 10 to 15 years of rigorous research, testing, and clinical trials before treatments become broadly available. It requires significant investment and the dedication of scientists working to better understand disease and develop innovative new solutions
Cutting-Edge Innovation: GrooveTech™ and Spatial Computing
Behind this powerful experience is a series of advanced technologies developed by Groove Jones:
GrooveTech™ Generative AI Pipeline: This proprietary technology creates ultra-high-resolution stereoscopic images representing real patient stories, preserving identities while authentically capturing emotion. Custom depth maps enhance emotional engagement through lifelike 3D imagery.
Custom depth maps enhance emotional engagement through lifelike 3D imagery.
XR Avatar Scanner: Utilizing our XR Avatar Scanner rig of 50 iPhone 15 Pros, we captured team members to create photorealistic 3D scientists. Combined with Apple Object Capture, our custom software optimized these avatars, placing them seamlessly into the Pfizer lab environment.
RealityKit Transform Entity System: Over 1,000 transform entities dynamically instantiate and manage 44+ unique objects within the lab environment, enabling rich interactivity and immersion. This system provides true spatial freedom, allowing users to naturally navigate and engage within the experience.
Custom Spatial Audio Design: Custom-recorded voice-overs, meticulous sound design, and original music scoring were spatially integrated to enrich every narrative and interactive element, significantly enhancing the immersive experience.
Individual audio files were then placed in the 3d space to create an immersive, life-like spatial audio experience.
Optimized for visionOS 2: Engineered specifically for Apple’s visionOS 2 platform, the app maintains exceptional performance, high frame rates, and minimal latency—ensuring a comfortable, seamless user experience.
Impact and Future Potential
This immersive Apple Vision Pro experience marks just the beginning of Pfizer’s exploration into mixed reality. It establishes a powerful new benchmark for communicating complex medical innovation and offers expansive potential for future educational, training, recruitment, and research applications.
Pfizer Oncology continues gaining ground against cancer through breakthrough science and innovation, bringing science to life to create more moments that matter. Its purpose is clear: breakthroughs that change the lives of people with cancer. Learn more here – https://cancer.pfizer.com/
