Leap Motion Co-Founders Talk ‘Dragonfly’ Made-for-VR Motion Input Camera

In this opening segment, Ben from Road to VR introduces the audience to the Leap Motion showcase at CES 2015. Ben speaks with David and Michael from Leap about their latest technology, specifically the Dragonfly sensor prototype. This device offers high-definition resolution with RGB and infrared pixels, enabling interesting VR/AR applications. The conversation touches on new user interface (UI) elements for developers, including holographic interfaces for more immersive control over VR/AR settings.

Michael discusses Leap Motion's growing focus on virtual reality (VR) and why the company sees VR as a major opportunity. They recognized the potential of Leap’s input technology as a key tool in the evolving VR space. Michael explains the excitement around VR development and how Leap’s technology can help define new methods of interaction with VR devices. The conversation highlights the aim to embed Leap sensors directly into VR headsets for a seamless user experience and cost efficiency, rather than relying on peripheral attachments.

The discussion moves into the advantages of embedding Leap Motion directly into VR headsets, including cost reduction and the ability to add features that aren't feasible with standalone peripherals. Michael mentions that embedding the technology helps developers by ensuring more users have access to Leap’s hand-tracking capabilities. Embedding also allows manufacturers to present a complete, integrated device to consumers, avoiding the limitations of external peripherals.

Michael continues by discussing the future of the Dragonfly sensor and Leap Motion’s hand-tracking technology. He explains that Dragonfly is a prototype and not necessarily a second generation of Leap, but rather part of ongoing internal experiments. Michael emphasizes that Leap Motion’s core strength lies in software, which is continuously evolving, and that even current hardware peripherals will benefit from future software updates.

The conversation delves deeper into the technical advancements of the Dragonfly prototype. The Dragonfly includes RGB cameras, enhancing the immersive experience by allowing users to see the world in color. This creates a more realistic experience compared to infrared-only systems. The broader field of view provided by Dragonfly allows for more comprehensive hand-tracking, even when users extend their hands beyond the immediate vicinity. Michael also notes that the future could involve multiple cameras to achieve an even wider range of tracking.

Michael explains that while hardware is important, Leap Motion’s true innovation comes from its software. The company is focused on improving hand-tracking precision and interaction through software advancements. Software enhancements are key to bridging the gap between physical hand movements and virtual interactions, ensuring accuracy when touching surfaces and other virtual objects. These updates will also apply to the current Leap peripheral, which continues to evolve with new software features.

The introduction of RGB cameras in the Dragonfly sensor makes the VR/AR experience more convincing by allowing users to see the real world in color, which helps blend virtual and actual environments. Michael discusses how this creates opportunities for mixed reality experiences, where users can interact with both digital and physical elements simultaneously. The RGB feature enhances immersion, making it easier for users to perceive the virtual world as part of their surroundings.

Michael describes how Dragonfly is tailored for VR, with design considerations like human vision baseline and wider field of view to improve tracking at greater distances. The baseline between cameras, similar to the inter-pupillary distance (IPD) in human vision, enhances hand-tracking performance when users reach out in VR. This is a major improvement over earlier Leap peripherals, which had limitations in tracking hand movements over longer distances.

The conversation shifts to the challenges of maintaining accurate hand tracking, especially when hands move outside of the camera’s field of view. Michael outlines potential future solutions, such as wider field-of-view cameras or software that remembers hand positions when they leave the tracking zone. He hints at the possibility of integrating multiple cameras for even greater coverage in future VR systems.

The team demonstrates a swipe gesture that allows users to switch between the real world and the virtual world seamlessly. Michael discusses the thought process behind creating this gesture, which provides a smooth and intuitive transition between realities. He explains how matching the gesture to the visual transition makes the experience more cohesive and memorable for users, highlighting Leap’s focus on creating fluid and natural user experiences.

In the final part of the discussion, Michael shares that there will be upcoming announcements about Dragonfly’s integration with other devices. While Dragonfly is just the beginning, Leap Motion’s software will continue to evolve, improving hand tracking and user interaction in VR and AR. The team is excited about future opportunities to refine the technology and integrate it into more consumer products.