Leap Motion hands-on

The Verge
26 Jun 201205:00

TLDRDavid Holes, CTO of Leap Motion, demonstrates their next-generation 3D motion-sensing device. The device accurately tracks all ten fingers and subtle hand movements, allowing users to interact with digital content in 3D. It simulates a virtual touchscreen, offering effortless backward compatibility with legacy applications. The demo shows precision drawing, object manipulation, and multitasking using both hands. The device enhances dynamic feedback rather than relying on predefined gestures, providing a more intuitive user experience. Holes emphasizes the potential for specialized apps and new interactions with virtual objects, making digital manipulation feel natural and seamless.

Takeaways

  • 👋 Leap Motion is developing a next-generation 3D motion-sensing device.
  • ✍️ The device allows for fast and responsive 2D and 3D interactions, even for detailed actions like drawing within millimeters.
  • 👐 It can track all 10 fingers, recognizing the shape and subtle motions of each finger.
  • 🔄 Users can draw in 3D, rotate objects, and interact with virtual space naturally using multiple hands and fingers.
  • 📱 The device provides effortless backward compatibility by simulating a virtual touchscreen for legacy touch applications.
  • 🎮 It supports dynamic interactions, like pinch-to-zoom and multi-hand operations, without the need for specific gestures.
  • 🖊️ The technology can sense the orientation of tools like pens, making it possible to perform more precise tasks, such as drawing cursive letters.
  • 💻 A specialized app store will offer apps specifically designed for the device, as well as compatible apps.
  • 🎨 The device allows for a more intuitive manipulation of virtual objects, bridging the gap between physical interaction and virtual interfaces.
  • 🤯 The immersive experience feels natural, moving beyond gestures to a sensation of physically handling objects in virtual space.

Q & A

  • What is Leap Motion developing?

    -Leap Motion is developing a next-generation 3D motion sensing device.

  • What is the capability of the device in terms of finger recognition?

    -The device can recognize all 10 fingers and see the actual shape of the hand.

  • Can the device track subtle motions?

    -Yes, the device can track very subtle motions, as demonstrated by drawing within a few millimeters.

  • How does the device handle 3D interactions?

    -The device can track multiple fingers and perform actions associated with touch, such as pinch to zoom, in a 3D space.

  • What is the concept of dynamic feedback mentioned in the script?

    -Dynamic feedback refers to the device showing what you're doing in real-time rather than instructing you on how to perform a certain action.

  • How does the device provide backward compatibility with legacy touch applications?

    -The device can project a virtual touchscreen above the actual screen, allowing legacy touch applications to work without modification.

  • What is the potential of the device for specialized app development?

    -There will be a specialized app store for apps designed for the device, offering compelling experiences.

  • How does the device handle direction sensing?

    -The device can sense the direction of a pen or tool, allowing for natural writing and drawing motions.

  • What is the advantage of the device over a traditional mouse?

    -The device offers the connectedness of touch with the responsiveness and sensitivity of a mouse, allowing for efficient interactions.

  • What kind of natural interactions does the device enable?

    -The device allows for natural interactions like reaching into a virtual space and moving things around in a physical way.

  • How does the device change the user's interaction with computers?

    -The device offers a new way of interacting with computers that feels intuitive and natural, unlike traditional interfaces.

Outlines

00:00

👨‍💻 Introduction to Leap Motion's CTO and Their 3D Motion-Sensing Device

David Holes, the CTO of Leap Motion, introduces the company's next-generation 3D motion-sensing device. He provides an overview of their various demos, starting with a simple 2D drawing to demonstrate how precise and responsive the device is. The demo allows users to draw accurately within millimeters and is powered by a version of the Leap Motion software.

🖐️ Tracking Hand Movements with Precision

David explains the device's capability to track all ten fingers, capturing even the subtle motions of individual fingers. Users can perform complex gestures like drawing spirals in 3D space, and the device accurately tracks these movements. The demo showcases the ability to perform actions typically associated with touch interfaces, such as pinch-to-zoom, using multiple fingers.

👾 Backward Compatibility with Touch Applications

Leap Motion’s device is backward compatible with existing touch applications, even those that weren't specifically designed for the technology. The system can create a 'virtual touchscreen' that allows users to interact with software designed for traditional touchscreens by projecting a touch interface in the air, making it easier to use legacy apps.

🖌️ Combining Tools and Hand Gestures

The demo highlights the device’s ability to simultaneously track hands and tools, allowing users to perform tasks like drawing with one hand while rotating the page with the other. The device can differentiate between various objects, such as mapping a pencil to drawing or a marker to a broader stroke, creating an intuitive experience even in 2D environments.

📱 Dedicated App Store for Leap Motion

David briefly mentions that there will be a specialized app store for the device, offering a wide range of applications built specifically for Leap Motion’s unique capabilities. He notes that third-party developers can also create compatible apps, expanding the ecosystem with compelling experiences.

🖊️ Pen Rotation and Fine-Tuned Control

The device can not only track the location of tools like pens but also detect their orientation in 3D space. David demonstrates this by rotating a pen and envisioning writing cursive letters with smooth, natural movements. This type of functionality, which is difficult for traditional touchscreens, is one of the key strengths of the Leap Motion device.

🎮 Enhanced Accuracy and Efficiency

David discusses how Leap Motion combines the precision of a touch interface with the efficiency of a mouse. The device can map a small physical movement to a larger screen area, allowing users to control the entire screen with minimal hand movement. This balance of accuracy and responsiveness offers a connected, intuitive experience.

🍎 Natural Interaction in Virtual Space

The device excels in virtual environments, where users can reach into space, move objects, and perform complex interactions that feel natural and physical. David demonstrates how easy it is to manipulate virtual objects, rotate them, and perform intricate tasks with hand gestures, highlighting the device’s potential for immersive, intuitive interaction.

🖼️ Manipulating Virtual Objects Intuitively

Instead of relying on predefined gestures, Leap Motion’s technology allows users to naturally manipulate virtual objects as if they were holding them in real life. David illustrates this by explaining how users can grab, rotate, and interact with objects seamlessly, without having to think about specific commands or modes. This natural interaction feels intuitive and fluid, similar to real-world actions.

Mindmap

Keywords

💡3D motion sensing

3D motion sensing refers to technology that detects and tracks movement in three dimensions, allowing for the recognition of hand gestures and movements in space. In the video, Leap Motion's device can sense the subtle motions of fingers, hands, and even tools, making it highly responsive to user input in 3D environments.

💡10 fingers recognition

The Leap Motion device can detect and track all 10 fingers of the user. This capability allows for a highly detailed and nuanced interaction with virtual objects, such as drawing or manipulating items in 3D space, as demonstrated in the video when the presenter rotates virtual spirals and shapes.

💡Pinch to zoom

Pinch to zoom is a common touch-based gesture where two fingers are pinched together or apart to zoom in or out on a screen. The video highlights that Leap Motion can replicate this gesture in mid-air, offering precise control without the need for a physical touchscreen.

💡Backwards compatibility

Backwards compatibility refers to the device's ability to work with older touch-based applications without the need for additional software. In the video, it is demonstrated how Leap Motion projects a virtual touchscreen, enabling touch-based apps to recognize hand motions as if they were on a physical touch device.

💡Virtual touchscreen

A virtual touchscreen is a non-physical projection of a touch interface that allows users to interact with applications without physically touching a screen. Leap Motion uses this concept to translate hand movements in the air into inputs that a touchscreen app can recognize, as shown when the presenter manipulates objects by pointing and touching virtually.

💡Effortless dynamic feedback

Dynamic feedback refers to the real-time response users receive from interacting with a system. In the Leap Motion device, feedback is intuitive and immediate, as the system tracks subtle motions and provides responsive actions, such as drawing or rotating objects in 3D space, without requiring predefined gestures.

💡Multiple hands and tools detection

This refers to the ability of the Leap Motion device to track more than one hand simultaneously, as well as different tools like pencils or markers. This allows for complex interactions, such as drawing with one hand while rotating an object with the other, enhancing multitasking in virtual environments.

💡Touchscreen limitations

The video discusses the limitations of traditional touchscreens, which are confined to 2D and can’t capture the depth or subtlety of hand movements. Leap Motion overcomes these limitations by allowing users to interact in 3D space, offering more natural and precise controls, such as rotating objects or drawing detailed shapes in the air.

💡Intuitive manipulation

Intuitive manipulation refers to the natural way users can interact with virtual objects using hand movements, without the need to learn complex gestures. In the video, the presenter describes how Leap Motion allows for smooth, instinctive interactions, like grabbing and rotating virtual objects, as if they were physical objects.

💡Specialized app store

The specialized app store is mentioned as a platform where users can access apps designed specifically for the Leap Motion device. This ensures that users can fully experience the device's capabilities with applications tailored to its 3D motion sensing technology, expanding its functionality beyond just basic gestures.

Highlights

Leap Motion is developing a next-generation 3D motion sensing device.

The device is fast, responsive, and capable of fine precision down to a few millimeters.

It can track all 10 fingers, recognizing the shape of the hand and subtle movements.

The technology allows for drawing in 3D space, showing both precision and depth.

The system supports dynamic feedback rather than relying on specific gestures.

Leap Motion's device can emulate a touchscreen, providing backward compatibility for legacy touch applications.

The device allows multiple hands and tools to be used simultaneously, like drawing with one hand and rotating with the other.

It can detect different objects, mapping tools like pencils and markers to specific actions.

A specialized app store will offer apps tailored for Leap Motion's device.

The system can sense not just the location of a tool but also its orientation, like rotating a pen for precise input.

The device maps small physical movements to large virtual actions, offering efficiency and precision.

It combines the connectedness of touch with the accuracy and sensitivity of a mouse.

The device enables intuitive, physical manipulation of objects in a virtual space, allowing users to reach in, move, and rotate objects naturally.

Leap Motion creates a new way of interacting with virtual objects, making it feel as though you're holding and manipulating them in your hands.

The experience is described as intuitive, blurring the lines between virtual gestures and real physical movements.