Proactive AI Agents on Smart Glasses

Kaden Pierce from the MIT Media Lab discusses the potential of smart glasses to surpass smartphones in ubiquity and utility, emphasizing the need for a new kind of app that is proactive, contextual, and intelligent. He argues that merely replicating smartphone functions on glasses won't drive adoption, but rather, a 100x improvement is needed. This improvement would come from apps that understand the user's context and act proactively, rather than reactively. He illustrates this with an example of a scenario where a smart system could anticipate a user's needs based on their environment and actions, such as finding a hotel address and arranging transportation without user prompts.

The speaker continues to explore the concept of proactive AI agents, using personal anecdotes to illustrate their potential. He describes a scenario where an AI agent, aware of the context of a late-night conversation about dark chocolate, proactively searches for and provides information on caffeine content, influencing the user's decision. He also discusses the value of AI agents in providing timely information during conversations, such as weather updates or definitions, without the user needing to ask. The emphasis is on the AI's ability to understand context and act preemptively to enhance user experience.

Kaden Pierce envisions a future where smart glasses equipped with augmented reality and AI provide contextual information in retail settings. He suggests that as users navigate a mall, AI could identify their needs and preferences, proactively highlighting relevant stores and products. This could include reviews, product information, and personal recommendations based on the user's past behavior and current context. The goal is to create a seamless and intuitive shopping experience where the AI acts as a personal assistant, anticipating and fulfilling user needs without interruption.

The speaker introduces 'Convos Scope', a system designed to augment conversations through proactive AI agents. These agents can answer questions, generate ideas, and even play the role of a devil's advocate to stimulate deeper discussion. The system uses natural language processing to understand the context of conversations and provide relevant information in real-time. This could include defining unfamiliar terms, providing historical context, or suggesting alternative viewpoints to prevent groupthink. The aim is to enhance communication and problem-solving by integrating AI into the conversational flow.

Kaden Pierce discusses the technical challenges in developing proactive AI agents that can operate effectively. He highlights the need for a semantic layer or natural language interface on top of current operating systems and APIs. This layer would allow applications to describe their functionality and usefulness in context, enabling the operating system to decide when and how to present information to the user. The challenge lies in creating a system that can understand and act on user context without being explicitly told, thereby providing a seamless and intuitive user experience.

In the conclusion of his talk, Kaden Pierce reflects on the timely convergence of wearable technology and advanced AI. He predicts that lightweight, all-day wearables combined with increasingly intelligent AI will enable the development of proactive agents that feel like an extension of the user. He sees this as a shift from AI as a separate entity to AI as an exocortex, enhancing human capabilities. The speaker expresses excitement about the potential of human intelligence augmentation technology and its role in enabling us to learn, understand, and achieve more than ever before.