How to Build Multi-Agent AI Systems with OpenAI Swarm & Secure Them Using Portkey

This paragraph introduces the topic of the video, which is Open AI Swarm, an educational multi-agent system, and Portkey, an AI Gateway. The speaker, JD, explains that Open AI Swarm allows orchestration and handoffs between different AI routines and highlights the benefits of integrating it with Portkey for observability, security, and advanced configurations like Canary tests. The video aims to explore the setup of an agent in Swarm, which involves naming, instructions (prompts), functions (tools), and message history. The importance of the Swarm client is emphasized, as it can be customized to use different models compliant with the Open AI specification, which is crucial for setting up observability.

The speaker demonstrates how to connect Open AI Swarm to Portkey, an AI Gateway that acts as a proxy, providing observability and security features. By passing a custom client to Swarm, one can set up the base URL and different models. The video shows an example where the Open AI key and base URL are parameterized, and headers are passed, allowing the system to hit the AI Gateway provided by Portkey instead of going directly to a model. Portkey's benefits include a control panel for AI apps, open-source availability for self-hosting, and a freemium model for analytics and feature access. The logs show the request sequence and token costs, similar to systems like Link Smith or Link Fuse, providing transparency into the system's operations.

The video moves on to a more complex example involving a trip planner, showcasing the multi-agent architecture in action. The speaker explains that the interaction with the OpenAI client remains largely the same, but the client is passed to Swarm. The example includes functions for getting weather, booking flights, and booking hotels, with each agent having specific tools and responsibilities. The agents can operate independently or communicate with each other, allowing for flexibility in application design. The trip planner example demonstrates how agents can transfer between each other based on the context and user instructions, such as booking a flight and then a hotel for a 5-day trip to Japan, based on weather conditions.

The speaker discusses the potential to incorporate payment APIs like Stripe into the multi-agent system to create a checkout link for booking hotels or flights. This integration allows users to purchase tickets directly from a link provided in a web interface or text message. The paragraph also revisits Portkey to examine the observability of the system, showing how requests are processed and how the system maintains a history of interactions. The speaker highlights the ability to see handoffs between agents and the parameters passed to book hotels, as well as the system's dynamic function building based on requirements. Portkey also provides feedback mechanisms to evaluate response quality and the ability to set guardrails for responses to prevent issues like PHI detection or biases.

The final paragraph covers additional security features of Portkey, such as virtual keys for rotating API keys without hardcoding them into the application, and load balancing to distribute requests evenly across different keys. The speaker also explains how to set up Canary tests or AB tests using Portkey, allowing a small percentage of requests to test new models or prompts while the majority use the standard setup. This feature is useful for observing the impact of changes in a controlled manner. The video concludes with a summary of the exploration of multi-agent architecture with Open AI Swarm and Portkey, focusing on security, observability, and the ability to perform Canary tests.