Adobe Answers the Sora Question & New SD3 Features!
TLDRIn this video, the host discusses several advancements in AI technology. They start with the release of Stable Diffusion 3, which excels in prompt understanding and text generation within images. The host highlights new features like a creative upscaler, search and replace, and background removal. They also compare Stable Diffusion 3 with Stable Diffusion XL, noting the improved clarity and detail in the former. Next, they discuss Adobe's integration of generative AI video features into Adobe Premiere, including object removal and addition, and generative extend. Adobe's commitment to an open ecosystem and partnerships with AI model providers is emphasized. The host then explores Microsoft's entry into the AI avatar space with Vasa 1, which uses audio to drive realistic facial expressions and lip sync. Lastly, they mention their experience as a judge in the first AI art esports competition, which utilized Leonardo's real-time drawing tool for a unique and exciting event.
Takeaways
- ๐ Stable Diffusion 3 has been released with improved prompt understanding and text-to-image generation capabilities.
- ๐ The new version includes features like a creative upscaler up to 4K, in-painting, out-painting, search and replace, and background removal.
- ๐จ There's an image-to-video feature that connects with Stable Diffusion Video, and an image-to-image editing option with a mask for detailed modifications.
- ๐ Adobe discussed their upcoming generative AI video features for Adobe Premiere, focusing on object removal, addition, and generative extend functionalities.
- ๐ค Microsoft entered the AI avatar market with VASA 1, which generates lifelike, audio-driven talking faces in real time with impressive lip sync and facial nuances.
- ๐ Adobe is working on integrating third-party AI models, like OpenAI's Sora, into Premiere Pro for more specialized use cases.
- ๐น Adobe is also exploring generative audio to accompany video clip extensions, offering more control and granularity over the output.
- ๐ธ Adobe's Firefly video model will continue to be commercially safe, and any third-party models used will receive content credentials.
- ๐ Adobe is aiming to improve smart and manual masking tools within Premiere, enhancing the user experience for video editing.
- ๐ฎ The first AI art esports competition was held, showcasing the potential for community events centered around AI technology.
- ๐ The event featured real-time drawing using Leonardo's tool, where contestants had to generate images based on prompts in just one minute.
Q & A
What was the main focus of the NAB convention in Las Vegas that the speaker attended?
-The main focus of the NAB convention in Las Vegas, as mentioned by the speaker, was on generative AI and collaboration, with a particular emphasis on Adobe's new AI video generation features and the integration of these technologies into Adobe Premiere.
What are the two versions of Stable Diffusion 3 mentioned in the transcript?
-The two versions of Stable Diffusion 3 mentioned are Stable Diffusion 3 and Stable Diffusion 3 Turbo.
What new feature does Stable Diffusion 3 offer that is considered a time-saver for image editing?
-Stable Diffusion 3 offers a 'remove background' feature built right in, which is considered a time-saver as it eliminates the need for manual masking tools.
How does Stable Diffusion 3 improve upon its predecessor, Stable Diffusion XL, in terms of image generation?
-Stable Diffusion 3 improves upon Stable Diffusion XL by offering better prompt understanding, sharper and crisper images, more detailed patterns in the generated images, and a more accurate representation of objects as described in the prompt.
What is the name of the new platform announced by Stability AI?
-The new platform announced by Stability AI is called Stable Assistant Beta.
What kind of access does the Stable Assistant Beta offer to its paying subscribers?
-Stable Assistant Beta offers paying subscribers access to the latest models, the ability to generate images, write content, and match photos to text through conversation.
What is the significance of the Firefly image model mentioned in the context of Adobe?
-The Firefly image model is a collection of AI models within Adobe that the company is using to develop its video model. It signifies Adobe's commitment to integrating advanced AI capabilities into their creative software suite.
What are some of the AI video generation features that Adobe is planning to integrate into Adobe Premiere?
-Adobe is planning to integrate features such as object removal, object addition, generative extend (to match the beat of music), and clip extensions that allow for more control over video editing through AI.
How does Microsoft's Vasa AI differ from other AI avatar models?
-Microsoft's Vasa AI differs from other models as it is audio-driven rather than text-based, which allows for more natural and realistic lip sync and a wider range of facial nuances and head motions.
What was the role of the speaker in the first AI art esports competition?
-The speaker served as a judge in the first AI art esports competition, which was organized by Creative Refuge.
What is the potential impact of community events based around AI technology like the AI art esports competition?
-Community events based around AI technology, like the AI art esports competition, illustrate the potential for AI to bring people together in creative and competitive ways, fostering a sense of community and showcasing the capabilities of AI in a fun and engaging format.
Outlines
๐ Stable Diffusion 3 Release and Features
The video begins with the host returning from the NAB convention in Las Vegas and immediately diving into the latest developments in AI. The main highlight is the release of Stable Diffusion 3, which is praised for its prompt understanding and text-to-image generation capabilities. The host discusses the new features of Stable Diffusion 3, including a creative upscaler that can reach up to 4K resolution, in-painting and out-painting tools, search and replace functionality, background removal, and an image-to-video feature. A comparison is made between Stable Diffusion 3 and Stable Diffusion XL, showcasing the improved clarity and detail in the examples provided. The host also mentions Stable Assistant Beta, a new platform offering a chatbot service for generating images, writing content, and matching photos to text through conversation, with pricing plans available for subscribers.
๐ฅ๏ธ Adobe Premiere's AI Video Generation Features
The host follows up on a previous video by discussing Adobe's announcement of generative AI video features in Adobe Premiere. An interview with Kyle from Adobe provides insights into the upcoming video model, which will allow for object removal, object addition, generative extend, and other AI-powered editing capabilities. The conversation touches on Adobe's open ecosystem and partnerships with other AI model providers, such as Open AI, Sora, Model Runway ML, and Pika. The host inquires about the level of control users will have over the output and the potential for a standalone platform for video, to which Kyle responds that the features will be integrated within the Adobe ecosystem. They also discuss the commercial safety of third-party models and the importance of content credentials for transparency in media generation.
๐ค Microsoft's Vasa 1: AI Avatars with Real-Time Audio-Driven Faces
The host introduces Microsoft's entry into the AI avatar space with Vasa 1, which generates lifelike, audio-driven talking faces in real time. Unlike text-based models, Vasa 1 is noted for its impressive lip-sync and the natural range of facial expressions and head movements it can produce. The host shares examples of Vasa 1 in action, highlighting the realistic feel when the avatars are driven by actual recorded audio. The host also notes some minor issues with the technology, such as occasional jerking head movements and hair that doesn't track perfectly with head movements. The segment ends with a mention of camera controls within Vasa 1 and the potential use of the technology in applications like Microsoft Teams.
๐ฎ First AI Art Esports Competition
The host shares his experience as a judge in the first AI art esports competition organized by Creative Refuge. The event involved contestants using Leonardo's real-time drawing tool to create images based on prompts given by the audience and selected by the judges. The host describes the competition as more exciting and nail-biting than expected, emphasizing the potential for community events centered around AI technology. He encourages viewers to check out Creative Refuge's channel for a full rundown of the event and concludes the video by expressing his exhaustion from the NAB convention in Las Vegas.
Mindmap
Keywords
๐กNAB convention
๐กAI video generation features
๐กMicrosoft AI avatar
๐กStable Diffusion 3
๐กAdobe Premiere
๐กStable Assistant Beta
๐กAI art esport competition
๐กFirefly image model
๐กContent credentials
๐กSmart masking
๐กAI-generated lip sync
Highlights
Stable Diffusion 3 has been released with improved prompt understanding and text within images generation.
Stable Diffusion 3 offers two versions: standard and 'turbo', with the latter being optimized for speed.
The new version includes a creative upscaler that can upscale images up to 4K resolution.
In-painting and out-painting features have been enhanced with a search and replace function that doesn't require a mask.
A built-in background removal feature is included for convenience.
Image-to-video functionality connects directly to Stable Diffusion Video.
Image-to-image editing with a mask allows for text or image prompts to modify specific parts of an image.
Comparisons between Stable Diffusion 3 and XL show significant improvements in image quality and detail.
Adobe answers questions about new AI video generation features coming to Premiere Pro.
Adobe's generative AI video features will allow object removal, addition, and generative extend functionalities.
Adobe is working with big video AI model providers like Open AI, Sora, Model Runway ML, and Pika.
Stable Assistant Beta is a new platform offering a friendly chatbot for subscribers to generate images and content.
Microsoft has entered the AI avatar game with VASA, an audio-driven, lifelike talking face generation system.
VASA's lip sync and facial nuances contribute to a high level of authenticity in the generated avatars.
The first AI art esports competition was held, showcasing the potential for community events based on AI technology.
The competition used Leonardo's real-time drawing tool, emphasizing the potential for creative applications of AI.
Adobe is focusing on bringing smart masking and improved manual masking tools to Premiere Pro.
Generative audio will be included alongside clip extensions in Adobe's upcoming features.