GEN-3: The Ultimate Prompting Guide
TLDRRunway ML's Gen 3 model represents a significant leap forward in AI video capabilities, marking the beginning of a new era. This video offers an ultimate prompting guide for Gen 3, showcasing its advanced features compared to Gen 2. The presenter shares insights on crafting effective prompts, emphasizing the importance of descriptive language over keyword spamming. Examples are provided to illustrate how adding details and structure to prompts can vastly improve video generation. The video also discusses the use of keywords related to subject, action, setting, shot, and style to maximize output quality. It explores community ideas, such as using the word 'suddenly' for dramatic effects, and the potential of Gen 3 to handle text and mimic styles like the MCU opening. The presenter encourages experimentation with prompts and rating outputs to aid in model improvement, promising more features like image to video in future updates.
Takeaways
- ๐ Runway ML's Gen 3 has been released, marking a significant advancement in AI video technology and solidifying the 2.0 era of AI.
- ๐ The presenter has spent considerable time researching, testing, and studying Gen 3 to provide an ultimate prompting guide for users.
- ๐ฌ A comparison shows the evolution from Gen 2's text-to-video capabilities to Gen 3's enhanced video generation, demonstrating substantial progress in a short time.
- ๐ Gen 3 allows for more descriptive prompting, moving away from keyword spamming to a more modern style of prompting.
- ๐ The importance of including details like subject, action, setting, and shot in the prompt is emphasized for better video generation.
- ๐ The use of adjectives and mood characteristics in prompts can enhance the generated video's atmosphere and style.
- ๐ A PDF with a list of shot terms and prompts for Gen 3 is available for free on Gumroad, aiding users in experimenting with different prompt structures.
- ๐ฅ Gen 3 adheres closely to prompts, sometimes inserting cuts or dissolves to fulfill the user's request, even if it results in unusual transitions.
- ๐ If a generation is liked, users can reuse the seed and make minor adjustments to iterate on the output, maintaining the overall look while exploring variations.
- ๐ The community has noted that certain words like 'suddenly' can create interesting effects in Gen 3, and experimenting with these can lead to unique videos.
- ๐ Gen 3 can also handle text in prompts, as shown by examples mimicking the MCU opening and creating 3D letter reveals, demonstrating the model's versatility.
Q & A
What is the main topic of the video transcript?
-The main topic of the video transcript is an introduction and guide to Runway ML's Gen 3 model, which is a significant advancement in AI video technology, providing an ultimate prompting guide for using Gen 3 effectively.
How does the narrator describe the progress from Gen 2 to Gen 3 in AI video technology?
-The narrator describes the progress from Gen 2 to Gen 3 as a significant step forward that has brought them into a new 2.0 era of AI, with a vast improvement in the quality and capabilities of AI-generated videos in a very short amount of time.
What is the key difference between prompting in Gen 2 and Gen 3 according to the transcript?
-The key difference is that Gen 3 allows for more descriptive prompting and is less focused on spamming keywords, enabling users to be more detailed in their prompts and resulting in improved video generation.
What is an example of a prompt that was improved with additional details in Gen 3?
-An example given is the prompt 'the man in Black fled across the desert and the Gunslinger followed' which, with additional details, became 'long shot in the distance a man in Black robes calmly walks across a vast desert Wasteland the camera orbits to reveal a gunslinger watching him with steely eyes', resulting in a vastly improved video shot.
Why is it important to include keywords in your prompts according to the video?
-Including keywords in your prompts is important because it helps maximize the generation capabilities of Gen 3, ensuring that you cover essential aspects such as subject, action, setting, and shot type.
What is the significance of the 'style' keyword in prompts?
-The 'style' keyword is significant because it reinforces the overall look that the user is going for in the generated video, with examples like 'cinematic' or 'IMAX' improving the visual output.
How can users experiment with prompts to achieve different results?
-Users can experiment with prompts by changing the order of elements, such as putting the shot first or the subject in the middle, and iterating to see what results they get.
What is the narrator's advice on dealing with prompts that generate unexpected results?
-The narrator suggests that if a generation results in an unexpected output, users can reuse the seed from a previous generation and adjust the seed in the settings to generate a new output that maintains the overall look while avoiding the unwanted elements.
Can Gen 3 handle text in prompts and what is an example provided?
-Yes, Gen 3 can handle text in prompts. An example provided is a prompt that mimics the MCU Marvel opening, which involves close-ups of superhero comic book pages flipping and 3D lettering.
What is the narrator's advice on using the word 'suddenly' in prompts?
-The narrator notes that the word 'suddenly' can produce interesting effects, as demonstrated by a prompt that resulted in rain falling over a city suddenly, followed by a fast zoom down to the city street.
What is the narrator's suggestion for users who want to contribute to the improvement of Gen 3?
-The narrator suggests that users rate their outputs, as Gen 3 is still in the alpha phase and the model will continue to improve based on user feedback and interaction.
Outlines
๐ Introduction to Runway ML Gen 3
The video script introduces Runway ML's Gen 3, highlighting it as a significant advancement over the popular Gen 2 model, marking a new era in AI video technology. The speaker shares their extensive research, testing, and study of Gen 3 and promises to share their findings. They showcase the progress made since Gen 2 by revamping an old video, demonstrating the improved quality and capabilities. The script delves into the modern style of prompting for Gen 3, which allows for more descriptive prompts and less reliance on keyword spamming. Examples are given to illustrate how adding details and structuring prompts can lead to vastly improved video outputs. The speaker also emphasizes the importance of including keywords related to subject, action, setting, and shot, and provides a free PDF with a list of shot terms and prompts for Gen 3. The video concludes with a discussion on style keywords, such as 'cinematic' and 'IMAX,' and how they can enhance the overall look of the generated videos.
๐ฌ Exploring Runway ML Gen 3 Prompting Techniques
This paragraph discusses the quirks and capabilities of Runway ML Gen 3 in adhering to user prompts. The speaker notes that Gen 3 often includes cuts or dissolves to meet the prompt's requirements when it can't directly fulfill them. They share a trick for iterating on a generation by reusing the seed from a previous generation to maintain a similar style. The speaker also explores community ideas, such as using the word 'suddenly' to create dramatic changes in scenes. They mention Gen 3's ability to handle text, showcasing examples like mimicking the MCU opening and creating a miniature civilization on an ancient scroll. The speaker emphasizes the importance of experimenting with different prompts and finding ways around content restrictions. They also note that while Gen 3 can't turn script pages into video like Dream Factory, it can create impressive AI videos with the right prompts.
๐ Runway ML Gen 3's Timelapses and Future Features
The final paragraph of the script focuses on Runway ML Gen 3's ability to create timelapses, particularly those occurring at two different intervals. The speaker shares an example of a prompt that resulted in a successful timelapse of a woman watching the day turn into night. They encourage viewers to rate their outputs to help improve the model, which is still in its alpha phase. The speaker expresses excitement about upcoming features, such as image to video capabilities in Gen 3 and speculates on how tools like motion brush might be integrated. They conclude by inviting viewers to share their findings and favorite prompts, looking forward to exploring the new model together.
Mindmap
Keywords
๐กGen 3
๐กPrompting
๐กMorphing Issues
๐กDescriptive Prompting
๐กKeywords
๐กSeed
๐กStyle
๐กCommunity Ideas
๐กText and Video
๐กTime Lapses
Highlights
Runway ML's Gen 3 has arrived, a significant step forward in AI video technology.
The video provides an ultimate prompting guide for Gen 3.
Gen 3 allows for more descriptive prompting and less focus on spamming keywords.
A comparison of Gen 2 and Gen 3 shows significant improvement in video quality.
Prompt structuring is crucial for better results with Gen 3.
The importance of including color grading in the prompt for better visual results.
Keywords associated with subject, action, setting, and shot are essential for maximizing generation.
Adjectives can enhance the action description in prompts.
Mood characteristics can add depth to setting descriptions.
A list of shot terms for Gen 3 is available for free download.
There is no right or wrong way to prompt; experimentation is encouraged.
Style keywords like 'cinematic' and 'IMAX' can enhance the overall look of the video.
Gen 3 adheres closely to the prompt, sometimes inserting cuts or dissolves to meet the description.
Reusing a seed can help maintain the overall look in iterative generation.
Community ideas, such as using 'suddenly' in prompts, can yield interesting results.
Gen 3 can handle text prompts, as demonstrated by mimicking the MCU opening.
Prompts can be adapted to create unique videos, like a miniature civilization on an ancient scroll.
Gen 3 cannot create videos from actual script pages.
Gen 3 performs well with time lapses, especially at different intervals.
Rating outputs is crucial for the improvement of Gen 3 in its alpha phase.
There is much to explore with Gen 3, including potential future features like image to video.