How to Prompt for Text to Video | Runway Academy
TLDRThis Runway Academy tutorial covers the fundamentals of text-to-video prompting in Gen 2. It guides viewers on crafting detailed prompts to control video output, including specific dog breeds, settings, and times of day. The video emphasizes the importance of focusing on 'what it looks like' and 'what it is', and demonstrates the use of style presets and prompt modifiers to refine the generation process, ultimately allowing for greater creative control and achieving the desired video style.
Takeaways
- 🐾 Start with a simple prompt for text to video in Gen 2, like 'a dog'.
- 🔍 Add specificity to your prompts by including details like dog breed, color, location, and time of day.
- 🎨 As you add more detail, the images and videos generated become more reflective of those details.
- 📸 Include details about the type of shot, such as 'extreme close-up' or 'shallow depth of field', for more control over output.
- 🖼️ Understand the difference between 'what it looks like' and 'what it is' for clear prompting.
- 🖌️ Experiment with style presets in Gen 2 to influence the style of your output.
- 🔧 Use prompt modifiers to fine-tune control over the output, adding them as comma-separated values.
- 🌆 Try combining prompts with modifiers for a stylized direction, such as 'a photo of a city skyline made out of paper' with modifiers like 'burnt texture'.
- 🎭 Remember to focus prompts on appearance and essence, avoiding run-on sentences or contradictory modifiers.
- 📚 Join the Runway Academy community on Discord for more resources and experimentation.
- 🔄 Keep prompts focused and avoid unnecessary filler words or too many concepts in a single prompt.
Q & A
What is the main topic discussed in the Runway Academy video?
-The main topic discussed in the video is the basics of prompting for text to video in Gen 2, including how to create specific and detailed prompts to control the output of images and videos.
How does the video guide viewers to start with text to video in Gen 2?
-The video instructs viewers to navigate to Gen 2, click on 'text to video', and enter a simple prompt such as 'a dog', then click 'free preview' to see the starting images.
What is the purpose of adding specific details to the prompt?
-Adding specific details to the prompt helps the images and resulting videos to reflect those details more meaningfully, giving the user more control over the outputs.
Can you give an example of a detailed prompt mentioned in the video?
-An example of a detailed prompt given in the video is 'a light-colored French Bulldog laying on a couch during the day'.
What are the two main things to focus on when creating a prompt according to the video?
-The two main things to focus on when creating a prompt are 'what it looks like' and 'what it is', which help in achieving a specific look and understanding the subject of the prompt.
What is the role of style presets in Gen 2 as mentioned in the video?
-Style presets in Gen 2, found at the bottom of the prompt box, have a significant influence on the style of the output results and can be used even with the most simplified prompts.
How can prompt modifiers be used to gain further control over the output?
-Prompt modifiers are descriptive words that can be added to the end of a prompt to help control the output further, and they can be used with style presets for even more control.
What is an example of a prompt modifier mentioned in the video?
-Examples of prompt modifiers mentioned in the video include 'burnt texture', 'black and gray color palette', and 'flat composition'.
What advice does the video give for creating effective prompts?
-The video advises to focus on what the output looks like and what it is, include relevant prompt modifiers, keep the prompt focused, and avoid run-on sentences, unnecessary filler words, too many concepts in a single prompt, or contradictory prompt modifiers.
How can viewers find more helpful resources and join the community for further learning?
-Viewers can join the community on Discord for more information and experimentation using Runway, or they can find specific answers to their questions by using the 'help' button on their dashboard.
What is the final piece of advice given in the video for using Gen 2 effectively?
-The final advice is to remember the formula and rules discussed in the video to take more control over Gen 2 and start creating in the specific style desired.
Outlines
📝 Introduction to Text-to-Video Prompting in Gen 2
The video script begins with an introduction to Runway Academy's tutorial on the basics of creating prompts for text-to-video in Gen 2. It guides viewers on how to navigate the platform, enter simple prompts, and utilize the 'free preview' feature to generate initial images. The script emphasizes the importance of specificity in prompts to refine the output, including details such as dog breed, location, and time of day. It also introduces the concept of adding descriptive details to achieve a particular look and style in the generated videos.
🔍 Enhancing Prompts for Detailed Video Outputs
This paragraph delves deeper into the art of prompting, illustrating how to enhance prompts with more specific details such as the type of shot, lighting conditions, and the setting to gain better control over the video output. The script uses the example of a light-colored French Bulldog on a couch to demonstrate how adding details like 'extreme close-up' and 'shallow depth of field' can influence the final result. It also touches on the importance of distinguishing between the visual appearance ('what it looks like') and the subject matter ('what it is') in crafting effective prompts.
🎨 Utilizing Style Presets and Prompt Modifiers
The script introduces the use of style presets and prompt modifiers in Gen 2 to achieve a desired style or to further control the output. Style presets are located at the bottom of the prompt box and can significantly influence the style of the generated content. Prompt modifiers are descriptive words that can be added to the end of a prompt to refine the output, such as 'burnt texture,' 'black and gray color palette,' and 'flat composition.' The paragraph provides an example of how to use these modifiers with a prompt to create a stylized city skyline made out of paper.
🛠️ Mastering Prompting Techniques for Creative Control
The final paragraph of the script offers guidance on mastering the prompting process. It advises viewers to focus on the visual appearance and the essence of the subject, to include relevant prompt modifiers, and to keep prompts focused and clear. The script warns against using run-on sentences, unnecessary filler words, too many concepts in a single prompt, or contradictory prompt modifiers. It concludes by encouraging viewers to join the community on Discord for further resources and experimentation, and to use the dashboard button for specific questions.
Mindmap
Keywords
💡Prompting
💡Gen 2
💡Text to Video
💡Free Preview
💡Specificity
💡Type of Shot
💡Style Presets
💡Prompt Modifiers
💡Control
💡Run-on Sentences
💡Discord Community
Highlights
Introduction to the basics of prompting for text to video in Gen 2.
Navigate to Gen 2 and click on text to video to start.
Begin with a simple prompt like 'a dog' to generate initial images.
Use free preview to see starting images and select for video conversion.
Adding specificity to prompts improves image and video detail.
Include dog breed, color, location, and time of day for more detailed prompts.
Example prompt: 'a light-colored French Bulldog laying on a couch during the day'.
Details about shot type, such as extreme close-up or portrait, can be included.
The more information added, the more control over the output.
Focus on 'what it looks like' and 'what it is' for effective prompting.
Example of contrasting prompts: 'a painting' vs. 'a bowl of fruit'.
Adding detail to prompts can lead to a specific look, such as a textured historical oil painting.
Style presets in Gen 2 can significantly influence the style of output results.
Experiment with style presets even with simplified prompts.
Prompt modifiers can be used to gain further control over outputs.
Modifiers are descriptive words added to the end of a prompt.
Example prompt with modifiers: 'a photo of a city skyline made out of paper'.
Modifiers can be combined with style presets for more control.
When prompting, avoid run-on sentences and contradictory modifiers.
Join the Runway Academy community on Discord for more resources and support.