Run Stable Diffusion 3 Locally! | ComfyUI Tutorial
TLDRThis tutorial demonstrates how to locally run Stable Diffusion 3 Medium using ComfyUI. The process involves downloading necessary files from Hugging Face, updating ComfyUI, and installing models into the correct folders. After setup, the user can generate images with natural language prompts, showcasing the model's ability to create detailed and ethereal images like a female character with aurora-like hair. The video also touches on the need for community feedback regarding licensing issues.
Takeaways
- ๐ The tutorial is about using Stable Diffusion 3 Medium with ComfyUI.
- ๐ To access the model, one must fill out a form on Hugging Face and agree to access the repository.
- ๐ Necessary files to download include sd3 medium safe tensors, text encoders like CLIP G, CLIP L, T5 XXL, and the ComfyUI workflows.
- ๐ If ComfyUI is already running, it needs to be closed for an update, which can be done through the 'update_comfy_ui.bat' script.
- ๐ After updating, install the CLIP models into the 'clip' folder and the sd3 medium safe tensor into the 'checkpoints' folder.
- ๐ผ๏ธ The generation process involves loading the checkpoint and using a natural language prompt for image creation.
- ๐ The example prompt provided is for a female character with hair resembling the northern lights.
- ๐ The model's weights have been released for free, which is a significant development.
- ๐ There are licensing issues that need to be addressed, and the community is encouraged to open issues or contact Stability AI for updates.
- ๐ The script suggests a more natural language style prompt is more effective than a tag style for this model.
- ๐จ The generated image is described as 'amazing,' indicating high satisfaction with the model's capabilities.
Q & A
What is the main topic of the tutorial video?
-The tutorial video is about how to use Stable Diffusion 3 medium and integrate it with ComfyUI.
Why is the model referred to as 'gated'?
-The model is called 'gated' because access to it requires filling out a form on Hugging Face, indicating it is restricted and not freely available to everyone.
What files does the user need to download from Hugging Face for Stable Diffusion 3 medium?
-The user needs to download the 'sd3 medium.safetensors', 'clip G', 'clip L', and 'T5 XXL' text encoders in fp16 format.
What is the purpose of the 'update comfy ui.bat' file?
-The 'update comfy ui.bat' file is used to update the ComfyUI software to the latest version.
Why is it necessary to close ComfyUI before updating it?
-Closing ComfyUI before updating ensures that the update process is not interrupted and that the software is not running while changes are being made.
What is the recommended way to organize the downloaded models in the ComfyUI directory?
-The recommended way is to place the downloaded models in the respective folders, such as 'clip' for the text encoders and 'checkpoints' for the 'sd3 medium.safetensors' file.
What is the 'basic inference workflow' mentioned in the script?
-The 'basic inference workflow' is a ComfyUI workflow that the user can download and use for generating images with Stable Diffusion 3 medium.
What does the user need to do after updating ComfyUI and organizing the models?
-The user needs to start ComfyUI using the 'Nvidia GPU dobat' and then load the checkpoint 'sd3 medium.safetensors' to begin using the software.
What is the example prompt provided in the video for generating an image?
-The example prompt is 'a female character with long flowing hair that appears to be made of ethereal swirling patterns resembling the northern lights or Aurora Borealis'.
What issue is mentioned regarding the licensing of Stable Diffusion 3 medium?
-The licensing is described as 'a little messed up', and the user is encouraged to open an issue or contact Stability AI to update the license.
How does the video suggest the community can help with the licensing issue?
-The video suggests that it should be a community effort to let Stability AI know about the licensing issue so that it can be updated.
Outlines
๐จ Introduction to Using Stable Diffusion 3 Medium
The video begins with an introduction to the Stable Diffusion 3 Medium model, which has just been released. The host explains the process of accessing the gated model by filling out a form on Hugging Face and agreeing to access the repository. The viewer is guided through downloading necessary files such as the 'sd3 medium.safetensors', 'clip G', 'clip L', 'T5 XXL', and 'fp16', as well as the 'comfy UI workflows' for basic inference.
๐ ๏ธ Updating Comfy UI and Installing Models
This section details the steps required to update Comfy UI, which includes closing the running application, navigating to the directory, and executing the 'update comfy ui.bat' file. The host emphasizes the importance of updating to the latest version for compatibility with the new models. Following the update, the process of installing the CLIP models into the 'clip' folder and placing the 'sd3 medium.safetensors' file into the 'checkpoints' folder is described, ensuring the user is prepared to start using Comfy UI with the new models.
๐ Starting Comfy UI and Testing the Model
The final part of the script instructs the viewer on how to start Comfy UI using the 'Nvidia GPU dobat' and then load the newly downloaded 'sd3 medium.safetensors' checkpoint. The host demonstrates the use of the model by inputting a descriptive prompt for a female character with hair resembling the Northern Lights. The video showcases the model's capabilities by generating an impressive image, highlighting the model's ability to understand and respond to natural language prompts effectively. The host concludes by expressing satisfaction with the release of the model's weights for free and encourages the community to help address licensing issues by opening issues or contacting Stability AI.
Mindmap
Keywords
๐กStable Diffusion 3
๐กComfyUI
๐กHugging Face
๐กGated Model
๐กTensors
๐กText Encoders
๐กWorkflow
๐กCheckpoints
๐กNvidia GPU
๐กQ prompt
๐กAurora Borealis
Highlights
Introduction to using Stable Diffusion 3 Medium and ComfyUI.
Accessing the gated model on Hugging Face by filling out a form and agreeing to terms.
Downloading necessary files such as sd3 medium safe tensors, text encoders, and ComfyUI workflows.
Updating ComfyUI by running the 'update comfy ui.bat' script.
Installing CLIP models into the ComfyUI models directory.
Creating a new folder for sd3 medium safe tensors and adding the file to the checkpoints.
Starting ComfyUI with the Nvidia GPU 'dobat' to ensure optimal performance.
Loading the sd3 medium safe tensors as the checkpoint in ComfyUI.
Using the example prompt to generate an image with a natural language description.
Observing the generated image with a female character and ethereal swirling patterns.
Comparing the prompt style to SDXL but closer to the natural language.
Expressing excitement about the release of the model's weights for free.
Discussing the licensing issue and the need for community effort to update it.
Encouraging users to open an issue or contact Stability AI about the license.
Concluding the tutorial with a reminder to have a great day.