Stable Diffusion 3 via API in comfyUI with Stability AI official nodes - SD Experimental
TLDRIn this video, Andrea Baioni guides viewers through using Stable Diffusion 3 (SD3) via API key in ComfyUI, a process that requires purchasing credits for image generation, costing about 6 cents per image. He demonstrates installing the necessary nodes from Stability AI's GitHub, inputting the API key for each node, and generating images using the SD3 and SD3 Turbo models. Baioni also explores additional features like upscaling, outpainting, inpainting, and search and replace, noting that the base model provides impressive results despite some minor issues. The video concludes with a slideshow of generated images and an invitation for viewers to suggest prompts for testing.
Takeaways
- 📈 Stable Diffusion 3 (SD3) is available for use via API keys but not as a free checkpoint, requiring the purchase of credits for image generation.
- 💵 The cost for generating an image with SD3 is around 6 cents USD per image, with the option to wait for a future free checkpoint release.
- 🛠️ To use SD3 with ComfyUI, you need to install missing custom nodes from the Stability AI GitHub page and input your Stability AI API key into each node.
- 🔑 API keys are found on the Stability AI account page, and additional credits can be purchased if the initial 25 free credits are insufficient.
- 💡 The SD3 node offers options to select between SD3 and SD3 Turbo models, with the latter being slightly less costly in terms of credits.
- 👗 A positive prompt for image generation was tested, resulting in a comparison between the Core and SD3 models, with SD3 producing more accurate results to the prompt.
- 🎨 Additional nodes such as 'stability creative upscale' and 'stability outpainting' were explored, demonstrating impressive results with base models.
- 🖼️ The 'stability inpainting' feature allowed for changes in the image, such as a quick change of clothes and model, albeit with some issues like hair transformation.
- 🔍 The 'stability search and replace' node was tested, successfully replacing a person with a cat in an image, albeit with some unexpected perspective changes.
- 🚫 The 'stability remove background' node was not functional in the demonstration, possibly due to a missing field for API key input.
- 🤖 The entire process was conducted within ComfyUI, providing a seamless workflow for users familiar with the platform and offering the ability to test SD3 without relying on external interfaces.
- 📸 The presenter, Andrea Baioni, offered to test additional prompts provided by the audience and shared a link to view the generated SD3 images.
Q & A
What is the current status of Stable Diffusion 3 (SD3) in terms of availability and usage?
-Stable Diffusion 3 has been released for use with API keys but has not been released as a checkpoint. This means it can only be used with API calls and is not free. Users need to purchase credits to use it.
How much does it cost to generate an image using SD3 via API?
-It costs around 6 cents of a US dollar for image generation using SD3 via API.
What is the process of setting up a workflow in ComfyUI to use SD3?
-To set up a workflow in ComfyUI for SD3, you need to install missing custom nodes from the Stability AI GitHub page, restart ComfyUI, and then input your Stability AI API key into the API key override field for each node you want to use.
What are the different nodes available for use with SD3 in ComfyUI?
-The available nodes include stability image core, stability SD3, preview image, stability remove background, stability creative upscale, stability outpainting, stability inpainting, and stability research and replace.
How can one obtain Stability AI API keys and credits?
-You can obtain Stability AI API keys and credits by signing up or logging into your Stability AI account, navigating to the account page, and purchasing additional credits if needed. The initial 25 free credits are provided upon account creation.
What is the cost of an API call for different models in Stability AI?
-The cost per API call varies by model: SD3 costs 6.5 credits, SD3 turbo costs 4 credits, and Core costs 3 credits.
How can one test SD3 without spending their own money?
-New users can test SD3 with the initial 25 free credits provided by Stability AI upon account creation, which should cover at least a couple of generations.
What was the outcome of the first image generation test using SD3 and Core models?
-The first image generation test resulted in two images: the Core model translated the environment settings into the clothing and general mood, while the SD3 model produced clothing more similar to Miu Miu and included a skylight as prompted.
What issues were encountered when trying to use the stability creative upscale and outpainting nodes?
-The upscaled image was too large to be handled by the outpainting node, resulting in a payload error. Adjusting the outpainting settings to a smaller expansion size resolved the issue.
What was the result of the search and replace node when trying to replace a person with a cat?
-The search and replace node successfully replaced the person with a giant cat, although the perspective and some details were altered in the process.
Why was the remove background node not functional in the given demonstration?
-The remove background node was not functional because it kept asking for an API key, but there was no field provided to input it. A potential workaround was mentioned but not demonstrated.
How can users share their prompts for testing SD3 with the presenter?
-Users can leave comments with their prompts, and the presenter may test them out and share the results on imgur for everyone to see.
Outlines
📝 Introduction to Stable Diffusion 3 with API Key in ComfyUI
The video begins with an introduction to Stable Diffusion 3 (SD3), a tool released by Stability AI for image generation via API keys. It's clarified that while SD3 is available for use, it isn't free and requires the purchase of credits for image generation, costing approximately 6 cents per image. The video promises a walkthrough on setting up the workflow in ComfyUI and using the API keys for image generation. The process includes installing missing nodes from the Stability AI GitHub page and using the provided workflow image to set up the ComfyUI workspace.
💳 Purchasing Credits and Setting Up API Key for Stability AI SD3
The speaker guides viewers on how to purchase credits and set up an API key for Stability AI's SD3. They explain the pricing structure for different models and how to buy additional credits if the initial 25 free credits are insufficient. The process includes navigating to the account page, revealing the API key, and copying it to the clipboard for use in ComfyUI. The video then demonstrates how to input the API key into each node within ComfyUI that requires it.
🖼️ Generating Images with Core and SD3 Models in ComfyUI
The video continues with a demonstration of generating images using the Core and SD3 models within ComfyUI. The speaker inputs a positive prompt for a fashion scene and compares the results from both models. They note that the SD3 model produces a more accurate representation of the Miu Miu fashion style and includes details like a skylight, which was absent in the Core model's output. The speaker also discusses the cost of credits used for these image generations.
🎨 Exploring Additional Nodes for Image Manipulation
The speaker explores additional nodes for creative image manipulation, including upscaling, outpainting, inpainting, and search and replace functionalities. They activate the nodes, input prompts, and discuss the results of each operation. The video highlights the impressive detail and quality of the base model's output, despite not having community fine-tuned models or additional tools like RealVIS. The speaker also humorously addresses an error encountered while trying to perform a search and replace operation, providing a solution to the issue.
🚀 Conclusion and Future Testing with SD3 in ComfyUI
The video concludes with the speaker's positive first impressions of the SD3 models and a summary of the nodes tested. They mention an issue with the remove background node but do not attempt a workaround. The speaker offers to test prompts provided by viewers and shares links to view the generated images. They sign off with contact information and a slideshow of SD3 images generated during the video.
Mindmap
Keywords
💡Stable Diffusion 3
💡API key
💡ComfyUI
💡Image generation
💡Credits
💡Positive prompt
💡Negative prompt
💡Upscaling
💡Outpainting
💡Inpainting
💡Search and Replace
Highlights
Stable Diffusion 3 (SD3) is accessible via API key in ComfyUI, but not yet available as a free checkpoint.
Using SD3 requires purchasing credits, costing approximately 6 US cents per image generation.
Stability AI's official nodes and workflow for ComfyUI are available on their GitHub page.
Missing nodes in ComfyUI can be installed through the manager and the application may need to be restarted.
Each Stability AI node features an API key override field for user-specific keys.
The Stability AI API key can be obtained and managed through the user's account on the Stability AI website.
Credits for image generation can be purchased in increments starting at $10 for 1000 credits.
The Core and SD3 models differ in their level of refinement and cost, with SD3 Turbo being a cost-effective alternative.
Positive and negative prompt fields are used to guide the image generation process in SD3.
The output format for SD3 should be set correctly to avoid errors in the generation process.
The generated images from the Core and SD3 models are of high quality, even without community fine-tuning.
Stability Creative Upscale node enhances image details and corrects anatomy with impressive results.
Outpainting node expands the image while maintaining perspective and ambience, though it has limitations with large payload sizes.
Inpainting node allows for changes in the image, such as altering clothing or models, with some minor issues.
Search and Replace node can substitute elements within an image, as demonstrated by replacing a person with a cat.
The Remove Background node was not functional in the demonstration, possibly due to a missing API key input field.
Users can utilize the 25 free credits provided by Stability AI to test out several generations of image generation.
ComfyUI provides a seamless workflow for using SD3 without relying on external web interfaces.