Complete Guide to Setting Up Control Net in SDXL for Enhanced Image Generation

Table of Contents

• Downloading Required Control Net Files
• Integrating Control Net into SDXL Workflow
• Configuring Settings for Optimal Performance
• Rendering Images with Control Net Applied
• Troubleshooting Common Issues
• Conclusion

Downloading Required Control Net Files for SDXL

To use control net with SDXL, you first need to download the required model files. There are full size control net models available for both the Candy and Depth Map methods. You can find download links under the video accompanying this post - be sure to download the fp16 versions to save storage space.

The full size control net models should be placed in your ConvoUI models folder, inside a controlnet subfolder. It's best to rename the files on download for clarity. For example, name the Candy model file 'SDXL_Candy' and the Depth Map file 'SDXL_Depth'. This will make selecting the right model easier later.

Getting Full Size Control Net Models

On the download page, scroll down to find full size control net models for both Candy and Depth Map. Click the 'fp16 version' link to get the smaller file size. Use the download button to save the file to your ConvoUI models/controlnet folder.

Installing Additional Nodes

In addition to the model files, you need to install some custom nodes for working with control net in ConvoUI. There is a git clone command on the video's accompanying page to do this automatically. Simply copy the command, open a CMD window in ConvoUI's custom_nodes folder, paste the command and hit enter.

Integrating Control Net into SDXL Workflow

With the files installed, load the SDXL build JSON from the video page to see how control net integrates with the existing SDXL nodes like Candy and Depth Map preprocessors. The key nodes for control net are the loader, apply control net, and inserting prompts.

Connect your image input to the Candy/Depth Map preprocessors as usual, then feed that output to the control net apply node along with the loaded control net model. Control net prompts can be added through the positive/negative prompt node connected to the base sampler.

Configuring Settings for Optimal Performance

Getting good results with control net requires some experimentation with the threshold settings. Start with default values then adjust up or down if your outputs lack detail or are too noisy/distorted. The prompts also have a huge impact, so refine those over many generations for best quality.

Be sure to use prompts that specifically describe the desired adjustments to the image, rather than just 'better quality'. The more direct, the better control net can latch onto the correct improvements.

Rendering Images with Control Net Applied

When ready to generate images, simply fill your prompts, set thresholds/sampling parameters as desired, and click the Queue Prompt button. Rendered outputs will be saved to ConvoUI's output folder for easy access.

Remember, control net works best on the base image from the sampler rather than the refined output. So connect prompts to the first sampler in your pipeline for optimal control.

Troubleshooting Common Issues

If your outputs don't show control net's improvements, double check your model selection, thresholds aren't set too extreme, and prompts describe specific desired changes vs just 'better quality'. It may take some iterations to find the right balance.

Also ensure control net prompts are connected to the base sampler prompt inputs, not further downstream after refinements are applied. Putting prompts later can diminish control net's impact.

Conclusion

Control net is a powerful new tool for steering SDXL results in helpful directions. With the right workflow set up, it can help take your creations to the next level. Expect rapid ongoing improvements in capabilities too. We've only begun tapping into control net's immense potential for creative AI to positively impact the world.

FAQ

Q: Where do I download control net files?
A: You can download control net files from the links provided in the video description. Make sure to use the full size FP16 models.

Q: How do I install additional nodes?
A: Use the git clone command provided to download extra nodes into the custom nodes folder. Double click the instyle.py file to install.

Q: Where are control net nodes located?
A: You can find all control net related nodes by typing 'control' in the background search bar.

Q: How do I set positive/negative prompts?
A: Set base prompts in the text prompts node. Output positive/negative prompts to prompts in first case sampler.

Q: Where are rendered images saved?
A: Generated images are saved to the output folder inside Confi UI directory.