How to AI Upscale with ControlNet Tiles - High Resolution for Everyone!

Sebastian Kamph
4 May 202318:16

TLDRThe video tutorial demonstrates how to upscale low-resolution images to high-resolution formats like 4K and 8K using an older PC with a GPU that has at least 4GB of VRAM. The process involves using ControlNet Tiles and the Ultimate Stable Fusion Upscale extension. The host guides viewers through installing the necessary extensions, setting up the image-to-image process with a specific model, and adjusting denoising strength for multiple passes. The video also discusses the importance of ControlNet in maintaining the coherence of the upscaled image and addresses common issues like tile lines and image smoothing. The host shares their workflow, which includes upscaling in steps to retain more detail and texture, and encourages viewers to experiment with different settings to find the best results for their images.

Takeaways

  • 🚀 **AI Upscaling with ControlNet Tiles**: You can upscale low-resolution images to HD, 4K, or even 8K using ControlNet Tiles, even with older hardware that has a GPU with at least 4GB of VRAM.
  • 💡 **Install Ultimate Stable Fusion Upscale**: To begin, install the 'ultimate stable fusion upscale' extension in your Stable Fusion, which is crucial for the upscaling process.
  • 🖼️ **Load Your Image**: Start by loading a low-resolution image into the 'image to image' section of Stable Fusion.
  • 🔍 **Use Denoising Strength**: Adjust the denoising strength for the initial pass (between 0.1 and 0.4) and lower it for the second pass to maintain image quality.
  • 🧩 **Tile Resample Preprocessor**: Utilize the 'tile resample' preprocessor in ControlNet to manage the tiling of the image during upscaling.
  • 📚 **Update ControlNet**: Ensure you have the latest version of ControlNet (with '1.1' in the filename) for optimal results.
  • 🔄 **Control Mode**: Set the control mode to 'ControlNet' to ensure the coherence of the upscaled image.
  • 🔢 **Target Size**: Choose to scale from the image size to ignore other settings and resize the image by the desired multiple (e.g., 2x or 4x).
  • 📈 **Iterative Upscaling**: Perform iterative upscaling by loading the upscaled image back into Stable Fusion and repeating the process for higher resolutions.
  • 🕒 **Time Consideration**: Be aware that upscaling to higher resolutions, such as 8K, will take significantly longer due to the increased number of tiles being processed.
  • 🆕 **ControlNet's Role**: ControlNet ensures that the upscaled image retains coherence and detail, even when using high denoising strengths.
  • 📝 **Workflow Preferences**: Experiment with different upscaling steps and denoising strengths to find the workflow that best suits your specific image requirements.

Q & A

  • What is the minimum GPU VRAM required to run stable Fusion for upscaling images?

    -The minimum GPU VRAM required is 4 gigabytes.

  • How can one install the 'ultimate stable Fusion upscale' extension?

    -To install the 'ultimate stable Fusion upscale' extension, go to your extensions, check available, press the load from search button, search for 'upscale', find 'ultimate as the upscale', and then install and apply it.

  • What is the recommended denoising strength for the first pass of upscaling?

    -For the first pass of upscaling, a denoising strength between 0.1 and 0.4 is recommended.

  • What is the role of ControlNet in the upscaling process?

    -ControlNet helps to maintain the coherence and structure of the upscaled image, ensuring that the details and features of the original image are preserved even when higher denoising strengths are used.

  • What is the recommended model for the second pass of upscaling?

    -The recommended model for the second pass is the '4X Ultra sharp' model.

  • How does the 'ultimate stable Fusion upscale' extension work with ControlNet tiles?

    -The 'ultimate stable Fusion upscale' extension works with ControlNet tiles by upscaling the image in multiple steps, using ControlNet to maintain the image's coherence and structure, and then combining the tiles to form a larger, high-resolution image.

  • What is the maximum image resolution that can be achieved with this method?

    -The method can upscale images to resolutions as high as 8K, depending on the GPU's capabilities and the number of tiles generated.

  • What are the potential issues with upscaling images to very high resolutions like 8K?

    -Upscaling to very high resolutions like 8K can result in a loss of detail and the introduction of 'fake detail', where the upscaled image may appear sharper but the details are not accurate to the original image.

  • How can one reduce the visible tile lines in the upscaled image?

    -The video suggests experimenting with different settings such as band pass, padding, and half tile offset. However, the speaker found that using none of the seams fixes provided the best results in their testing.

  • What is the advantage of using a step-by-step upscaling approach?

    -A step-by-step upscaling approach allows for a more controlled process, where one can monitor the changes and quality of the image at each step, rather than waiting for a long render time and then assessing the final result.

  • How can one find the best workflow for upscaling their specific images?

    -One can find the best workflow by experimenting with different settings, such as denoising strength, target size, and the number of upscaling steps, and then comparing the results to determine which settings yield the most desirable outcome for their specific images.

Outlines

00:00

🚀 Upscaling Low Resolution Images with Control Net and Stable Fusion

The video introduces a method for upscaling images to high resolutions like 4K and 8K using Control Net and Stable Fusion, which is particularly useful for users with older PCs equipped with a GPU that has at least 4 GB of VRAM. The process involves installing the 'ultimate stable Fusion upscale' extension, using Control Net tiles, and adjusting denoising strength for better detail. The speaker also provides instructions on how to install necessary models and extensions, and emphasizes the importance of using the correct version of the Control 1.1 SD 1.5 tile model for optimal results.

05:00

🧩 Testing Various Seams Fixes for Image Tiling

The speaker discusses the challenges of image tiling when upscaling, such as visible tile lines and seams. They share their testing experience with different settings for seams fixes, including band pass and padding adjustments, and report that none of these settings significantly improved the image quality over using no seams fix at all. The video demonstrates the process of generating larger images by combining multiple tiles and the impact on rendering time and image quality as the image size increases.

10:04

🔍 The Role of Control Net in Retaining Image Coherence

This section explains the importance of Control Net in maintaining the coherence of upscaled images. Without Control Net, the upscaled image loses its resemblance to the original due to the increased denoising strength, resulting in a completely different image. The speaker demonstrates the difference by disabling Control Net and showing how the image changes drastically. They also compare the results of upscaling in steps versus a single large step, highlighting the trade-offs between detail retention and processing time.

15:04

🎨 Generating 8K Resolution Images and Evaluating the Results

The video concludes with an attempt to upscale a 512x512 image directly to an 8K resolution. The speaker acknowledges the time-consuming nature of this process and provides a comparison between the 4K and 8K images. While the 8K image has sharper details, it also exhibits signs of tiling and loss of detail, suggesting that the direct upscale from such a low resolution introduces 'fake detail'. The speaker expresses their preference for a step-by-step approach to upscaling, as it allows for more control over the final image quality and texture.

Mindmap

Invitation for feedback and sharing of preferred workflows
Encouragement for users to learn and experiment
Even on lower-end GPUs
Open to suggestions for better workflows
Different results for different images
Advantages of iterative scaling for texture and detail retention
Increased rendering time and detail loss
Direct upscale from 512x512 to 8K
Loss of detail with each upscaling step
Increasing rendering time with higher resolutions
2048x2048 and 4096x4096 resolutions
Role of ControlNet in maintaining image coherence
No significant improvement with tested settings
Testing various settings for reducing tile visibility
Scaling from image size
Control 1.1 SD 1.5 Tile Model
Tile Resample Preprocessor
ControlNet Integration
Denoising Strength Adjustment
512x512 image as a starting point
Installation from extensions
Installation and update process
Use of Stable Fusion for image rendering
Viewer Engagement
Learning and Experimentation
Accessibility of High-Resolution Image Generation
Community Input
User Preferences
Step-by-Step Scaling
8K Resolution Attempt
Detail Comparison
Scaling to Higher Resolutions
Coherence Retention
Seams Fixing
Target Size Scaling
Model Selection
Upscaling Steps
Initial Image
Ultimate Stable Fusion Upscale
ControlNet
Stable Fusion
Requirements: GPU with at least 4GB VRAM
Potential for High Resolution Upscaling
Low Resolution Imagery Issue
Conclusion
Workflow and Tips
Resolution and Quality
Seams and Coherence
Image Upscaling Process
Software and Extensions
Introduction
AI Upscaling with ControlNet Tiles
Alert

Keywords

💡AI Upscale

AI Upscale refers to the process of using artificial intelligence to increase the resolution of an image or video. In the context of the video, AI Upscale is used to generate high-resolution images, such as 4K or 8K, from lower resolution images using a PC with a GPU capable of running the necessary software.

💡ControlNet Tiles

ControlNet Tiles is a specific technique or tool used in the AI upscaling process mentioned in the video. It helps in managing the tiling of images during the upscaling process, ensuring that the upscaled image maintains coherence and detail across different sections.

💡GPU

GPU stands for Graphics Processing Unit. It is a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. In the video, a GPU with at least 4 gigabytes of VRAM is required to run the upscaling software, indicating the importance of GPU in rendering high-resolution images.

💡VRAM

VRAM stands for Video Random Access Memory. It is a type of memory used by some graphics processing units (GPUs) to store image data close to the GPU for faster access. The video mentions the need for a GPU with at least 4 gigabytes of VRAM, highlighting the importance of sufficient memory for high-quality image processing.

💡Denoising Strength

Denoising Strength is a parameter in the upscaling process that controls the level of noise reduction applied to the image. A higher denoising strength results in a cleaner image but can also lead to loss of detail. The video discusses adjusting this parameter for different passes of the upscaling process.

💡Ultimate Stable Fusion Upscale

Ultimate Stable Fusion Upscale is an extension or feature within the software used for upscaling images. It is mentioned as a tool that, when installed, allows for the upscaling of images to higher resolutions than typically possible with the hardware at hand.

💡Tile Resample

Tile Resample is a preprocessing step in the upscaling process that involves dividing the image into smaller sections or 'tiles' and processing each tile individually before recombining them into a larger image. It is a crucial part of the upscaling process described in the video.

💡Control 1.1 SD 1.5 Tile

Control 1.1 SD 1.5 Tile refers to a specific model or file used within the ControlNet framework for the upscaling process. It is important to use the correct version of the file, as it has been updated to work with the ControlNet 1.1 software release.

💡Seams Fix

Seams Fix is a feature or technique used to address the issue of visible seams between the tiles in an upscaled image. The video discusses various settings and tests to minimize the visibility of these seams, although it concludes that for many images, no setting significantly outperforms not using a fix at all.

💡8K Resolution

8K Resolution refers to a display resolution of approximately 8000 pixels along the horizontal axis. In the video, the presenter demonstrates the capability of the upscaling process to achieve an 8K resolution image from a much lower resolution starting point, showcasing the power of AI upscaling.

💡Workflow

Workflow in the context of the video refers to the sequence of steps or processes followed to upscale an image using AI. The presenter discusses different workflows, such as upscaling in multiple steps versus a single large step, and the pros and cons of each approach.

Highlights

You can upscale low resolution images to full HD, 4K, or even 8K using ControlNet Tiles and a GPU with at least 4GB of VRAM.

ControlNet is used alongside Ultimate Stable Fusion Upscale for high-resolution image generation.

The process involves installing the Ultimate Upscale extension and using it within the Image-to-Image feature of Stable Fusion.

Using ControlNet with the preprocessor set to 'tile resample' ensures coherence in the upscaled image.

The Control 1.1 SD 1.5 tile model is crucial for the process and should be updated if necessary.

Denoising strength can be adjusted based on the image for better results, with a recommended range of 0.1 to 0.4 for the first pass.

Seams between tiles can be minimized, but may still be visible upon close inspection.

The Ultimate SD Upscale script is used to change the target size and upscale the image.

Different models like the 4X Ultra Sharp can be used for better image quality.

Seams fixes can be experimented with, but may not always lead to improved results for all images.

Upscaling in steps can help retain more detail and texture in the final image.

Direct 8K upscaling from a 512x512 image is possible but may result in a loss of detail and an increase in rendering time.

ControlNet ensures that the upscaled image retains the spatial coherence and details of the original.

The workflow presented is a combination of community tips, personal testing, and may vary in effectiveness based on the specific image.

Upscaling in fewer, larger steps might retain more image texture and detail compared to multiple smaller steps.

The process is considered game-changing as it allows users with lower-end GPUs to generate high-resolution images.

The video provides a detailed tutorial on how to install and use ControlNet Tiles and Ultimate Stable Fusion Upscale for image upscaling.

The presenter encourages viewers to share their preferred workflows for image upscaling in the comments for community benefit.