How to AI Upscale with ControlNet Tiles - High Resolution for Everyone!

Sebastian Kamph
4 May 202318:16

TLDRThe video tutorial demonstrates how to upscale low-resolution images to high-resolution formats like 4K and 8K using an older PC with a GPU that has at least 4GB of VRAM. The process involves using ControlNet Tiles and the Ultimate Stable Fusion Upscale extension. The host guides viewers through installing the necessary extensions, setting up the image-to-image process with a specific model, and adjusting denoising strength for multiple passes. The video also discusses the importance of ControlNet in maintaining the coherence of the upscaled image and addresses common issues like tile lines and image smoothing. The host shares their workflow, which includes upscaling in steps to retain more detail and texture, and encourages viewers to experiment with different settings to find the best results for their images.

Takeaways

  • 🚀 **AI Upscaling with ControlNet Tiles**: You can upscale low-resolution images to HD, 4K, or even 8K using ControlNet Tiles, even with older hardware that has a GPU with at least 4GB of VRAM.
  • 💡 **Install Ultimate Stable Fusion Upscale**: To begin, install the 'ultimate stable fusion upscale' extension in your Stable Fusion, which is crucial for the upscaling process.
  • 🖼️ **Load Your Image**: Start by loading a low-resolution image into the 'image to image' section of Stable Fusion.
  • 🔍 **Use Denoising Strength**: Adjust the denoising strength for the initial pass (between 0.1 and 0.4) and lower it for the second pass to maintain image quality.
  • 🧩 **Tile Resample Preprocessor**: Utilize the 'tile resample' preprocessor in ControlNet to manage the tiling of the image during upscaling.
  • 📚 **Update ControlNet**: Ensure you have the latest version of ControlNet (with '1.1' in the filename) for optimal results.
  • 🔄 **Control Mode**: Set the control mode to 'ControlNet' to ensure the coherence of the upscaled image.
  • 🔢 **Target Size**: Choose to scale from the image size to ignore other settings and resize the image by the desired multiple (e.g., 2x or 4x).
  • 📈 **Iterative Upscaling**: Perform iterative upscaling by loading the upscaled image back into Stable Fusion and repeating the process for higher resolutions.
  • 🕒 **Time Consideration**: Be aware that upscaling to higher resolutions, such as 8K, will take significantly longer due to the increased number of tiles being processed.
  • 🆕 **ControlNet's Role**: ControlNet ensures that the upscaled image retains coherence and detail, even when using high denoising strengths.
  • 📝 **Workflow Preferences**: Experiment with different upscaling steps and denoising strengths to find the workflow that best suits your specific image requirements.

Q & A

  • What is the minimum GPU VRAM required to run stable Fusion for upscaling images?

    -The minimum GPU VRAM required is 4 gigabytes.

  • How can one install the 'ultimate stable Fusion upscale' extension?

    -To install the 'ultimate stable Fusion upscale' extension, go to your extensions, check available, press the load from search button, search for 'upscale', find 'ultimate as the upscale', and then install and apply it.

  • What is the recommended denoising strength for the first pass of upscaling?

    -For the first pass of upscaling, a denoising strength between 0.1 and 0.4 is recommended.

  • What is the role of ControlNet in the upscaling process?

    -ControlNet helps to maintain the coherence and structure of the upscaled image, ensuring that the details and features of the original image are preserved even when higher denoising strengths are used.

  • What is the recommended model for the second pass of upscaling?

    -The recommended model for the second pass is the '4X Ultra sharp' model.

  • How does the 'ultimate stable Fusion upscale' extension work with ControlNet tiles?

    -The 'ultimate stable Fusion upscale' extension works with ControlNet tiles by upscaling the image in multiple steps, using ControlNet to maintain the image's coherence and structure, and then combining the tiles to form a larger, high-resolution image.

  • What is the maximum image resolution that can be achieved with this method?

    -The method can upscale images to resolutions as high as 8K, depending on the GPU's capabilities and the number of tiles generated.

  • What are the potential issues with upscaling images to very high resolutions like 8K?

    -Upscaling to very high resolutions like 8K can result in a loss of detail and the introduction of 'fake detail', where the upscaled image may appear sharper but the details are not accurate to the original image.

  • How can one reduce the visible tile lines in the upscaled image?

    -The video suggests experimenting with different settings such as band pass, padding, and half tile offset. However, the speaker found that using none of the seams fixes provided the best results in their testing.

  • What is the advantage of using a step-by-step upscaling approach?

    -A step-by-step upscaling approach allows for a more controlled process, where one can monitor the changes and quality of the image at each step, rather than waiting for a long render time and then assessing the final result.

  • How can one find the best workflow for upscaling their specific images?

    -One can find the best workflow by experimenting with different settings, such as denoising strength, target size, and the number of upscaling steps, and then comparing the results to determine which settings yield the most desirable outcome for their specific images.

Outlines

00:00

🚀 Upscaling Low Resolution Images with Control Net and Stable Fusion

The video introduces a method for upscaling images to high resolutions like 4K and 8K using Control Net and Stable Fusion, which is particularly useful for users with older PCs equipped with a GPU that has at least 4 GB of VRAM. The process involves installing the 'ultimate stable Fusion upscale' extension, using Control Net tiles, and adjusting denoising strength for better detail. The speaker also provides instructions on how to install necessary models and extensions, and emphasizes the importance of using the correct version of the Control 1.1 SD 1.5 tile model for optimal results.

05:00

🧩 Testing Various Seams Fixes for Image Tiling

The speaker discusses the challenges of image tiling when upscaling, such as visible tile lines and seams. They share their testing experience with different settings for seams fixes, including band pass and padding adjustments, and report that none of these settings significantly improved the image quality over using no seams fix at all. The video demonstrates the process of generating larger images by combining multiple tiles and the impact on rendering time and image quality as the image size increases.

10:04

🔍 The Role of Control Net in Retaining Image Coherence

This section explains the importance of Control Net in maintaining the coherence of upscaled images. Without Control Net, the upscaled image loses its resemblance to the original due to the increased denoising strength, resulting in a completely different image. The speaker demonstrates the difference by disabling Control Net and showing how the image changes drastically. They also compare the results of upscaling in steps versus a single large step, highlighting the trade-offs between detail retention and processing time.

15:04

🎨 Generating 8K Resolution Images and Evaluating the Results

The video concludes with an attempt to upscale a 512x512 image directly to an 8K resolution. The speaker acknowledges the time-consuming nature of this process and provides a comparison between the 4K and 8K images. While the 8K image has sharper details, it also exhibits signs of tiling and loss of detail, suggesting that the direct upscale from such a low resolution introduces 'fake detail'. The speaker expresses their preference for a step-by-step approach to upscaling, as it allows for more control over the final image quality and texture.

Mindmap

Keywords

💡AI Upscale

AI Upscale refers to the process of using artificial intelligence to increase the resolution of an image or video. In the context of the video, AI Upscale is used to generate high-resolution images, such as 4K or 8K, from lower resolution images using a PC with a GPU capable of running the necessary software.

💡ControlNet Tiles

ControlNet Tiles is a specific technique or tool used in the AI upscaling process mentioned in the video. It helps in managing the tiling of images during the upscaling process, ensuring that the upscaled image maintains coherence and detail across different sections.

💡GPU

GPU stands for Graphics Processing Unit. It is a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. In the video, a GPU with at least 4 gigabytes of VRAM is required to run the upscaling software, indicating the importance of GPU in rendering high-resolution images.

💡VRAM

VRAM stands for Video Random Access Memory. It is a type of memory used by some graphics processing units (GPUs) to store image data close to the GPU for faster access. The video mentions the need for a GPU with at least 4 gigabytes of VRAM, highlighting the importance of sufficient memory for high-quality image processing.

💡Denoising Strength

Denoising Strength is a parameter in the upscaling process that controls the level of noise reduction applied to the image. A higher denoising strength results in a cleaner image but can also lead to loss of detail. The video discusses adjusting this parameter for different passes of the upscaling process.

💡Ultimate Stable Fusion Upscale

Ultimate Stable Fusion Upscale is an extension or feature within the software used for upscaling images. It is mentioned as a tool that, when installed, allows for the upscaling of images to higher resolutions than typically possible with the hardware at hand.

💡Tile Resample

Tile Resample is a preprocessing step in the upscaling process that involves dividing the image into smaller sections or 'tiles' and processing each tile individually before recombining them into a larger image. It is a crucial part of the upscaling process described in the video.

💡Control 1.1 SD 1.5 Tile

Control 1.1 SD 1.5 Tile refers to a specific model or file used within the ControlNet framework for the upscaling process. It is important to use the correct version of the file, as it has been updated to work with the ControlNet 1.1 software release.

💡Seams Fix

Seams Fix is a feature or technique used to address the issue of visible seams between the tiles in an upscaled image. The video discusses various settings and tests to minimize the visibility of these seams, although it concludes that for many images, no setting significantly outperforms not using a fix at all.

💡8K Resolution

8K Resolution refers to a display resolution of approximately 8000 pixels along the horizontal axis. In the video, the presenter demonstrates the capability of the upscaling process to achieve an 8K resolution image from a much lower resolution starting point, showcasing the power of AI upscaling.

💡Workflow

Workflow in the context of the video refers to the sequence of steps or processes followed to upscale an image using AI. The presenter discusses different workflows, such as upscaling in multiple steps versus a single large step, and the pros and cons of each approach.

Highlights

You can upscale low resolution images to full HD, 4K, or even 8K using ControlNet Tiles and a GPU with at least 4GB of VRAM.

ControlNet is used alongside Ultimate Stable Fusion Upscale for high-resolution image generation.

The process involves installing the Ultimate Upscale extension and using it within the Image-to-Image feature of Stable Fusion.

Using ControlNet with the preprocessor set to 'tile resample' ensures coherence in the upscaled image.

The Control 1.1 SD 1.5 tile model is crucial for the process and should be updated if necessary.

Denoising strength can be adjusted based on the image for better results, with a recommended range of 0.1 to 0.4 for the first pass.

Seams between tiles can be minimized, but may still be visible upon close inspection.

The Ultimate SD Upscale script is used to change the target size and upscale the image.

Different models like the 4X Ultra Sharp can be used for better image quality.

Seams fixes can be experimented with, but may not always lead to improved results for all images.

Upscaling in steps can help retain more detail and texture in the final image.

Direct 8K upscaling from a 512x512 image is possible but may result in a loss of detail and an increase in rendering time.

ControlNet ensures that the upscaled image retains the spatial coherence and details of the original.

The workflow presented is a combination of community tips, personal testing, and may vary in effectiveness based on the specific image.

Upscaling in fewer, larger steps might retain more image texture and detail compared to multiple smaller steps.

The process is considered game-changing as it allows users with lower-end GPUs to generate high-resolution images.

The video provides a detailed tutorial on how to install and use ControlNet Tiles and Ultimate Stable Fusion Upscale for image upscaling.

The presenter encourages viewers to share their preferred workflows for image upscaling in the comments for community benefit.