【爆速！】TensorRTのstable diffusion webuiへのインストール方法と効果的な使い方

AI is in wonderland
19 Oct 202319:38

TLDRIn this video, Alice and Yuki introduce the integration of NVIDIA's TensorRT with the stable diffusion webUI, which significantly accelerates image generation. They guide viewers through the installation process on an RTX4090 GPU using the dev branch of stable diffusion webUI. After setting up the environment, they demonstrate how to export TensorRT engines for various image sizes and models, including Dreamshaper, Magic Mix, and anime bluepencil. The video showcases the speed improvement with TensorRT, achieving around 1.5 times faster image generation compared to normal mode. However, they also note that using high-resolution fixes with TensorRT may increase generation time. The video concludes with a look at SDXL and the potential for even faster image generation in the future. The hosts encourage viewers to try TensorRT, despite the initial installation challenges, and to look forward to upcoming improvements.

Takeaways

  • 🚀 TensorRT is a high-performance deep learning inference engine by NVIDIA that can significantly speed up image generation with stable diffusion webUI.
  • ⚠️ The operation with TensorRT may still be unstable, so it's recommended to wait before using it unless you want to try it out immediately.
  • 📦 TensorRT is specifically for NVIDIA's GPU and cannot be used with other GPUs. The demonstration used an RTX4090.
  • 💡 To use TensorRT, a new stable diffusion webUI from the dev branch needs to be installed, as it's a development version.
  • 🛠️ The installation process involves cloning the webUI from GitHub, switching to the dev branch, and editing the batch file for integration.
  • 🔄 After setting up the environment, TensorRT requires the installation of NVIDIA's cuDNN and the development version of TensorRT, followed by uninstalling the initial cuDNN.
  • 📌 The TensorRT engine needs to be exported to the desired checkpoint for each image size, such as 512x512, 1024x1024, and 512x768.
  • ⏱️ Using TensorRT can result in faster image generation speeds, approximately 1.5 times faster than normal mode, as demonstrated in the video.
  • 🖼️ When generating high-resolution images with Hi-Res Fix, the use of TensorRT may increase the image generation time, suggesting potential areas for future improvement.
  • 🔍 TensorRT also helps in reducing VRAM consumption compared to normal mode, which is beneficial for systems with limited VRAM.
  • 🔧 The video suggests that future improvements could include optimizing the automatic calculation method for tiling to allow for faster image generation without compromising on quality.

Q & A

  • What is TensorRT and how does it relate to stable diffusion webUI?

    -TensorRT is a high-performance deep learning inference engine developed by NVIDIA that optimizes deep learning models to run quickly. It is used with stable diffusion webUI to significantly increase the speed of image generation.

  • Why might someone be advised to wait before using TensorRT with stable diffusion webUI?

    -The operation may still be unstable, so users who are not in a hurry are recommended to wait a little longer before using it to ensure more reliability and fewer potential issues.

  • What GPU is required to use TensorRT?

    -TensorRT is an engine for NVIDIA's GPU, so it cannot be used with other GPUs. The environment used in the script is RTX4090.

  • How does one install the stable diffusion webUI for use with TensorRT?

    -To install the stable diffusion webUI, one must create a new folder under the C drive, open a command prompt from this folder, and use git clone with the provided code from automatic1111's GitHub page. After the folder is created, change to the dev branch using the commit hash from the stable diffusion commit hash page.

  • What is the significance of using the dev branch instead of the master branch for SDXL?

    -The dev branch is a development branch, which means it is under development and may contain the latest features and updates. For using SDXL with TensorRT, the dev branch is necessary as it includes the required updates.

  • How does the installation process of TensorRT affect the stable diffusion webUI?

    -The installation of TensorRT involves several steps, including activating venv, installing a new version of pip, installing NVIDIA's cuDNN, installing the development version of TensorRT, and uninstalling the initially installed cuDNN. This process modifies the webUI to incorporate TensorRT for faster image generation.

  • What is the impact of TensorRT on image generation speed?

    -Using TensorRT, the image generation speed is significantly increased. For example, with an RTX4090 GPU, the iterations per second can reach 51.12 for a 512x512 image, which is about 1.5 times faster than the normal mode.

  • How does exporting the TensorRT engine to different checkpoints affect the image generation process?

    -Exporting the TensorRT engine to different checkpoints allows for the optimization of image generation for specific models. Each image size requires a specific TensorRT engine, and exporting these engines enables faster image generation for those sizes.

  • What is the VRAM consumption when using TensorRT for image generation?

    -When generating a 512x512 image using TensorRT and upscaling it to 2x with Hi-Res Fix, the VRAM usage was 5.04GB, which is less than the 6.19GB used when generating the image with SD Unet set to none.

  • What are the limitations when using high resolution fixes with TensorRT?

    -Using high resolution fixes with TensorRT can increase the image generation time, suggesting that there might be some compatibility issues or areas for improvement in the integration of these two features.

  • How does the image generation process differ when using img to img upscaling compared to high resolution fix?

    -Using img to img upscaling is faster than using high resolution fix. For example, upscaling 5 1024x1024 images took about 10 seconds with TensorRT, compared to 13 seconds in normal mode, indicating a speed increase of about 1.5 times.

  • What are the current limitations of TensorRT when used with SDXL models?

    -In the given environment, the TensorRT engine could not be exported to SDXL models other than the SDXL base model, indicating that there may be limitations or additional requirements for using TensorRT with other SDXL models.

Outlines

00:00

🚀 Introduction to TensorRT and Stable Diffusion WebUI

Alice from AI’s Wonderland introduces the integration of TensorRT, NVIDIA's high-performance deep learning inference engine, with the stable diffusion webUI. Yuki explains the potential for significant speed improvements in image generation. They caution about potential instability and recommend waiting for further stabilization, but provide a guide for eager users. The process involves installing the stable diffusion webUI on an RTX4090 environment, switching to the development branch, and setting up the environment with specific commands and steps.

05:01

🔧 Installation and Configuration of TensorRT

The video script details the steps for installing and configuring TensorRT with the stable diffusion webUI. It covers the initial setup, including creating a new folder, cloning the webUI from GitHub, switching to the dev branch, and editing the batch file for installation. The guide also addresses how to revert changes if TensorRT was installed by mistake. It proceeds to explain the installation of necessary packages like pip, NVIDIA's cuDNN, and the development version of TensorRT, followed by uninstalling the initial cuDNN installation.

10:10

🖼️ Exporting TensorRT Engines and Image Generation

After preparing the environment, the script moves on to exporting TensorRT engines for different image sizes and models, such as Dreamshaper, Magic Mix, and anime bluepencil. It demonstrates how to adjust settings in the webUI, including selecting the appropriate model and engine. The video shows the speed of image generation with TensorRT compared to normal mode, highlighting the performance benefits. It also discusses the use of Hi-Res Fix for high-resolution image generation and the impact on speed and VRAM usage.

15:10

📈 Performance Comparison and Future Outlook

The final paragraph compares the performance of image generation using TensorRT versus normal mode, focusing on speed and VRAM consumption. It explores the use of img to img upscaling and discusses the potential for faster image generation without tiling. The script also touches on SDXL capabilities and the challenges encountered when exporting the TensorRT engine for SDXL models. The video concludes with a teaser for future improvements and a call to action for viewers to subscribe and like the video.

Mindmap

Keywords

💡TensorRT

TensorRT is a high-performance deep learning inference engine developed by NVIDIA. It is designed to optimize deep learning models for rapid execution, which is particularly beneficial for applications that require real-time inference. In the context of the video, TensorRT is used to significantly increase the speed of image generation with stable diffusion webUI, making it a core technology for the discussed improvements in performance.

💡stable diffusion webUI

Stable diffusion webUI refers to a user interface for the stable diffusion model, which is a type of machine learning model used for generating images from textual descriptions. The webUI facilitates the interaction with the model through a web-based interface. In the video, the installation and usage of TensorRT with this webUI are detailed to enhance the speed of image generation.

💡NVIDIA GPU

NVIDIA GPU refers to a Graphics Processing Unit designed and manufactured by NVIDIA Corporation. GPUs are specialized hardware accelerators that are highly efficient for performing the complex mathematical operations required in deep learning and graphic rendering. The video mentions that TensorRT is an engine optimized for NVIDIA's GPUs, emphasizing the compatibility and performance benefits when using this specific hardware for image generation tasks.

💡RTX4090

RTX4090 is a specific model of NVIDIA's graphics card, representing a high-end GPU used for gaming and professional applications that require intense graphical and computational power. In the video, it is the environment used for demonstrating the capabilities of TensorRT in conjunction with the stable diffusion webUI, highlighting its role in achieving high-speed image generation.

💡dev branch

The dev branch, short for development branch, is a version control term referring to an ongoing work area in a software project where new features are developed and tested. In the context of the video, the dev branch of the stable diffusion webUI is used to install and test the TensorRT engine before it becomes available in the main or 'master' branch.

💡cuDNN

cuDNN, which stands for CUDA Deep Neural Network, is a GPU-accelerated library developed by NVIDIA that provides highly tuned routines for deep learning applications. It is mentioned in the video as a prerequisite for the installation of TensorRT, indicating its role in optimizing deep learning operations on NVIDIA GPUs.

💡VRAM

VRAM, or Video RAM, is the dedicated memory in a graphics card used for storing image data. It is a critical resource for graphics-intensive applications, including deep learning models that require large amounts of memory for processing. The video discusses how TensorRT can help reduce VRAM consumption during image generation, which is beneficial for maintaining performance on systems with limited graphics memory.

💡Hi-Res Fix

Hi-Res Fix is a term used in the context of image generation to refer to a method or feature that improves the resolution of generated images. The video explores the use of Hi-Res Fix in conjunction with TensorRT, noting that while it can increase image generation time, it is an important aspect of achieving higher quality images.

💡img to img

Img to img, or image-to-image, is a process where an input image is used to generate a new image, often with modifications or enhancements. In the video, the presenter demonstrates how TensorRT can be used to speed up the img to img upscaling process, showcasing its utility in improving the efficiency of image manipulation tasks.

💡SDXL

SDXL likely refers to a specific model or version of the stable diffusion technology that is capable of handling larger and more complex image generation tasks. The video discusses the potential of using TensorRT with SDXL, indicating that it can significantly improve the speed of image generation for this more advanced model.

💡Dynamic Export

Dynamic Export in the context of the video refers to a feature that allows for the TensorRT engine to be exported with settings that can adapt to different image sizes and batch sizes. This provides flexibility in generating images at various resolutions without needing to export a separate engine for each specific size. The video mentions this feature as a way to optimize the use of TensorRT for different image generation scenarios.

Highlights

TensorRT is now compatible with stable diffusion webUI, potentially increasing image generation speed significantly.

TensorRT is a high-performance deep learning inference engine developed by NVIDIA that optimizes models for faster execution.

The current operation may be unstable, suggesting users wait for further stability before use unless they wish to experiment.

TensorRT is exclusive to NVIDIA's GPU, demonstrated here using an RTX4090.

A new stable diffusion webUI and dev branch are installed to use TensorRT without affecting existing functions.

The dev branch is a development version of the software, indicating it may have the latest features but could be less stable.

The process of installing the webUI and switching to the dev branch is detailed, including command prompt instructions.

After installing, users should check if the webUI starts up successfully before proceeding.

If TensorRT was previously installed incorrectly, specific folders need to be deleted and the process restarted.

Commands for installing TensorRT, including activating venv and installing necessary NVIDIA components, are provided.

The TensorRT engine can be exported to different models and image sizes, with specific instructions given for the process.

The webUI's Extensions tab is used to install TensorRT from a GitHub URL, following a successful engine export.

After installation, a TensorRT tab is created in the webUI for users to access and utilize TensorRT functionalities.

Image generation speed is compared between normal mode and TensorRT mode, with TensorRT showing a 1.5 times faster iteration.

The impact of using high resolution fixes in conjunction with TensorRT on image generation time is discussed.

VRAM consumption is compared when using TensorRT versus normal mode, with TensorRT showing less usage.

The possibility of faster image generation without tiling is suggested as a future improvement for TensorRT.

Upscaling using img to img is shown to be faster with TensorRT compared to normal mode, approximately 1.5 times faster.

SDXL integration with TensorRT is demonstrated, noting that it is currently only possible on the dev branch.

The video concludes with a note on the future simplification of TensorRT installation and its potential for even faster image generation.