ADetailer in A1111: How to auto inpaint and fix multiple faces, hands, and eyes with After Detailer.

Keyboard Alchemist
6 Oct 202319:29

TLDRIn this tutorial, the presenter introduces After Detailer, an AI extension that automatically enhances and fixes details such as faces, hands, and eyes in generated images. The video demonstrates how to install the extension, configure its settings, and use various models for different object detection and inpainting tasks. It also covers advanced usage, including sorting bounding boxes and using the sep token for inpainting multiple objects with distinct prompts. The presenter guides viewers through an example of transforming a simple image into a detailed scene featuring characters from Final Fantasy 7, highlighting the extension's capabilities and potential applications.

Takeaways

  • 😀 The tutorial introduces After Detailer, an extension for stable diffusion that automates the inpainting of faces, bodies, hands, and eyes in generated images.
  • 🛠️ To install After Detailer, search for it in the extensions tab, install it, and then apply and restart the UI.
  • ⚙️ After Detailer allows for multiple instances to be run, with the number adjustable in the settings menu.
  • 🔍 The extension includes various detection models like face YOLO, person YOLO, and media pipe face for different inpainting tasks.
  • 👥 The 'sort bounding boxes by' option is crucial for controlling the inpainting order, especially when multiple objects are present.
  • 🔄 After Detailer can be used to enhance character details by using specific prompts for each detected object, separated by 'sep' tokens.
  • 🎨 The tutorial demonstrates how to use After Detailer to transform a simple image into a complex scene with characters from Final Fantasy 7.
  • 🔧 Advanced settings like mask dilation and control net models are used to refine the inpainting process and maintain image composition.
  • 🖌️ The inpainting process may require adjustments to denoising strength and the use of control net models to achieve coherent results.
  • ⏱️ The final image generation can be time-consuming due to multiple inpainting tasks, but the results are significantly improved.
  • 🔄 The video concludes with a demonstration of how to clean up artifacts and details in the final image using other inpainting methods.

Q & A

  • What is After Detailer and what does it do?

    -After Detailer is an automatic extension for AI image generation that saves time and effort by automatically inpainting faces, bodies, hands, and eyes after the initial image has been generated.

  • How do you install the After Detailer extension?

    -To install After Detailer, go to the extensions tab, then the available subtab, search for 'after', click the install button next to After Detailer, wait for the installation to finish, and then apply and restart the UI.

  • How can you increase the number of After Detailer instances?

    -You can increase the number of After Detailer instances from the settings menu by scrolling down to 'a detailer', using the slider to select the desired number of models, and then applying the settings.

  • What is the significance of the detection model's confidence value?

    -The detection model's confidence value indicates how certain the model is about detecting a particular object, such as a face or body. A higher value means greater confidence in the detection.

  • Why might you want to change the 'sort bounding boxes by' option to 'position left to right'?

    -Changing the 'sort bounding boxes by' option to 'position left to right' ensures that the inpaint order goes from left to right when there are multiple objects in the image, providing a consistent inpaint order.

  • What are the different models available in After Detailer for face detection?

    -After Detailer offers several face detection models including Face YOLO v8n, Face YOLO V8s, MediaPipe Face Full, MediaPipe Face Short, and MediaPipe Face Mesh.

  • How can you improve hand detection in After Detailer?

    -To improve hand detection, you can adjust the detection confidence threshold or use the Hand YOLO v8n model, which is effective at detecting hands even when they are small in the image.

  • What is the purpose of the 'positive prompt' and 'negative prompt' fields in After Detailer?

    -The 'positive prompt' and 'negative prompt' fields in After Detailer allow you to specify what should be inpainted and what should be avoided during the inpainting process, providing more control over the final output.

  • How does the 'noise multiplier' setting affect the inpainting process?

    -The 'noise multiplier' setting controls the amount of random noise added to the latent tensor during inpainting, which in turn affects the level of detail and changes introduced in the output image.

  • What is the role of the control net section in After Detailer?

    -The control net section in After Detailer helps maintain the composition of the output image consistent with the input image by applying various control net models that guide the diffusion process.

  • Can you provide an example of an advanced usage of After Detailer?

    -An advanced usage of After Detailer is inpainting multiple characters with different features simultaneously, such as turning a generic image into one featuring specific characters from Final Fantasy 7, by using the sep token to input different prompts for each detected object.

Outlines

00:00

🔧 Introduction to After Detailer Extension

The video introduces the After Detailer extension for stable diffusion, a tool that automates the refinement of generated images, focusing on details like faces, bodies, hands, and eyes. The tutorial covers the installation process, starting with accessing the extensions tab and searching for 'after detailer' to install it. Once installed, the extension is found in the installed tab, and the user is guided to apply and restart the UI. The video also explains how to adjust settings to increase the number of After Detailer instances and change the sort bounding boxes by option for a more controlled inpaint order. The extension's interface is explored, highlighting the ability to run multiple instances of different models sequentially.

05:01

👥 Exploring After Detailer Models

The tutorial delves into the various models available within the After Detailer extension, starting with face detection models like Face YOLO v8n and v8s, which are compared for their detection capabilities. It's noted that while the v8s model is larger, the output difference is minimal. The video also discusses body detection models, the importance of running a second instance of the face model to fix faces after body detection, and the significance of detection confidence values. Alternative face detection methods like MediaPipe Face and the Eyes model are introduced, with a recommendation for the YOLO face detection methods due to their reliability. An alternative eye detection model, 'eyes.PT', is suggested for more consistent eye detection, which is not included in the extension but can be downloaded and added.

10:04

🎨 Advanced Inpainting Techniques with After Detailer

This section of the video focuses on the advanced functionalities of After Detailer, including the use of positive and negative prompt fields for inpainting, and the four main sections of the extension: detection, pre-processing, inpainting, and control net. The detection section covers mask size ratios and the mask only the top K largest objects setting. The pre-processing section discusses offsets, erosion, dilation, and mask merge options. The inpainting section is compared to the image to image inpainting interface, with settings like mask blur, inpaint only masked, and denoising strength explained. The control net section introduces models that help maintain the original image composition during inpainting, with examples of how different control net models affect the output.

15:05

🖌️ Practical Application and Final Touches

The final part of the video demonstrates a practical application of After Detailer by inpainting an image of three girls into characters from Final Fantasy 7. The process involves using the person YOLO v8n model and the sep token to input different prompts for each character. The video advises on adjusting denoising strength and using control net models to maintain composition. It also addresses the common issue of faces not being properly inpainted after body detection, suggesting running the face YOLO model to fix this. The tutorial concludes with tips on cleaning up artifacts and performing a latent upscale for final image refinement, showcasing the final image and thanking viewers for their support.

Mindmap

Keywords

💡After Detailer

After Detailer is an automatic extension for the AI image generation tool, Stable Diffusion. It is designed to enhance and fix details in generated images, particularly focusing on faces, bodies, hands, and eyes. In the video, the tutorial demonstrates how to install and use After Detailer to improve the quality of AI-generated images by inpainting these specific areas. The extension is shown to be a time-saving tool that can automatically refine the details of an image post-generation.

💡Inpainting

Inpainting refers to the process of filling in missing or damaged parts of an image with new content that matches the surrounding area. In the context of the video, After Detailer uses inpainting to automatically generate and fix details such as faces, hands, and eyes. The script mentions how After Detailer can inpainted faces, bodies, and hands, and how it can be adjusted to control the inpainting process, such as by setting the denoising strength or using control nets to maintain image composition.

💡YOLO

YOLO stands for 'You Only Look Once' and is a type of AI algorithm used for object detection in images. In the video, YOLO models are mentioned as part of After Detailer's functionality, with specific versions like 'face YOLO v8n' and 'person YOLO v8n' being used to detect and inpaint faces and full bodies, respectively. The script explains how these models can be selected within After Detailer to target specific areas of an image for enhancement.

💡Detection Model

A detection model in the context of After Detailer is an AI model that identifies and locates specific objects or features within an image. The video script describes how different detection models like 'face YOLO' and 'media pipe face' are used to find faces in the generated images. These models provide a bounding box and a confidence value to indicate the likelihood of a successful detection, which is crucial for the inpainting process to target the right areas.

💡Denoising Strength

Denoising strength is a parameter in image generation that controls the amount of change introduced to the output image during the inpainting process. A higher denoising strength value results in more significant alterations to the image, while a lower value results in subtler changes. The video explains how adjusting denoising strength can affect the quality and coherence of the inpainted areas, and how it needs to be balanced with other settings like control nets.

💡Control Net

A control net in After Detailer is a model that helps maintain the overall composition of the image during the inpainting process. It ensures that the generated content fits harmoniously with the existing image. The video gives examples of different control nets, such as 'Global harmonious' and 'Open pose', and demonstrates how they can be used to correct issues like unwanted new details being generated within the face area when denoising strength is high.

💡Stable Diffusion

Stable Diffusion is an AI model used for generating images from text descriptions. It is the underlying technology on which After Detailer is built as an extension. The video tutorial assumes that viewers are familiar with Stable Diffusion and focuses on how to integrate After Detailer into an existing workflow that uses this AI model.

💡CFG Scale

CFG Scale, short for 'Control Flow Guidance Scale', is a parameter in AI image generation that influences the level of detail and structure in the output image. In the video, it is mentioned as one of the settings that can be adjusted within After Detailer to fine-tune the inpainting process. The script does not go into detail about CFG Scale, but it is implied that it works similarly to its role in other image generation contexts.

💡Noise Multiplier

The noise multiplier is a setting that controls the amount of random noise added to the latent tensor during image generation. In the video, the presenter explains that adjusting the noise multiplier can affect the level of noise in the final image, with higher values leading to noisier outputs and lower values resulting in cleaner images. This setting is part of the advanced options within After Detailer for fine-tuning the inpainting process.

💡Latent Upscale

Latent upscale is a process mentioned in the video that involves enhancing the quality and resolution of an AI-generated image. It is used as a final step after the main inpainting process to improve the overall appearance of the image. The script does not provide a detailed explanation of latent upscale, but it is presented as a complementary technique to After Detailer's inpainting capabilities.

Highlights

Introduction to After Detailer, an automatic 1111 extension for image inpainting.

After Detailer can automatically inpaint faces, bodies, hands, and eyes in generated images.

Tutorial on installing and using the After Detailer extension.

Guide on incorporating After Detailer into your workflow.

How to install After Detailer from the extensions tab.

Instructions to increase the number of After Detailer instances from settings.

Explanation of the sort bounding boxes by option for inpaint order control.

Overview of the nine different After Detailer models available for detection.

Demonstration of the face YOLO v8n and face YOLO V8s models for face detection.

Comparison of person YOLO v8n and person YOLO V8s models for body detection.

Details on the detection model's confidence value and its significance.

How to adjust the detection confidence threshold in the web UI.

Description of the media pipe face and media pipe face mesh models for face detection.

Introduction to the media pipe face mesh Eyes Only model for eye detection.

Alternative eye detection model, eyes.PT, from civit AI.

Hand YOLO v8n model for detecting hands in images.

Explanation of the positive prompt and negative prompt fields in After Detailer.

Details on the detection, mask pre-processing, inpainting, and control net sections of After Detailer.

How to use the sep token for inpainting multiple objects with different prompts.

Advanced usage case: Inpainting three girls into Tifa, Aith, and Yui from Final Fantasy 7.

Techniques for maintaining image composition while inpaintings with control net models.

Final results and tips for cleaning up artifacts in inpainted images.