How to Boost Your ComfyUI Skills: SDXL Refiner and ControlNet

Building an SDXL workflow is like building a complex masterpiece ๐ŸŽจ. With SDXL base and refiner models, we’re creating stunning images step by step. But wait, the real magic is when we use Control Net ๐ŸŒŸ. It helps us direct the stability models to create images just the way we want ๐Ÿš—๐Ÿ“ธ. It’s like giving the models a special recipe ๐Ÿฐ. Stay tuned for more Control Net adventures! #SDXL #ControlNet #WorkflowWizardry

In this video tutorial, Vishnu Suban, founder of JarvisLabs, demonstrates how to build an SDXL workflow using the Stability Excel base and refiner models. This workflow is slightly more complicated than building a basic workflow as it involves using two different models – Stability Excel base and refiner – and ControlNet. ControlNet is a technology that allows for the use of different techniques, like edge detection and depth, to instruct Stability diffusion models to generate specific images based on pose or density.

Building the Workflow

To build the workflow, the first step is to load a checkpoint. In this example, we will have two checkpoints – SDXL base and SDXL refiner. We will use a positive and negative prompt shared between the two checkpoints. To do this, we convert the text as an input and add a primitive node. We create two text encoders – one for positive and one for negative – and connect them to the checkpoints. We then add a sampler using K sampler Advance and make some adjustments to use the base model for the first 20 steps and the refiner model for the next 5 steps. We connect the positive and negative prompts to the conditioning, and the latent image to the sampler. We add a decoder and preview the image.

ControlNet

ControlNet is used to create images of a particular type. We use a depth detector and a load image node to create a depth map of an image of a car on a racetrack. We then use the apply control net Advanced node to connect the depth map to the ControlNet and attach the positive and negative prompts. We then reconnect to the K sampler. We can adjust the prompts to get an image of similar texture.

Key Takeaways

  • SDXL workflows involve using Stability Excel base and refiner models.
  • ControlNet allows for the use of different techniques to instruct Stability diffusion models to generate specific images.
  • K sampler Advance allows for the use of different models for different steps.
  • Apply control net Advanced node is used to connect ControlNet to the workflow.
  • Adjusting prompts can result in images of similar texture.

FAQ

Q: What is ControlNet?
A: ControlNet is a technology that allows for the use of different techniques to instruct Stability diffusion models to generate specific images.

Q: What is K sampler Advance?
A: K sampler Advance allows for the use of different models for different steps in the workflow.

Q: How do you connect ControlNet to the workflow?
A: Use the apply control net Advanced node to connect the ControlNet to the workflow.

Q: How do you adjust prompts in the workflow?
A: Adjusting prompts can result in images of similar texture.

About the Author

About the Channel๏ผš

Share the Post:
en_GBEN_GB