Qwen Image and Edit - Complete LoRA Training Guide.md

This guide provides a comprehensive walkthrough for training `Qwen Image` and `Qwen Image Edit` models using the **SECourses Musubi Tuner** application. It covers both LoRA and full Fine-Tuning (DreamBooth) methods.

This documentation is a reference guide based on the [full video tutorial](https://www.youtube.com/watch?v=DPX3eBTuO_Y), which provides critical visual context.

### Primary Resources

- **Main Patreon Post (for App Download):** [Qwen Trainer App & Presets](https://www.patreon.com/posts/qwen-trainer-app-137551634)

- **Video Tutorial (Source):** [Full Comprehensive Qwen Training Tutorial](https://www.youtube.com/watch?v=DPX3eBTuO_Y)

- **Mandatory Requirements Tutorial:** [How to Install Python, CUDA, etc.](https://www.youtube.com/watch?v=DrhUHnYfwC0)

- **Inference App (SwarmUI) Tutorial:** [How to Install and Use SwarmUI](https://www.youtube.com/watch?v=c3gEoAyL2IE)


---

## Part 1: Initial Setup & Installation

Before you can train, you must set up the environment and the training application.

### Mandatory Prerequisites

You **must** install all required base dependencies _before_ installing the trainer.

1. You can find detailed information in the [[Installation of Python and Dependencies]] note. Watch the [Mandatory Requirements Tutorial](https://www.youtube.com/watch?v=DrhUHnYfwC0) if you need more details, though keep in mind that it's outdated in terms of what specific versions to install.

2. Install all components shown, including:

    - Python 3.10.11

    - CUDA (e.g., 12.9) & cuDNN

    - FFmpeg

    - C++ tools & MSVC

    - Git


### Core Application Installation

This process installs the **SECourses Musubi Tuner** application and downloads the necessary base models.

1. **Download & Extract:**

    - Download the "Latest Zip File" (e.g., `SECourses_Musubi_Trainer_v19_1.zip`) from the [main Patreon post](https://www.patreon.com/posts/qwen-trainer-app-137551634).

    - Move the zip file to your desired installation drive (e.g., `Q:\`).

    - Extract the file (e.g., "Extract Here").

2. **Install Application:**

    - Enter the extracted folder.

    - Double-click `windows_install_and_update.bat` to run it. **Do not run as administrator**.

    - This creates a secure, isolated virtual environment (`venv`) and installs all libraries.

3. **Download Training Models:**

    - After installation finishes, run `windows_download_training_models.bat`.

    - You will be prompted to choose which models to download:

        - **Option 1: Qwen base model.** (Used for standard `Qwen Image` training).

        - **Option 2: Qwen Image Edit Plus 2509 model.** (Used for `Qwen Image Edit` training).

    - The downloader is highly robust: it's resumable, uses your full internet speed, and verifies file integrity with SHA256 hashes to prevent corruption.

    - The models will be saved in the `training/models/Qwen` folder.

4. **Start the Application:**

    - Run `windows_start_app.bat` to launch the Gradio web UI.

    - Your web browser will automatically open to the application's interface.


---

## Part 2: Dataset Preparation

The quality and structure of your dataset are the most critical factors for a successful training.

### General Guidelines & Tools

These tools and principles apply to all training scenarios.

#### Dataset Quality

The model will only be as good as your input images. Pay close attention to:

- **Lighting & Focus:** Images should be clear, well-lit, and in-focus. Do not use images with fuzzy or artificially blurry (bokeh) backgrounds, as the model may learn this blur as part of the subject.

- **Quality:** Use the highest possible quality images you can get. Even medium-quality images can produce good results, but higher quality is always better.

- **Variety:** Do not use images that are too similar. Variety is key to helping the model learn and not just memorize.


#### Image Processing: Ultimate Batch Image Processing Tool

This is a separate tool, linked in the Patreon post, for pre-processing your images. It is highly recommended.

- **Stage 1: Auto-Cropping:** This stage intelligently finds the subject (e.g., "person") and zooms in to frame it, attempting to match your target aspect ratio _without_ cropping key parts.

- **Stage 2: Resizing:** This stage takes the cropped images and resizes them to your _exact_ final resolution, cropping if necessary to fit the dimensions.

- **Recommended Resolution:** The recommended resolution for `Qwen Image` is **1328x1328**.


#### Internal Pre-processing Tool

The main training app has a tab called "Image Pre-processing". This tool is not for batch processing, but for _verification_. It allows you to see the _exact_ final image the trainer will use, including bucketing and orientation. This is extremely useful for:

- Verifying your images are oriented correctly (it can fix EXIF orientation issues).

- Seeing how bucketing (multiple aspect ratios) will be applied.


#### Dataset Folder Structure

This is the mandatory folder structure the trainer expects.

1. Create a main folder (e.g., `training_images_dataset`).

2. Inside that folder, create a subfolder using this naming convention: `[repeats]_[trigger_word] [class_word]`.

    - **Example:** `1_ohwx man`.

    - `1_`: This is the number of repeats. For single-concept training (one person, one style), this should always be `1`.

    - `ohwx`: This is your **trigger word**. It should be a rare, random, single word.

    - `man`: This is the class word.

3. Place all your processed training images inside this subfolder.

4. When setting up the trainer, you will point it to the _parent folder_ (`training_images_dataset`), not the subfolder.


---

### Scenario 1: Style Training Dataset

This is for teaching the model an artistic style (e.g., "GTA 5 style").

- **Key Concept:** Consistency of the _style_ is the number one priority. All images must look like they belong to the same style.

- **Rules:** The most important rule is to **avoid repetition** of anything _except_ the style.

    - **DO NOT** repeat characters.

    - **DO NOT** repeat objects or items.

    - **DO NOT** repeat specific scenes, places, or buildings.

- **Dataset Size:** For style training, more images are better. The more examples you provide, the better the model will learn the style's nuances.

- **Captioning:** Use a single trigger word (e.g., `ohwx`). The trainer will use the folder name (`1_ohwx style`) to generate captions. Detailed captions are _not_ recommended and may perform worse.


#### Style Dataset Examples

- **Bad Dataset (Too Repetitive):** A dataset of 30 "GTA 5 style" images, but 15 of them are of the character Michael and 15 are of the character Franklin. The model will learn to generate _Michael_ and _Franklin_ (overfitting on the characters) instead of learning the _style_ itself.

- **Good Dataset (Consistent Style):** A dataset of 30 "GTA 5 style" images, each one featuring a _different_ subject: one image of a car, one of a building, one of a sunset, one of a random NPC, one of a street, etc. No single character, object, or location repeats. This forces the model to find the only common element: the art style.

- **Good Tactic:** One way you can make fewer images go farther is by taking one large, high-resolution screenshot of a complex scene (e.g., a city street) and **cropping it into 3-4 smaller, unique images** (a close-up of a car, a building, a pedestrian). This builds a larger dataset that has consistent style without repetition.


### Scenario 2: Subject Training Dataset (e.g., a person)

This is for teaching the model a specific person, character, or object.

- **Key Concept:** Variety is essential to prevent _memorization_ (overfitting). You want the model to learn the _subject_, not the specific clothes or background they were in.

- **Rules:** Your dataset should include a wide mix of:

    - **Clothing:** Use as many different outfits as possible.

    - **Backgrounds:** Shoot in different locations and environments.

    - **Angles & Poses:** Include all angles you want to generate later (close-ups, full body, looking down, etc.).

    - **Emotions:** Include all emotions you want the model to be able to replicate (smiling, laughing, serious, etc.).

- **Captioning:** A single trigger word is **strongly recommended** (e.g., `ohwx`).

    - After testing detailed captions vs. a single trigger word, the single word (`ohwx`) worked best.

    - If you _must_ use detailed captions, the app includes an "Image Captioning" tab that uses the Qwen 2.5-VL model, but the single-word method is preferred.


#### Subject Dataset Examples

- **Bad Dataset (Too Similar):** A dataset of 30 images, all taken from the same 5-minute "selfie session". Even if the smile is slightly different, the subject is wearing the **same shirt**, in the **same room**, with the **same lighting**. The model will memorize the shirt and the room, and you'll struggle to generate the subject in any other context.

- **Good Dataset (Good Variety):** A dataset of 28 "medium quality" images is effective _because_ it includes:

    - **Angles:** "Full body pose," "half body pose," and "very close shot".

    - **Context:** "Different clothings," "different times," and "different backgrounds".

    - **Emotions:** Images that are "smiling" or show other desired emotions. If you want the model to laugh, you must provide images of laughing.

    - **Poses:** A specific pose like "looks down". If you don't include this, the model won't know how to generate it.


### Scenario 3: Product Training Dataset

This is for teaching the model a specific product, focusing on its details and shape.

- **Key Concept:** The model must learn both the fine _details_ (like text) and the overall _proportions_ of the product.

- **Rules:** Your dataset must include a mix of:

    1. **Close-up Shots:** Very close shots to capture small details, logos, and text. `Qwen Image` is excellent at learning small text.

    2. **Contextual/Distant Shots:** Shots that show the product's full-body proportion, ideally next to other objects (e.g., a person holding it, a glass next to it) so the model learns its relative size.

- **Captioning:** Use a single trigger word (e.g., `ohwx`).


#### Product Dataset Examples

- **Bad Dataset (Poor Proportion):** A dataset of 20 images of a perfume bottle, all "extremely close shot" macros focusing on the logo and the text.

- **The Consequence:** The model will learn the text and logo _perfectly_. But when you prompt "a photo of the perfume bottle on a table," the generated bottle's **size and shape will be inaccurate** and distorted. The model learned the _details_, but not the _proportions_.

- **Good Dataset (Good Proportion):** The "Bad Dataset" (for details) _plus_ 10 more images: 5 images showing the "perfume bottle in a person's hand" and 5 images showing it "next to a glass on a table".

- **The Result:** These "contextual shots" teach the model the product's _scale_ and _proportion_ relative to other objects, fixing the problem.


### Scenario 4: Qwen Image Edit Dataset (Advanced)

This is a completely different process for training a `Qwen Edit` or `Qwen Edit 2509` model to perform a specific _command_ or _action_ (e.g., "replace clothing," "colorize sketch").

- **Use Case:** You are not teaching _what_ something looks like, but _how to do_ something.

- **Structure:** Your dataset requires three components, linked by file name. An example dataset is included in the zip file.

    1. **Final Image (The Result):** The "after" image.

    2. **Control Image(s) (The Input):** The "before" image(s). You can provide up to 3.

    3. **Caption (The Command):** A text file containing the command.

- **Image Sizing:** All images (input and output) should be resized to your target resolution (e.g., 1328x1328).


#### Example Command

- **Caption (`final_image_A.txt`):** `make him wear the headphones`

- **Final Image (`final_image_A.jpg`):** Photo of a man wearing headphones.

- **Control Image (`final_image_A0.jpg`):** The _same_ photo, but with the man _not_ wearing headphones.


---

## Part 3: Training Configuration & Settings

This section covers the settings in the SECourses Musubi Tuner's Gradio UI.

### Loading a Preset (Crucial First Step)

**Do this first.** The app comes with pre-made `.toml` configuration files for different scenarios and hardware tiers.

1. In the "Qwen Image Training" tab, click the "Load config" folder icon.

2. Navigate to the `Qwen-Training-Configs` folder in your installation directory.

3. Choose your configuration:

    - **`LoRA_Training` vs. `Fine_Tuning_Training-DreamBooth`:** Choose your method.

    - **Epochs (e.g., `200_epoch`):** Higher epochs (200) use a lower learning rate for higher quality, but are slower. Lower epochs (50) are faster but may be lower quality.

    - **GPU Tiers (e.g., `Tier_2_30000_MB`):** These are set for different VRAM amounts. The `LoRA_Configs_Explanation.jpg` file inside the folder explains the tiers.

4. Select your desired `.toml` file (e.g., `Qwen_Image_LoRA_200_epoch_Tier_2_30000_MB.toml`).

5. Click the blue "Load parameters from file" icon to apply the settings.


### VRAM & System Preparation

Training is VRAM-intensive. You must prepare your system.

- **Check VRAM:** Open a command prompt and type `nvidia-smi` to see your total VRAM. Type `nvitop` (if installed) to see your _current_ VRAM usage.

- **Minimize VRAM Usage:** Your goal is to have the maximum VRAM free.

    - **Restart your PC** before a training session.

    - **Close all other applications** (browsers, Discord, etc.).

    - Disable unnecessary startup apps in Task Manager and restart.


### Core Training Parameters (Gradio UI Tabs)

After loading a preset, you only need to change a few paths.

1. **`Checkpoint Save Settings` Tab:**

    - **Save Path (`output_dir`):** Click the folder icon and select the directory where your checkpoints will be saved. For SwarmUI, this is `SwarmUI/models/lora` (for LoRA) or `SwarmUI/models/diffusion_models` (for Fine-Tuning).

    - **Save Frequency (`save_every_n_epochs`):** The preset (e.g., 25) is usually fine. A LoRA checkpoint is about 2.3GB.

2. **`Qwen Image Training Dataset` Tab:**

    - **Dataset Path:** Click the folder icon and select the _parent folder_ of your dataset (e.g., `training_images_dataset`).

    - **Resolution:** Ensure this matches your preset and dataset (e.g., 1328x1328).

    - **Generate Config:** Click the **"Generate dataset configuration"** button. This will find your subfolder, read your trigger word, and create the caption files (`.txt`) for you.

3. **`Qwen Image Model Settings` Tab:**

    - **Base Model:** Click the folder icon and select your base model (e.g., `qwen_image_bf16.safetensors`).

    - **VAE:** Select the VAE (e.g., `qwen_train_vae.safetensors`).

    - **Text Encoder:** Select the text encoder (e.g., `qwen_2.5_vl_7b_bf16.safetensors`).

4. **`Training Settings` Tab:**

    - **Epochs (`max_train_epochs`):** This will be set by your preset (e.g., 200). For a dataset with <50 images, 200 epochs is recommended.

5. **Save Your Config:**

    - Click the "Save" icon to save your new configuration (with the correct paths) as a new `.toml` file. This allows you to easily re-load this exact setup later.


---

### Parameter Differences: LoRA vs. Fine-Tuning (DreamBooth)

The process is nearly identical, but the key differences are:

- **Preset:** You must load a preset from the `Fine_Tuning_Training-DreamBooth` folder.

- **Speed:** Fine-Tuning is significantly slower than LoRA on Windows.

- **Checkpoint Size:** Fine-Tuning checkpoints are massive (**40GB**).

- **Post-Training Step:** After training, you **must** use the "FP8 Model Converter" tab to convert your 40GB BF16 models to FP8 scaled. This reduces them to ~20GB, makes them "tensor-wise" scaled for high quality, and allows consumer GPUs to load them.


### Parameter Differences: Qwen Image vs. Qwen Image Edit

You can train on both the base `Qwen Image` model and the `Qwen Edit` models.

- **`Qwen Edit` (Without Control Images):**

    - This trains the `Qwen Edit 2509` model to learn a new subject or style, just like base training.

    - The setup is identical to base training, with two changes:

        1. Check the **"Enable Qwen-Image-Edit-2509 Mode"** box.

        2. Set the "Base Model" path to the `Qwen Edit 2509` model.

- **`Qwen Edit` (With Control Images):**

    - This is for teaching the model a new command, using the `[[#Scenario 4: Qwen Image Edit Dataset (Advanced)|special dataset]]`.

    - This process uses _significantly_ more VRAM.

    - You must go to the `Qwen Image Model Settings` tab and **increase the `block_swap_size`** value (e.g., from 7 to 35, 40, or more). If you get an out-of-memory (OOM) error, stop the training and increase this value.


---

## Part 4: Running & Monitoring Training

### Starting and Monitoring

1. Click the **"Start Training"** button.

2. Watch the console window. The first time you run a dataset, the app will **cache** your images and text encodings. This can take a while, but it only happens once per dataset.

3. The training will begin, showing you the progress, steps, and speed (s/it).


### Performance Optimization & Troubleshooting

If your training is extremely slow (e.g., much slower than the 8-15 s/it shown in the video), it is likely due to block swapping on Windows.

- **Fastest Fix:** Use a lower-tier preset. The `50_epoch` configs are 4x faster because they use a higher learning rate and fewer steps.

- **Enable Pinned Memory:** In the `Qwen Image Model Settings` tab, check **"Use pinned memory for block swapping"**. This can improve speed but uses more system RAM.

- **Disable HAGS:** In Windows "Graphics Settings," turn **off** "Hardware-accelerated GPU scheduling" and **restart your PC**.

- **Overclocking:** Advanced users can use MSI Afterburner to overclock the GPU core and memory clock, which can further improve speed.


### Resuming an Incomplete Training

- **To Resume a LoRA:**

    1. Load your saved config file.

    2. Go to the `LoRA Settings` tab.

    3. In the **"Network Weights (LoRA Weight)"** field, set the full path to your _last saved checkpoint file_ (e.g., `.../Qwen-Image-Lora-Tutorial-000200.safetensors`).

    4. Go to the `Training Settings` tab. Calculate the _remaining_ epochs and enter that number.

        - Example: Your original training was 200 epochs. You want to train for 50 more. Set `max_train_epochs` to **50**.

    5. Change the output folder or name to avoid overwriting.

- **To Resume a Fine-Tuning (DreamBooth):**

    1. Load your saved config file.

    2. Go to the `Qwen Image Model Settings` tab.

    3. Set the **"Base Model"** path to your _last saved checkpoint_ (e.g., your 100-epoch model).

    4. Go to the `Training Settings` tab and set the _remaining_ epochs, just like with LoRA.


---

## Part 5: Post-Training Workflow (Analysis & Usage)

After training, you must test your checkpoints in [SwarmUI](https://www.youtube.com/watch?v=c3gEoAyL2IE) to find the best one.

### Setting up SwarmUI

1. Install SwarmUI with the ComfyUI backend as shown in its dedicated tutorial.

2. **Move Checkpoints:**

    - Copy your LoRA `.safetensors` files to `SwarmUI/models/lora/`.

    - Copy your FP8-converted Fine-Tune models to `SwarmUI/models/diffusion_models/`.

3. Start SwarmUI and load the `Qwen` generation presets.


### Analyzing Training Results (Finding Your Best Checkpoint)

This is the most important _skill_ in the training process. Your goal is to let the training run and save multiple checkpoints (e.g., every 25 epochs), then use this process to analyze them and find the "sweet spot".

1. In SwarmUI, go to **Tools -> Grid Generator**.

2. Set the **Parameter** dropdown to:

    - `LoRAs` (if testing a LoRA).

    - `Model` (if testing a Fine-Tune).

3. **Select Checkpoints:** In the list below, select all the checkpoints you want to compare (e.g., 100, 125, 150, 175, 200).

4. **Add Prompts:** Paste several test prompts into the `Prompt` box.

5. Click **Generate Grid**.

6. **Analyze the Grid:**

    - Open the finished grid and change the **"View By"** dropdown to `Prompt`.

    - Now, each row is a prompt, and the columns are your different epochs.


#### Interpreting Your Results & How to Iterate

Analyze the grid to decide your next steps. You will generally see one of three outcomes:

- **Outcome 1: Under-training**

    - **How to Spot:** Even your _final_ checkpoint (e.g., 200 epochs) looks weak, "not great," and doesn't strongly resemble your subject or style.

    - **The Solution:** You didn't train for long enough. You must **resume training** from your last checkpoint. Go back to `[[#Resuming an Incomplete Training]]`, load your 200-epoch checkpoint, and train for 50-100 _more_ epochs.

- **Outcome 2: Over-training (Overfitting)**

    - **How to Spot:** Your subject looks "burnt in" or "fried," and the model struggles to generate anything _different_ from your training images. The 200-epoch checkpoint looks _worse_ and less flexible than the 175-epoch one.

    - **The Solution (Simple):** This is why you saved multiple checkpoints. Don't use the final one. Your "best" model is the **earlier checkpoint** (e.g., 175-epoch) that looked good _before_ the quality started to degrade.

    - **The Solution (Inference):** If a checkpoint is just a _little_ too strong, you can fix it at generation time. In SwarmUI, simply **lower the LoRA's weight** from `1.0` to `0.9` or `0.8`.

- **Outcome 3: "Just Right" (The "Sweet Spot")**

    - **How to Spot:** You find a checkpoint (e.g., 175-epoch) that looks "much better" than the ones before it and is not overfit. It strongly represents your subject/style and is flexible.

    - **The Solution:** This is your "best" model. You are finished. Use this checkpoint for `[[#Generating High-Quality Images]]`.


### Generating High-Quality Images

This is a two-stage workflow: **Generate > Select > Upscale**.

1. **Stage 1: Generation**

    - Load your base preset (e.g., `Qwen-Image-Realism-Tier-2`).

    - Select your **best LoRA** (if LoRA) or set the **Base Model** to your best Fine-Tune checkpoint.

    - **Disable Refine (Upscale)**.

    - Set **Images** to 4 or 8.

    - Write your prompt and click **Generate**. This will quickly create many low-res previews.

2. **Stage 2: Upscale & Refine**

    - Find an image you like from the batch.

    - Click **"Reuse Parameters"** on that image to lock in its seed.

    - Load the **Upscale preset**.

    - Set **Images** back to 1.

    - Click **Generate** again. This will re-run the _same seed_ but apply the high-quality upscale and refinement steps.


#### Fixing Faces & Details

- **Automatic (Segmentation):** At the end of your prompt, add `segment:face` (e.g., `a photo of ohwx man segment:face`). This will auto-detect the face and inpaint it.

- **Manual:** Click **"Edit Image"**, use the mask tool to draw over the face/area you want to fix, and click Generate. This gives you more control.


#### Prompting Strategy: The "Gemini" Method

Even though you trained with a single word, you should use **detailed, descriptive prompts** at inference time.

[Watch a visual of this process](https://www.google.com/search?q=https://www.youtube.com/watch%3Fv%3DDPX3eBTuO_Y%23t%3D1h8m49s)

1. Go to **Google AI Studio (Gemini)**.

2. If training a style or product, upload 10-20 of your training images.

3. Open the example prompt files from the tutorial (e.g., `Gemini generate trained style prompts`).

4. Paste this "mega-prompt" into Gemini.

5. Gemini will generate 10s or 100s of high-quality, descriptive prompts (including your `ohwx` trigger word) for you to use in SwarmUI.

6. This works because the model _internally_ encodes all the details from your images during training, even without captions.


---

## Part 6: Resources & Links

- **Main Patreon Post (for App Download):** [https://www.patreon.com/posts/qwen-trainer-app-137551634](https://www.patreon.com/posts/qwen-trainer-app-137551634)

- **Video Tutorial (Source):** [https://www.youtube.com/watch?v=DPX3eBTuO_Y](https://www.youtube.com/watch?v=DPX3eBTuO_Y)

- **Mandatory Requirements Tutorial:** [https://www.youtube.com/watch?v=DrhUHnYfwC0](https://www.youtube.com/watch?v=DrhUHnYfwC0)

- **Inference App (SwarmUI) Tutorial:** [https://www.youtube.com/watch?v=c3gEoAyL2IE](https://www.youtube.com/watch?v=c3gEoAyL2IE)

- **Community Links:**

    - **Discord:** [SECourses Discord](https://discord.com/servers/software-engineering-courses-secourses-772774097734074388)

    - **GitHub:** [Stable Diffusion & Generative AI Repo](https://github.com/FurkanGozukara/Stable-Diffusion)

    - **Reddit:** [r/SECourses Subreddit](https://www.reddit.com/r/SECourses/)