Masked training by default

[HuskedHull]

Why is masked training not enabled by default?

I thought that masked training was useful instead of cropping for privacy of those who made photos, and to reduce the restrictions for the LoRA on CivitAI, but it is disabled by default, and even when enabled, OneTrainer has "unmasked_probability": 0.1 and "unmasked_weight": 0.1.

https://github.com/Nerogar/OneTrainer/wiki/Training#masked-training

To enable masked training, you need to add a mask for every training image.

What happens if an image does not have a mask?

https://github.com/Nerogar/OneTrainer/wiki/Lessons-Learnt-and-Tutorials

Enabling masked training didn't increase VRAM used by much, maybe a few MB, but currently it does not work when latent caching is enabled.

What happens if both latent caching and masked training are enabled?

I tried training with both issues:

epoch:  75%|████████████████████████████████████████████████████████████████████████████████████                            | 3/4 [1:52:22<37:27, 2247.38s/it]
Traceback (most recent call last):
  File "/OneTrainer/modules/ui/TrainUI.py", line 577, in __training_thread_function
    trainer.train()
  File "/OneTrainer/modules/trainer/GenericTrainer.py", line 674, in train
    loss = self.model_setup.calculate_loss(self.model, batch, model_output_data, self.config)
  File "/OneTrainer/modules/modelSetup/BaseFluxSetup.py", line 588, in calculate_loss
    return self._flow_matching_losses(
  File "/OneTrainer/modules/modelSetup/mixin/ModelSetupDiffusionLossMixin.py", line 328, in _flow_matching_losses
    losses = self.__masked_losses(batch, data, config)
  File "/OneTrainer/modules/modelSetup/mixin/ModelSetupDiffusionLossMixin.py", line 89, in __masked_losses
    mask=batch['latent_mask'].to(dtype=torch.float32),
KeyError: 'latent_mask'

But next time I clicked "Start Training", it continued training.

[O-J1]

It’s not enabled by default because it’s very clearly not free lunch and only useful in specific circumstances. How did you even come to this conclusion? Please be honest if there is a YouTuber or guide that has influenced you

[HuskedHull]

It’s not enabled by default because it’s very clearly not free lunch

What if there are no masks in the dataset, as would normally be?

How did you even come to this conclusion? Please be honest if there is a YouTuber or guide that has influenced you

I train on photos of objects that sometimes contain people and places who might not want to be in a LoRA. Also, you can't use CivitAI's online generator with a LoRA that "depicts an actual person". So I have a choice: replace the person and the place with blob of color and tag them as what, a blob of color? or use masks.

[O-J1]

Going to bullet point because I’m tired:

1. This has been pinned for months: Am I misunderstanding how masked training works? #347
2. Masked training is not a free lunch and requires careful consideration
3. Creating masks is a large effort
4. We aren’t civitai, why are you mentioning them?
5. You shouldn’t be worrying about it learning someone if your dataset is diverse enough

[HuskedHull]

1. This has been pinned for months: [Am I misunderstanding how masked training works? #347](https://github.com/Nerogar/OneTrainer/discussions/347)

Do you mean people would be confused if they don't specify a background?
Or that solo/duo/trio/1girl are meaningless in a caption for a masked image?
Or is it so bad that "Training non-Face altering LoRas : Full workflow" is bad advice and does not actually work most of the time?

2. Masked training is not a free lunch and requires careful consideration

Does it use a lot of VRAM? A lot of time?
If I don't have any filenames ending in "-masklabel.ext", then mask training would just be disabled, and if I do, then I want it to be enabled unless explicitly disabled, so what's the problem?

3. Creating masks is a large effort

You don't have to create actual masks for all images. Why couldn't full masks be implied?

4. We aren’t civitai, why are you mentioning them?

Because I upload my LoRAs there, and that's an additional reason to avoid people in images.

5. You shouldn’t be worrying about it learning someone if your dataset is diverse enough

Sometimes there are only a few photos of something. Although the reason my LoRA is generating trees suspiciously similar to the ones on photos may be that I didn't tag them as trees/forest.
And there are other reasons to use masks, aren't there?

[HuskedHull]

Or is it so bad that "Training non-Face altering LoRas : Full workflow" is bad advice and does not actually work most of the time?

There's this though:

Masks with both the guy and the girl faces : OK as well, not worth fixing. Although I’ve made a LoRa recently where most of the guys faces were masked and some guys not. It seems the remaining guys “have won” and now appear frequently. I think ideally it would be best to be consistent, either never include guys faces, or always, but YMMV. For safety's sake I would rather remove every single face.

[HuskedHull]

Am I getting this right?

The word "tree" in a prompt can mean one tree, two trees, many trees, a forest. An image with either of those will match "tree".
If I train on an image of one tree surrounded by grass, tagged "tree, grass", then when generating, the grass will be used as a likely context for "tree", and that will make other trees less likely to appear. If I mask the grass out and only tag "tree", the sampler might decide that a forest including that tree will match "tree", too.

If only a face is masked out, and the tag is "human", then the user likely unambiguously wants a face there anyway. If the user prompted for a specific face, that will be the best place for it. But if there are multiple face masks, the model will be confused like it is with tags for multiple characters.