Trying to have validation loss while training Anima LoRa does not work.
traceback (most recent call last):
File "/ai/sd-scripts/anima_train_network.py", line 448, in <module>
trainer.train(args)
File "/ai/sd-scripts/train_network.py", line 1558, in train
loss = self.process_batch(
^^^^^^^^^^^^^^^^^^^
File "/ai/sd-scripts/anima_train_network.py", line 353, in process_batch
text_encoder_outputs_list = anima_text_encoding_strategy.drop_cached_text_encoder_outputs(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: AnimaTextEncodingStrategy.drop_cached_text_encoder_outputs() missing 1 required positional argument: 't5_attn_mask'
steps: 0%| | 4/5000 [00:12<4:18:41, 3.11s/it, avr_loss=0.0595]
validation steps: 0%|▏ | 1/300 [00:01<08:13, 1.65s/it, val_avg_loss=0.102, timestep=200]
Traceback (most recent call last):
File "/ai/kohya_ss/.venv/bin/accelerate", line 10, in <module>
sys.exit(main())
^^^^^^
File "/ai/kohya_ss/.venv/lib/python3.11/site-packages/accelerate/commands/accelerate_cli.py", line 50, in main
args.func(args)
File "/ai/kohya_ss/.venv/lib/python3.11/site-packages/accelerate/commands/launch.py", line 1199, in launch_command
simple_launcher(args)
File "/ai/kohya_ss/.venv/lib/python3.11/site-packages/accelerate/commands/launch.py", line 785, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/ai/kohya_ss/.venv/bin/python', 'anima_train_network.py', '--config_file=/ai/training/anima_test.toml']' returned non-zero exit status 1.
Trying to have validation loss while training Anima LoRa does not work.
Enabling validation loss with
validation_split = 0.5andvalidate_every_n_steps = 4results in the following error: