Is the "displacement" useful?

[shawn-AI-Tech]

I found multiple instances of “displacement” in the code. This will cause an error in the code. Is this part useful or not?

[wonjunior]

Hi, this was part of an extension I was working on but it didn't make it to the paper. The idea was to output an additional displacement map and supervise that branch to improve the material quality. The current implementation does not require that key do you can discard it!

[wonjunior]

Can you let me know which file and line threw the error? Thanks!

[shawn-AI-Tech]

Can you let me know which file and line threw the error? Thanks!
You can search “displacement” in these files to get which line. I add a switch solve the error.
capture/callbacks/visualize.py
capture/render/main.py
capture/source/loss.py

[shawn-AI-Tech]

I have some problem when finetuning the network for a further 100 epochs on AmbientCG and TexSD,
when compute loss (capture/source/loss.py line105)
if self.use_target: for k, w, loss_fn in self.weights: loss[f'tgt_{k}'] = self.pl_reg_weight*loss_fn(y_tgt[k], x_tgt[k])
the 'x_tgt' keys is dict_keys(['dir', 'name', 'path', 'input']), but it needed dict_keys(['albedo', 'roughness', 'normals']) .Is the dataset class StableDiffusion is wrong version or I need to preprocess the TexSD before training start?

[wonjunior]

Indeed, TexSD only contains the texture images. To train the fine-tuning stage, you first need to output pseudo-labels using the checkpoint from the pertaining stage. Then you must load the labels in capture/data/target.py.

For your information, if you are interested in using our published weights, you can do so by downloading this release.