[Film] Sorting the aovs in specfilm.cpp

[Angom8]

Hello,

I am currently working on a spectral albedo implementation for Mitsuba3 and noticed some weird interactions with the specfilm plugin. My main question revolves around this line in the prepare function.

std::sort(sorted.begin(), sorted.end());

What is its purpose ? I don't see any clear use that would require a sorted array of channels. It may also cause some unintended behaviors if the loaded scene tries to name its SRFs with the letter y or z, as their content will be placed last and mess with this instruction later in prepare_sample

aovs[m_channels.size() - 1] = weight;

I noticed this as my aov channels were being placed last instead of the weights and consequently created a film plugin without the sorting. I wanted to make sure nothing would go wrong without it.

Have a great day,

[njroussel]

@mcrescas Do you mind taking a look at this?
I have no idea how the specfilm works.

[mcrescas]

Hi @Angom8 ,

The sorting happening in the prepare function has the objective of checking if there is any duplicate channel name.
The question here is why your channels are being sorted, given that this operation only affects a copy of themselves.

Could be possible to have more information about your setup or even a reproducer?
Notice that by default the spectral sensitivities passed in the properties of the plugin will be sorted by its name (this is a hardcoded behaviour of Mitsuba).

Best regards,

[Angom8]

Hi again !
I may have mixed in two issues together with the sorting function (which do apply on an unused copy).
I applied 16 SRFs to a specfilm (named s0.srfs_N) that would sample a derived form of the aov plugin (implementing a spectral Albedo + XYZ Albedo). However, when rendering with the base aov integrator + the base specfilm, i noticed that the resulting exr would contain incorrect channels.

Regular aov uuv render (base aov integrator + hdr film)

Anormal uuv aov render (base aov integrator + base specfilm)

The results would differ (with some overlapping) based on the channels names (aovs included) with more or less artifacts. I tried for example "zrfs" (without the "s0.").

Boundaries aov render (base aov integrator + base specfilm) with one or more SRF named Y or Z. It displays a complete black screen otherwise.

I'm trying to look deeper to determine what exactly causes this behavior. I can't exactly say that the names have an impact or if the specfilm tries to apply the srfs to the aov integrator channels.

SRFs used (base specfilm)

<spectrum name="s0.srf_0" value="360.0:1,387.6470588235294:1,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_1" value="360.0:0,387.6470588235294:1,415.29411764705884:1,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_2" value="360.0:0,387.6470588235294:0,415.29411764705884:1,442.94117647058823:1,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_3" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:1,470.5882352941177:1,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_4" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:1,498.2352941176471:1,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_5" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:1,525.8823529411765:1,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_6" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:1,553.5294117647059:1,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_7" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:1,581.1764705882354:1,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_8" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:1,608.8235294117648:1,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_9" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:1,636.4705882352941:1,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_10" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:1,664.1176470588235:1,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_11" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:1,691.7647058823529:1,719.4117647058824:0,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_12" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:1,719.4117647058824:1,747.0588235294117:0,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_13" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:1,747.0588235294117:1,774.7058823529412:0,802.3529411764706:0"/>
<spectrum name="s0.srf_14" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:1,774.7058823529412:1,802.3529411764706:0"/>
<spectrum name="s0.srf_15" value="360.0:0,387.6470588235294:0,415.29411764705884:0,442.94117647058823:0,470.5882352941177:0,498.2352941176471:0,525.8823529411765:0,553.5294117647059:0,581.1764705882354:0,608.8235294117648:0,636.4705882352941:0,664.1176470588235:0,691.7647058823529:0,719.4117647058824:0,747.0588235294117:0,774.7058823529412:1,802.3529411764706:1"/>

Base aov integrator parameters
<string name="aovs" value="albedo:albedo,pos:position,dep:depth,uuv:uv,geom:geo_normal,nn:sh_normal,dp:dp_du,dup:duv_dx,boun:boundary_test"/>

EDIT : The output is perfectly fine when and only when 4 SRFs are given

[Angom8]

I think i found a fix. By default, integrator.cpp skips 4 or 5 channels of RGBA via render_sample. However, this specifically derails AOVs if used with a spectralfilm. A simple check on the film Flag (Special or Spectral) to adapt the skip to the number of channels is enough.
The naming issue was more or less linked, as channels would overlap depending on their output index.

Applied fix :
I added the m_srfs parameter from specfilm to the film class (as m_srf is already declared here) and a count_srfs function. Instead of always skipping 5 or 4 channels of RGBA, integrator.cpp checks the flags. If the film has the Special Flag, then it calls for the number of srfs.