Extract number of signal electrons from a spectral image

[Ketaries]

Hi, I am currently working on a research project where we want to simulate spectral images of different objects.

In order to validate our tool I would like to extract the number of photons or the number of signal electrons for each pixel in the sensor (for comparison with rough hand calculations). If I am not mistaken, the radiance stored in the final .exr output already accounts for the quantum efficiency of the sensor (the srf provided for each band) therefore it's directly linked to the number of signal electrons.

I'm not sure how exactly I can extract this number but I think I'm missing a few things:

Whats the sensor's pixel size (depending on the height and width parameters) ?
What shutter time is used during a render ? I noticed the shutter_open and shutter_open_time methods but I believe I can't use them.

Thank you in advance for your help !

[Speierers]

Hi @Ketaries ,

You are only interested in the number of photons hitting a pixel regardless of the amount of energy they carry, is that correct? In this case, you should be able to use Film.develop(raw=True) and look at the last channel of this tensor (e.g. the weight channel).

The sensor's pixel size can be computed based on the height, width, fov and near_clip parameters of the perspective sensor plugin. Take a look at the m_sample_to_camera matrix in perspective.cpp.

During rendering, every light path will be generated for a specific time in [shutter_open, shutter_open + shutter_open_time]. Those are parameters of the sensor you are using.

[Ketaries]

Hi @Speierers, thank you for your reply !

The goal would be to compare the number of photons reaching the pixels for each wavelengths, I guess I need to take the energy they carry into account in this case. I tried Film.develop(raw=True) and the last channel returns a roughly constant value for the entire image whereas the object imaged only occupies a square in the center, does that make sense ?

I've had a look at the m_sample_to_camera matrix but I don't think I understand how the pixel size is computed. Am I correct assuming m_dx and m_dy are the x and y sizes of the pixels ?
Also in my scene I am using the thinlens camera, I'm assuming this doesn't cause a problem because they are fairly similar right ?

Regarding the shutter time, I got the shutter opening time and it's duration by calling shutter_open and shutter_open_time but they both return a value of 0.0 which seems weird to me.

[Speierers]

I tried Film.develop(raw=True) and the last channel returns a roughly constant value for the entire image whereas the object imaged only occupies a square in the center, does that make sense ?

Yes you might need to take into account the alpha channel as well (channel before last).

I've had a look at the m_sample_to_camera matrix but I don't think I understand how the pixel size is computed. Am I correct assuming m_dx and m_dy are the x and y sizes of the pixels ?

Yes they should be IIRC.

Regarding the shutter time, I got the shutter opening time and it's duration by calling shutter_open and shutter_open_time but they both return a value of 0.0 which seems weird to me.

Did you specify the shutter_open and shutter_close properties when constructing your sensor (e.g. in the XML file)?

[Ketaries]

Yes you might need to take into account the alpha channel as well (channel before last).

The image I generate has 51 channels (400 nm to 900 nm with 10 nm steps), when using Film.develop(raw=True) the tensor returned only has 52 channels, the additional one being the weight channel. If the alpha channel is the channel before last, I am missing a channel no ?

Did you specify the shutter_open and shutter_close properties when constructing your sensor (e.g. in the XML file)?

I didn't specify those parameters indeed, my bad. They are nowhere in the mitsuba 3 documentation, is that volontary ? The only place I could find reference to them is the mitsuba 0.5.0 documentation.
I played a bit with these parameters and it seems like a longer exposure doesn't affect the values stored in the weight channel...

[Speierers]

Actually it looks like the specfilm plugin doesn't support alpha channel. @mcrescas is that correct?

I didn't specify those parameters indeed, my bad. They are nowhere in the mitsuba 3 documentation, is that volontary ? The only place I could find reference to them is the mitsuba 0.5.0 documentation.

We will fix that.

I played a bit with these parameters and it seems like a longer exposure doesn't affect the values stored in the weight channel...

You shouldn't expect a longer exposure. The shutter times are meant to be used to produce a motion blur effect, which isn't implemented yet. The resulting energy will then be divided by the shutter time, hence the exposure should remain constant.

Given the specific needs of your experiment, it sounds to me that you will need to implement a custom pipeline on top of Mitsuba 3 (e.g. integrator, film, ...). This can easily be done directly in Python as shown in this tutorial or by implement your own plugins (C++ or Python).

[Ketaries]

Thanks for the answers !

You shouldn't expect a longer exposure. The shutter times are meant to be used to produce a motion blur effect, which isn't implemented yet. The resulting energy will then be divided by the shutter time, hence the exposure should remain constant.

But a longer exposure (= larger shutter time) should cause a larger number of photons to be captured isn't it ?

Given the specific needs of your experiment, it sounds to me that you will need to implement a custom pipeline on top of Mitsuba 3 (e.g. integrator, film, ...). This can easily be done directly in Python as shown in this tutorial or by implement your own plugins (C++ or Python).

Thanks for the tip, I'll have a look a that. In the mean time I managed to do some sort of validation by directly comparing radiances values.

[mcrescas]

But a longer exposure (= larger shutter time) should cause a larger number of photons to be captured isn't it ?

Longer exposure / shutter time is not implemented yet. What that means is that there is no notion of time inside the renderer. Because of that, we are not simulating photons hitting the sensor at different times. Modifying these parameters should not have right now any effect on the simulation.

[Ketaries]

Alright, that explains why nothing was changing. thanks for the clarification :)

[mcrescas]

Hi 👋 ,

Actually it looks like the specfilm plugin doesn't support alpha channel. @mcrescas is that correct?

It is correct that specfilm currently does not support alpha channel.