DataType and input files

[francopestilli]

It would be great to discuss the needs for the workflow behind this App so to design a proper INPUT/OUTPUT and DataType.
@soichih @aarya22

[francopestilli]

I attach our communication here:

INPUTS.
Inputs and out puts should theoretically be a DataType containing multishell DWI data. BVECS/BVALS. Ideally the App should create its own internal files that dipy might need to read these files, for example like the GTAB. Does this make sense to you? Perhaps we can start a thread with Ariel and Soichi about this issue? I Started an issue for us on GitHub.

PARAMETERS.
Parameters should be set by the user but have default that work well with the HCP 3T dataset. For example parameters such as radial order.

OUTPUTS.
Outputs should be all NIFTI files with the same header orientation of the input file. No plots.
We should not be worried if many output files are created, but instead, create as many ask possible.

Below all the outputs for which we should create a NIFTI:
• 1 Mean Squared Displacement (MSD) is a measure of how far protons are able to diffuse. a decrease in MSD indicates protons are hindered/restricted more, as can be seen by the high MSD in the CSF, but low in the white matter.
• 2 Q-space Inverse Variance (QIV) is a measure of variance in the signal, which is said to have higher contrast to white matter than the MSD [Hosseinbor2013]. We also showed that QIV has high sensitivity to tissue composition change in a simulation study [Fick2016b].
• 3 Return to origin probability (RTOP) quantifies the probability that a proton will be at the same position at the first and second diffusion gradient pulse. A higher RTOP indicates that the volume a spin is inside of is smaller, meaning more overall restriction. This is illustrated by the low values in CSF and high values in white matter.
• 4 Return to axis probability (RTAP) is a directional index that quantifies the probability that a proton will be along the axis of the main eigenvector of a diffusion tensor during both diffusion gradient pulses. RTAP has been related to the apparent axon diameter [Ozarslan2013] [Fick2016a] under several strong assumptions on the tissue composition and acquisition protocol.
• 5 Return to plane probability (RTPP) is a directional index that quantifies the probability that a proton will be on the plane perpendicular to the main eigenvector of a diffusion tensor during both gradient pulses. RTPP is related to the length of a pore [Ozarslan2013] but in practice should be similar to that of Gaussian diffusion.
• 6 Non-Gaussian diffusion in every voxel [Ozarslan2013]. This quantity is estimated through the ratio between Gaussian volume (MAPMRI’s first basis function) and the non-Gaussian volume (all other basis functions) of the fitted signal. For this value to be physically meaningful we must use a b-value threshold in the MAPMRI model. This threshold makes the scale estimation in MAPMRI only use samples that realistically describe Gaussian diffusion, i.e., at low b-values.

So 6 NIFTIs as output in the DataType for this App.

[arokem]

For reference, here is the PR for the workflow on DIPY: dipy/dipy#1350

…

On Mon, Jan 1, 2018 at 8:21 AM, Franco Pestilli ***@***.***> wrote: I attach our communication here: INPUTS. Inputs and out puts should theoretically be a DataType containing multishell DWI data. BVECS/BVALS. Ideally the App should create its own internal files that dipy might need to read these files, for example like the GTAB. Does this make sense to you? Perhaps we can start a thread with Ariel and Soichi about this issue? I Started an issue for us on GitHub. PARAMETERS. Parameters should be set by the user but have default that work well with the HCP 3T dataset. For example parameters such as radial order. OUTPUTS. Outputs should be all NIFTI files with the same header orientation of the input file. No plots. We should not be worried if many output files are created, but instead, create as many ask possible. Below all the outputs for which we should create a NIFTI: • 1 Mean Squared Displacement (MSD) is a measure of how far protons are able to diffuse. a decrease in MSD indicates protons are hindered/restricted more, as can be seen by the high MSD in the CSF, but low in the white matter. • 2 Q-space Inverse Variance (QIV) is a measure of variance in the signal, which is said to have higher contrast to white matter than the MSD [Hosseinbor2013]. We also showed that QIV has high sensitivity to tissue composition change in a simulation study [Fick2016b]. • 3 Return to origin probability (RTOP) quantifies the probability that a proton will be at the same position at the first and second diffusion gradient pulse. A higher RTOP indicates that the volume a spin is inside of is smaller, meaning more overall restriction. This is illustrated by the low values in CSF and high values in white matter. • 4 Return to axis probability (RTAP) is a directional index that quantifies the probability that a proton will be along the axis of the main eigenvector of a diffusion tensor during both diffusion gradient pulses. RTAP has been related to the apparent axon diameter [Ozarslan2013] [Fick2016a] under several strong assumptions on the tissue composition and acquisition protocol. • 5 Return to plane probability (RTPP) is a directional index that quantifies the probability that a proton will be on the plane perpendicular to the main eigenvector of a diffusion tensor during both gradient pulses. RTPP is related to the length of a pore [Ozarslan2013] but in practice should be similar to that of Gaussian diffusion. • 6 Non-Gaussian diffusion in every voxel [Ozarslan2013]. This quantity is estimated through the ratio between Gaussian volume (MAPMRI’s first basis function) and the non-Gaussian volume (all other basis functions) of the fitted signal. For this value to be physically meaningful we must use a b-value threshold in the MAPMRI model. This threshold makes the scale estimation in MAPMRI only use samples that realistically describe Gaussian diffusion, i.e., at low b-values. So 6 NIFTIs as output in the DataType for this App. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHPNhNDb0yYVN3PsF-sBOhTJbAzqUnlks5tGQX-gaJpZM4RQG4v> .

[arokem]

FWIW, it seems that there are just a couple more unaddressed issues there, mostly around documentation (See e.g., https://github.com/nipy/dipy/pull/1350/files#r148071818).

[aarya22]

@arokem I notice that there is no Non-Gaussian Diffusion yet in the mapmri. I think we will have to make this function. @francopestilli We can make all of those outputs now except the last one. The Mapmri workflow also needs to input a model type: (Laplacian, Positivity, Both). Should we have this as a user input for the application? Or should we just keep one as default?

[francopestilli]

@aarya22 Sounds good. -1 is acceptable. Model type should be an input the BL App can allow a selection and perhaps we could keep the Laplacian as Default.

[soichih]

For input, this app already takes dwi(& bvecs/bvals) as an input. dipy has a function to generate the gtab easily from bvecs/bvals, so I don't think it needs to be generated outside the app itself?

@aarya22 Please let me know if you need help with adding new parameters via Brainlife GUI - especially the enum parameter which is a bit tricky.

For output, I recommend outputting those nifti-s under "raw" output for now - with an appropriate output datatype tag (maybe like "mrimap"?) When we know exactly which output files should be generated, and have another app that can make use of the output, we can then formalize a new dedicated datatype with specific filename, meaning, etc.

[arokem]

Just to clarify: the workflow takes as input a nifti file with the data and standard ("FSL format") bvec/bval files. @aarya22 and I took a look and non-gaussianity is (of course) implemented in MAPMRI, and the three NG images (NG, NG parllel and NG perpendicular) will all be generated in the workflow as well.

…

On Thu, Jan 4, 2018 at 1:42 PM, Soichi Hayashi ***@***.***> wrote: For input, this app already takes dwi(& bvecs/bvals) as an input. dipy has a function to generate the gtab easily from bvecs/bvals, so I don't think it needs to be generated outside the app itself? @aarya22 <https://github.com/aarya22> Please let me know if you need help with adding new parameters via Brainlife GUI - especially the enum parameter which is a bit tricky. For output, I recommend outputting those nifti-s under "raw" output for now - with an appropriate output datatype tag (maybe like "mrimap"?) When we know exactly which output files should be generated, and have another app that can make use of the output, we can then formalize a new dedicated datatype with specific filename, meaning, etc. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHPNkG_Cy14gzcmz0oKwSnFCjbM6VeSks5tHUXjgaJpZM4RQG4v> .

[francopestilli]

Thanks for working on this!

[aarya22]

@francopestilli @arokem I was told that having pre-set values for small-delta and big-delta for the mapmri application can lead to a faulty model. Rather the user should know the small-delta and big-delta from the acquisition parameters of the data they are using. You can see the discussion here dipy/dipy#1395

I'm not exactly sure what small-delta and big-delta are or how difficult it might be to acquire these values. Do you think that it is a good idea to include small-delta and big-delta as an input to this application?

[arokem]

Yeah - it seems that users should know their small delta, big delta values. These are usually things you can read out from your acquisition settings (e.g., in the dicom header). I think that it's fine to set this as a required input.

…

On Thu, Jan 25, 2018 at 3:33 PM, Aman Arya ***@***.***> wrote: @francopestilli <https://github.com/francopestilli> @arokem <https://github.com/arokem> I was told that having pre-set values for small-delta and big-delta for the mapmri application can lead to a faulty model. Rather the user should know the small-delta and big-delta from the acquisition parameters of the data they are using. You can see the discussion here dipy/dipy#1395 <dipy/dipy#1395> I'm not exactly sure what small-delta and big-delta are or how difficult it might be to acquire these values. Do you think that it is a good idea to include small-delta and big-delta as an input to this application? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHPNqTh2lddPxXLcARVYhZIRlxv1wLaks5tOQ8-gaJpZM4RQG4v> .

[francopestilli]

Hi Ariel Thanks. Do you Know the Dicom fields for these parameters?

…

On Jan 27, 2018 2:25 PM, "Ariel Rokem" ***@***.***> wrote: Yeah - it seems that users should know their small delta, big delta values. These are usually things you can read out from your acquisition settings (e.g., in the dicom header). I think that it's fine to set this as a required input. On Thu, Jan 25, 2018 at 3:33 PM, Aman Arya ***@***.***> wrote: > @francopestilli <https://github.com/francopestilli> @arokem > <https://github.com/arokem> I was told that having pre-set values for > small-delta and big-delta for the mapmri application can lead to a faulty > model. Rather the user should know the small-delta and big-delta from the > acquisition parameters of the data they are using. You can see the > discussion here dipy/dipy#1395 <dipy/dipy#1395> > > I'm not exactly sure what small-delta and big-delta are or how difficult > it might be to acquire these values. Do you think that it is a good idea to > include small-delta and big-delta as an input to this application? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#3 issuecomment-360636280>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ AAHPNqTh2lddPxXLcARVYhZIRlxv1wLaks5tOQ8-gaJpZM4RQG4v> > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACBYc_MfAtGAkE3_6VBmRtlBkJ_B7raWks5tO3gjgaJpZM4RQG4v> .

[arokem]

It varies from dataset to dataset, and depends a bit on the way things are set up on the scanner. I just remembered that in some cases it's not there at all. For example, for the Stanford dataset, Bob and I had to measure this directly from the machine... But it is something you can control when you set up an acquisition. For public datasets (e.g., HCP) this is information that should be available in the relevant references. On Sat, Jan 27, 2018 at 11:29 AM, Franco Pestilli <notifications@github.com> wrote:

…

Hi Ariel Thanks. Do you Know the Dicom fields for these parameters? On Jan 27, 2018 2:25 PM, "Ariel Rokem" ***@***.***> wrote: > Yeah - it seems that users should know their small delta, big delta values. > > These are usually things you can read out from your acquisition settings > (e.g., in the dicom header). > > I think that it's fine to set this as a required input. > > On Thu, Jan 25, 2018 at 3:33 PM, Aman Arya ***@***.***> > wrote: > > > @francopestilli <https://github.com/francopestilli> @arokem > > <https://github.com/arokem> I was told that having pre-set values for > > small-delta and big-delta for the mapmri application can lead to a faulty > > model. Rather the user should know the small-delta and big-delta from the > > acquisition parameters of the data they are using. You can see the > > discussion here dipy/dipy#1395 <dipy/dipy#1395 > > > > > I'm not exactly sure what small-delta and big-delta are or how difficult > > it might be to acquire these values. Do you think that it is a good idea > to > > include small-delta and big-delta as an input to this application? > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > <#3 > issuecomment-360636280>, > > or mute the thread > > <https://github.com/notifications/unsubscribe-auth/ > AAHPNqTh2lddPxXLcARVYhZIRlxv1wLaks5tOQ8-gaJpZM4RQG4v> > > . > > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#3 issuecomment-361008603>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ACBYc_MfAtGAkE3_ 6VBmRtlBkJ_B7raWks5tO3gjgaJpZM4RQG4v> > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHPNuMnU87yLplFtlFr1Yg7MXTs3uLjks5tO3kPgaJpZM4RQG4v> .

[francopestilli]

agree. I ask because it is not in our sequences. Is there a way go guesstimate it? AMICOx does that, it guesstimates the params. Franco

…

On Jan 27, 2018, at 2:33 PM, Ariel Rokem ***@***.***> wrote: It varies from dataset to dataset, and depends a bit on the way things are set up on the scanner. I just remembered that in some cases it's not there at all. For example, for the Stanford dataset, Bob and I had to measure this directly from the machine... But it is something you can control when you set up an acquisition. For public datasets (e.g., HCP) this is information that should be available in the relevant references. On Sat, Jan 27, 2018 at 11:29 AM, Franco Pestilli ***@***.***> wrote: > Hi Ariel > > Thanks. Do you Know the Dicom fields for these parameters? > > > On Jan 27, 2018 2:25 PM, "Ariel Rokem" ***@***.***> wrote: > > > Yeah - it seems that users should know their small delta, big delta > values. > > > > These are usually things you can read out from your acquisition settings > > (e.g., in the dicom header). > > > > I think that it's fine to set this as a required input. > > > > On Thu, Jan 25, 2018 at 3:33 PM, Aman Arya ***@***.***> > > wrote: > > > > > @francopestilli <https://github.com/francopestilli> @arokem > > > <https://github.com/arokem> I was told that having pre-set values for > > > small-delta and big-delta for the mapmri application can lead to a > faulty > > > model. Rather the user should know the small-delta and big-delta from > the > > > acquisition parameters of the data they are using. You can see the > > > discussion here dipy/dipy#1395 <dipy/dipy#1395 > > > > > > > > I'm not exactly sure what small-delta and big-delta are or how > difficult > > > it might be to acquire these values. Do you think that it is a good > idea > > to > > > include small-delta and big-delta as an input to this application? > > > > > > — > > > You are receiving this because you were mentioned. > > > Reply to this email directly, view it on GitHub > > > <#3 > > issuecomment-360636280>, > > > or mute the thread > > > <https://github.com/notifications/unsubscribe-auth/ > > AAHPNqTh2lddPxXLcARVYhZIRlxv1wLaks5tOQ8-gaJpZM4RQG4v> > > > . > > > > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > <#3 > issuecomment-361008603>, > > or mute the thread > > <https://github.com/notifications/unsubscribe-auth/ACBYc_MfAtGAkE3_ > 6VBmRtlBkJ_B7raWks5tO3gjgaJpZM4RQG4v> > > . > > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#3 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAHPNuMnU87yLplFtlFr1Yg7MXTs3uLjks5tO3kPgaJpZM4RQG4v> > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACBYc3vDKet-Q5T6Lur08q3rC8UdQ4p6ks5tO3odgaJpZM4RQG4v>.

[arokem]

Do you have details on *how* it guesstimates? Seems like it would be hard to do just based on knowing the b-values/TE/etc. If we knew that better, we might be able to do that here as well. On Sat, Jan 27, 2018 at 12:28 PM, Franco Pestilli <notifications@github.com> wrote:

…

agree. I ask because it is not in our sequences. Is there a way go guesstimate it? AMICOx does that, it guesstimates the params. Franco > On Jan 27, 2018, at 2:33 PM, Ariel Rokem ***@***.***> wrote: > > It varies from dataset to dataset, and depends a bit on the way things are > set up on the scanner. > > I just remembered that in some cases it's not there at all. For example, > for the Stanford dataset, Bob and I had to measure this directly from the > machine... > > But it is something you can control when you set up an acquisition. For > public datasets (e.g., HCP) this is information that should be available in > the relevant references. > > On Sat, Jan 27, 2018 at 11:29 AM, Franco Pestilli < ***@***.***> > wrote: > > > Hi Ariel > > > > Thanks. Do you Know the Dicom fields for these parameters? > > > > > > On Jan 27, 2018 2:25 PM, "Ariel Rokem" ***@***.***> wrote: > > > > > Yeah - it seems that users should know their small delta, big delta > > values. > > > > > > These are usually things you can read out from your acquisition settings > > > (e.g., in the dicom header). > > > > > > I think that it's fine to set this as a required input. > > > > > > On Thu, Jan 25, 2018 at 3:33 PM, Aman Arya ***@***.*** > > > > wrote: > > > > > > > @francopestilli <https://github.com/francopestilli> @arokem > > > > <https://github.com/arokem> I was told that having pre-set values for > > > > small-delta and big-delta for the mapmri application can lead to a > > faulty > > > > model. Rather the user should know the small-delta and big-delta from > > the > > > > acquisition parameters of the data they are using. You can see the > > > > discussion here dipy/dipy#1395 <https://github.com/nipy/dipy/ pull/1395 > > > > > > > > > > > I'm not exactly sure what small-delta and big-delta are or how > > difficult > > > > it might be to acquire these values. Do you think that it is a good > > idea > > > to > > > > include small-delta and big-delta as an input to this application? > > > > > > > > — > > > > You are receiving this because you were mentioned. > > > > Reply to this email directly, view it on GitHub > > > > <#3 > > > issuecomment-360636280>, > > > > or mute the thread > > > > <https://github.com/notifications/unsubscribe-auth/ > > > AAHPNqTh2lddPxXLcARVYhZIRlxv1wLaks5tOQ8-gaJpZM4RQG4v> > > > > . > > > > > > > > > > — > > > You are receiving this because you were mentioned. > > > Reply to this email directly, view it on GitHub > > > <#3 > > issuecomment-361008603>, > > > or mute the thread > > > <https://github.com/notifications/unsubscribe-auth/ACBYc_MfAtGAkE3_ > > 6VBmRtlBkJ_B7raWks5tO3gjgaJpZM4RQG4v> > > > . > > > > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > <#3 issuecomment-361008864>, > > or mute the thread > > <https://github.com/notifications/unsubscribe-auth/ AAHPNuMnU87yLplFtlFr1Yg7MXTs3uLjks5tO3kPgaJpZM4RQG4v> > > . > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub < #3 issuecomment-361009206>, or mute the thread <https://github.com/ notifications/unsubscribe-auth/ACBYc3vDKet-Q5T6Lur08q3rC8UdQ4p6ks5tO3odga JpZM4RQG4v>. > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHPNoSpfIkVlOQA6Gxc6k2HycWb2ZHkks5tO4cBgaJpZM4RQG4v> .