Add Downloading Results Files

[ptth222]

There are around 1000 analysis files out of 4000 that don't have METABOLITES and DATA sections. They instead have a RESULTS_FILE. It needs to be downloaded separately from the MWTAB file. In the MWTAB specification the file is supposed to be a tab-delimited .txt file with the sample information. There is nothing about metabolite information being in this file in the specification, and the few that I have looked at do not contain any metabolite information, even though those sections are missing from the MWTAB file.

The download URL looks similar to downloading MWTAB files though, so at least that part should not be too difficult:
https://www.metabolomicsworkbench.org/studydownload/ST000071_AN000111_Results.txt

The specification does not specify that the files must be named like "ST000071_AN000111_Results.txt", so relying on that to create the URL is probably not a good idea. I think we are going to have to download the analysis file and parse it to get the results file out of it.

I am wondering if we should modify the MWTAB file and just put the results file info in there where it should be.

[hunter-moseley]

Yes, the plan is to augment the mwtab file with a metabolites block.

…

On Wed, Feb 12, 2025, 3:44 AM ptth222 ***@***.***> wrote: There are around 1000 analysis files out of 4000 that don't have METABOLITES and DATA sections. They instead have a RESULTS_FILE. It needs to be downloaded separately from the MWTAB file. In the MWTAB specification the file is supposed to be a tab-delimited .txt file with the sample information. There is nothing about metabolite information being in this file in the specification, and the few that I have looked at do not contain any metabolite information, even though those sections are missing from the MWTAB file. The download URL looks similar to downloading MWTAB files though, so at least that part should not be too difficult: https://www.metabolomicsworkbench.org/studydownload/ST000071_AN000111_Results.txt The specification does not specify that the files must be named like "ST000071_AN000111_Results.txt", so relying on that to create the URL is probably not a good idea. I think we are going to have to download the analysis file and parse it to get the results file out of it. I am wondering if we should modify the MWTAB file and just put the results file info in there where it should be. — Reply to this email directly, view it on GitHub <#12>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADEP7B7H2MYM2DCNWJKWP7T2PMCVPAVCNFSM6AAAAABW7ATCMCVHI2DSMVQWIX3LMV43ASLTON2WKOZSHA2DONJVGAYTOOI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[rmflight]

From testing a few of these (I grepped for "RESULTS_FILE" in the repaired data, the file only exists at the /studydownload/... if it's named ST.._AN.._RESULTS.txt. Anything else doesn't exist at the URL, which seems odd, and annoying.

[rmflight]

Here is an example of an analysis that the file doesn't resolve:

AN003030, Noesy_binned_data.txt, the file should be at: https://www.metabolomicsworkbench.org/studydownload/Noesy_binned_data.txt

but I get a 404.

[rmflight]

And here is an example of a file that fits the ST.._AN.. pattern but doesn't seem to exist:

https://www.metabolomicsworkbench.org/studydownload/ST002683_AN004356_results.txt

[rmflight]

I get about 85 files I extracted using the regex that don't want to download. Out of 1783. So I can download 1700 of the files.

[ptth222]

Thanks Robert!

The file for AN003030 is at https://www.metabolomicsworkbench.org/studydownload/ST001868_AN003030_Results.txt

The file for AN004356 is at https://www.metabolomicsworkbench.org/studydownload/ST002683_AN004356_Results.txt (Note the capital 'R' in "Results")

I think you don't need to bother trying to pull the file names from the RESULTS_FILE. Just construct the ST.._AN..Results.txt URL.

Here is the webpage for downloading data for AN003030, https://www.metabolomicsworkbench.org/data/DRCCStudySummary.php?Mode=SetupRawDataDownload&StudyID=ST001868. The raw file has "noesy" in the name.

[rmflight]

OK. So to get the full proper file listings, I should be checking for the existence of a results file in the MS or NM section, and then construct the expected file name from the study and analysis keys.

[ptth222]

Yes. Let me know if there are any problems trying that.