How to resolve analysis metadata/names and map them to tables (ANALYSIS METADATA EXTRACTION)

[jdkent]

Problem

we extract study level metadata in one workflow, and parse tables into separate analyses in another workflow. The rub is when we want to map those analyses back to the information we've extracted from the tables. The parsed tables have little metadata (partly because we do not pass a lot of context when parsing the tables).

Ideation:

We want to use the tables to specify analyses, not full text. Partly because we have a difficult time explaining to an LLM what an analysis is and have it reliably use that definition while extracting information in full text.

there is relevant metadata to be applied at the analysis level, like whether it was connectivity/contrast ROI/wholebrain what software was used.

Can we tie confidence estimates to our extractions? If so, can we create a workflow where users review the low confidence extractions/mappings of metadata onto analyses for the analysis extraction.

Solutions not selected

study level extraction -> analysis level extraction
Using the study level extracted entities (like populations/conditions) to inform the analysis level metadata extractions.
We don't like this one since constraining the answers in the analysis level extraction with the study level extraction means any failures in the study extraction will be propagated to the next step.

study/analysis extraction with full text+tables
We don't like this since the additional context in the full text will detract from being able to identify/label analyses in the paper appropriately

Solution selected

study level extraction & analysis level extraction -> alignment/normalization of metadata
Run study level and analysis level extraction independently and then run a normalization step to adjudicate any discrepencies.