Skip to content

🐕 Refactor ordering of attempted fetch methods#69

Merged
fwkoch merged 1 commit intomainfrom
feat/fetch-order-refactor
Jun 10, 2025
Merged

🐕 Refactor ordering of attempted fetch methods#69
fwkoch merged 1 commit intomainfrom
feat/fetch-order-refactor

Conversation

@fwkoch
Copy link
Member

@fwkoch fwkoch commented Jun 10, 2025

After #67 - there was still the case where a DOI that resolved to (1) a URL that had a custom JATS resolver and (2) a PMCID with degraded JATS would prioritize (2). This was because in jatsFetch we were doing the DOI -> PMC conversion before even attempting any resolving, so even though the DOI resolution was in the correct order, we only ever tried to resolve it as an already-converted PMC.

I think that's a pretty bad explanation... but hopefully the code changes make some sense. I'll leave comments.

session.log.debug(`Resolved input ${input} to PMC ID: ${pmcid}`);
input = pmcid;
// We still may be able to use original DOI input, so keep both DOI and PMC
altInput = pmcid;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep track of original input too...

result = await downloadJatsFromUrl(session, input);
}
if (!result?.data && altInput) {
result = await downloadJatsFromUrl(session, altInput);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... so we can resolve against both original doi and new pmc.

} catch (error) {
session.log.debug((error as Error).message);
}
if (doi.validate(urlOrId)) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function I split up to make the doi vs pmc vs other paths clearer.

This appears to reduce some parallelism, but I don't think it actually makes much of a difference, since the original parallel functions were often failing fast based on "is doi" vs "is pmc" checks.

@fwkoch
Copy link
Member Author

fwkoch commented Jun 10, 2025

I'm still not totally happy with the order things happen in jatsFetch - if you follow through everything carefully, there are edge cases where the doi -> pmc conversion may happen a couple times in different places.

But I think this is an improvement, so I'm going to release it, and we can iterate to resolve cases where certain DOIs (for example) are unnecessarily slow or something...

@fwkoch fwkoch merged commit f0bc51a into main Jun 10, 2025
5 checks passed
@fwkoch fwkoch deleted the feat/fetch-order-refactor branch June 10, 2025 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant