Parent merge leaves nested split artifacts in catalog metadata

[Jay2006sawant]

Describe the bug

When merging a parent element, trestle merge successfully writes merged content back into catalog.json, but nested split files/directories are not fully cleaned up.

I verified this locally after full dev setup. After splitting catalog.metadata and then splitting nested metadata.roles.*, running trestle merge -e 'catalog.metadata' leaves stale files under catalog/metadata/ (including deeper roles children), even though catalog.json already contains merged metadata.

This creates an inconsistent workspace state and can confuse follow-up split/merge operations.

To Reproduce

Steps to reproduce the behavior:

1. Create a fresh workspace and import a model:
   • trestle init
   • trestle import -f tests/data/json/minimal_catalog_with_groups.json -o mycatalog
2. Go to catalogs/mycatalog and split top-level catalog parts:
   • trestle split -f ./catalog.json -e 'catalog.metadata,catalog.groups,catalog.back-matter'
3. Go to catalogs/mycatalog/catalog and split metadata:
   • trestle split -f ./metadata.json -e 'metadata.roles,metadata.responsible-parties'
4. Go to catalogs/mycatalog/catalog/metadata and split roles deeper:
   • trestle split -f ./roles.json -e 'roles.*'
5. Return to catalogs/mycatalog and merge metadata:
   • trestle merge -e 'catalog.metadata'
6. Inspect remaining files with ls -R catalog and see stale artifacts still present under catalog/metadata/....

Expected behavior

After trestle merge -e 'catalog.metadata', obsolete nested split artifacts for that subtree should be cleaned up (or at least consistently handled), so the workspace does not retain stale catalog/metadata/... split content after parent merge.

Screenshots / Logs.

Repro output after merge (ls -R catalog):

catalog:
back-matter.json
groups.json
metadata

catalog/metadata:
responsible-parties.json
roles

catalog/metadata/roles:
00000__role.json

Also confirmed catalog.json already contains merged metadata.

Environment

• OS: Linux 6.8.0-101-generic
• Python version: 3.10.12 (hatch test environment)
• Installed packages: local editable install (pip install -e ".[dev]"), make develop, make test passed (1255 passed, 3 skipped)

Are you going to solve this issue?

Yes

[Jay2006sawant]

PTAL at this too @degenaro @vikas-agarwal76

[Acuspeedster]

#2143 fixes this issue

[Jay2006sawant]

Hi @Acuspeedster, thanks for taking the initiative and raising PR #2143.

However, since this is an open-source project, issues usually need to go through review and discussion with the maintainers first. I had opened this issue to get maintainer feedback before implementing a fix, and I’m already planning to work on resolving it.

[Jay2006sawant]

Could you please close this PR for now and wait for maintainer review on the issue. Once the maintainers confirm the approach, I’ll proceed with the implementation.

Thanks for understanding and for contributing to the project 🙂

[Acuspeedster]

Hi @Jay2006sawant ,

I appreciate the kind tone, but I respectfully disagree with the premise here.

Open-source does not mean "one person claims an issue and nobody else can touch it." There is no assignment, no lock, and no rule that says a PR must wait for maintainer pre-approval before being submitted that is the review process. The PR is the maintainer discussion.

A few points worth noting:

The fix was already done by the time you commented. This is +60/-3 lines a small, surgical change that removes stale artifacts and cleans up a # FIXME that has been sitting in the codebase for years. There was no ambiguity about the correct approach.

I didn't just open this one. I've also submitted fixes for issue #2135 (docs/OSCAL version consistency), a misc bugs PR (type check bug, unused imports, exit() → sys.exit(), timezone-naive datetime), and the JCS canonicalization feature (#2013). Contributing multiple fixes is how open source works.

"Wait for maintainer review on the issue before implementing" is not a standard OSS workflow. Maintainers review code, not issue plans. If my implementation has a technical flaw, please point it out and I'll fix it. That's what the review process is for.

If you'd like to contribute something here, you're welcome to review the PR and leave technical feedback. I'm happy to collaborate. But asking me to discard working, tested, linted code so you can start from scratch is not a reasonable ask.

[Acuspeedster]

Hi @Jay2006sawant ,

Since this was a clear bug rather than a new feature, I felt it would be more helpful to demonstrate a concrete implementation via a PR rather than only discuss the approach in the issue. That way maintainers can review the actual fix and suggest any changes directly on the code.

Given that #2143 already implements the fix, it may be simplest to review and iterate there if any adjustments are needed.

If you'd like to contribute in the meantime, there are also several other open issues in the repository that could benefit from work. Otherwise, feel free to review the PR and leave any technical feedback you may have.

Maintainers can then decide the preferred path forward.

[Jay2006sawant]

Hi @Acuspeedster, thanks for contributing the fix.

My intention wasn’t to block contributions I raised the point mainly to ensure maintainers are aligned on the approach before multiple parallel fixes happen.

Since you already have a working implementation in #2143, it makes sense for maintainers to review it there.

[Jay2006sawant]

Sounds good ! let’s collaborate on it .I’ll go through the implementation and add any suggestions if needed.

Looking forward to working together on this 🙂