Skip to content

Global or local whitening/normalization for refine_template#103

Open
jdickerson95 wants to merge 40 commits intodevelopment_v1.2from
mdg_global_whitening
Open

Global or local whitening/normalization for refine_template#103
jdickerson95 wants to merge 40 commits intodevelopment_v1.2from
mdg_global_whitening

Conversation

@jdickerson95
Copy link
Contributor

Refine template now either fully uses full micrograph for filters and normalization (making it same as match results), or uses the particle for filters and normalization.
The everything is loaded in utils.py and passed into particle stack, which will make it easy to refine from a movie.
Closes #99

@jdickerson95 jdickerson95 changed the title Mdg global whitening Global or local whitening/normalization for refine_template Nov 7, 2025
@jdickerson95 jdickerson95 requested a review from mgiammar November 8, 2025 16:12
@mgiammar
Copy link
Member

mgiammar commented Jan 3, 2026

The loading and pre-processing logic is getting a bit messy within the codebase given all the options that a user could turn on/off now. Separate to this PR I'm going to think about utility organization within Leopard-EM to see if there's ways to make the code easier to reason about.

@jdickerson95
Copy link
Contributor Author

I've refactored a little bit from the differentiable refine PR onwards to have the utils a bit more explicit.
I've found it a lot nicer when using it, but not necessarily the best solution.

Copy link
Member

@mgiammar mgiammar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdickerson95 I've gone through the smaller files in the PR and have made some change requests. Still need to go over the larger changes, namely core_differentiable_refine.py, particle_stack.py, refine_template_manager.py, and backend_setup.py.

Something I'm noticing with this PR is we're staring to leverage the result CSV file heavily for particle parsing when there are many elements that are hard to fit into a tabular data format (e.g. Zernike dict, aniso mag params). I'm wondering if there's utility in working with JSON data more frequently for describing data results. Open question and not in-scope for this PR.

Copy link
Member

@mgiammar mgiammar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdickerson95 Besides the outlying comments, PR looks good to me. Once those are merged and have verification that the code is working for the optimization workflows happy to merge

@jdickerson95
Copy link
Contributor Author

I think I've handled all the changes.
I agree the list/dict in the results csv is getting a bit much, and a json format could be more readable.

I'll test that nothing broke the optimization workflows before merging with dev 1.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants