Skip to content

Conversation

@ndg63276
Copy link
Collaborator

@ndg63276 ndg63276 commented Sep 5, 2025

JIRA ticket: LIMS-1836

Summary:

Improve upstream and downstream reprocessing, particularly for non-MX beamlines

Changes:

  • Move available reprocessing pipelines into the config
  • Automatically select all images for upstream reprocessing
  • Add placeholder values for d_min and d_max to avoid confusion between the two
  • Show/hide the unit cell fields correctly if 'Process Individually' is ticked/unticked
  • Only allow 'real space grid search' indexing if the unit cell is defined
  • Only allow certain downstream pipelines if no sample id is defined for a data collection (and therefore no protein id)
  • Fix bug where Downstream Reprocessing dialog would fail for 'sm' beamlines
  • Fix bug where unavailable filesystem would crash Shelxt results

To test:

  • Add the following to the config:
$upstream_reprocessing_pipelines = array(
    'sm' => array(
        array('NAME' => 'Xia2 DIALS', 'VALUE' => 'xia2-dials'),
    ),
    'default' => array(
        array('NAME' => 'Xia2 DIALS', 'VALUE' => 'xia2-dials'),
        array('NAME' => 'Xia2 3dii', 'VALUE' => 'xia2-3dii'),
        array('NAME' => 'Fast DP', 'VALUE' => 'fast_dp'),
        array('NAME' => 'autoPROC', 'VALUE' => 'autoPROC'),
    ),
);

$downstream_reprocessing_pipelines = array(
    'sm' => array(
        array('NAME' => 'shelxt', 'VALUE' => 'trigger-shelxt'),
    ),
    'default' => array(
        array('NAME' => 'Dimple', 'VALUE' => 'trigger-dimple'),
        array('NAME' => 'Fast EP', 'VALUE' => 'trigger-fastep'),
        array('NAME' => 'Big EP', 'VALUE' => 'trigger-bigep'),
        array('NAME' => 'MrBUMP', 'VALUE' => 'trigger-mrbump'),
    ),
);
  • Use the production database as reprocessing only works on prod
  • Go to an mx data collection, eg /dc/visit/cm40607-3/id/19521838, click the cog button for (upstream) reprocessing, check all images are selected by default (1 to 3600)
  • Check 4 pipelines are displayed.
  • Check the High Res and Low Res boxes have suggested values
  • Click to show "Space Group / Cell", then tick "Process Individually", check the Space Group / Cell boxes are hidden again
  • While "Process Individually" is ticked, click the Space Group / Cell button next to the graph. Untick "Process Individually", check the Space Group / Cell boxes are hidden again
  • Click "Xia2 Options", try to change the Indexing Method to "Real Space Grid Search". Check it gives an error asking you to fill in the unit cell. Fill in the unit cell and then check you can now select "Real Space Grid Search". Repeat the check with "Process Individually" ticked.
  • Test upstream reprocessing runs successfully, suggest using Fast DP as it is quicker.
  • Go to an i19-1 data collection (eg /dc/visit/cm40638-3/id/19579252), click the cog button, check only one pipeline is displayed.
  • Open a successful MX processing job, click the button for "Downstream Reprocessing", check 4 pipelines are displayed
  • Open a successful I19-1 processing job, click the button for "Downstream Reprocessing", check only 1 pipeline is displayed
  • Open a successful MX processing job for a data collection without a sample id (eg /dc/visit/mx23694-115/id/14212014), click the button for "Downstream Reprocessing", check 4 pipelines are displayed but each is disabled as protein information is missing

@gfrn gfrn added Ready to Release Configuration Change Pull requests which necessitate changes to SynchWeb configuration labels Oct 13, 2025
@ndg63276 ndg63276 changed the base branch from master to pre-release/2025-R4.4 October 14, 2025 09:08
@ndg63276 ndg63276 merged commit e021992 into pre-release/2025-R4.4 Oct 14, 2025
2 checks passed
@ndg63276 ndg63276 deleted the improvement/LIMS-1836/improvements-to-reprocessing branch October 14, 2025 09:27
ndg63276 added a commit that referenced this pull request Oct 22, 2025
* LIMS-1924: Fix update of container owner (#988)

* LIMS-1923: Fix downstream accordion if files missing from disk (#987)

* LIMS-1876: Fix URL parameters on fault report list (#979)

* LIMS-1841: Link downstream jobs to their upstream counterpart (#985)

* LIMS-1836: Improvements to reprocessing (#978)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Configuration Change Pull requests which necessitate changes to SynchWeb configuration Ready to Release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants