Merge remote-tracking branch 'remotes/origin/main'

Author: vondreele

.github/copilot-instructions.md

@@ -0,0 +1,195 @@
+# GSAS-II - Crystallography Software Package
+
+GSAS-II is a comprehensive Python package for analysis of x-ray and neutron diffraction data, including single-crystal, powder, and time-of-flight data. It includes both GUI (wxPython) and scriptable interfaces, with compiled Fortran extensions for performance-critical computations.
+
+Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.
+
+## Working Effectively
+
+### Bootstrap and Build the Repository
+```bash
+# Install core build dependencies
+python -m pip install --upgrade pip setuptools wheel
+python -m pip install meson-python ninja numpy cython scipy pycifrw
+
+# Build the package (takes ~16 seconds, NEVER CANCEL, set timeout to 30+ minutes)
+python -m build -wnx --no-isolation
+
+# OR install in editable mode for development (takes ~12 seconds, NEVER CANCEL, set timeout to 30+ minutes)  
+python -m pip install -e . --no-build-isolation
+```
+
+### Install Optional Dependencies
+```bash
+# For GUI functionality
+python -m pip install wxpython matplotlib pyopengl
+
+# For additional features
+python -m pip install pillow h5py imageio requests gitpython pybaselines
+
+# For testing and development
+python -m pip install pytest nox
+```
+
+### Run Tests
+```bash
+# Add compiled executables to PATH before running tests
+export PATH="$PATH:$(pwd)/build/cp312/sources"
+
+# Alternative: Copy executables to a location in PATH (one-time setup)
+# mkdir -p ~/.local/bin
+# cp build/cp312/sources/{LATTIC,convcell} ~/.local/bin/
+
+# Run all tests (some require network connectivity, ~3-4 seconds for working tests)
+python -m pytest tests/ -v
+
+# Run specific test modules that work offline
+python -m pytest tests/test_lattice.py tests/test_nistlat.py tests/test_elm.py -v
+
+# Using nox for testing workflow
+python -m nox -s tests
+```
+
+### Validation Scenarios
+Always run these validation steps after making changes:
+
+1. **Basic Import Test**:
+   ```bash
+   python -c "import GSASII; print('GSAS-II import successful')"
+   ```
+
+2. **Scriptable Interface Test**:
+   ```bash
+   python -c "
+   import GSASII.GSASIIscriptable as G2sc
+   gpx = G2sc.G2Project(newgpx='/tmp/test.gpx')
+   print('GSASIIscriptable working correctly')
+   "
+   ```
+
+3. **Binary Extensions Test**:
+   ```bash
+   python -c "
+   import GSASII.GSASIIlattice as G2lat
+   import GSASII.GSASIIspc as G2spc
+   print('Compiled extensions working')
+   "
+   ```
+
+4. **Run Core Functionality Tests**:
+   ```bash
+   export PATH="$PATH:$(pwd)/build/cp312/sources"
+   python -m pytest tests/test_lattice.py::test_Brav -v
+   ```
+
+## Build System Details
+
+### Timing Expectations
+- **NEVER CANCEL**: Build takes ~16 seconds on typical hardware but may take up to 30 minutes on slower systems
+- **NEVER CANCEL**: Editable install takes ~12 seconds but may take up to 30 minutes  
+- **NEVER CANCEL**: Test suite takes 3-4 seconds for working tests but up to 10 minutes with all tests
+- Set explicit timeouts of 30+ minutes for build commands and 15+ minutes for test commands
+
+### Build Architecture
+- Uses **meson** build system with **f2py** for Fortran compilation
+- Compiles Fortran extensions: pyspg, pydiffax, pypowder, pytexture, pack_f, histogram2d, unpack_cbf
+- Builds standalone executables: LATTIC, convcell
+- Requires gfortran compiler
+
+### Build Troubleshooting
+- If binaries are missing: run `python -m pip install -e . --no-build-isolation` 
+- For PATH issues with executables: `export PATH="$PATH:$(pwd)/build/cp312/sources"`
+- Build output location: `./build/cp312/sources/` (for Python 3.12)
+
+## Running Applications
+
+### Scriptable Interface (Recommended for Development)
+```bash
+python -c "
+import GSASII.GSASIIscriptable as G2sc
+gpx = G2sc.G2Project(newgpx='project.gpx')
+# Add your GSAS-II scripting code here
+"
+```
+
+### GUI Application
+```bash
+# Note: GUI requires display/X11 - will not work in headless environments
+python -m GSASII
+```
+
+## Development Workflow
+
+### Linting and Code Quality
+```bash
+# Using nox (recommended)
+python -m nox -s lint      # Pre-commit hooks and formatting
+python -m nox -s pylint    # Static code analysis
+
+# Manual linting
+python -m pip install pre-commit
+pre-commit run --all-files
+```
+
+### Documentation
+```bash
+# Build documentation  
+python -m nox -s docs
+
+# Build and serve docs locally
+python -m nox -s docs -- --serve
+```
+
+### Common Development Tasks
+Always validate changes by running:
+1. Build the package: `python -m build -wnx --no-isolation` 
+2. Install in editable mode: `python -m pip install -e . --no-build-isolation`
+3. Add executables to PATH: `export PATH="$PATH:$(pwd)/build/cp312/sources"`
+4. Run working tests: `python -m pytest tests/test_lattice.py tests/test_elm.py -v`
+5. Test core functionality: Run the validation scenarios above
+
+## Troubleshooting
+
+### Common Issues
+- **"binary load error: pyspg not found"**: Run editable install to compile binaries
+- **"FileNotFoundError: 'LATTIC'"**: Add `./build/cp312/sources` to PATH  
+- **Network connectivity test failures**: Normal in restricted environments, focus on offline tests
+- **wxPython GUI issues**: GUI requires display, use scriptable interface in headless environments
+
+### Dependencies Not Found
+Install the specific missing dependency:
+```bash
+# For seekpath k-vector functionality
+python -m pip install seekpath
+
+# For complete GUI experience  
+python -m pip install wxpython matplotlib pyopengl pillow h5py
+```
+
+## Key Projects in Codebase
+
+### Main Modules
+- **GSASII/**: Main package directory
+  - **GSASIIscriptable.py**: Primary API for programmatic use
+  - **GSASIIGUI.py**: Main GUI application entry point
+  - **GSASIIdata.py**: Core data structures and file I/O
+  - **GSASIImath.py**: Mathematical routines for crystallography
+  - **GSASIIlattice.py**: Lattice parameter and space group operations
+  - **GSASIIspc.py**: Space group and symmetry operations
+
+### Build Files  
+- **meson.build**: Main build configuration
+- **sources/meson.build**: Fortran extension build configuration
+- **pyproject.toml**: Package metadata and dependencies
+
+### Testing
+- **tests/**: Self-contained test suite
+  - **test_lattice.py**: Lattice and crystallography tests (always work offline)
+  - **test_nistlat.py**: NIST lattice utility tests (require executables in PATH)
+  - **test_scriptref.py**: Scripting interface integration tests
+
+### Documentation  
+- **docs/**: Sphinx documentation source
+- **README.md**: Project overview and links
+
+Always check binary compilation status and add executables to PATH before running tests or using advanced functionality.

GSASII/GSASIIdataGUI.py

@@ -7056,7 +7056,7 @@ def _makemenu():     # routine to create menu when first used
         # IMG / Masks
         G2G.Define_wxId('wxID_MASKCOPY', 'wxID_MASKSAVE', 'wxID_MASKLOAD', 'wxID_NEWMASKSPOT', 'wxID_NEWMASKARC', 'wxID_NEWMASKRING',
             'wxID_NEWMASKFRAME', 'wxID_NEWMASKPOLY','wxID_NEWMASKXLINE','wxID_NEWMASKYLINE','wxID_MASKLOADNOT',
-            'wxID_FINDSPOTS', 'wxID_AUTOFINDSPOTS', 'wxID_DELETESPOTS','wxID_MASKCOPYSELECTED')
+            'wxID_FINDSPOTS', 'wxID_AUTOFINDSPOTS', 'wxID_DELETESPOTS','wxID_MASKCOPYSELECTED', 'wxID_LOADSPOTS')
         def _makemenu():     # routine to create menu when first used
             self.MaskMenu = wx.MenuBar()
             self.PrefillDataMenu(self.MaskMenu)
@@ -7069,7 +7069,8 @@ def _makemenu():     # routine to create menu when first used
             self.MaskEdit.Append(G2G.wxID_MASKSAVE,'Save mask','Save mask to file')
             self.MaskEdit.Append(G2G.wxID_MASKLOADNOT,'Load mask','Load mask from file; ignoring threshold')
             self.MaskEdit.Append(G2G.wxID_MASKLOAD,'Load mask w/threshold','Load mask from file keeping the threshold value')
-            self.MaskEdit.Append(G2G.wxID_FINDSPOTS,'Pixel mask search','Search for pixels to mask; NB: slow')
+            self.MaskEdit.Append(G2G.wxID_FINDSPOTS,'Pixel mask search','Search for pixels to mask')
+            self.MaskEdit.Append(G2G.wxID_LOADSPOTS,'Load pixel mask','Use non-zero pixels in an image to mask')
             self.MaskEdit.Append(G2G.wxID_AUTOFINDSPOTS,'Multi-IMG pixel mask search','Search multiple images for pixels to mask; NB: slow')
             self.MaskEdit.Append(G2G.wxID_DELETESPOTS,'Delete spot masks','Delete all spot masks')
             submenu.Append(G2G.wxID_NEWMASKARC,'Arc mask','Create an arc mask with mouse input')

GSASII/GSASIIimgGUI.py

@@ -2000,6 +2000,55 @@ def OnLoadMask(event):
         finally:
             dlg.Destroy()
 
+    def OnLoadPixelMask(event):
+        Names = G2gd.GetGPXtreeDataNames(G2frame,['IMG ',])
+        img = G2frame.GPXtree.GetItemText(G2frame.Image)
+        if img not in Names: # remove current entry from list of names
+            print(f'Strange: {img} not in IMG entries')
+        else:
+            del Names[Names.index(img)]
+        # if nothing left, provide error & quit
+        if not Names:
+            G2G.G2MessageBox(G2frame,
+                             'Before you can use this command you must import the mask you want to use as an image',
+                             'No mask image')
+            return
+        elif len(Names) == 1:
+            sel = Names[0]
+        else:
+            dlg = G2G.G2SingleChoiceDialog(G2frame,
+                    'Select an image as a pixel mask:',
+                    'Load Pixel Mask',Names)
+            dlg.CenterOnParent()
+            if dlg.ShowModal() == wx.ID_OK:
+                sel = dlg.GetSelection()
+                dlg.Destroy()
+            else:
+                dlg.Destroy()
+                return
+        # Got our mask identified
+        maskID = G2gd.GetGPXtreeItemId(G2frame,G2frame.root,sel)
+        maskdata = G2frame.GPXtree.GetItemPyData(
+            G2gd.GetGPXtreeItemId(G2frame,maskID,'Image Controls'))
+        formatName = maskdata.get('formatName','')
+        Npix,maskfile,imagetag = G2IO.GetCheckImageFile(G2frame,maskID)
+        # check it matches
+        imgdim = G2frame.GPXtree.GetItemPyData(G2frame.Image)[0]
+        if Npix != imgdim:
+            G2G.G2MessageBox(G2frame,
+                             f'Mask dimensions ({Npix}) != Image ({imgdim}). Images do not match',
+                             'Mask image wrong dimensions')
+            return
+        maskImage = np.array(G2fil.GetImageData(G2frame,maskfile,True,ImageTag=imagetag,FormatName=formatName),dtype=np.float32)
+        data['SpotMask']['MaskLoaded'] = sel # provides a record that this
+                                             # was loaded rather than a search
+        data['SpotMask']['spotMask'] = maskImage > 0
+        nmasked = sum(data['SpotMask']['spotMask'].flatten())
+        frac = nmasked/data['SpotMask']['spotMask'].size
+        G2G.G2MessageBox(G2frame,
+                             f'Mask removes {nmasked} pixels ({frac:.3f}%)',
+                             'Mask loaded')
+
     def OnFindPixelMask(event):
         '''Do auto search for pixels to mask
         Called from (Masks) Operations->"Pixel mask search"
@@ -2343,6 +2392,7 @@ def OnAzimuthPlot(event):
     G2frame.Bind(wx.EVT_MENU, OnLoadMask, id=G2G.wxID_MASKLOADNOT)
     G2frame.Bind(wx.EVT_MENU, OnSaveMask, id=G2G.wxID_MASKSAVE)
     G2frame.Bind(wx.EVT_MENU, OnFindPixelMask, id=G2G.wxID_FINDSPOTS)
+    G2frame.Bind(wx.EVT_MENU, OnLoadPixelMask, id=G2G.wxID_LOADSPOTS)
     G2frame.Bind(wx.EVT_MENU, OnAutoFindPixelMask, id=G2G.wxID_AUTOFINDSPOTS)
     G2frame.Bind(wx.EVT_MENU, OnDeleteSpotMask, id=G2G.wxID_DELETESPOTS)
     G2frame.Bind(wx.EVT_MENU, ToggleSpotMaskMode, id=G2G.wxID_NEWMASKSPOT)

GSASII/GSASIIpwdplot.py

@@ -1386,7 +1386,9 @@ def onPlotFormat(event):
 
     def refPlotUpdate(Histograms,cycle=None,restore=False):
         '''called to update an existing plot during a Rietveld fit; it only 
-        updates the curves, not the reflection marks or the legend
+        updates the curves, not the reflection marks or the legend. 
+        It should be called with restore=True to reset plotting 
+        parameters after the refinement is done.
         '''
         if restore:
             (G2frame.SinglePlot,G2frame.Contour,G2frame.Weight,
@@ -1396,7 +1398,7 @@ def refPlotUpdate(Histograms,cycle=None,restore=False):
         if plottingItem not in Histograms:
             histoList = [i for i in Histograms.keys() if i.startswith('PWDR ')]
             if len(histoList) == 0:
-                print('Skipping plot, no PWDR item found!')
+                print('Skipping plot update, no PWDR items!')
                 return
             plotItem = histoList[0]
         else:
@@ -1418,6 +1420,9 @@ def refPlotUpdate(Histograms,cycle=None,restore=False):
             Ifin = np.searchsorted(X,limits[1][1])
         if Ibeg == Ifin: # if no points are within limits bad things happen 
             Ibeg,Ifin = 0,None
+        elif Ibeg > Ifin:
+            Ifin, Ibeg = Ibeg, Ifin # TOF with order reversed
+
         if Page.plotStyle['sqrtPlot']:
             olderr = np.seterr(invalid='ignore') #get around sqrt(-ve) error
             Y = np.where(xye[1]>=0.,np.sqrt(xye[1]),-np.sqrt(-xye[1]))
@@ -1454,6 +1459,7 @@ def refPlotUpdate(Histograms,cycle=None,restore=False):
             else:
                 Plot.set_title(Title)
         Page.canvas.draw()
+        wx.GetApp().Yield()
 
     def incCptn(string):
         '''Adds a underscore to "hide" a MPL object from the legend if