Merge pull request #138 from paulmueller/develop

Develop

Author: Paul Müller

ChangeLog.txt

@@ -1,3 +1,24 @@
+0.9.3
+- Fitting: migrate to lmfit
+  - This introduces a new dependency for building PyCorrFit.
+    (e.g. in Debian, the package "python-lmfit" is required)
+  - Improved fitting behavior at parameter boundaries
+  - Removed "Polak-Ribiere" fitting algorithm
+  - Added "Sequential Linear Squares Programming" algorithm
+- Heuristic fit (#109):
+  - Detect parameters that are stuck during fitting
+  - Fit each curve five times or less and check
+    whether the fit converges.
+  - If two diffusion time parameter exist in a model, always
+    make sure that one parameter is the larger one. This
+    feature can currently not be disabled (#110).
+- Allow infinity ("inf" and "-inf") parameters for models
+  and boundaries.
+- New model: confocal T+T+3D+3D
+- Bugfixes:
+  - Sessions saved with 64bit Windows were not opened (#136)
+  - Old sessions and "KeyError: 'chi2'"
+  - Old session file extension was not recognized (#106)
 0.9.2
 - Bugfixes:
   - "Slider Simulation"/"Parm Range" broken (#133)

appveyor.yml

@@ -65,17 +65,15 @@ install:
   # CONDA installs
   # Pinned versions are defined in freeze_appveyor\pinned
   - xcopy freeze_appveyor\pinned %PYTHON%\conda-meta\ /Y
-  - "conda install --yes numpy"
-  - "conda install --yes pip"
-  - "conda install --yes scipy"
-  - "conda install --yes wxpython"
+  - "conda install --yes --quiet numpy pip scipy wxpython"
   # PIP installs
   # Install the build dependencies of the project. 
   - "pip install cython"
   - "pip install wheel"
   # Install package-specific libraries  
   - "pip install matplotlib"
   - "pip install sympy"
+  - "pip install lmfit"
   # Install pyinstaller
   - 'ECHO Downloading %PYWIN_DL%'
   - ps: (new-object net.webclient).DownloadFile("$env:PYWIN_DL", 'C:/pywin_inst.exe')
@@ -103,11 +101,11 @@ after_test:
   - ps: "ls dist"
   # Run pyinstaller
   # This will create the "win7_innosetup.iss" file
-  - "%WITH_COMPILER% pyinstaller -y freeze_pyinstaller\\PyCorrFit_win7.spec"
+  - "%WITH_COMPILER% pyinstaller -y --log-level=WARN freeze_pyinstaller\\PyCorrFit_win7.spec"
   # Create InnoSetup installers
   # Set InnoSetup path here, because Cython complained about it.
   - set PATH=%PATH%;"C:\\Program Files (x86)\\Inno Setup 5"
-  - iscc win7_innosetup.iss
+  - iscc /VERYSILENT win7_innosetup.iss
 
 artifacts:
   # Archive the generated wheel package in the ci.appveyor.com build report.

doc/PyCorrFit_doc_content.tex

@@ -57,17 +57,7 @@ \subsubsection{Software}
 \item\textbf{PyPI.} To run \textit{PyCorrFit} on any other operating system, the installation of Python v.2.7 is required. \textit{PyCorrFit} is included in the Python package index (PyPI, \url{http://pypi.python.org/pypi/pip}) and can be installed via\footnote{See also the wiki article at \url{https://github.com/paulmueller/PyCorrFit/wiki/Installation_pip}}
 \texttt{pip~install~pycorrfit$\!$[GUI]}.
 \item \textbf{Sources.}
-You can also directly download the source code at any developmental stage\footnote{See also the wiki article at \url{https://github.com/paulmueller/PyCorrFit/wiki/Running-from-source}}. \textit{PyCorrFit} depends on the following python modules:
-\texttt{
-\begin{itemize}
-\item matplotlib ($\geq$ 1.0.1)
-\item NumPy ($\geq$ 1.5.1)
-\item PyYAML
-\item SciPy ($\geq$ 0.8.0)
-\item sympy ($\geq$ 0.7.2)
-\item wxPython3
-\end{itemize}
-}
+You can also directly download the source code at any developmental stage\footnote{See also the wiki article at \url{https://github.com/paulmueller/PyCorrFit/wiki/Running-from-source}}.
 \end{itemize}
 
 
@@ -80,7 +70,7 @@ \subsection{Workflow}
 
 The following chapter introduces the general idea of how to start and accomplish a fitting project. FCS experiments produce different sets of experimental correlation functions which must be interpreted with appropriate physical models (\hyref{Chapter}{sec:theor}). Each correlation function refers to a single contiguous signal trace or ``run''. In \textit{PyCorrFit}, the user must assign a mathematical model function to each correlation function during the loading procedure. The assignment is irreversible in the sense that within an existing \textit{PyCorrFit} session it cannot be changed. This feature assures the stability of the batch processing routine for automated fitting of large data sets. Nevertheless, the fit of different models to the same data can be explored by loading the data twice or simply by creating two different sessions.
 
-Let's briefly discuss a typical example: To determine the diffusion coefficient of a fluorescently labeled protein in free solution, one has to deal with two sets of autocorrelation data: measurements of a diffusion standard (e.g. free dye for which a diffusion coefficient has been published) to calibrate the detection volume and measurements of the protein sample. The protein sample may contain small amounts of slowly diffusing aggregates. While the calibration measurements can be fitted with a one-component diffusion model (T-3D), the protein sample displays two mobility states, monomers and aggregates, which are taken into account by a two-component diffusion model (T-3D-3D). With \textit{PyCorrFit} such a situation can be treated in three ways, having different pros and cons: 
+Let's briefly discuss a typical example: To determine the diffusion coefficient of a fluorescently labeled protein in free solution, one has to deal with two sets of autocorrelation data: measurements of a diffusion standard (e.g. free dye for which a diffusion coefficient has been published) to calibrate the detection volume and measurements of the protein sample. The protein sample may contain small amounts of slowly diffusing aggregates. While the calibration measurements can be fitted with a one-component diffusion model (T+3D), the protein sample displays two mobility states, monomers and aggregates, which are taken into account by a two-component diffusion model (T+3D+3D). With \textit{PyCorrFit} such a situation can be treated in three ways, having different pros and cons: 
 
 
 \begin{enumerate}
@@ -306,9 +296,9 @@ \subsection{Models}
 \rule{0pt}{3ex} - Confocal (Gaussian): T+3D+3D & Triplet with two diffusive components \\
 %Confocal (Gaussian): T+3D+3D+3D & [Triplet with three diffusive components]
 %Confocal (Gaussian): 2D &  2D diffusion, e.g. in membranes \\
-\rule{0pt}{3ex} - Confocal (Gaussian): T-2D &  Triplet blinking and 2D diffusion \\
-\rule{0pt}{3ex} - Confocal (Gaussian): T-2D-2D & Triplet with two diffusive components \\
-\rule{0pt}{3ex} - Confocal (Gaussian): T-3D-2D &  Triplet with mixed 3D and 2D diffusion \\
+\rule{0pt}{3ex} - Confocal (Gaussian): T+2D &  Triplet blinking and 2D diffusion \\
+\rule{0pt}{3ex} - Confocal (Gaussian): T+2D+2D & Triplet with two diffusive components \\
+\rule{0pt}{3ex} - Confocal (Gaussian): T+3D+2D &  Triplet with mixed 3D and 2D diffusion \\
 \rule{0pt}{3ex}
 \end{tabular}
 
@@ -734,9 +724,8 @@ \subsubsection{Algorithms}
 \item The \textbf{BFGS} method uses the quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno (BFGS) \cite{Nocedal2006} (pp. 136). It uses the first derivatives only. BFGS has proven good performance even for non-smooth optimizations.
 \item The \textbf{Levenberg-Marquardt} algorithm \cite{Levenberg1944} uses the first derivatives and combines the Gauss–Newton algorithm with a trust region approach. It is very robust compared to other algorithms and it is very popular in curve-fitting. \textit{PyCorrFit} uses this algorithm by default. If this algorithm is used, \textit{PyCorrFit} can estimate an error of the fit parameters using the covariance matrix.
 \item The \textbf{Nelder-Mead} method uses the Simplex algorithm \cite{Nelder1965,Wright1996}. This algorithm has been successful in many applications but other algorithms using the first and/or second derivatives information might be preferred for their better performances and robustness in general.
-\item The method \textbf{Polak-Ribiere} uses a nonlinear conjugate gradient algorithm by Polak and Ribiere, a variant of the Fletcher-Reeves method described in \cite{Nocedal2006} pp.
-120-122. Only the first derivatives are used.
 \item The method \textbf{Powell} is a modification of Powell's method \cite{Powell1964, Press} which is a conjugate direction method. It performs sequential one-dimensional minimizations along each vector of the directions set, which is updated at each iteration of the main minimization loop. The function need not be differentiable, and no derivatives are taken.
+\item \textbf{Sequential Linear Squares Programming}  inherently accepts boundaries and thus might behave better than other algorithms for problems with  bounded parameters.
 \end{itemize}
 
 \input{PyCorrFit_doc_models}

doc/PyCorrFit_doc_models.tex

@@ -167,6 +167,18 @@ \subsection{Confocal FCS}
 \end{center}
 \vspace{2em}
 
+
+% 3D+3D diffusion + T+T
+\noindent \begin{tabular}{lp{.7\textwidth}}
+Name & \textbf{Confocal (Gaussian) T+T+3D+3D} \\ 
+ID & \textbf{6043} \\ 
+Descr. &  Two-component three-dimensional free diffusion with a Gaussian laser profile, including two triplet components. 
+The correlation function is a superposition of three-dimensional model functions of the type \textbf{T+3D} (6011).
+\end{tabular}
+\vspace{2em}
+
+
+
 \subsection{TIR-FCS}
 \label{sec:imple.tirfc}
 The model functions make use of the Faddeeva function (complex error function)\footnote{In user-defined model functions (\hyref{Section}{sec:hacke.extmod}), the Faddeeva function is accessible through \texttt{wofz()}. For convenience, the function \texttt{wixi()} can be used which only takes $\xi$ as an argument and the imaginary $i$ can be omitted.}:

freeze_travis/requirements.txt

@@ -1,4 +1,5 @@
 Cython==0.22
+lmfit
 matplotlib==1.4.3
 numpy==1.9.2
 PyYAML==3.11

pycorrfit/doc.py

@@ -26,6 +26,7 @@ def __init__(self):
 with warnings.catch_warnings():
     warnings.simplefilter("ignore")
     matplotlib.use('WXAgg') # Tells matplotlib to use WxWidgets for dialogs
+import lmfit
 import numpy
 import os
 import platform
@@ -148,6 +149,7 @@ def SoftwareUsed():
     text = "Python "+sys.version+\
            "\n\nModules:"+\
            "\n - cython "+\
+           "\n - lmfit "+lmfit.__version__+\
            "\n - matplotlib "+matplotlib.__version__+\
            "\n - NumPy "+numpy.__version__+\
            "\n - PyYAML "+yaml.__version__ +\