Final updates to parmest.py, test_parmest.py, and covariance.rst

Author: slilonfe5

doc/OnlineDocs/explanation/analysis/parmest/covariance.rst

@@ -9,30 +9,36 @@ methods which have been implemented in parmest.
 
 1. Reduced Hessian Method
 
-    When the objective function is the sum of squared errors (SSE) between the
-    observed and predicted values of the measured variables, the covariance matrix is:
+    When the objective function is the sum of squared errors (SSE):
+    :math:`\text{SSE} = \sum_{i = 1}^n \left(y_{i} - \hat{y}_{i}\right)^2`,
+    the covariance matrix is:
 
     .. math::
        V_{\boldsymbol{\theta}} = 2 \sigma^2 \left(\frac{\partial^2 \text{SSE}}
         {\partial \boldsymbol{\theta} \partial \boldsymbol{\theta}}\right)^{-1}_{\boldsymbol{\theta}
         = \boldsymbol{\theta}^*}
 
-    When the objective function is the weighted SSE (WSSE), the covariance matrix is:
+    When the objective function is the weighted SSE (WSSE):
+    :math:`\text{WSSE} = \frac{1}{2} \left(\mathbf{y} - f(\mathbf{x};\boldsymbol{\theta})\right)^\text{T}
+    \mathbf{W} \left(\mathbf{y} - f(\mathbf{x};\boldsymbol{\theta})\right)`,
+    the covariance matrix is:
 
     .. math::
        V_{\boldsymbol{\theta}} = \left(\frac{\partial^2 \text{WSSE}}
         {\partial \boldsymbol{\theta} \partial \boldsymbol{\theta}}\right)^{-1}_{\boldsymbol{\theta}
         = \boldsymbol{\theta}^*}
 
     Where :math:`V_{\boldsymbol{\theta}}` is the covariance matrix of the estimated
-    parameters, :math:`\boldsymbol{\theta}` are the unknown parameters,
-    :math:`\boldsymbol{\theta^*}` are the estimates of the unknown parameters, and
-    :math:`\sigma^2` is the variance of the measurement error. When the standard
+    parameters, :math:`y` are the observed measured variables, :math:`\hat{y}` are the
+    predicted measured variables, :math:`n` is the number of data points,
+    :math:`\boldsymbol{\theta}` are the unknown parameters, :math:`\boldsymbol{\theta^*}`
+    are the estimates of the unknown parameters, :math:`\mathbf{x}` are the decision
+    variables, and :math:`\mathbf{W}` is a diagonal matrix containing the inverse of the
+    variance of the measurement error, :math:`\sigma^2`. When the standard
     deviation of the measurement error is not supplied by the user, parmest
     approximates the variance of the measurement error as
-    :math:`\sigma^2 = \frac{1}{n-l} \sum e_i^2` where :math:`n` is the number of data
-    points, :math:`l` is the number of fitted parameters, and :math:`e_i` is the
-    residual for experiment :math:`i`.
+    :math:`\sigma^2 = \frac{1}{n-l} \sum e_i^2` where :math:`l` is the number of
+    fitted parameters, and :math:`e_i` is the residual for experiment :math:`i`.
 
     In parmest, this method computes the inverse of the Hessian by scaling the
     objective function (SSE or WSSE) with a constant probability factor.

doc/OnlineDocs/explanation/analysis/parmest/overview.rst

@@ -41,23 +41,32 @@ that for most experiments, only small parts of :math:`x` will change
 from one experiment to the next.
 
 The following least squares objective can be used to estimate parameter
-values, where data points are indexed by :math:`s=1,\ldots,S`
+values assuming Gaussian independent and identically distributed measurement
+errors, where data points are indexed by :math:`s=1,\ldots,S`
 
 .. math::
 
    \min_{{\theta}} Q({\theta};{\tilde{x}}, {\tilde{y}}) \equiv \sum_{s=1}^{S}q_{s}({\theta};{\tilde{x}}_{s}, {\tilde{y}}_{s}) \;\;
 
-where
+where :math:`q_{s}({\theta};{\tilde{x}}_{s}, {\tilde{y}}_{s})` can be:
 
-.. math::
+1. Sum of squared errors
+
+    .. math::
+
+       q_{s}({\theta};{\tilde{x}}_{s}, {\tilde{y}}_{s}) =
+        \sum_{i=1}^{m}\left({\tilde{y}}_{s,i} - g_{i}({\tilde{x}}_{s};{\theta})\right)^{2}
+
+2. Weighted sum of squared errors
+
+    .. math::
 
-   q_{s}({\theta};{\tilde{x}}_{s}, {\tilde{y}}_{s}) = \sum_{i=1}^{m}w_{i}\left[{\tilde{y}}_{si} - g_{i}({\tilde{x}}_{s};{\theta})\right]^{2}, 
+       q_{s}({\theta};{\tilde{x}}_{s}, {\tilde{y}}_{s}) =
+        \sum_{i=1}^{m}\left(\frac{{\tilde{y}}_{s,i} - g_{i}({\tilde{x}}_{s};{\theta})}{w_i}\right)^{2}
 
 i.e., the contribution of sample :math:`s` to :math:`Q`, where :math:`w
-\in \Re^{m}` is a vector of weights for the responses. For
-multi-dimensional :math:`y`, this is the squared weighted :math:`L_{2}`
-norm and for univariate :math:`y` the weighted squared deviation.
-Custom objectives can also be defined for parameter estimation.
+\in \Re^{m}` is a vector containing the standard deviation of the measurement
+errors of :math:`y`. Custom objectives can also be defined for parameter estimation.
 
 In the applications of interest to us, the function :math:`g(\cdot)` is
 usually defined as an optimization problem with a large number of