Merge branch 'main' into db/instevalfits

Author: palday

.github/workflows/current.yml

@@ -19,12 +19,13 @@ jobs:
   ci:
     runs-on: ${{ matrix.os }}
     strategy:
+      fail-fast: false
       matrix:
         julia-version: [1]
         # julia-arch: [x64]
         os:
-         - ubuntu-22.04
-         - macOS-14 # apple silicon!
+         - ubuntu-24.04
+         - macOS-15 # apple silicon!
     steps:
       - uses: actions/checkout@v5
       - uses: julia-actions/setup-julia@v2

NEWS.md

@@ -2,12 +2,14 @@ MixedModels v5.0.0 Release Notes
 ==============================
 - Optimization is now performed _without constraints_. In a post-fitting step, the Cholesky factor is canonicalized to have non-negative diagonal elements. [#840]
 - The default optimizer has changed to NLopt's implementation of NEWUOA where possible. NLopt's implementation fails on 1-dimensional problems, so in the case of a single, scalar random effect, BOBYQA is used instead. In the future, the default optimizer backend will likely change to PRIMA and NLopt support will be moved to an extension. Blocking this change in backend is an issue with PRIMA.jl when running in VSCode's built-in REPL on Linux. [#840]
-- [BREAKING] Support for constrained optimization has been completely removed, i.e. the field `lowerbd` has been removed from `OptSummary`.
+- [BREAKING] Support for constrained optimization has been completely removed, i.e. the field `lowerbd` has been removed from `OptSummary`. [#849]
 - [BREAKING] A fitlog is always kept -- the deprecated keyword argument `thin` has been removed as has the `fitlog` keyword argument. [#850]
 - The fitlog is now stored as Tables.jl-compatible column table. [#850]
 - Internal code around the default optimizer has been restructured. In particular, the NLopt backend has been moved to a submodule, which will make it easier to move it to an extension if we promote another backend to the default. [#853]
 - Internal code around optimization in profiling has been restructuring so that fitting done during calls to `profile` respect the `backend` and `optimizer` settings. [#853]
 - The `prfit!` convenience function has been removed. [#853]
+- The `dataset` and `datasets` functions have been removed. They are now housed in `MixedModelsDatasets`.[#854]
+- One argument `predict(::GeneralizedLinearMixedModel)`, i.e. prediction on the original data, now supports the `type` keyword argument. [#856]
 
 MixedModels v4.38.0 Release Notes
 ==============================

benchmark/benchmarks.jl

@@ -1,5 +1,5 @@
 using BenchmarkTools, MixedModels
-using MixedModels: dataset
+using MixedModelsDatasets: dataset
 
 const SUITE = BenchmarkGroup()
 

docs/Project.toml

@@ -9,6 +9,7 @@ ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"
 FreqTables = "da1fdf0e-e0ff-5433-a45f-9bb5ff651cb1"
 Gadfly = "c91e804a-d5a3-530f-b6f0-dfbca275c004"
 MixedModels = "ff71e718-51f3-5ec2-a782-8ffcbfa3c316"
+MixedModelsDatasets = "7e9fb7ac-9f67-43bf-b2c8-96ba0796cbb6"
 ProgressMeter = "92933f4c-e287-5a05-a399-4b506db050ca"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
 StatsAPI = "82ae8749-77ed-4fe6-ae5f-f523153014b0"

docs/src/GaussHermite.md

@@ -73,7 +73,7 @@ The definition of `MixedModels.GHnorm` is similar to the `gausshermitenorm` func
 GHnorm
 ```
 ```@example Main
-using MixedModels
+using MixedModels, MixedModelsDatasets
 GHnorm(3)
 ```
 
@@ -100,7 +100,7 @@ Several covariates were recorded including the woman's age (centered at the mean
 The version of the data used here is that used in review of multilevel modeling software conducted by the Center for Multilevel Modelling, currently at University of Bristol (http://www.bristol.ac.uk/cmm/learning/mmsoftware/data-rev.html).
 These data are available as the `:contra` dataset.
 ```@example Main
-contra = DataFrame(MixedModels.dataset(:contra))
+contra = DataFrame(MixedModelsDatasets.dataset(:contra))
 describe(contra)
 ```
 

docs/src/bootstrap.md

@@ -27,12 +27,12 @@ package for [`R`](https://www.r-project.org) is fit by
 ```@example Main
 using DataFrames
 using Gadfly          # plotting package
-using MixedModels
+using MixedModels, MixedModelsDatasets
 using Random
 ```
 
 ```@example Main
-dyestuff = MixedModels.dataset(:dyestuff)
+dyestuff = MixedModelsDatasets.dataset(:dyestuff)
 m1 = fit(MixedModel, @formula(yield ~ 1 + (1 | batch)), dyestuff)
 ```
 
@@ -88,7 +88,7 @@ However, it is not as straightforward to detect singularity in vector-valued ran
 
 For example, if we bootstrap a model fit to the `sleepstudy` data
 ```@example Main
-sleepstudy = MixedModels.dataset(:sleepstudy)
+sleepstudy = MixedModelsDatasets.dataset(:sleepstudy)
 contrasts = Dict(:subj => Grouping())
 m2 = let f = @formula reaction ~ 1+days+(1+days|subj)
     fit(MixedModel, f, sleepstudy; contrasts)

docs/src/constructors.md

@@ -14,8 +14,8 @@ using DisplayAs
 ```
 
 ```@example Main
-using DataFrames, MixedModels, StatsModels
-dyestuff = MixedModels.dataset(:dyestuff)
+using DataFrames, MixedModels, MixedModelsDatasets, StatsModels
+dyestuff = MixedModelsDatasets.dataset(:dyestuff)
 ```
 
 ```@example Main
@@ -71,7 +71,7 @@ Notice that both are equivalent.
 
 ```@example Main
 using BenchmarkTools
-dyestuff2 = MixedModels.dataset(:dyestuff2)
+dyestuff2 = MixedModelsDatasets.dataset(:dyestuff2)
 @benchmark fit(MixedModel, $fm, $dyestuff2)
 ```
 
@@ -110,7 +110,7 @@ It corresponds to a shift in the intercept for each level of the grouping factor
 The *sleepstudy* data are observations of reaction time, `reaction`, on several subjects, `subj`, after 0 to 9 days of sleep deprivation, `days`.
 A model with random intercepts and random slopes for each subject, allowing for within-subject correlation of the slope and intercept, is fit as
 ```@example Main
-sleepstudy = MixedModels.dataset(:sleepstudy)
+sleepstudy = MixedModelsDatasets.dataset(:sleepstudy)
 fm2 = fit(MixedModel, @formula(reaction ~ 1 + days + (1 + days|subj)), sleepstudy)
 DisplayAs.Text(ans) # hide
 ```
@@ -120,14 +120,14 @@ DisplayAs.Text(ans) # hide
 A model for the *Penicillin* data incorporates random effects for the plate, and for the sample.
 As every sample is used on every plate these two factors are *crossed*.
 ```@example Main
-penicillin = MixedModels.dataset(:penicillin)
+penicillin = MixedModelsDatasets.dataset(:penicillin)
 fm3 = fit(MixedModel, @formula(diameter ~ 1 + (1|plate) + (1|sample)), penicillin)
 DisplayAs.Text(ans) # hide
 ```
 
 In contrast, the `cask` grouping factor is *nested* within the `batch` grouping factor in the *Pastes* data.
 ```@example Main
-pastes = DataFrame(MixedModels.dataset(:pastes))
+pastes = DataFrame(MixedModelsDatasets.dataset(:pastes))
 describe(pastes)
 ```
 This can be expressed using the solidus (the "`/`" character) to separate grouping factors, read "`cask` nested within `batch`":
@@ -147,7 +147,7 @@ In observational studies it is common to encounter *partially crossed* grouping
 For example, the *InstEval* data are course evaluations by students, `s`, of instructors, `d`.
 Additional covariates include the academic department, `dept`, in which the course was given and `service`, whether or not it was a service course.
 ```@example Main
-insteval = MixedModels.dataset(:insteval)
+insteval = MixedModelsDatasets.dataset(:insteval)
 fm5 = fit(MixedModel, @formula(y ~ 1 + service * dept + (1|s) + (1|d)), insteval)
 DisplayAs.Text(ans) # hide
 ```
@@ -222,7 +222,7 @@ To create a GLMM representation, the distribution family for the response, and p
 You can either use `fit(MixedModel, ...)` or `glmm(...)` to fit the model. For instance:
 
 ```@example Main
-verbagg = MixedModels.dataset(:verbagg)
+verbagg = MixedModelsDatasets.dataset(:verbagg)
 verbaggform = @formula(r2 ~ 1 + anger + gender + btype + situ + mode + (1|subj) + (1|item));
 gm1 = fit(MixedModel, verbaggform, verbagg, Bernoulli())
 DisplayAs.Text(ans) # hide
@@ -258,7 +258,7 @@ The optional argument `nAGQ=k` causes evaluation of the deviance function to use
 adaptive Gauss-Hermite quadrature rule.
 This method only applies to models with a single, simple, scalar random-effects term, such as
 ```@example Main
-contraception = MixedModels.dataset(:contra)
+contraception = MixedModelsDatasets.dataset(:contra)
 contraform = @formula(use ~ 1 + age + abs2(age) + livch + urban + (1|dist));
 bernoulli = Bernoulli()
 deviances = Dict{Symbol,Float64}()
@@ -511,7 +511,7 @@ sum(leverage(fm2))
 
 When a model converges to a singular covariance, such as
 ```@example Main
-fm3 = fit(MixedModel, @formula(yield ~ 1+(1|batch)), MixedModels.dataset(:dyestuff2))
+fm3 = fit(MixedModel, @formula(yield ~ 1+(1|batch)), MixedModelsDatasets.dataset(:dyestuff2))
 DisplayAs.Text(ans) # hide
 ```
 the effective degrees of freedom is the lower bound.
@@ -522,7 +522,7 @@ sum(leverage(fm3))
 Models for which the estimates of the variances of the random effects are large relative to the residual variance have effective degrees of freedom close to the upper bound.
 ```@example Main
 fm4 = fit(MixedModel, @formula(diameter ~ 1+(1|plate)+(1|sample)),
-    MixedModels.dataset(:penicillin))
+    MixedModelsDatasets.dataset(:penicillin))
 DisplayAs.Text(ans) # hide
 ```
 ```@example Main
@@ -532,7 +532,7 @@ sum(leverage(fm4))
 Also, a model fit by the REML criterion generally has larger estimates of the variance components and hence a larger effective degrees of freedom.
 ```@example Main
 fm4r = fit(MixedModel, @formula(diameter ~ 1+(1|plate)+(1|sample)),
-    MixedModels.dataset(:penicillin), REML=true)
+    MixedModelsDatasets.dataset(:penicillin), REML=true)
 DisplayAs.Text(ans) # hide
 ```
 ```@example Main

docs/src/derivatives.md

@@ -15,9 +15,9 @@ ForwardDiff.hessian(::LinearMixedModel{T}, ::Vector{T}) where {T}
 ### Exact zero at optimum for trivial models
 
 ```@example Derivatives
-using MixedModels, ForwardDiff
+using MixedModels, MixedModelsDatasets, ForwardDiff
 using DisplayAs # hide
-fm1 = lmm(@formula(yield ~ 1 + (1|batch)), MixedModels.dataset(:dyestuff2))
+fm1 = lmm(@formula(yield ~ 1 + (1|batch)), MixedModelsDatasets.dataset(:dyestuff2))
 DisplayAs.Text(ans) # hide
 ```
 
@@ -32,7 +32,7 @@ ForwardDiff.hessian(fm1)
 ### Approximate zero at optimum for non trivial models
 
 ```@example Derivatives
-fm2 = lmm(@formula(reaction ~ 1 + days + (1+days|subj)), MixedModels.dataset(:sleepstudy))
+fm2 = lmm(@formula(reaction ~ 1 + days + (1+days|subj)), MixedModelsDatasets.dataset(:sleepstudy))
 DisplayAs.Text(ans) # hide
 ```
 

docs/src/index.md

@@ -16,8 +16,8 @@ You can fit a model using a `lmer`-style model formula using `@formula` and a da
 Here is a short example of how to fit a linear mixed-effects modeling using the `dyestuff` dataset:
 
 ```@example Main
-using DataFrames, MixedModels           # load packages
-dyestuff = MixedModels.dataset(:dyestuff);              # load dataset
+using DataFrames, MixedModels, MixedModelsDatasets  # load packages
+dyestuff = MixedModelsDatasets.dataset(:dyestuff);   # load dataset
 
 lmod = lmm(@formula(yield ~ 1 + (1|batch)), dyestuff)   # fit the model!
 DisplayAs.Text(ans) # hide
@@ -28,7 +28,7 @@ A quick example of generalized linear model using the `verbagg` dataset:
 
 ```@example Main
 using DataFrames, MixedModels               # load packages
-verbagg = MixedModels.dataset(:verbagg);    # load dataset
+verbagg = MixedModelsDatasets.dataset(:verbagg);    # load dataset
 
 frm = @formula(r2 ~ 1 + anger + gender + btype + situ + mode + (1|subj) + (1|item));
 bernmod = glmm(frm, verbagg, Bernoulli())   # fit the model!

docs/src/mime.md

@@ -12,7 +12,7 @@ Packages like `IJulia` and `Documenter` can often detect the presence of these d
 
 
 ```@example Main
-using MixedModels
+using MixedModels, MixedModelsDatasets
 form = @formula(rt_trunc ~ 1 + spkr * prec * load +
                           (1 + load | item) +
                           (1 + spkr + prec + load | subj))
@@ -21,7 +21,7 @@ contr = Dict(:spkr => EffectsCoding(),
              :load => EffectsCoding(),
              :item => Grouping(),
              :subj => Grouping())
-kbm = fit(MixedModel, form, MixedModels.dataset(:kb07); contrasts=contr)
+kbm = fit(MixedModel, form, MixedModelsDatasets.dataset(:kb07); contrasts=contr)
 ```
 
 Note that the display here is more succinct than the standard REPL display:
@@ -52,8 +52,8 @@ kbm.optsum
 ```
 
 ```@example Main
-m0 = fit(MixedModel, @formula(reaction ~ 1 + (1|subj)), MixedModels.dataset(:sleepstudy))
-m1 = fit(MixedModel, @formula(reaction ~ 1 + days + (1+days|subj)), MixedModels.dataset(:sleepstudy))
+m0 = fit(MixedModel, @formula(reaction ~ 1 + (1|subj)), MixedModelsDatasets.dataset(:sleepstudy))
+m1 = fit(MixedModel, @formula(reaction ~ 1 + days + (1+days|subj)), MixedModelsDatasets.dataset(:sleepstudy))
 MixedModels.likelihoodratiotest(m0,m1)
 ```