`optimizing()` returns a misaligned `par` vector, breaking constrained parameters

[wjakethompson]

In rstan 2.36.0.9000, optimizing() fails for any model whose first parameter is constrained, with an error like lub_free: Bounded variable is ... must be in the interval [0, 1] (or simplex_free: ... is not a valid simplex). sampling() on the same model works fine. The optimizer itself converges normally. The error is thrown afterward, in rstan’s post-processing, because the parameter vector returned by the optimizer is shifted by one position (a spurious leading element), so every parameter is mislabeled and the unconstrain_pars() round-trip rejects the first one.

Reprex

library(rstan)

# Example model with no data; the mode is p = 0.5, a = 0.
code <- "
parameters {
  real<lower=0, upper=1> p;
  real a;
}
model {
  p ~ beta(2, 2);
  a ~ normal(0, 1);
}
"
mod <- stan_model(model_code = code)

# sampling() works ------------------------------------------------------------
fit_mcmc <- sampling(mod, chains = 1, iter = 200, refresh = 0, seed = 1)
round(colMeans(as.matrix(fit_mcmc)), 3)
#>      p      a   lp__ 
#>  0.522 -0.010 -3.676


# optimizing() errors ---------------------------------------------------------
fit_optim <- optimizing(mod, seed = 1)
#> Error:
#> ! Exception: lub_free: Bounded variable is 31, but must be in the interval [0, 1] (in 'anon_model', line 2, column 2 to column 27)

Root cause

The raw vector returned by call_sampler(method = "optim") has a spurious leading element, so it has one element too many and everything is shifted right:

sampler <- new(mod@mk_cppmodule(mod), list(), 0L, rstan:::grab_cxxfun(mod@dso))
o <- sampler$call_sampler(list(init = "0", seed = 1L, method = "optim",
                               algorithm = "LBFGS", iter = 2000L, refresh = 0L))

# 20.0 is spurious
o$par
#> [1] 20.0  0.5  0.0

sampler$param_fnames_oi()
#> [1] "p"    "a"    "lp__"

I'm not sure where the 20.0 is coming from, or what it is intended to represent, but the real optimum is p = 0.5, a = 0 (elements 2 and 3). In rstan::optimizing(), the names are then assigned with

names(optim$par) <- fnames[-length(fnames)] 

which labels the spurious 20.0 as p. The subsequent theta <- sampler$unconstrain_pars(theta) call then tries to unconstrain p = 20, which fails the [0, 1] bound. For a model whose first parameter is a simplex, the same shift produces simplex_free: ... sum(...) = <big>, but should be 1.

sampling() is unaffected because it uses a different return path.

^{Created on 2026-07-17 with reprex v2.1.1}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.6.1 (2026-06-24)
#>  os       macOS Tahoe 26.5.2
#>  system   aarch64, darwin23
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       America/Chicago
#>  date     2026-07-17
#>  pandoc   3.10 @ /opt/homebrew/bin/ (via rmarkdown)
#>  quarto   1.10.7 @ /usr/local/bin/quarto
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version     date (UTC) lib source
#>  callr          3.8.0       2026-06-05 [1] CRAN (R 4.6.0)
#>  cli            3.6.6       2026-04-09 [1] CRAN (R 4.6.0)
#>  codetools      0.2-20      2024-03-31 [2] CRAN (R 4.6.1)
#>  curl           7.1.0       2026-04-22 [1] CRAN (R 4.6.0)
#>  digest         0.6.39      2025-11-19 [1] CRAN (R 4.6.0)
#>  dplyr          1.2.1       2026-04-03 [1] CRAN (R 4.6.0)
#>  evaluate       1.0.5       2025-08-27 [1] CRAN (R 4.6.0)
#>  farver         2.1.2       2024-05-13 [1] CRAN (R 4.6.0)
#>  fastmap        1.2.0       2024-05-15 [1] CRAN (R 4.6.0)
#>  fs             2.1.0       2026-04-18 [1] CRAN (R 4.6.0)
#>  generics       0.1.4       2025-05-09 [1] CRAN (R 4.6.0)
#>  ggplot2        4.0.3       2026-04-22 [1] CRAN (R 4.6.0)
#>  glue           1.8.1       2026-04-17 [1] CRAN (R 4.6.0)
#>  gridExtra      2.3.1       2026-06-25 [1] CRAN (R 4.6.1)
#>  gtable         0.3.6       2024-10-25 [1] CRAN (R 4.6.0)
#>  htmltools      0.5.9       2025-12-04 [1] CRAN (R 4.6.0)
#>  inline         0.3.21      2025-01-09 [1] CRAN (R 4.6.0)
#>  jsonlite       2.0.0       2025-03-27 [1] CRAN (R 4.6.0)
#>  knitr          1.51        2025-12-20 [1] CRAN (R 4.6.0)
#>  lifecycle      1.0.5       2026-01-08 [1] CRAN (R 4.6.0)
#>  loo            2.10.0.9000 2026-06-30 [1] https://stan-dev.r-universe.dev (R 4.6.1)
#>  magrittr       2.0.5       2026-04-04 [1] CRAN (R 4.6.0)
#>  matrixStats    1.5.0       2025-01-07 [1] CRAN (R 4.6.0)
#>  otel           0.2.0       2025-08-29 [1] CRAN (R 4.6.0)
#>  pillar         1.11.1      2025-09-17 [1] CRAN (R 4.6.0)
#>  pkgbuild       1.4.8       2025-05-26 [1] CRAN (R 4.6.0)
#>  pkgconfig      2.0.3       2019-09-22 [1] CRAN (R 4.6.0)
#>  processx       3.9.0       2026-04-22 [1] CRAN (R 4.6.0)
#>  QuickJSR       1.10.0      2026-05-17 [1] CRAN (R 4.6.0)
#>  R6             2.6.1       2025-02-15 [1] CRAN (R 4.6.0)
#>  RColorBrewer   1.1-3       2022-04-03 [1] CRAN (R 4.6.0)
#>  Rcpp           1.1.2       2026-07-05 [1] CRAN (R 4.6.1)
#>  RcppParallel   5.1.11-2    2026-03-05 [1] CRAN (R 4.6.0)
#>  reprex         2.1.1       2024-07-06 [1] CRAN (R 4.6.0)
#>  rlang          1.3.0       2026-07-05 [1] CRAN (R 4.6.1)
#>  rmarkdown      2.31        2026-03-26 [1] CRAN (R 4.6.0)
#>  rstan        * 2.36.0.9000 2026-07-17 [1] https://stan-dev.r-universe.dev (R 4.6.1)
#>  S7             0.2.2       2026-04-22 [1] CRAN (R 4.6.0)
#>  scales         1.4.0       2025-04-24 [1] CRAN (R 4.6.0)
#>  sessioninfo    1.2.4       2026-06-04 [1] CRAN (R 4.6.0)
#>  StanHeaders  * 2.36.0.9000 2026-07-17 [1] https://stan-dev.r-universe.dev (R 4.6.1)
#>  tibble         3.3.1       2026-01-11 [1] CRAN (R 4.6.0)
#>  tidyselect     1.2.1       2024-03-11 [1] CRAN (R 4.6.0)
#>  V8             8.2.0       2026-04-21 [1] CRAN (R 4.6.0)
#>  vctrs          0.7.3       2026-04-11 [1] CRAN (R 4.6.0)
#>  withr          3.0.3       2026-06-19 [1] CRAN (R 4.6.0)
#>  xfun           0.60        2026-07-09 [1] CRAN (R 4.6.1)
#>  yaml           2.3.12      2025-12-10 [1] CRAN (R 4.6.0)
#> 
#>  [1] /Users/jakethompson/Library/R/arm64/4.6/library
#>  [2] /Library/Frameworks/R.framework/Versions/4.6/Resources/library
#>  * ── Packages attached to the search path.
#> 
#> ──────────────────────────────────────────────────────────────────────────────

[bob-carpenter]

Thanks so much for the careful bug report. Maybe 20.0 is the log density at the optimum?

[jgabry]

I think @andrjohns is right in #1192 that this is because of the new converged column that is included in the output from optimization in the latest Stan. RStan isn’t set up to handle this yet but the experimental version of RStan that is being used in this case is probably recent enough that it includes the extra variable in the output.