[R] Quarto (knitr) output doesn't match script output

[nader-hotait]

Hey everybody,

I am using the quarto framwork to publish my code for a journal publication. I'm running a topic model via stm and displaying some model diagnostics. Unfortunetly, the output created within my script does not match what is rendered in the html file through quarto. Let me show you what I mean:

Script output:

Markdown output:

Here are the codes:
YAML (exluding personal information):

format:
  html:
    code-fold: true
    toc: true
    toc-location: left

Packages:

pacman::p_load(tidyverse, readxl, ggrepel, lubridate, ggpubr, quanteda,
               quanteda.textplots, quanteda.textstats, quanteda.textmodels,
               grid, gridExtra, Cairo, ldatuning, stm, tidytext, seededlda,
               caret, glmnet, tibble, kableExtra, caretEnsemble, methods)

Preprocessing:

dfm_post <- corpus(df) %>%
  tokens(remove_punct=T,
         remove_numbers = T,
         remove_url = T, 
         split_hyphens = T,
         remove_symbols = T) %>%
  tokens_remove(stopwords("english", source = "marimo")) %>%
  tokens_remove(stopwords("de", source = "marimo")) %>%
  tokens_remove(stopwords("ar", source = "stopwords-iso")) %>%
  tokens_remove(c("à", "un", "la", "le", "en", "et", "été", "les", "avec")) %>%
  tokens_remove(pattern = "^[\\p{script=Arab}]+$", valuetype = "regex") %>%
  dfm() %>%
  dfm_remove(pattern = c("*.tt", "*.uk", "*.com", "rt", "#*", "@*", ".de")) %>%
  dfm_trim(max_termfreq = .99,termfreq_type = "quantile",verbose = T) %>%
  dfm_trim(min_termfreq = .7,termfreq_type = "quantile",verbose = T) %>%
  dfm_wordstem()

Execution:

set.seed(1234)
dfm2stm <- convert(dfm_post, to = "stm")
searchk <- searchK(dfm2stm$documents, dfm2stm$vocab, K = seq(6,18,2))
plot(searchk)

I am grateful for every hint or information. Things I have tried so far:

• Including cache and freeze options
• Checking the data type throughout the code (stays consistent)
• Set seeds differently, less and more frequently
• Clearing cache before executing code
• Update packages and RStudio version

[mcanouil]

Could you share a reproducible example exhibiting this behaviour which seems to be tied up with the random number generator.
Does your script with the seed set properly generates the exact same output? If this is not the case, then there is absolutely no way Quarto can.

Also, a workaround could be to externalise the computation (as pre-render script if you really want it to be linked to the document), then load the results in Quarto for rendering only.

[mcanouil]

Did you try knitr/rmarkdown? (Not a solution, but the issue might not be in Quarto itself)

[nader-hotait]

Yes I did. Knitr Markdown and Quarto produce the same results. Pure script and script within quarto/markdown another one consistently. So you right, probably is not an inherent Quarto problem, damn.

[mcanouil]

Some seed issue with knitr then. 🤔

[nader-hotait]

I have a reproducible example now but you are right anyways - must relate to the kntir engine, cache, and random seed. I will experiment further and let you know

[mcanouil]

I did a quick search, this issue was reported several times over the last decade.
There are some "solutions" but not sure if those apply here and if they actually work.