Stable hashing for `pins`: file-byte hashes cause false versioning with `qs`; `siphash13`/`qs2` fix it

[randrescastaneda]

First, thank you for the pins package; In mty team, at the World Bank pins had become central to our team’s pipelines. Because we rely on it heavily, the behavior below is creating a lot of unnecessary version churn and downstream instability for us.

Summary
pins appears to decide “has this changed?” by hashing the file bytes (via digest::digest()), not the object. With qs, two saves of the same object can produce slightly different bytes, so the file hash changes and pins creates unnecessary new versions. Using secretbase::siphash13 yields stable file hashes; using qs2 (deterministic settings) also helps. I think an opt-in stable hashing path could be a good solution.

Why it happens

• digest::digest(file) is sensitive to byte-level variations in serialized output.
• qs may change bytes across saves despite identical content.
• siphash13(file=…) is stable in these cases; qs2 also enables deterministic serialization.

Reprex (simulates pins’ file-hash check)

# Same object, saved twice with qs
x <- list(a=1:3, b=list(y="z"))
f1 <- tempfile(fileext=".qs") 
f2 <- tempfile(fileext=".qs")
qs::qsave(x, f1, check_hash=TRUE) 
qs::qsave(x, f2, check_hash=TRUE)

# File-level digests differ → would look like "changed" to pins
digest::digest(f1, algo="xxhash64")
#> [1] "54db9bb61f3033fa"
digest::digest(f2, algo="xxhash64")
#> [1] "65c7307f0cebca90"

# Stable alternative: siphash13 (file-level stable for identical content)
secretbase::siphash13(file=f1)
#> [1] "06df1cf71318cbac"
secretbase::siphash13(file=f2)
#> [1] "06df1cf71318cbac"

# Objects are identical
waldo::compare(qs::qread(f1), qs::qread(f2))
#> ✔ No differences
# ✔ No differences

^{Created on 2025-10-27 with reprex v2.1.1}

(I also tested qs2::qs_save(..., shuffle = FALSE): object-level hashes match deterministically; file-byte digest can still differ, while siphash13 remains stable.)

Proposal (any of the following)

• A. Allow custom hash function (e.g., options(pins.hash_fn = ...) or a board arg hash_fn=) so users can choose secretbase::siphash13 or an object-level hasher.
• B. Built-in stable option: expose hash = c("digest", "siphash13", "object"); keep current default for compatibility.
• C. Serializer profile for qs2 with deterministic settings (e.g., shuffle = FALSE), optionally combined with B.
• D. Object-level hashing (deserialize then hash) as an opt-in for maximal reproducibility.

I think this suggestion will avoid spurious versions, storage churn, and non-reproducible pipelines when using pins as a content-addressed cache. Thank you so much for your support.

[juliasilge]

Thank you so much for this feature request and detailed explanation! 🙌

Just so we're totally on the same page, this only applies to pins stored via the R package qs?

[randrescastaneda]

Hi @juliasilge ,

I tested for a few other formats and I got the same results as with qs. Basically, the problem is in digest package. All the hashes remain the same with secretbase.

Thank you so much!
BEst,

R> # Test object serialization stability across different formats
+ x <- list(a=1:3, b=list(y="z"))
+ 
+ # For fst, we need a data frame
+ df <- data.frame(
+     col_a = 1:5,
+     col_b = letters[1:5],
+     col_c = runif(5)
+ )
+ 
+ # Create temporary files for each format
+ f_qs1 <- tempfile(fileext=".qs") 
+ f_qs2 <- tempfile(fileext=".qs")
+ f_fst1 <- tempfile(fileext=".fst")
+ f_fst2 <- tempfile(fileext=".fst")
+ f_csv1 <- tempfile(fileext=".csv")
+ f_csv2 <- tempfile(fileext=".csv")
+ f_rds1 <- tempfile(fileext=".rds")
+ f_rds2 <- tempfile(fileext=".rds")
+ 
+ # Save same objects twice with each format
+ # QS format
+ qs::qsave(x, f_qs1, check_hash=TRUE) 
+ qs::qsave(x, f_qs2, check_hash=TRUE)
+ 
+ # FST format (requires data frame)
+ fst::write_fst(df, f_fst1)
+ fst::write_fst(df, f_fst2)
+ 
+ # CSV format
+ write.csv(df, f_csv1, row.names=FALSE)
+ write.csv(df, f_csv2, row.names=FALSE)
+ 
+ # RDS format
+ saveRDS(x, f_rds1)
+ saveRDS(x, f_rds2)
+ 
+ # Compare file-level digests for each format
+ cat("=== File-level digests (xxhash64) ===\n")
+ cat("QS format:\n")
+ cat("  File 1:", digest::digest(f_qs1, algo="xxhash64"), "\n")
+ cat("  File 2:", digest::digest(f_qs2, algo="xxhash64"), "\n")
+ 
+ cat("FST format:\n")
+ cat("  File 1:", digest::digest(f_fst1, algo="xxhash64"), "\n")
+ cat("  File 2:", digest::digest(f_fst2, algo="xxhash64"), "\n")
+ 
+ cat("CSV format:\n")
+ cat("  File 1:", digest::digest(f_csv1, algo="xxhash64"), "\n")
+ cat("  File 2:", digest::digest(f_csv2, algo="xxhash64"), "\n")
+ 
+ cat("RDS format:\n")
+ cat("  File 1:", digest::digest(f_rds1, algo="xxhash64"), "\n")
+ cat("  File 2:", digest::digest(f_rds2, algo="xxhash64"), "\n")
+ 
+ cat("\n=== Stable alternative: siphash13 ===\n")
+ cat("QS format:\n")
+ cat("  File 1:", secretbase::siphash13(file=f_qs1), "\n")
+ cat("  File 2:", secretbase::siphash13(file=f_qs2), "\n")
+ 
+ cat("FST format:\n")
+ cat("  File 1:", secretbase::siphash13(file=f_fst1), "\n")
+ cat("  File 2:", secretbase::siphash13(file=f_fst2), "\n")
+ 
+ cat("CSV format:\n")
+ cat("  File 1:", secretbase::siphash13(file=f_csv1), "\n")
+ cat("  File 2:", secretbase::siphash13(file=f_csv2), "\n")
+ 
+ cat("RDS format:\n")
+ cat("  File 1:", secretbase::siphash13(file=f_rds1), "\n")
+ cat("  File 2:", secretbase::siphash13(file=f_rds2), "\n")
+ 
+ cat("\n=== Object comparison ===\n")
+ cat("QS objects identical:", identical(qs::qread(f_qs1), qs::qread(f_qs2)), "\n")
+ cat("FST objects identical:", identical(fst::read_fst(f_fst1), fst::read_fst(f_fst2)), "\n")
+ cat("CSV objects identical:", identical(read.csv(f_csv1), read.csv(f_csv2)), "\n")
+ cat("RDS objects identical:", identical(readRDS(f_rds1), readRDS(f_rds2)), "\n")
+ 
+ # Clean up
+ unlink(c(f_qs1, f_qs2, f_fst1, f_fst2, f_csv1, f_csv2, f_rds1, f_rds2))
fstcore package v0.10.0
(OpenMP detected, using 14 threads)
=== File-level digests (xxhash64) ===
QS format:
  File 1: a54b9a6ae7fec54b 
  File 2: aa2cd18fc83b7003 
FST format:
  File 1: 62e5d618da0cedcb 
  File 2: 3bba69690478a9a8 
CSV format:
  File 1: 7078b4875b9ea44f 
  File 2: 89576d0dff2dc9b0 
RDS format:
  File 1: d4a267c31341bdd4 
  File 2: 94e9e6f4399b976a 

=== Stable alternative: siphash13 ===
QS format:
  File 1: 06df1cf71318cbac 
  File 2: 06df1cf71318cbac 
FST format:
  File 1: fb9e70fed6614025 
  File 2: fb9e70fed6614025 
CSV format:
  File 1: 2cc7928e8151f819 
  File 2: 2cc7928e8151f819 
RDS format:
  File 1: 71392f8dc0427928 
  File 2: 71392f8dc0427928 

=== Object comparison ===
QS objects identical: TRUE 
FST objects identical: TRUE 
CSV objects identical: TRUE 
RDS objects identical: TRUE 

[juliasilge]

Can you explain to me how this plays out in pins specifically? For example, I see:

library(pins)

b <- board_temp()
df <- data.frame(
  col_a = 1:5,
  col_b = letters[1:5],
  col_c = runif(5)
)

b  |> pin_write(df, "really-great-data-csv", type = "csv")
#> Creating new version '20251031T204801Z-6f9d5'
#> Writing to pin 'really-great-data-csv'
b  |> pin_write(df, "really-great-data-csv", type = "csv")
#> ! The hash of pin "really-great-data-csv" has not changed.
#> • Your pin will not be stored.

b  |> pin_write(df, "really-great-data-rds", type = "rds")
#> Creating new version '20251031T204801Z-f9903'
#> Writing to pin 'really-great-data-rds'
b  |> pin_write(df, "really-great-data-rds", type = "rds")
#> ! The hash of pin "really-great-data-rds" has not changed.
#> • Your pin will not be stored.

b  |> pin_write(df, "really-great-data-qs", type = "qs")
#> Creating new version '20251031T204801Z-114b0'
#> Writing to pin 'really-great-data-qs'
b  |> pin_write(df, "really-great-data-qs", type = "qs")
#> ! The hash of pin "really-great-data-qs" has not changed.
#> • Your pin will not be stored.

^{Created on 2025-10-31 with reprex v2.1.1}

I see the same if I restart R between different attempts at doing pin_write().

[juliasilge]

I think you need to use digest::digest() in the same way we do here in pins:

f_rds1 <- tempfile(fileext=".rds")
f_rds2 <- tempfile(fileext=".rds")

x <- sample(1:100, 10)
saveRDS(x, f_rds1)
saveRDS(x, f_rds2)

digest::digest(file = f_rds1, algo="xxhash64")
#> [1] "70e4f8ab83c1300a"
digest::digest(file = f_rds2, algo="xxhash64")
#> [1] "70e4f8ab83c1300a"

^{Created on 2025-10-31 with reprex v2.1.1}

I see the same thing for .qs and all other file types.

[randrescastaneda]

Hi @juliasilge, thank you for your reply.

The challenge we're facing with pins is difficult to fully replicate, as it appears to depend on the data structure and storage format. We're running a large process that saves and loads approximately 2,000 files, all are nested lists saved in .qs format. We've observed that for some files each run generated new versions using pins despite the data remaining unchanged.

Unfortunately, we cannot reproduce this issue with toy data, and our actual data is confidential. We suspect the problem originates from either the serialization mechanism in qs of nested lists or how the digest package computes file hashes.

I am sorry for not being able to provide a minimal reproducible example. If you have any suggestions on how we might further investigate this issue or alternative approaches to ensure data integrity with pins, we would greatly appreciate your insights.

[juliasilge]

Thank you for those details! If you are able to come up with an example that demonstrates the problem, we would be very happy to take a look. 👀

[dana89co]

Hi @juliasilge,

Thank you so much for considering our concerns regarding the pins package. Unfortunately, as @randrescastaneda mentioned earlier, we were unable to create a reproducible example to illustrate the issue in detail. I spent several days trying to replicate the problem without success. My current hypothesis is that it may be related to how the check_pin_version function uses last() with pin_versions :

pins-r/R/pin-meta.R

Line 179 in 0d6d7f0

last(pin_versions(board, name)$version) %||% abort("No versions found")

It seems this function assumes the last item corresponds to the most recent file, which is not always the case. I’m not sure how the version order is determined, which is why I couldn’t reproduce the issue with a simple example.

In the meantime, we’ve included a screenshot from our console that demonstrates the problem for a specific file. As shown, we have multiple versions of the same file even though we use force_identical_write = FALSE in our pipeline when calling pin_write. We believe this happens because when pin_meta is called without specifying a version, it does not return the latest version but rather the last item in the list, which is not necessarily the most recent. This is evident in the comparison between meta_1 and meta_wv in the screenshot: the latter should return the latest version but instead retrieves the first version ever created. This behavior seems to be the reason we end up with multiple versions of the same file.

I wish we could provide a more concrete example or even a solution, as we were eager to integrate pins into our pipeline. However, we need to move forward with other improvements and adopt a different versioning system for now. We truly appreciate your attention to our feedback and apologize for not being able to offer more actionable details. Thank you again for all your work on this and other fantastic packages, which make our jobs so much easier :)

[juliasilge]

Another thing to be aware of here is that the qs package is being deprecated, i.e. archived on CRAN, fairly soon (see the note at the top of the README):
https://github.com/qsbase/qs

I'll need to remove the pins support for qs fairly soon. Do you all find that qs2 works as you expect?

[dana89co]

Dear @juliasilge ,

Apologies for the delayed response. We’ve been working hard to resolve the hashing issue, which we also encountered when using only secretbase. We can now share a reproducible example illustrating the problem.

The issue appears to stem from the use of data.table and the ordering of its attributes. After certain manipulations, the data.table content remained the same, but the attribute order changed, resulting in a different hash and a new version. Additionally, this behavior seems to affect the pin_meta function, which retrieves the first version’s hash and causes a new version to be created each time we save.

# Load libraries
library(pins)
library(data.table)

# Set pin name
pin_name <- "BFA"

# Create temp board
board <- board_temp()

# Create data.table and show attr
dt_BFA <- data.table(BFA = 1:3, CDA = 4:6, RFG = 7:9)

attributes(dt_BFA)
#> $names
#> [1] "BFA" "CDA" "RFG"
#> 
#> $row.names
#> [1] 1 2 3
#> 
#> $class
#> [1] "data.table" "data.frame"
#> 
#> $.internal.selfref
#> <pointer: 0x000001d80ad36050>

# Save version pin
board |> pin_write(dt_BFA, pin_name, versioned = TRUE,
                   type = "qs", force_identical_write = FALSE)
#> Creating new version '20251121T182104Z-daee4'
#> Writing to pin 'BFA'

# Show meta and pin hash
meta_1 <- board |> pin_meta(name = pin_name)
meta_1$pin_hash
#> [1] "daee4073fa46bc67"

# Create copy of dt_BFA and manipulate data.table which makes reorder attr
dt_BFA_2 <- dt_BFA |>
  rbind(dt_BFA)|>
  unique()

attributes(dt_BFA_2)
#> $row.names
#> [1] 1 2 3
#> 
#> $class
#> [1] "data.table" "data.frame"
#> 
#> $.internal.selfref
#> <pointer: 0x000001d80ad36050>
#> 
#> $names
#> [1] "BFA" "CDA" "RFG"

# Save version pin (creates new version)
board |> pin_write(dt_BFA_2, pin_name, versioned = TRUE,
                   type = "qs", force_identical_write = FALSE)
#> Creating new version '20251121T182104Z-bcd4c'
#> Writing to pin 'BFA'

# Show meta and pin hash
meta_2 <- board |> pin_meta(name = pin_name)
meta_2$pin_hash
#> [1] "daee4073fa46bc67"

# Same pin_hashes
waldo::compare(meta_1$pin_hash, meta_2$pin_hash)
#> ✔ No differences

# However, two versions of pin
vers <- board |> pin_versions(name = pin_name)
vers
#> # A tibble: 2 × 3
#>   version                created             hash 
#>   <chr>                  <dttm>              <chr>
#> 1 20251121T182104Z-bcd4c 2025-11-21 19:21:04 bcd4c
#> 2 20251121T182104Z-daee4 2025-11-21 19:21:04 daee4

# And both data.tables are the same
waldo::compare(dt_BFA, dt_BFA_2)
#> ✔ No differences

^{Created on 2025-11-21 with reprex v2.1.0}

We hope this example clearly illustrates the issue and helps others understand why multiple versions may occur when working with data.table. Regarding your question about qs2, yes, we have implemented it in our pipeline, and it is working as expected for now. Happy to answer any other questions and as always, thank you for creating such great packages like pins.

[juliasilge]

Ah interesting, that is great news, then! I see that .rds does actually record different binary objects if you have changed the attributes in that way:

library(data.table)
df1 <- data.table(BFA = 1:3, CDA = 4:6, RFG = 7:9)
df2 <- df1 |> rbind(df1)|> unique()

f_rds1 <- tempfile(fileext=".rds")
f_rds2 <- tempfile(fileext=".rds")

saveRDS(df1, f_rds1)
saveRDS(df2, f_rds2)

digest::digest(file = f_rds1, algo="xxhash64")
#> [1] "ad6c87287f0e6236"
digest::digest(file = f_rds2, algo="xxhash64")
#> [1] "2e77c786aeaf49b7"

^{Created on 2025-11-21 with reprex v2.1.1}

Since those files are literally different on disk (what is saved in the .rds or .qs2 is different in one case compared to the other) pins is doing the correct thing.

Thanks for all the information you shared here!

[randrescastaneda]

Hi @juliasilge ,

Thanks for your message. I really appreciate you looking into this. I wanted to gently push back on the idea that "pins is doing the correct thing." As could see from the examples above, the files’ contents and attribute values are exactly the same; the only difference is the order of attributes on disk. Those ordering-only differences feel too minor to justify a new version in pins, and tools like waldo::compare also ignore them. This issue is so critical, that we ended up writing a custom hashing function to avoid spurious versions, which feels like extra work for an edge case. Would it be possible to add an option to ignore attribute-order changes, or at least document this behavior so users know what to expect? Thanks for considering it.

[dana89co]

Hi @juliasilge,

Maybe to portray @randrescastaneda comment, here is a clear example of the extra versioning we get with pins :

library(pins)
library(data.table)

board <- board_temp()

df1 <- data.table(BFA = 1:3, CDA = 4:6, RFG = 7:9)

# Save version pin
board |> pin_write(df1, "data", versioned = TRUE,
                   type = "qs", force_identical_write = FALSE)
#> Creating new version '20251124T141136Z-daee4'
#> Writing to pin 'data'

# Modify data.table as before
df2 <- df1 |> rbind(df1)|> unique()

# Save new version pin
board |> pin_write(df2, "data", versioned = TRUE,
                   type = "qs", force_identical_write = FALSE)
#> Creating new version '20251124T141136Z-bcd4c'
#> Writing to pin 'data'

Sys.sleep(1)

# Save the same version pin 
board |> pin_write(df2, "data", versioned = TRUE,
                   type = "qs", force_identical_write = FALSE)
#> Creating new version '20251124T141137Z-bcd4c'
#> Writing to pin 'data'

# We get three versions of pin instead of two
vers <- board |> pin_versions(name = "data")
vers
#> # A tibble: 3 × 3
#>   version                created             hash 
#>   <chr>                  <dttm>              <chr>
#> 1 20251124T141136Z-bcd4c 2025-11-24 15:11:36 bcd4c
#> 2 20251124T141136Z-daee4 2025-11-24 15:11:36 daee4
#> 3 20251124T141137Z-bcd4c 2025-11-24 15:11:37 bcd4c

^{Created on 2025-11-24 with reprex v2.1.0}

I hope this is useful.