[R][Parquet] Error: Invalid: Invalid number of indices: 0 with `read_parquet`

[anashen]

Describe the bug, including details regarding any error messages, version, and platform.

Hi,

It looks like the team may already be aware of this (#47981), but I wanted to create a separate issue for the R package.

Our R package Seurat depends on read_parquet. With arrow v22.0.0, read_parquet throws Error: Invalid: Invalid number of indices: 0 on only some files. I can confirm that arrow v21.0.0.1 has no issues on the same files. An example of the type of file raising an error is included below.

Any fixes for this, or tips on getting around it, will be much appreciated. However, the data in question is not generated by us, so we have no control regarding file format.

# publicly available at 
# https://www.10xgenomics.com/support/software/xenium-onboard-analysis/latest/resources/xenium-example-data#test-data-v4-0
curl -O https://cf.10xgenomics.com/samples/xenium/4.0.0/Xenium_V1_Protein_Human_Kidney_tiny/Xenium_V1_Protein_Human_Kidney_tiny_outs.zip
# `transcripts.parquet` is located inside the zipped folder

read_parquet("/fakepath/Xenium_V1_Protein_Human_Kidney_tiny_outs/transcripts.parquet")
# Error: Invalid: Invalid number of indices: 0
traceback()
# 6: stop(e)
# 5: value[[3L]](cond)
# 4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
# 3: tryCatchList(expr, classes, parentenv, handlers)
# 2: tryCatch(reader$ReadTable(), error = read_compressed_error)
# 1: read_parquet("/fakepath/Xenium_V1_Protein_Human_Kidney_tiny_outs/transcripts.parquet")

sessionInfo

R version 4.5.1 (2025-06-13)
Platform: aarch64-apple-darwin20
Running under: macOS Sequoia 15.6.1

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRblas.0.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] arrow_22.0.0

loaded via a namespace (and not attached):
 [1] tidyselect_1.2.1 bit_4.6.0        compiler_4.5.1   magrittr_2.0.3  
 [5] assertthat_0.2.1 R6_2.6.1         cli_3.6.5        glue_1.8.0      
 [9] bit64_4.6.0-1    vctrs_0.6.5      lifecycle_1.0.4  rlang_1.1.6     
[13] purrr_1.1.0  

Thanks!

Component(s)

R

[thisisnic]

From that PR, it looks like the problem looks like it resides in the file itself not fully conforming to the Parquet spec, but this will be fixed in the next release so that we can read files which are non-compliant in this way without crashing - #47992

As a temporary workaround I'd suggest using an older version of Arrow to read and re-write it the file, or using an older version of Arrow if the issue didn't show up there.

Hope that helps; closing the ticket but happy to brainstorm alternative workaround here if it does not!

[anashen]

Got it, thanks for the update @thisisnic!

[azzurradep]

Dear team,
I am analysing two different xenium files. Yesterday I run the pipeline for the first file and it worked fine no issue with arrow. Today I run the other and I am getting the same error as above. I tried to fix it multiple ways, including downgrading (which is not working) but it did not help the issue.
I would appreciate any help.
Best,
AD

[thisisnic]

@azzurradep Thanks for reporting; are you able to provide sessionInfo information similar to above, with more info on your operating system, Arrow version etc?

[azzurradep]

@azzurradep Thanks for reporting; are you able to provide sessionInfo information similar to above, with more info on your operating system, Arrow version etc?

Thank you so much for your reply. Here are the information from both when I use the xenium file that works and when I use the other one which gives me the error. I am struggling to understand why is working with one but not the other.

Xenium file 1- working

library(future)
options(future.seed = TRUE)
RNGkind("L'Ecuyer-CMRG")
set.seed(1234)
plan("multisession", workers = 6)
options(future.globals.maxSize = 8 * 1024^3)

library(ggplot2)
library(arrow)

Attaching package: ‘arrow’

The following object is masked from ‘package:utils’:

timestamp

library(Seurat)
Loading required package: SeuratObject
Loading required package: sp

Attaching package: ‘SeuratObject’

The following objects are masked from ‘package:base’:

intersect, t

path <- "/Users/azzurradepace/Desktop/Xenium/output-XETG00160__0032909__Region_1__20240911__163848/output-XETG00160__0032909__Region_1__20240911__163848"

Load the Xenium data

xenium.obj <- LoadXenium(path, fov = "fov", segmentations = "cell")
Cell_seg columns: cell_id, segmentation_method
Genome matrix has multiple modalities, returning a list of matrices for this genome
Warning: Feature names cannot have underscores (''), replacing with dashes ('-')
Warning: Feature names cannot have underscores (''), replacing with dashes ('-')
Warning: Feature names cannot have underscores (''), replacing with dashes ('-')
Warning: Feature names cannot have underscores (''), replacing with dashes ('-')
Warning: Feature names cannot have underscores (''), replacing with dashes ('-')
Warning: Feature names cannot have underscores (''), replacing with dashes ('-')

packageVersion("arrow")
[1] ‘22.0.0.1’
R.version.string
[1] "R version 4.5.2 (2025-10-31)"

Xenium file 2- not working

library(future)
options(future.seed = TRUE)
RNGkind("L'Ecuyer-CMRG")
set.seed(1234)
plan("multisession", workers = 6)
options(future.globals.maxSize = 8 * 1024^3)
library(ggplot2)
library(arrow)

Attaching package: ‘arrow’

The following object is masked from ‘package:utils’:

timestamp

library(Seurat)
Loading required package: SeuratObject
Loading required package: sp

Attaching package: ‘SeuratObject’

The following objects are masked from ‘package:base’:

intersect, t

path <- "/Users/azzurradepace/Desktop/Xenium/output-XETG00160__0061567__CTS__20250919__114240/output-XETG00160__0061567__CTS__20250919__114240"

Load the Xenium data

xenium.obj <- LoadXenium(path, fov = "fov", segmentations = "cell")
Cell_seg columns: cell_id, segmentation_method
Genome matrix has multiple modalities, returning a list of matrices for this genome
Error : Invalid: Invalid number of indices: 0
Error in file(file, "rt") : cannot open the connection
Error in FUN(X[[i]], ...) :
Xenium outputs were incomplete: missing transcripts. Xenium outputs no longer include transcripts.csv.gz. Instead, please install arrow to read transcripts.parquet

packageVersion("arrow")
[1] ‘22.0.0.1’
R.version.string
[1] "R version 4.5.2 (2025-10-31)"

sessionInfo()
R version 4.5.2 (2025-10-31)
Platform: aarch64-apple-darwin20
Running under: macOS Sequoia 15.6.1

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.12.1

Random number generation:
RNG: L'Ecuyer-CMRG
Normal: Inversion
Sample: Rejection

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Europe/London
tzcode source: internal

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] Seurat_5.4.0 SeuratObject_5.3.0 sp_2.2-1 arrow_22.0.0.1 ggplot2_4.0.2 future_1.69.0

loaded via a namespace (and not attached):
[1] deldir_2.0-4 pbapply_1.7-4 gridExtra_2.3 rlang_1.1.7 magrittr_2.0.4 RcppAnnoy_0.0.23
[7] otel_0.2.0 matrixStats_1.5.0 ggridges_0.5.7 compiler_4.5.2 spatstat.geom_3.7-0 png_0.1-8
[13] vctrs_0.7.1 reshape2_1.4.5 hdf5r_1.3.12 stringr_1.6.0 pkgconfig_2.0.3 fastmap_1.2.0
[19] promises_1.5.0 purrr_1.2.1 bit_4.6.0 jsonlite_2.0.0 goftest_1.2-3 later_1.4.6
[25] spatstat.utils_3.2-1 irlba_2.3.7 parallel_4.5.2 cluster_2.1.8.2 R6_2.6.1 ica_1.0-3
[31] stringi_1.8.7 RColorBrewer_1.1-3 spatstat.data_3.1-9 reticulate_1.45.0 parallelly_1.46.1 spatstat.univar_3.1-6
[37] lmtest_0.9-40 scattermore_1.2 Rcpp_1.1.1 assertthat_0.2.1 tensor_1.5.1 future.apply_1.20.1
[43] zoo_1.8-15 sctransform_0.4.3 httpuv_1.6.16 Matrix_1.7-4 splines_4.5.2 igraph_2.2.2
[49] tidyselect_1.2.1 rstudioapi_0.18.0 abind_1.4-8 spatstat.random_3.4-4 codetools_0.2-20 miniUI_0.1.2
[55] spatstat.explore_3.7-0 listenv_0.10.0 lattice_0.22-9 tibble_3.3.1 plyr_1.8.9 shiny_1.12.1
[61] withr_3.0.2 S7_0.2.1 ROCR_1.0-12 Rtsne_0.17 fastDummies_1.7.5 survival_3.8-6
[67] polyclip_1.10-7 fitdistrplus_1.2-6 pillar_1.11.1 KernSmooth_2.23-26 plotly_4.12.0 generics_0.1.4
[73] RcppHNSW_0.6.0 scales_1.4.0 globals_0.19.0 xtable_1.8-4 glue_1.8.0 lazyeval_0.2.2
[79] tools_4.5.2 data.table_1.18.2.1 RSpectra_0.16-2 RANN_2.6.2 dotCall64_1.2 cowplot_1.2.0
[85] grid_4.5.2 tidyr_1.3.2 nlme_3.1-168 patchwork_1.3.2 cli_3.6.5 spatstat.sparse_3.1-0
[91] spam_2.11-3 viridisLite_0.4.3 dplyr_1.2.0 uwot_0.2.4 gtable_0.3.6 digest_0.6.39
[97] progressr_0.18.0 ggrepel_0.9.6 htmlwidgets_1.6.4 farver_2.1.2 htmltools_0.5.9 lifecycle_1.0.5
[103] httr_1.4.8 mime_0.13 bit64_4.6.0-1 MASS_7.3-65

[thisisnic]

Thanks @azzurradep - looks like you're using 22.0.0.1 there, can you try updating to 23.0.0 and showing us the output? It should be resolved in 23.0.0 and above