Skip to content

[Go][Parquet] slice bounds out of range - panic in writeDenseArrow #622

@thoro

Description

@thoro

Describe the bug, including details regarding any error messages, version, and platform.

I'm getting this error when writing large amounts of strings:

> panic: runtime error: slice bounds out of range [:-2147360192]

  goroutine 186374 [running]:
  github.com/apache/arrow/go/v17/parquet/pqarrow.writeDenseArrow(0xddae04a9b0, {0x20e96a8, 0xc4ccf9ef00}, {0x20e5320, 0xc4d1a6e5f0}, {0x0, 0x0, 0x0},
  {0x0, 0x0, ...}, ...)
      /go/pkg/mod/github.com/apache/arrow/go/[email protected]/parquet/pqarrow/encode_arrow.go:491 +0x2d2d
  github.com/apache/arrow/go/v17/parquet/pqarrow.WriteArrowToColumn({0x20c7298, 0xdeb4627020}, {0x20e96a8, 0xc4ccf9ef00}, {0x20e5320, 0xc4d1a6e5f0},
  {0x0, 0x0, 0x0}, {0x0, ...}, ...)
      /go/pkg/mod/github.com/apache/arrow/go/[email protected]/parquet/pqarrow/encode_arrow.go:232 +0x325
  github.com/apache/arrow/go/v17/parquet/pqarrow.(*arrowColumnWriter).Write(0xc4fb951710, {0x20c7298, 0xdeb4627020})
      /go/pkg/mod/github.com/apache/arrow/go/[email protected]/parquet/pqarrow/encode_arrow.go:193 +0x533
  github.com/apache/arrow/go/v17/parquet/pqarrow.(*FileWriter).WriteColumnChunked(0xdeb4634a10, 0x31b3220?, 0xc4fb9517d8?, 0x1?)
      /go/pkg/mod/github.com/apache/arrow/go/[email protected]/parquet/pqarrow/file_writer.go:330 +0xe5
  github.com/apache/arrow/go/v17/parquet/pqarrow.(*FileWriter).WriteColumnData(0xdeb4634a10, {0x20e5320, 0xc1c0963360})
      /go/pkg/mod/github.com/apache/arrow/go/[email protected]/parquet/pqarrow/file_writer.go:339 +0xc5
  github.com/apache/arrow/go/v17/parquet/pqarrow.(*FileWriter).Write(0xdeb4634a10, {0x20e1c20, 0xc4d1a16060})
      /go/pkg/mod/github.com/apache/arrow/go/[email protected]/parquet/pqarrow/file_writer.go:242 +0x272

haven't exactly pinpointed what my data looks like at that moment, but this shouldn't happen regardless.

Component(s)

Go

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type: bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions