GH-47648: [Archery][Integration] More granularity in JSON test cases

[pitrou]

Rationale for this change

Separate binary columns from other primitives in the integration test cases, to make life easier when writing a new Arrow implementation.

Are these changes tested?

Yes, by integration CI test.

Are there any user-facing changes?

No, only for implementers who would use the generated files directly.

• GitHub Issue: [Archery][Integration] More granularity in JSON test cases #47648

[pitrou]

@Alex-PLACET does this look ok?

[Alex-PLACET]

@pitrou perfect 👌

[pitrou]

@alamb @tustvold @zeroshade Is it ok to you?

[pitrou]

@paleolimbot

[paleolimbot]

Sounds good!

It was already not 1:1, but I found it confusing when I started out that the files in arrow-testing didn't quite align with the ones generated by Archery and I suppose this further widens that gap? The golden files in arrow-testing are easier to use than a full-on integration test for those getting started.

[pitrou]

It was already not 1:1, but I found it confusing when I started out that the files in arrow-testing didn't quite align with the ones generated by Archery and I suppose this further widens that gap?

For the old files, yes, but the plan is to add a set of more recent "golden" files generated with Arrow 21: #47635

[pitrou]

@zeroshade How can we make these errors go away on the integration tests? They've been happening randomly for quite some time.

  + go install -v ./...
  type_traits_boolean.go:20:2: cannot find package "github.com/apache/arrow-go/v18/arrow/bitutil" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/bitutil (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/bitutil (from $GOPATH)
  datatype_fixedwidth.go:25:2: cannot find package "github.com/apache/arrow-go/v18/arrow/decimal" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/decimal (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/decimal (from $GOPATH)
  datatype_viewheader.go:23:2: cannot find package "github.com/apache/arrow-go/v18/arrow/endian" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/endian (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/endian (from $GOPATH)
  type_traits.go:24:2: cannot find package "github.com/apache/arrow-go/v18/arrow/float16" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/float16 (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/float16 (from $GOPATH)
  datatype.go:24:2: cannot find package "github.com/apache/arrow-go/v18/arrow/internal/debug" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/internal/debug (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/internal/debug (from $GOPATH)

[etc.]

[pitrou]

Ok, it should be fixed by #47674

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit eb9d519.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.