Skip to content

ctlstore regularly produces races detectable by the Go race detector #125

@extemporalgenome

Description

@extemporalgenome
WARNING: DATA RACE
--
  | Write at 0x000000e93ac0 by goroutine 22:
  | database/sql.convertAssignRows()
  | /go/src/database/sql/convert.go:381 +0x1aa7
  | database/sql.(*Rows).Scan()
  | /go/src/database/sql/sql.go:3326 +0x6e4
  | github.com/segmentio/ctlstore/pkg/scanfunc.New.scanFuncStruct.func1()
  | /var/lib/buildkite-agent/go/pkg/mod/github.com/segmentio/[email protected]/pkg/scanfunc/scan_func.go:88 +0xa7
  | github.com/segmentio/ctlstore.(*Rows).Scan()
  | /var/lib/buildkite-agent/go/pkg/mod/github.com/segmentio/[email protected]/rows.go:58 +0xb9
  | github.com/segmentio/profile-data-services/internal/traitcatalog.(*Client).scanTraitCatalog()
  | /var/lib/buildkite-agent/builds/buildkite-ci-0f362bc367e6f2b76/segment/profile-data-services/internal/traitcatalog/traitcatalog.go:88 +0x27c
  | github.com/segmentio/profile-data-services/internal/traitcatalog.(*Client).TraitspaceDefinitions()
  | /var/lib/buildkite-agent/builds/buildkite-ci-0f362bc367e6f2b76/segment/profile-data-services/internal/traitcatalog/traitcatalog.go:76 +0x1b1
  | github.com/segmentio/profile-data-services/internal/traitcatalog.TestTraitCatalog.func7()
  | /var/lib/buildkite-agent/builds/buildkite-ci-0f362bc367e6f2b76/segment/profile-data-services/internal/traitcatalog/traitcatalog_test.go:187 +0x351
  | testing.tRunner()
  | /go/src/testing/testing.go:1595 +0x238
  | testing.(*T).Run.func1()
  | /go/src/testing/testing.go:1648 +0x44
  |  
  | Previous write at 0x000000e93ac0 by goroutine 19:
  | database/sql.convertAssignRows()
  | /go/src/database/sql/convert.go:381 +0x1aa7
  | database/sql.(*Rows).Scan()
  | /go/src/database/sql/sql.go:3326 +0x6e4
  | github.com/segmentio/ctlstore/pkg/scanfunc.New.scanFuncStruct.func1()
  | /var/lib/buildkite-agent/go/pkg/mod/github.com/segmentio/[email protected]/pkg/scanfunc/scan_func.go:88 +0xa7
  | github.com/segmentio/ctlstore.(*Rows).Scan()
  | /var/lib/buildkite-agent/go/pkg/mod/github.com/segmentio/[email protected]/rows.go:58 +0xb9
  | github.com/segmentio/profile-data-services/internal/traitcatalog.(*Client).scanTraitCatalog()
  | /var/lib/buildkite-agent/builds/buildkite-ci-0f362bc367e6f2b76/segment/profile-data-services/internal/traitcatalog/traitcatalog.go:88 +0x27c
  | github.com/segmentio/profile-data-services/internal/traitcatalog.(*Client).Definitions()
  | /var/lib/buildkite-agent/builds/buildkite-ci-0f362bc367e6f2b76/segment/profile-data-services/internal/traitcatalog/traitcatalog.go:67 +0x165
  | github.com/segmentio/profile-data-services/internal/traitcatalog.TestTraitCatalog.func4()
  | /var/lib/buildkite-agent/builds/buildkite-ci-0f362bc367e6f2b76/segment/profile-data-services/internal/traitcatalog/traitcatalog_test.go:113 +0x344
  | testing.tRunner()
  | /go/src/testing/testing.go:1595 +0x238
  | testing.(*T).Run.func1()
  | /go/src/testing/testing.go:1648 +0x44
  |  
  | Goroutine 22 (running) created at:
  | testing.(*T).Run()
  | /go/src/testing/testing.go:1648 +0x82a
  | github.com/segmentio/profile-data-services/internal/traitcatalog.TestTraitCatalog()
  | /var/lib/buildkite-agent/builds/buildkite-ci-0f362bc367e6f2b76/segment/profile-data-services/internal/traitcatalog/traitcatalog_test.go:163 +0x516
  | testing.tRunner()
  | /go/src/testing/testing.go:1595 +0x238
  | testing.(*T).Run.func1()
  | /go/src/testing/testing.go:1648 +0x44
  |  
  | Goroutine 19 (running) created at:
  | testing.(*T).Run()
  | /go/src/testing/testing.go:1648 +0x82a
  | github.com/segmentio/profile-data-services/internal/traitcatalog.TestTraitCatalog()
  | /var/lib/buildkite-agent/builds/buildkite-ci-0f362bc367e6f2b76/segment/profile-data-services/internal/traitcatalog/traitcatalog_test.go:89 +0x356
  | testing.tRunner()
  | /go/src/testing/testing.go:1595 +0x238
  | testing.(*T).Run.func1()
  | /go/src/testing/testing.go:1648 +0x44
  | ==================
  | FAIL: TestTraitCatalog (0.01s) | 0s
  | --- FAIL: TestTraitCatalog/GetTraitDefinitionForNamespace_found (0.00s)
  | testing.go:1465: race detected during execution of test
  | --- FAIL: TestTraitCatalog/GetTraitDefinitionForNamespaceInTraitSpace_found (0.00s)
  | testing.go:1465: race detected during execution of test

This seems to be due to avoidable cleverness around use of unsafe. We should consider carefully how much our use of unsafe is actually worth, i.e. whether saving an allocation or avoiding normal reflection in a loop is actually sufficiently valuable to outweigh the inherent risk in our approach.

We should also seriously consider using community-supported options, such as https://pkg.go.dev/github.com/jmoiron/sqlx, over our home-grown alternatives.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions