Skip to content

feat: skip batch insert when unique index exists #1440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
qqxhb merged 4 commits into from
Feb 27, 2026
Merged

feat: skip batch insert when unique index exists #1440

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
cdcbeac
feat: skip batch insert when unique index exists
qqxhb Feb 27, 2026
7d82d76
feat: support custom query unit test template
qqxhb Feb 27, 2026
d1bf5e3
fix: avoid extra blank line in unit test template
qqxhb Feb 27, 2026
ad4c750
fix: keep blank lines around conditional batch insert
qqxhb Feb 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ type Config struct {

Mode GenerateMode // generate mode

UnitTestTemplate string

queryPkgName string // generated query code's package name
modelPkgPath string // model pkg path in target project
dbNameOpts []model.SchemaNameOpt
Expand Down
10 changes: 9 additions & 1 deletion generator.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -475,7 +475,15 @@ func (g *Generator) generateQueryUnitTestFile(data *genInfo) (err error) {
return err
}

err = render(tmpl.CRUDMethodTest, &buf, data.QueryStructMeta)
crudTmpl := tmpl.CRUDMethodTest
if g.UnitTestTemplate != "" {
content, readErr := os.ReadFile(g.UnitTestTemplate)
if readErr != nil {
return readErr
}
crudTmpl = string(content)
}
err = render(crudTmpl, &buf, data.QueryStructMeta)
if err != nil {
return err
}
Expand Down
12 changes: 12 additions & 0 deletions internal/generate/query.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,18 @@ func (b *QueryStructMeta) appendOrUpdateField(f *model.Field) {

func (b *QueryStructMeta) appendField(f *model.Field) { b.Fields = append(b.Fields, f) }

func (b *QueryStructMeta) HasUniqueIndex() bool {
for _, f := range b.Fields {
if f == nil {
continue
}
if len(f.GORMTag[field.TagKeyGormUniqueIndex]) > 0 {
Copy link

@propel-code-bot propel-code-bot bot Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

[Logic] The PR description says to skip CreateInBatches when any field has a gorm tag, but HasUniqueIndex() only checks uniqueIndex. Models using other uniqueness-related tags (e.g., unique, primaryKey) will still run the batch insert and can still fail with duplicate zero values. If the intent is truly “any gorm tag,” consider checking the tag map directly or expanding the tag set you check.

Example (if you want any gorm tag to disable batching):

if len(f.GORMTag) > 0 {
    return true
}

Or include other uniqueness tags as needed.

Context for Agents 
The PR description says to skip `CreateInBatches` when **any** field has a gorm tag, but `HasUniqueIndex()` only checks `uniqueIndex`. Models using other uniqueness-related tags (e.g., `unique`, `primaryKey`) will still run the batch insert and can still fail with duplicate zero values. If the intent is truly “any gorm tag,” consider checking the tag map directly or expanding the tag set you check.

Example (if you want any gorm tag to disable batching):

```go
if len(f.GORMTag) > 0 {
    return true
}
```

Or include other uniqueness tags as needed.

File: internal/generate/query.go
Line: 145

Copy link

@propel-code-bot propel-code-bot bot Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

[Logic] HasUniqueIndex only checks model.Field.GORMTag, but for struct-based generation (ConvertStructs → parseStruct) GORMTag is never populated. That means uniqueIndex tags on model structs won’t be detected and CreateInBatches will still run, reintroducing the duplicate failure. Consider populating GORMTag from schema.Field.TagSettings in parseStruct, or extend HasUniqueIndex to also check the struct tag settings so struct-derived models are covered.

Context for Agents 
HasUniqueIndex only checks `model.Field.GORMTag`, but for struct-based generation (`ConvertStructs` → `parseStruct`) `GORMTag` is never populated. That means `uniqueIndex` tags on model structs won’t be detected and `CreateInBatches` will still run, reintroducing the duplicate failure. Consider populating `GORMTag` from `schema.Field.TagSettings` in `parseStruct`, or extend `HasUniqueIndex` to also check the struct tag settings so struct-derived models are covered.

File: internal/generate/query.go
Line: 145

return true
}
}
return false
}

// HasField check if BaseStruct has fields
func (b *QueryStructMeta) HasField() bool { return len(b.Fields) > 0 }

Expand Down
3 changes: 2 additions & 1 deletion internal/template/method.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -308,11 +308,12 @@ func Test_{{.QueryStructName}}Query(t *testing.T) {
t.Error("create item in table <{{.TableName}}> fail:", err)
}

{{ if not .HasUniqueIndex }}
err = _do.CreateInBatches([]*{{.StructInfo.Package}}.{{.ModelStructName}}{ {}, {} }, 10)
if err != nil {
t.Error("create item in table <{{.TableName}}> fail:", err)
}

{{ end }}
_, err = _do.Select({{.QueryStructName}}.ALL).Take()
if err != nil {
t.Error("Take() on table <{{.TableName}}> fail:", err)
Expand Down
6 changes: 6 additions & 0 deletions tools/gentool/README.ZH_CN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@
only generate models (without query file)
-withUnitTest
generate unit test for query code
-unitTestTemplate string
custom unit test template file path for query code
-fieldSignable
detect integer field's unsigned type, adjust generated data type

Expand Down Expand Up @@ -126,6 +128,10 @@ eg :

生成单元测试。

#### unitTestTemplate

自定义 query 单测模板文件路径

#### fieldSignable

Value : False / True
Expand Down
6 changes: 6 additions & 0 deletions tools/gentool/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ Install GEN as a binary tool
only generate models (without query file)
-withUnitTest
generate unit test for query code
-unitTestTemplate string
custom unit test template file path for query code
-fieldSignable
detect integer field's unsigned type, adjust generated data type

Expand Down Expand Up @@ -120,6 +122,10 @@ Value : False / True

Generate unit test.

#### unitTestTemplate

custom unit test template file path for query code

#### fieldSignable

Value : False / True
Expand Down
1 change: 1 addition & 0 deletions tools/gentool/gen.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ database:
outFile : ""
# generate unit test for query code
withUnitTest : false
unitTestTemplate : ""
# generated model code's package name
modelPkgName : ""
# generate with pointer when field is nullable
Expand Down
20 changes: 13 additions & 7 deletions tools/gentool/gentool.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,14 @@ const (

// CmdParams is command line parameters
type CmdParams struct {
DSN string `yaml:"dsn"` // consult[https://gorm.io/docs/connecting_to_the_database.html]"
DB string `yaml:"db"` // input mysql or postgres or sqlite or sqlserver. consult[https://gorm.io/docs/connecting_to_the_database.html]
Tables []string `yaml:"tables"` // enter the required data table or leave it blank
OnlyModel bool `yaml:"onlyModel"` // only generate model
OutPath string `yaml:"outPath"` // specify a directory for output
OutFile string `yaml:"outFile"` // query code file name, default: gen.go
WithUnitTest bool `yaml:"withUnitTest"` // generate unit test for query code
DSN string `yaml:"dsn"` // consult[https://gorm.io/docs/connecting_to_the_database.html]"
DB string `yaml:"db"` // input mysql or postgres or sqlite or sqlserver. consult[https://gorm.io/docs/connecting_to_the_database.html]
Tables []string `yaml:"tables"` // enter the required data table or leave it blank
OnlyModel bool `yaml:"onlyModel"` // only generate model
OutPath string `yaml:"outPath"` // specify a directory for output
OutFile string `yaml:"outFile"` // query code file name, default: gen.go
WithUnitTest bool `yaml:"withUnitTest"` // generate unit test for query code
UnitTestTemplate string `yaml:"unitTestTemplate"`
ModelPkgName string `yaml:"modelPkgName"` // generated model code's package name
FieldNullable bool `yaml:"fieldNullable"` // generate with pointer when field is nullable
FieldCoverable bool `yaml:"fieldCoverable"` // generate with pointer when field has default value
Expand Down Expand Up @@ -149,6 +150,7 @@ func argParse() *CmdParams {
outPath := flag.String("outPath", defaultQueryPath, "specify a directory for output")
outFile := flag.String("outFile", "", "query code file name, default: gen.go")
withUnitTest := flag.Bool("withUnitTest", false, "generate unit test for query code")
unitTestTemplate := flag.String("unitTestTemplate", "", "custom unit test template file path for query code")
modelPkgName := flag.String("modelPkgName", "", "generated model code's package name")
fieldNullable := flag.Bool("fieldNullable", false, "generate with pointer when field is nullable")
fieldCoverable := flag.Bool("fieldCoverable", false, "generate with pointer when field has default value")
Expand Down Expand Up @@ -185,6 +187,9 @@ func argParse() *CmdParams {
if *withUnitTest {
cmdParse.WithUnitTest = *withUnitTest
}
if *unitTestTemplate != "" {
cmdParse.UnitTestTemplate = *unitTestTemplate
}
if *modelPkgName != "" {
cmdParse.ModelPkgName = *modelPkgName
}
Expand Down Expand Up @@ -226,6 +231,7 @@ func main() {
OutFile: config.OutFile,
ModelPkgPath: config.ModelPkgName,
WithUnitTest: config.WithUnitTest,
UnitTestTemplate: config.UnitTestTemplate,
FieldNullable: config.FieldNullable,
FieldCoverable: config.FieldCoverable,
FieldWithIndexTag: config.FieldWithIndexTag,
Expand Down
Loading