Skip to content

Commit ab5f261

Browse files
erezrokaherezrokah
authored
Merge branch 'main' into fix/plugin_version_embed
2 parents 8278f37 + 608f700 commit ab5f261

File tree

29 files changed

+629
-165
lines changed

29 files changed

+629
-165
lines changed

‎.release-please-manifest.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
{
2-
".": "4.77.0"
2+
".": "4.80.2"
33
}

‎CHANGELOG.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,77 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [4.80.2](https://github.com/cloudquery/plugin-sdk/compare/v4.80.1...v4.80.2) (2025-05-19)
9+
10+
11+
### Bug Fixes
12+
13+
* Change logic for batch writing to write when batch size is reached, not exceeded ([#2153](https://github.com/cloudquery/plugin-sdk/issues/2153)) ([58c8a1e](https://github.com/cloudquery/plugin-sdk/commit/58c8a1e35d8d77f7cb1ae1c73e70e4a21b23e0a7))
14+
* Flush DeleteRecord messages when batch writer is flushed ([#2154](https://github.com/cloudquery/plugin-sdk/issues/2154)) ([791c865](https://github.com/cloudquery/plugin-sdk/commit/791c8658a0b0224f080dce8ea0cf734dbc9ce911))
15+
16+
## [4.80.1](https://github.com/cloudquery/plugin-sdk/compare/v4.80.0...v4.80.1) (2025-05-12)
17+
18+
19+
### Bug Fixes
20+
21+
* **deps:** Update module github.com/cloudquery/codegen to v0.3.27 ([#2148](https://github.com/cloudquery/plugin-sdk/issues/2148)) ([1fd7b1e](https://github.com/cloudquery/plugin-sdk/commit/1fd7b1e95a262017152616290da791e08fc497b8))
22+
* **deps:** Update module github.com/cloudquery/codegen to v0.3.28 ([#2150](https://github.com/cloudquery/plugin-sdk/issues/2150)) ([4d1409c](https://github.com/cloudquery/plugin-sdk/commit/4d1409c402fd5e8b793ee0586e9d2ec8c3812cd5))
23+
* **deps:** Update module github.com/cloudquery/plugin-pb-go to v1.26.11 ([#2149](https://github.com/cloudquery/plugin-sdk/issues/2149)) ([a904b46](https://github.com/cloudquery/plugin-sdk/commit/a904b461a5ca7eebb0c6f4c807db9aa23011a2b5))
24+
* **deps:** Update module github.com/samber/lo to v1.49.1 ([#2139](https://github.com/cloudquery/plugin-sdk/issues/2139)) ([f11b5e6](https://github.com/cloudquery/plugin-sdk/commit/f11b5e67d95af1f54410236bbefb071c01c5df82))
25+
26+
## [4.80.0](https://github.com/cloudquery/plugin-sdk/compare/v4.79.1...v4.80.0) (2025-05-09)
27+
28+
29+
### Features
30+
31+
* Add SensitiveColumns to tables schema ([#2134](https://github.com/cloudquery/plugin-sdk/issues/2134)) ([e95674f](https://github.com/cloudquery/plugin-sdk/commit/e95674f255c7225a9b5d593daf54c0a373c2ef50))
32+
33+
34+
### Bug Fixes
35+
36+
* **deps:** Update module github.com/cloudquery/cloudquery-api-go to v1.13.9 ([#2143](https://github.com/cloudquery/plugin-sdk/issues/2143)) ([77d4b9b](https://github.com/cloudquery/plugin-sdk/commit/77d4b9b317dfa06ccbf4c0696adcdaa6de724173))
37+
38+
## [4.79.1](https://github.com/cloudquery/plugin-sdk/compare/v4.79.0...v4.79.1) (2025-05-03)
39+
40+
41+
### Bug Fixes
42+
43+
* **deps:** Update golang.org/x/exp digest to 7e4ce0a ([#2135](https://github.com/cloudquery/plugin-sdk/issues/2135)) ([efb8813](https://github.com/cloudquery/plugin-sdk/commit/efb8813182f813a25ec9bf0ba465f5a1419c937d))
44+
* **deps:** Update module github.com/grpc-ecosystem/go-grpc-middleware/v2 to v2.3.1 ([#2136](https://github.com/cloudquery/plugin-sdk/issues/2136)) ([5534187](https://github.com/cloudquery/plugin-sdk/commit/553418708ebc1fcbc1eb13ea29907346258aa36b))
45+
* **deps:** Update module github.com/rs/zerolog to v1.34.0 ([#2138](https://github.com/cloudquery/plugin-sdk/issues/2138)) ([e1bcb05](https://github.com/cloudquery/plugin-sdk/commit/e1bcb0532b7b1b42e38571254c655769296240e4))
46+
* **deps:** Update module google.golang.org/grpc to v1.72.0 ([#2141](https://github.com/cloudquery/plugin-sdk/issues/2141)) ([a0f27a3](https://github.com/cloudquery/plugin-sdk/commit/a0f27a3ca01a870912ad7afb9ec92fb8e52f78b7))
47+
* Pass installation ID from env for usage report ([#2140](https://github.com/cloudquery/plugin-sdk/issues/2140)) ([4d36bfb](https://github.com/cloudquery/plugin-sdk/commit/4d36bfba0a572995c11e10fc2e448e36beb67208))
48+
49+
## [4.79.0](https://github.com/cloudquery/plugin-sdk/compare/v4.78.0...v4.79.0) (2025-04-28)
50+
51+
52+
### Features
53+
54+
* Add transformer to update table description with its table options ([#2128](https://github.com/cloudquery/plugin-sdk/issues/2128)) ([2387b57](https://github.com/cloudquery/plugin-sdk/commit/2387b5765133a03e202f332290bfd94c2ac50eab))
55+
* Show plugin version in plugin server logs ([#2124](https://github.com/cloudquery/plugin-sdk/issues/2124)) ([be08606](https://github.com/cloudquery/plugin-sdk/commit/be08606413a4392d04d3e388ccc1edbe64439c14))
56+
57+
58+
### Bug Fixes
59+
60+
* **deps:** Update module github.com/cloudquery/plugin-pb-go to v1.26.10 ([#2132](https://github.com/cloudquery/plugin-sdk/issues/2132)) ([775d537](https://github.com/cloudquery/plugin-sdk/commit/775d537cc9ee69ee548d69be67e7172baa03fcab))
61+
* Prevent deadlock in transformer ([#2130](https://github.com/cloudquery/plugin-sdk/issues/2130)) ([a65b101](https://github.com/cloudquery/plugin-sdk/commit/a65b101d05ee43bc1f1f0033736ede756ee55604))
62+
63+
## [4.78.0](https://github.com/cloudquery/plugin-sdk/compare/v4.77.0...v4.78.0) (2025-04-22)
64+
65+
66+
### Features
67+
68+
* Add logger to context ([#2125](https://github.com/cloudquery/plugin-sdk/issues/2125)) ([718e8ed](https://github.com/cloudquery/plugin-sdk/commit/718e8ed781fb27130636a87c76bfeb6c00348383))
69+
70+
71+
### Bug Fixes
72+
73+
* **deps:** Update aws-sdk-go-v2 monorepo ([#2119](https://github.com/cloudquery/plugin-sdk/issues/2119)) ([5554039](https://github.com/cloudquery/plugin-sdk/commit/5554039d4358a66f21e765b8dc7c3203b7437f04))
74+
* **deps:** Update aws-sdk-go-v2 monorepo ([#2121](https://github.com/cloudquery/plugin-sdk/issues/2121)) ([7b54577](https://github.com/cloudquery/plugin-sdk/commit/7b54577964b523aba6ca93497d65c8bad6132149))
75+
* **deps:** Update aws-sdk-go-v2 monorepo ([#2123](https://github.com/cloudquery/plugin-sdk/issues/2123)) ([8f370f8](https://github.com/cloudquery/plugin-sdk/commit/8f370f80da7ba9f8c48896aa31414bf16c57fbf1))
76+
* **deps:** Update Google Golang modules ([#2118](https://github.com/cloudquery/plugin-sdk/issues/2118)) ([93d9203](https://github.com/cloudquery/plugin-sdk/commit/93d9203936fb499ab516fe9e847e078e758afb36))
77+
* **deps:** Update module golang.org/x/net to v0.38.0 [SECURITY] ([#2122](https://github.com/cloudquery/plugin-sdk/issues/2122)) ([0b0e187](https://github.com/cloudquery/plugin-sdk/commit/0b0e18763cccad1d01cf61c7a8b4c6c5e5ec343c))
78+
879
## [4.77.0](https://github.com/cloudquery/plugin-sdk/compare/v4.76.0...v4.77.0) (2025-04-03)
980

1081

‎CONTRIBUTING.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,8 @@ This is the repository for CloudQuery SDK. If you are looking for CloudQuery CLI
77
## Links
88

99
- [Homepage](https://cloudquery.io)
10-
- [Documentation](https://docs.cloudquery.io)
10+
- [Platform](https://docs.cloudquery.io)
11+
- [Documentation](https://cli-docs.cloudquery.io)
1112
- [CLI and Plugins Monorepo](https://github.com/cloudquery/cloudquery)
1213
- [Community](https://community.cloudquery.io)
1314

‎docs/table_options.go‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
package docs
2+
3+
import (
4+
"encoding/json"
5+
"errors"
6+
"reflect"
7+
"regexp"
8+
"strings"
9+
10+
schemaDocs "github.com/cloudquery/codegen/jsonschema/docs"
11+
"github.com/cloudquery/plugin-sdk/v4/schema"
12+
invoschema "github.com/invopop/jsonschema"
13+
)
14+
15+
func TableOptionsDescriptionTransformer(tableOptions any, jsonSchema string) (schema.Transform, error) {
16+
var sc invoschema.Schema
17+
if err := json.Unmarshal([]byte(jsonSchema), &sc); err != nil {
18+
return nil, err
19+
}
20+
tableNamesToOptionsDocs := make(map[string]string)
21+
tableOptionsType := reflect.ValueOf(tableOptions).Elem().Type()
22+
for i := range tableOptionsType.NumField() {
23+
field := tableOptionsType.Field(i)
24+
fieldType := field.Type.String()
25+
if strings.Contains(fieldType, ".") {
26+
fieldType = strings.Split(fieldType, ".")[1]
27+
}
28+
defValue, ok := sc.Definitions[fieldType]
29+
if !ok {
30+
return nil, errors.New("definition not found for " + field.Name)
31+
}
32+
tableName := strings.Split(field.Tag.Get("json"), ",")[0]
33+
if tableName == "" {
34+
return nil, errors.New("json tag not found for table " + field.Name)
35+
}
36+
newRoot := sc
37+
newRoot.ID = "Table Options"
38+
newRoot.Ref = "#/$defs/" + "Table Options"
39+
newRoot.Definitions["Table Options"] = defValue
40+
sch, _ := json.Marshal(newRoot)
41+
doc, _ := schemaDocs.Generate(sch, 1)
42+
tocRegex := regexp.MustCompile(`# Table of contents[\s\S]+?##`)
43+
tableNamesToOptionsDocs[tableName] = tocRegex.ReplaceAllString(doc, "##")
44+
}
45+
46+
return func(table *schema.Table) error {
47+
if tableNamesToOptionsDocs[table.Name] != "" {
48+
table.Description = table.Description + "\n\n" + tableNamesToOptionsDocs[table.Name]
49+
}
50+
return nil
51+
}, nil
52+
}

‎docs/table_options_test.go‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
package docs
2+
3+
import (
4+
_ "embed"
5+
"testing"
6+
7+
"github.com/cloudquery/plugin-sdk/v4/docs/testdata"
8+
"github.com/cloudquery/plugin-sdk/v4/schema"
9+
"github.com/stretchr/testify/require"
10+
)
11+
12+
//go:embed testdata/schema.json
13+
var testSchema string
14+
15+
type testTableOptions struct {
16+
Dummy testdata.DummyTableOptions `json:"dummy,omitempty"`
17+
}
18+
19+
var testTable = &schema.Table{
20+
Name: "dummy",
21+
Description: "This is a dummy table",
22+
}
23+
24+
func TestTableOptionsDescriptionTransformer(t *testing.T) {
25+
type args struct {
26+
tableOptions any
27+
jsonSchema string
28+
table *schema.Table
29+
}
30+
tests := []struct {
31+
name string
32+
args args
33+
wantDesc string
34+
wantErr bool
35+
}{
36+
{
37+
name: "adds table options to description",
38+
args: args{tableOptions: &testTableOptions{}, jsonSchema: testSchema, table: testTable},
39+
wantDesc: "This is a dummy table\n\n## <a name=\"Table Options\"></a>Table Options\n\n DummyTableOptions contains configuration for the dummy table\n\n* `filter` (`string`)\n",
40+
},
41+
{
42+
name: "leaves description unchanged when table doesn't have options",
43+
args: args{tableOptions: &testTableOptions{}, jsonSchema: testSchema, table: &schema.Table{Description: "Foobar"}},
44+
wantDesc: "Foobar",
45+
},
46+
{
47+
name: "errors out when table options don't match schema",
48+
wantErr: true,
49+
args: args{tableOptions: &testTableOptions{}, jsonSchema: "", table: testTable},
50+
},
51+
}
52+
for _, tt := range tests {
53+
t.Run(tt.name, func(t *testing.T) {
54+
transformer, err := TableOptionsDescriptionTransformer(tt.args.tableOptions, tt.args.jsonSchema)
55+
if tt.wantErr {
56+
require.Error(t, err)
57+
return
58+
}
59+
require.NoError(t, transformer(tt.args.table))
60+
require.Equal(t, tt.wantDesc, tt.args.table.Description)
61+
})
62+
}
63+
}

‎docs/testdata/schema.json‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
{
2+
"$schema": "https://json-schema.org/draft/2020-12/schema",
3+
"$id": "test/spec",
4+
"$ref": "#/$defs/Spec",
5+
"$defs": {
6+
"Spec": {
7+
"properties": {
8+
"table_options": {
9+
"$ref": "#/$defs/TableOptions",
10+
"description": "TableOptions is a set of options to override the defaults for certain tables."
11+
}
12+
},
13+
"additionalProperties": false,
14+
"type": "object"
15+
},
16+
"TableOptions": {
17+
"properties": {
18+
"dummy": {
19+
"oneOf": [
20+
{
21+
"$ref": "#/$defs/DummyTableOptions",
22+
"description": "Options for the dummy table."
23+
},
24+
{
25+
"type": "null"
26+
}
27+
]
28+
}
29+
},
30+
"additionalProperties": false,
31+
"type": "object"
32+
},
33+
"DummyTableOptions": {
34+
"properties": {
35+
"filter": {
36+
"type": "string"
37+
}
38+
},
39+
"additionalProperties": false,
40+
"type": "object",
41+
"description": "DummyTableOptions contains configuration for the dummy table"
42+
}
43+
}
44+
}

‎docs/testdata/tableoptions.go‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
package testdata
2+
3+
type DummyTableOptions struct {
4+
Filter string `json:"filter,omitempty"`
5+
}

‎examples/simple_plugin/go.mod‎

Lines changed: 24 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,40 @@
11
module github.com/cloudquery/plugin-sdk/examples/simple_plugin
22

3-
go 1.23.0
3+
go 1.23.4
44

55
toolchain go1.24.1
66

77
require (
88
github.com/apache/arrow-go/v18 v18.2.0
9-
github.com/cloudquery/plugin-sdk/v4 v4.77.0
10-
github.com/rs/zerolog v1.33.0
9+
github.com/cloudquery/plugin-sdk/v4 v4.80.1
10+
github.com/rs/zerolog v1.34.0
1111
)
1212

1313
require (
1414
github.com/adrg/xdg v0.5.3 // indirect
1515
github.com/apache/arrow/go/v13 v13.0.0-20230731205701-112f94971882 // indirect
1616
github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect
1717
github.com/aws/aws-sdk-go-v2 v1.36.3 // indirect
18-
github.com/aws/aws-sdk-go-v2/config v1.29.12 // indirect
19-
github.com/aws/aws-sdk-go-v2/credentials v1.17.65 // indirect
18+
github.com/aws/aws-sdk-go-v2/config v1.29.14 // indirect
19+
github.com/aws/aws-sdk-go-v2/credentials v1.17.67 // indirect
2020
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.30 // indirect
2121
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.34 // indirect
2222
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.34 // indirect
2323
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 // indirect
2424
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.3 // indirect
2525
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.15 // indirect
26-
github.com/aws/aws-sdk-go-v2/service/licensemanager v1.30.1 // indirect
27-
github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.28.0 // indirect
28-
github.com/aws/aws-sdk-go-v2/service/sso v1.25.2 // indirect
29-
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.0 // indirect
30-
github.com/aws/aws-sdk-go-v2/service/sts v1.33.17 // indirect
26+
github.com/aws/aws-sdk-go-v2/service/licensemanager v1.30.3 // indirect
27+
github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.29.0 // indirect
28+
github.com/aws/aws-sdk-go-v2/service/sso v1.25.3 // indirect
29+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.1 // indirect
30+
github.com/aws/aws-sdk-go-v2/service/sts v1.33.19 // indirect
3131
github.com/aws/smithy-go v1.22.2 // indirect
3232
github.com/bahlo/generic-list-go v0.2.0 // indirect
3333
github.com/buger/jsonparser v1.1.1 // indirect
3434
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
35-
github.com/cloudquery/cloudquery-api-go v1.13.8 // indirect
36-
github.com/cloudquery/plugin-pb-go v1.26.9 // indirect
35+
github.com/cloudquery/cloudquery-api-go v1.13.9 // indirect
36+
github.com/cloudquery/codegen v0.3.28 // indirect
37+
github.com/cloudquery/plugin-pb-go v1.26.11 // indirect
3738
github.com/cloudquery/plugin-sdk/v2 v2.7.0 // indirect
3839
github.com/davecgh/go-spew v1.1.1 // indirect
3940
github.com/ghodss/yaml v1.0.0 // indirect
@@ -42,7 +43,7 @@ require (
4243
github.com/goccy/go-json v0.10.5 // indirect
4344
github.com/google/flatbuffers v25.2.10+incompatible // indirect
4445
github.com/google/uuid v1.6.0 // indirect
45-
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.2.0 // indirect
46+
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.3.1 // indirect
4647
github.com/grpc-ecosystem/grpc-gateway/v2 v2.25.1 // indirect
4748
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
4849
github.com/hashicorp/go-retryablehttp v0.7.7 // indirect
@@ -56,7 +57,7 @@ require (
5657
github.com/oapi-codegen/runtime v1.1.1 // indirect
5758
github.com/pierrec/lz4/v4 v4.1.22 // indirect
5859
github.com/pmezard/go-difflib v1.0.0 // indirect
59-
github.com/samber/lo v1.47.0 // indirect
60+
github.com/samber/lo v1.49.1 // indirect
6061
github.com/santhosh-tekuri/jsonschema/v6 v6.0.1 // indirect
6162
github.com/spf13/cobra v1.8.1 // indirect
6263
github.com/spf13/pflag v1.0.5 // indirect
@@ -77,17 +78,17 @@ require (
7778
go.opentelemetry.io/otel/sdk/metric v1.34.0 // indirect
7879
go.opentelemetry.io/otel/trace v1.34.0 // indirect
7980
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
80-
golang.org/x/exp v0.0.0-20250305212735-054e65f0b394 // indirect
81+
golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0 // indirect
8182
golang.org/x/mod v0.24.0 // indirect
82-
golang.org/x/net v0.37.0 // indirect
83-
golang.org/x/sync v0.12.0 // indirect
84-
golang.org/x/sys v0.31.0 // indirect
85-
golang.org/x/text v0.23.0 // indirect
86-
golang.org/x/tools v0.31.0 // indirect
83+
golang.org/x/net v0.39.0 // indirect
84+
golang.org/x/sync v0.13.0 // indirect
85+
golang.org/x/sys v0.32.0 // indirect
86+
golang.org/x/text v0.24.0 // indirect
87+
golang.org/x/tools v0.32.0 // indirect
8788
golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
88-
google.golang.org/genproto/googleapis/api v0.0.0-20250115164207-1a7da9e5054f // indirect
89-
google.golang.org/genproto/googleapis/rpc v0.0.0-20250115164207-1a7da9e5054f // indirect
90-
google.golang.org/grpc v1.71.1 // indirect
89+
google.golang.org/genproto/googleapis/api v0.0.0-20250218202821-56aae31c358a // indirect
90+
google.golang.org/genproto/googleapis/rpc v0.0.0-20250218202821-56aae31c358a // indirect
91+
google.golang.org/grpc v1.72.0 // indirect
9192
google.golang.org/protobuf v1.36.6 // indirect
9293
gopkg.in/yaml.v2 v2.4.0 // indirect
9394
gopkg.in/yaml.v3 v3.0.1 // indirect

0 commit comments

Comments
 (0)