Skip to content

Commit e2863c9

Browse files
jsorianoAndrea Spacca
jsoriano
and
Andrea Spacca
authored
Update to Package Spec v3 and corpora generator 0.7.0 (#1508)
Co-authored-by: Andrea Spacca <[email protected]>
1 parent 97303ff commit e2863c9

File tree

11 files changed

+414
-91
lines changed

11 files changed

+414
-91
lines changed

‎docs/howto/sample_metric.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -630,7 +630,7 @@
630630
},
631631
"parameter": {
632632
"package": "system_benchmarks",
633-
"description": "Benchmark 20MiB of data ingested",
633+
"description": "Benchmark 20000 events ingested",
634634
"version": "999.999.999",
635635
"policy_template": "testpo",
636636
"input": "filestream",
@@ -648,7 +648,7 @@
648648
"wait_for_data_timeout": 60000000000,
649649
"corpora": {
650650
"generator": {
651-
"size": "20MiB",
651+
"total_events": "20000",
652652
"template": {
653653
"raw": "",
654654
"path": "./logs-benchmark/template.log",

‎docs/howto/system_benchmarking.md‎

Lines changed: 27 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -227,43 +227,43 @@ want to benchmark. There can be multiple scenarios defined for the same package.
227227
The `<scenario>.yml` files allow you to define various settings for the benchmark scenario
228228
along with values for package and data stream-level variables. These are the available configuration options for system benchmarks.
229229
230-
| Option | Type | Required | Description |
231-
|---|---|---|---|
232-
| package | string | | The name of the package. If omitted will pick the current package, this is to allow for future definition of benchmarks outside of the packages folders. |
233-
| description | string | | A description for the scenario. |
234-
| version | string | | The version of the package to benchmark. If omitted will pick the current version of the package. |
235-
| policy_template | string | | The policy template to test. If omitted will pick the first one. |
236-
| input | string | yes | Input type to test (e.g. logfile, httpjson, etc). Defaults to the input used by the first stream in the data stream manifest. |
237-
| vars | dictionary | | Package level variables to set (i.e. declared in `$package_root/manifest.yml`). If not specified the defaults from the manifest are used. |
238-
| data_stream.name | string | yes | The data stream to benchmark. |
239-
| data_stream.vars | dictionary | | Data stream level variables to set (i.e. declared in `package_root/data_stream/$data_stream/manifest.yml`). If not specified the defaults from the manifest are used. |
240-
| warmup_time_period | duration | | Warmup time period. All data prior to this period will be ignored in the benchmark results. |
241-
| benchmark_time_period | duration | | Amount of time the benchmark needs to run for. If set the benchmark will stop after this period even though more data is still pending to be ingested. |
242-
| wait_for_data_timeout | duration | | Amount of time to wait for data to be present in Elasticsearch. Defaults to 10m. |
243-
| corpora.generator.size | string | | String describing the amount of data to generate. Example: `20MiB` |
244-
| corpora.generator.template.raw | string | | Raw template for the corpus generator. |
245-
| corpora.generator.template.path | string | | Path to the template for the corpus generator. If a `path` is defined, it will override any `raw` template definition. |
246-
| corpora.generator.template.type | string | | Type of the template for the corpus generator. Default `placeholder`. |
247-
| corpora.generator.config.raw | dictionary | | Raw config for the corpus generator. |
248-
| corpora.generator.config.path | string | | Path to the config for the corpus generator. If a `path` is defined, it will override any `raw` config definition. |
249-
| corpora.generator.fields.raw | dictionary | | Raw fields for the corpus generator. |
250-
| corpora.generator.fields.path | string | | Path to the fields for the corpus generator. If a `path` is defined, it will override any `raw` fields definition. |
251-
| corpora.input_service.name | string | | Name of the input service to use (defined in the `deploy` folder). |
252-
| corpora.input_service.signal | string | | Signal to send to the input service once the benchmark is ready to start. |
230+
| Option | Type | Required | Description |
231+
|---------------------------------|------------|---|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
232+
| package | string | | The name of the package. If omitted will pick the current package, this is to allow for future definition of benchmarks outside of the packages folders. |
233+
| description | string | | A description for the scenario. |
234+
| version | string | | The version of the package to benchmark. If omitted will pick the current version of the package. |
235+
| policy_template | string | | The policy template to test. If omitted will pick the first one. |
236+
| input | string | yes | Input type to test (e.g. logfile, httpjson, etc). Defaults to the input used by the first stream in the data stream manifest. |
237+
| vars | dictionary | | Package level variables to set (i.e. declared in `$package_root/manifest.yml`). If not specified the defaults from the manifest are used. |
238+
| data_stream.name | string | yes | The data stream to benchmark. |
239+
| data_stream.vars | dictionary | | Data stream level variables to set (i.e. declared in `package_root/data_stream/$data_stream/manifest.yml`). If not specified the defaults from the manifest are used. |
240+
| warmup_time_period | duration | | Warmup time period. All data prior to this period will be ignored in the benchmark results. |
241+
| benchmark_time_period | duration | | Amount of time the benchmark needs to run for. If set the benchmark will stop after this period even though more data is still pending to be ingested. |
242+
| wait_for_data_timeout | duration | | Amount of time to wait for data to be present in Elasticsearch. Defaults to 10m. |
243+
| corpora.generator.total_events | uint64 | | Number of total events to generate. Example: `20000` |
244+
| corpora.generator.template.raw | string | | Raw template for the corpus generator. |
245+
| corpora.generator.template.path | string | | Path to the template for the corpus generator. If a `path` is defined, it will override any `raw` template definition. |
246+
| corpora.generator.template.type | string | | Type of the template for the corpus generator. Default `placeholder`. |
247+
| corpora.generator.config.raw | dictionary | | Raw config for the corpus generator. |
248+
| corpora.generator.config.path | string | | Path to the config for the corpus generator. If a `path` is defined, it will override any `raw` config definition. |
249+
| corpora.generator.fields.raw | dictionary | | Raw fields for the corpus generator. |
250+
| corpora.generator.fields.path | string | | Path to the fields for the corpus generator. If a `path` is defined, it will override any `raw` fields definition. |
251+
| corpora.input_service.name | string | | Name of the input service to use (defined in the `deploy` folder). |
252+
| corpora.input_service.signal | string | | Signal to send to the input service once the benchmark is ready to start. |
253253
254254
Example:
255255
256256
`logs-benchmark.yml`
257257
```yaml
258258
---
259-
description: Benchmark 100MiB of data ingested
259+
description: Benchmark 100000 events ingested
260260
input: filestream
261261
vars: ~
262262
data_stream.name: test
263263
data_stream.vars.paths:
264264
- "{{SERVICE_LOGS_DIR}}/corpus-*"
265265
warmup_time_period: 10s
266-
corpora.generator.size: 100MiB
266+
corpora.generator.total_events: 100000
267267
corpora.generator.template.path: ./logs-benchmark/template.log
268268
corpora.generator.config.path: ./logs-benchmark/config.yml
269269
corpora.generator.fields.path: ./logs-benchmark/fields.yml
@@ -322,7 +322,7 @@ elastic-package benchmark system --benchmark logs-benchmark -v
322322
│ info │
323323
├──────────────┬──────────────────────────────────────┤
324324
│ benchmark │ logs-benchmark │
325-
│ description │ Benchmark 100MiB of data ingested │
325+
│ description │ Benchmark 100000 events ingested │
326326
│ run ID │ d2960c04-0028-42c9-bafc-35e599563cb1 │
327327
│ package │ system_benchmarks │
328328
│ start ts (s) │ 1682320355 │
@@ -339,7 +339,7 @@ elastic-package benchmark system --benchmark logs-benchmark -v
339339
│ warmup time period │ 10s │
340340
│ benchmark time period │ 0s │
341341
│ wait for data timeout │ 0s │
342-
│ corpora.generator.size │ 100MiB │
342+
│ corpora.generator.total_events │ 100000 │
343343
│ corpora.generator.template.path │ ./logs-benchmark/template.log │
344344
│ corpora.generator.template.raw │ │
345345
│ corpora.generator.template.type │ │

‎go.mod‎

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,12 +11,12 @@ require (
1111
github.com/cbroglie/mustache v1.4.0
1212
github.com/cespare/xxhash/v2 v2.2.0
1313
github.com/dustin/go-humanize v1.0.1
14-
github.com/elastic/elastic-integration-corpus-generator-tool v0.5.0
14+
github.com/elastic/elastic-integration-corpus-generator-tool v0.7.0
1515
github.com/elastic/go-elasticsearch/v7 v7.17.10
1616
github.com/elastic/go-licenser v0.4.1
1717
github.com/elastic/go-resource v0.1.1
1818
github.com/elastic/go-ucfg v0.8.6
19-
github.com/elastic/package-spec/v3 v3.0.0-rc1
19+
github.com/elastic/package-spec/v3 v3.0.0
2020
github.com/fatih/color v1.15.0
2121
github.com/go-git/go-billy/v5 v5.5.0
2222
github.com/go-git/go-git/v5 v5.9.0
@@ -28,7 +28,7 @@ require (
2828
github.com/magefile/mage v1.15.0
2929
github.com/mholt/archiver/v3 v3.5.1
3030
github.com/olekukonko/tablewriter v0.0.5
31-
github.com/pmezard/go-difflib v1.0.0
31+
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2
3232
github.com/shirou/gopsutil/v3 v3.23.9
3333
github.com/spf13/cobra v1.7.0
3434
github.com/stretchr/testify v1.8.4
@@ -63,7 +63,7 @@ require (
6363
github.com/cloudflare/circl v1.3.3 // indirect
6464
github.com/creasty/defaults v1.7.0 // indirect
6565
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
66-
github.com/davecgh/go-spew v1.1.1 // indirect
66+
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
6767
github.com/dnephin/pflag v1.0.7 // indirect
6868
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
6969
github.com/elastic/gojsonschema v1.2.1 // indirect
@@ -95,15 +95,15 @@ require (
9595
github.com/hashicorp/errwrap v1.1.0 // indirect
9696
github.com/hashicorp/go-multierror v1.1.1 // indirect
9797
github.com/huandu/xstrings v1.4.0 // indirect
98-
github.com/imdario/mergo v0.3.15 // indirect
98+
github.com/imdario/mergo v0.3.16 // indirect
9999
github.com/inconshreveable/mousetrap v1.1.0 // indirect
100100
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
101101
github.com/joeshaw/multierror v0.0.0-20140124173710-69b34d4ec901 // indirect
102102
github.com/josharian/intern v1.0.0 // indirect
103103
github.com/json-iterator/go v1.1.12 // indirect
104104
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
105105
github.com/kevinburke/ssh_config v1.2.0 // indirect
106-
github.com/klauspost/compress v1.16.0 // indirect
106+
github.com/klauspost/compress v1.17.0 // indirect
107107
github.com/klauspost/pgzip v1.2.5 // indirect
108108
github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de // indirect
109109
github.com/lithammer/shortuuid/v3 v3.0.7 // indirect
@@ -136,7 +136,8 @@ require (
136136
github.com/shoenig/go-m1cpu v0.1.6 // indirect
137137
github.com/shopspring/decimal v1.3.1 // indirect
138138
github.com/skeema/knownhosts v1.2.0 // indirect
139-
github.com/spf13/cast v1.5.0 // indirect
139+
github.com/spf13/afero v1.10.0 // indirect
140+
github.com/spf13/cast v1.5.1 // indirect
140141
github.com/spf13/pflag v1.0.5 // indirect
141142
github.com/stretchr/objx v0.5.1 // indirect
142143
github.com/tklauser/go-sysconf v0.3.12 // indirect
@@ -154,14 +155,14 @@ require (
154155
golang.org/x/exp/typeparams v0.0.0-20221208152030-732eee02a75a // indirect
155156
golang.org/x/mod v0.13.0 // indirect
156157
golang.org/x/net v0.17.0 // indirect
157-
golang.org/x/oauth2 v0.8.0 // indirect
158+
golang.org/x/oauth2 v0.12.0 // indirect
158159
golang.org/x/sync v0.4.0 // indirect
159160
golang.org/x/sys v0.13.0 // indirect
160161
golang.org/x/term v0.13.0 // indirect
161162
golang.org/x/text v0.13.0 // indirect
162163
golang.org/x/time v0.3.0 // indirect
163164
google.golang.org/appengine v1.6.7 // indirect
164-
google.golang.org/protobuf v1.30.0 // indirect
165+
google.golang.org/protobuf v1.31.0 // indirect
165166
gopkg.in/inf.v0 v0.9.1 // indirect
166167
gopkg.in/warnings.v0 v0.1.2 // indirect
167168
gopkg.in/yaml.v2 v2.4.0 // indirect

0 commit comments

Comments
 (0)