Skip to content

Commit 02f7494

Browse files
authored
Merge for 6.3.5 Release (#169)
* Fix jinja template bug under SA-Eventgen app * Feature timeMultiple (#141) * Adjusting interval with timeMultiple * Update issue templates add the bug report and feature request issue templates * changing the stanzas to produce data * Windbag generator/count + end=0 edge cases (#145) * Generator handling for count = -1, end = 0 working properly * Update docs (#146) * Updated docs, added release notes to changelog * Bump version * add python code style lint and format * [Docs] update contribute docs (#148) * [Docs] update contribute docs * [Docs] update the contribute * Fix make docs bug and summary anchor link error * init unittest * [Build] add ut for timeparser functions * Add more UT for config module * Add more unit tests for config module * Pep8 (#151) * Format to code standards, addressed linter errors/warnings * Update docs * Post-PEP8 Fixes (#157) * skip sort, gitignore 3rd party libs * fixed yapf ignore, ran format across repo * Issue 160 (#163) * Fixed timer and token * Added a conditional for end=-1 * Update eventgentimer.py * Fixed timer and token (#162) * add extendIndexes feature (#154) * add extendIndexes feature * set extendIndexes as a list value * correct log level * upate doc, change num to weight * Test fix (#168) * Add sample functional test for replay mode * Add token replacement functional tests * skip failed case * Added a timeout * created a results dir * Update version.json
1 parent bdb4622 commit 02f7494

File tree

138 files changed

+11717
-2481
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

138 files changed

+11717
-2481
lines changed

.circleci/config.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ jobs:
1717
name: Run Tests
1818
command: |
1919
make test
20+
no_output_timeout: 30m
2021
- store_test_results:
2122
path: /home/circleci/project/tests/test-reports
2223
- store_artifacts:

.github/ISSUE_TEMPLATE/bug-report.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name: Bug report
33
about: Provide the details of the bug to us in order to further triage and fix
44
title: "[BUG]"
55
labels: bug
6-
assignees: li-wu, GordonWang
6+
assignees: lephino, arctan5x, jmeixensperger, li-wu, GordonWang
77

88
---
99

.github/ISSUE_TEMPLATE/feature_request.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name: Feature request
33
about: Suggest an idea or improvement for this project
44
title: "[FEATURE/IMPROVEMENT]"
55
labels: enhancement
6-
assignees: li-wu, GordonWang
6+
assignees: lephino, arctan5x, jmeixensperger, li-wu, GordonWang
77

88
---
99

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@ stage/
1717
local/
1818
eventgen_wsgi.conf
1919
*.log
20-
dist
2120
*.egg-info
2221
**/*.tgz
2322
.cache
@@ -27,3 +26,4 @@ _book
2726
*.result
2827
venv/*
2928
*.log.*
29+
splunk_eventgen-*/

.yapfignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
splunk_eventgen/lib/concurrent
2+
splunk_eventgen/lib/requests_futures

CONTRIBUTING.md

Lines changed: 4 additions & 87 deletions
Original file line numberDiff line numberDiff line change
@@ -1,99 +1,16 @@
11
# Contributing
22

3-
When contributing to this repository, please first discuss the change you wish to make via issue,
4-
email, or any other method with the owners of this repository before making a change.
3+
When contributing to this repository, please read over the [contributing document](http://splunk.github.io/eventgen/CONTRIBUTE.html). [Create a issue](http://splunk.github.io/eventgen/FILE_ISSUES.html) to discuss your demand and follow the [guidelines](http://splunk.github.io/eventgen/CONTRIBUTE_CODE.html) to make the code change
54

6-
Please note we have a code of conduct, please follow it in all your interactions with the project.
5+
Please note we have a [code of conduct](http://splunk.github.io/eventgen/CONTRIBUTE.html#code-of-conduct), please follow it in all your interactions with the project.
76

8-
## Pull Request Process
9-
10-
1. Ensure any install or build dependencies are removed before the end of the layer when doing a
11-
build.
12-
2. Update the splunk_eventgen/README/eventgen.conf.spec with details of changes to the interface, this includes new environment
13-
variables, exposed ports, useful file locations and container parameters. Also, update the necessary documentation.
14-
3. Increase the version numbers in splunk_eventgen/version.json. The versioning scheme we use is [SemVer](http://semver.org/).
15-
4. You may merge the Pull Request in once you have the sign-off of two other developers, or if you
16-
do not have permission to do that, you may request the second reviewer to merge it for you.
17-
18-
## Code of Conduct
19-
20-
### Our Pledge
21-
22-
In the interest of fostering an open and welcoming environment, we as
23-
contributors and maintainers pledge to making participation in our project and
24-
our community a harassment-free experience for everyone, regardless of age, body
25-
size, disability, ethnicity, gender identity and expression, level of experience,
26-
nationality, personal appearance, race, religion, or sexual identity and
27-
orientation.
28-
29-
### Our Standards
30-
31-
Examples of behavior that contributes to creating a positive environment
32-
include:
33-
34-
* Using welcoming and inclusive language
35-
* Being respectful of differing viewpoints and experiences
36-
* Gracefully accepting constructive criticism
37-
* Focusing on what is best for the community
38-
* Showing empathy towards other community members
39-
40-
Examples of unacceptable behavior by participants include:
41-
42-
* The use of sexualized language or imagery and unwelcome sexual attention or
43-
advances
44-
* Trolling, insulting/derogatory comments, and personal or political attacks
45-
* Public or private harassment
46-
* Publishing others' private information, such as a physical or electronic
47-
address, without explicit permission
48-
* Other conduct which could reasonably be considered inappropriate in a
49-
professional setting
50-
51-
### Our Responsibilities
52-
53-
Project maintainers are responsible for clarifying the standards of acceptable
54-
behavior and are expected to take appropriate and fair corrective action in
55-
response to any instances of unacceptable behavior.
56-
57-
Project maintainers have the right and responsibility to remove, edit, or
58-
reject comments, commits, code, wiki edits, issues, and other contributions
59-
that are not aligned to this Code of Conduct, or to ban temporarily or
60-
permanently any contributor for other behaviors that they deem inappropriate,
61-
threatening, offensive, or harmful.
62-
63-
### Scope
64-
65-
This Code of Conduct applies both within project spaces and in public spaces
66-
when an individual is representing the project or its community. Examples of
67-
representing a project or community include using an official project e-mail
68-
address, posting via an official social media account, or acting as an appointed
69-
representative at an online or offline event. Representation of a project may be
70-
further defined and clarified by project maintainers.
71-
72-
### Enforcement
73-
74-
Instances of abusive, harassing, or otherwise unacceptable behavior may be
75-
reported by contacting the project team at tonyl@splunk.com. All
76-
complaints will be reviewed and investigated and will result in a response that
77-
is deemed necessary and appropriate to the circumstances. The project team is
78-
obligated to maintain confidentiality with regard to the reporter of an incident.
79-
Further details of specific enforcement policies may be posted separately.
80-
81-
Project maintainers who do not follow or enforce the Code of Conduct in good
82-
faith may face temporary or permanent repercussions as determined by other
83-
members of the project's leadership.
84-
85-
### Attribution
86-
87-
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
88-
available at [http://contributor-covenant.org/version/1/4][version]
89-
90-
[homepage]: http://contributor-covenant.org
91-
[version]: http://contributor-covenant.org/version/1/4/
927

938
### Past / Active(marked as *) Contributors
949
bbingham*
9510
arctan5x*
9611
jmeixensperger*
12+
li-wu*
13+
GordonWang*
9714
coccyx
9815
Jaykul
9916
allanwsplk

Makefile

Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ XLARGE ?= 'tests/xlarge'
1313
NEWLY_ADDED_PY_FILES = $(shell git ls-files -o --exclude-standard | grep -E '\.py$$')
1414
CHANGED_ADDED_PY_FILES = $(shell git ls-files -mo --exclude-standard | grep -E '\.py$$')
1515

16-
.PHONY: tests, lint, format
16+
.PHONY: tests, lint, format, docs
1717

1818
all: egg
1919

@@ -24,7 +24,7 @@ image: setup_eventgen egg
2424
rm splunk_eventgen/default/eventgen_engine.conf || true
2525
docker build -f dockerfiles/Dockerfile . -t eventgen
2626

27-
test: egg image test_helper test_collection_cleanup
27+
test: egg image test_helper run_tests test_collection_cleanup
2828

2929
test_helper:
3030
docker run -d -t --net=host -v /var/run/docker.sock:/var/run/docker.sock --name ${EVENTGEN_TEST_IMAGE} eventgen:latest cat
@@ -41,15 +41,10 @@ test_helper:
4141
@echo 'Installing test requirements'
4242
docker exec -i ${EVENTGEN_TEST_IMAGE} /bin/sh -c "pip install -r $(shell pwd)/tests/requirements.txt" || true
4343

44+
run_tests:
4445
@echo 'Running the super awesome tests'
4546
docker exec -i ${EVENTGEN_TEST_IMAGE} /bin/sh -c "cd $(shell pwd); python tests/run_tests.py ${SMALL} ${MEDIUM} ${LARGE} ${XLARGE}" || true
4647

47-
echo 'Collecting results'
48-
#TODO: Should be paramaterized or generalized so that we don't need to add this here
49-
docker cp ${EVENTGEN_TEST_IMAGE}:$(shell pwd)/tests_results.xml tests_results.xml || echo "no tests_results.xml" || true
50-
51-
docker stop ${EVENTGEN_TEST_IMAGE} || true
52-
5348
test_collection_cleanup:
5449
@echo 'Collecting results'
5550
#TODO: Should be paramaterized or generalized so that we don't need to add this here
@@ -97,9 +92,7 @@ run_controller: eg_network
9792
docker run --name eg_controller --network eg_network -d -p 5672:5672 -p 15672:15672 -p 9500:9500 eventgen:latest controller
9893

9994
docs:
100-
npm install -g gitbook-serve
101-
cd docs/
102-
gitbookserve
95+
cd docs/; bundle install; bundle exec jekyll serve
10396

10497
build_spl: clean
10598
python -m splunk_eventgen build --destination ./
@@ -124,3 +117,9 @@ else
124117
@yapf -i $(NEWLY_ADDED_PY_FILES)
125118
endif
126119

120+
lint-all:
121+
@flake8 .
122+
123+
format-all:
124+
@isort -rc .
125+
@yapf -r -i .

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,4 +33,4 @@ The Splunk Event Generator is licensed under the Apache License 2.0. Details can
3333
### Support
3434

3535
This software is released as-is. Splunk provides no warranty and no support on this software.
36-
If you have any issues with the software, please file an issue (https://github.com/splunk/eventgen/issues/new)
36+
If you have any issues with the software, please read over the [guidelines](http://splunk.github.io/eventgen/FILE_ISSUES.md) and file an issue.

docs/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
_site

docs/ARCHITECTURE.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ These requirements have led to this new architecture in verison 4. Eventgen is
1919
* Handles flattening of configs from global defaults and individual config files
2020
* Loading Plugins
2121
* Load plugins from the `lib/plugins` directory or App's `bin` directories, with three types of Plugins: Rating, Generator and Output
22-
* For more detail on writing plugins, see the [Plugins documentation](Plugins.md)
22+
* For more detail on writing plugins, see the [Plugins documentation](PLUGINS.md)
2323
* Creating timers for each sample
2424
* For samples with queueable generators (meaning we can have multiple threads generating sample simultaneously), the timer wakes up every interval and puts a task in the queue to be generated by a generator worker
2525
* For samples which are not queueable (meaning we can only run one simultaneous thread of the generator), the thread runs the generator plugin single threaded and handles sleeping the proper amount to manage intervals.
@@ -38,13 +38,13 @@ Depending on tunable parameters, these can be threads or processes and can eithe
3838

3939
# Scaling
4040

41-
This queue architecture allows significant advantages by distributing processing. At each timer execution a generation job gets put in the queue. This queue by default is a Python Queue (either in queue.Queue or multiprocessing.Queue). For details, see the [Performance guide](Performance.md)
41+
This queue architecture allows significant advantages by distributing processing. At each timer execution a generation job gets put in the queue. This queue by default is a Python Queue (either in queue.Queue or multiprocessing.Queue). For details, see the [Performance guide](PERFORMANCE.md)
4242

4343
# Testing
4444

4545
Given the complexity and the reimplementation of a number of features during refactors, we've also built out a number of test examples. We currently don't have test scripts built with assertions etc, but there are a series of configs and samples under the `tests/` directory which should demonstrate and allow testing of basic functionality. Check out the ``tests/sample_eventgen_conf`` directory in the project for commonly-used plugins and configurations. You can execute a test config by doing:
4646

47-
python bin/eventgen.py generate tests/<dir>/<test>.conf
47+
python -m splunk_eventgen generate tests/<dir>/<test>.conf
4848

4949
# Server-Controller Architecture
5050

0 commit comments

Comments
 (0)