Skip to content

Commit 031570d

Browse files
luckyj5luckyj5
authored
Merge to master for 1.1.0 release (#182)
* Added CI integration. (#170) * Per app indexes feature(#132) * Refactor org/space/app caching (#172) * Convert from static to exponential backoff (#173) * Added issue and er templates (#178) * Remove Splunk version field from Tile (#174) * Updated readme and manifest template for 1.1 release (#180)
1 parent 853346d commit 031570d

25 files changed

+834
-119
lines changed

‎.circleci/ci_nozzle_manifest.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
applications:
3+
- name: splunk-firehose-nozzle
4+
memory: 512M
5+
instances: 2
6+
buildpack: go_buildpack
7+
cmd: splunk-firehose-nozzle
8+
env:
9+
GOPACKAGENAME: main
10+
API_ENDPOINT:
11+
API_USER:
12+
API_PASSWORD:
13+
SPLUNK_HOST:
14+
SPLUNK_TOKEN:
15+
SPLUNK_INDEX:
16+
SKIP_SSL_VALIDATION_CF: true
17+
SKIP_SSL_VALIDATION_SPLUNK: true
18+
JOB_NAME: splunk-nozzle
19+
JOB_INDEX: -1
20+
JOB_HOST: localhost
21+
ADD_APP_INFO: true
22+
IGNORE_MISSING_APP: true
23+
MISSING_APP_CACHE_INVALIDATE_TTL: 3600s
24+
APP_CACHE_INVALIDATE_TTL: 86440s
25+
APP_LIMITS: 1000
26+
BOLTDB_PATH: cache.db
27+
EVENTS: ValueMetric,CounterEvent,Error,LogMessage,HttpStartStop,ContainerMetric
28+
EXTRA_FIELDS: name:update-ci-test
29+
FIREHOSE_SUBSCRIPTION_ID: splunk-ci
30+
FLUSH_INTERVAL: 5s
31+
CONSUMER_QUEUE_SIZE: 10000
32+
HEC_BATCH_SIZE: 1000
33+
HEC_RETRIES: 5
34+
HEC_WORKERS: 8
35+
DEBUG: false
36+
ENABLE_EVENT_TRACING: true

‎.circleci/config.yml‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
version: 2 # use CircleCI 2.0
2+
jobs:
3+
build:
4+
docker:
5+
- image: circleci/golang:1.12
6+
working_directory: /go/src/github.com/cloudfoundry-community/splunk-firehose-nozzle
7+
steps: # steps that comprise the `build` job
8+
- checkout # check out source code to working directory
9+
- run:
10+
name: Install Dependencies
11+
command: |
12+
curl https://glide.sh/get | sh
13+
go get -t ./...
14+
- run:
15+
name: Builder
16+
command: make build
17+
- run:
18+
name: Run tests
19+
command: |
20+
make testall
21+
cp splunk-firehose-nozzle /tmp
22+
- persist_to_workspace:
23+
root: /tmp
24+
paths:
25+
- splunk-firehose-nozzle
26+
27+
deploy-nozzle:
28+
docker:
29+
- image: circleci/golang:1.12
30+
working_directory: /go/src/github.com/cloudfoundry-community/splunk-firehose-nozzle
31+
steps: # steps that comprise the `deploy` job
32+
- attach_workspace:
33+
at: /tmp
34+
- checkout # check out source code to working directory
35+
- run:
36+
name: Install dependencies
37+
command: |
38+
curl https://glide.sh/get | sh
39+
go get -t ./...
40+
cp -R /tmp/splunk-firehose-nozzle .
41+
- run:
42+
name: Deploy nozzle
43+
command: |
44+
.circleci/update_manifest.sh
45+
.circleci/pre-req.sh
46+
cf push -f .circleci/ci_nozzle_manifest.yml -u process --random-route
47+
- run:
48+
name: Teardown deployment env
49+
command: |
50+
sleep 10
51+
echo "Teardown deployment env"
52+
cf delete splunk-firehose-nozzle -f
53+
cf delete-org splunk-ci -f
54+
55+
tile-builder:
56+
docker:
57+
- image: circleci/golang:1.12
58+
working_directory: /go/src/github.com/cloudfoundry-community/splunk-firehose-nozzle
59+
steps:
60+
- attach_workspace:
61+
at: /tmp
62+
- checkout
63+
- run:
64+
name: Tile builder
65+
command: |
66+
cp -R /tmp/splunk-firehose-nozzle .
67+
.circleci/tile-builder.sh
68+
- run:
69+
name: Push tile to internal s3
70+
command: |
71+
cp -R /tmp/splunk-firehose-nozzle .
72+
.circleci/push_tile.sh
73+
74+
workflows:
75+
version: 2
76+
build-and-deploy-nozzle:
77+
jobs:
78+
- build
79+
- deploy-nozzle:
80+
requires:
81+
- build
82+
filters:
83+
branches:
84+
only:
85+
- develop
86+
- master
87+
- tile-builder:
88+
requires:
89+
- deploy-nozzle
90+
filters:
91+
branches:
92+
only: master

‎.circleci/pre-req.sh‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
#!/usr/bin/env bash
2+
set -e
3+
#update deps
4+
make
5+
#Install CF CLI
6+
wget -q -O - https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key | sudo apt-key add -
7+
echo "deb https://packages.cloudfoundry.org/debian stable main" | sudo tee /etc/apt/sources.list.d/cloudfoundry-cli.list
8+
#Add support for https apt sources
9+
sudo apt-get install apt-transport-https ca-certificates
10+
sudo apt-get update
11+
sudo apt-get install cf-cli
12+
#CF Login
13+
cf login --skip-ssl-validation -a $API_ENDPOINT -u $API_USER -p $API_PASSWORD -o system -s system
14+
#Create splunk-ci org and space
15+
if [ "`cf o | grep "splunk-ci"`" == "splunk-ci" ]; then
16+
echo "Its here"
17+
cf target -o "splunk-ci" -s "splunk-ci"
18+
else
19+
cf create-org splunk-ci
20+
cf target -o splunk-ci
21+
cf create-space splunk-ci
22+
cf target -o "splunk-ci" -s "splunk-ci"
23+
fi

‎.circleci/push_tile.sh‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/usr/bin/env bash
2+
set -e
3+
sudo apt-get install -y python-pip libpython-dev > /dev/null 2>&1
4+
echo "Installing aws cli..."
5+
sudo pip install awscli > /dev/null 2>&1
6+
echo "Push Splunk tile to s3..."
7+
aws s3 cp tile/product/splunk-nozzle-*.pivotal s3://pcf-ci-artifacts/

‎.circleci/tile-builder.sh‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#!/usr/bin/env bash
2+
set -e
3+
wget https://github.com/cf-platform-eng/tile-generator/releases/download/v13.0.2/pcf_linux-64bit > /dev/null 2>&1
4+
chmod +x pcf_linux-64bit
5+
sudo mv pcf_linux-64bit /usr/local/bin/tile
6+
sudo apt install python-pip > /dev/null 2>&1
7+
echo "installing vurtualenv"
8+
sudo /usr/bin/easy_install virtualenv > /dev/null 2>&1
9+
virtualenv -p python tile-generator-env
10+
source tile-generator-env/bin/activate
11+
echo "Installing tile-generator..."
12+
pip install tile-generator
13+
cd tile
14+
echo "Installing bosh..."
15+
wget https://github.com/cloudfoundry/bosh-cli/releases/download/v5.5.0/bosh-cli-5.5.0-linux-amd64 > /dev/null 2>&1
16+
mv bosh-cli-5.5.0-linux-amd64 bosh
17+
chmod +x ./bosh
18+
sudo mv ./bosh /usr/local/bin/bosh
19+
echo "Building PCF Tile for Splunk-firehose-nozzle"
20+
tile build

‎.circleci/update_manifest.sh‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#!/usr/bin/env bash
2+
set -e
3+
#Set below params in CircleCI env variable settings
4+
# API_ENDPOINT, API_USER, API_PASSWORD, SPLUNK_TOKEN, SPLUNK_HOST, SPLUNK_INDEX
5+
#Update manifest for deployment
6+
sed -i 's@API_ENDPOINT:.*@'"API_ENDPOINT: $API_ENDPOINT"'@' .circleci/ci_nozzle_manifest.yml
7+
sed -i 's@API_USER:.*@'"API_USER: $API_USER"'@' .circleci/ci_nozzle_manifest.yml
8+
sed -i 's@API_PASSWORD:.*@'"API_PASSWORD: $API_PASSWORD"'@' .circleci/ci_nozzle_manifest.yml
9+
sed -i 's@SPLUNK_HOST:.*@'"SPLUNK_HOST: $SPLUNK_HOST"'@' .circleci/ci_nozzle_manifest.yml
10+
sed -i 's@SPLUNK_TOKEN:.*@'"SPLUNK_TOKEN: $SPLUNK_TOKEN"'@' .circleci/ci_nozzle_manifest.yml
11+
sed -i 's@SPLUNK_INDEX:.*@'"SPLUNK_INDEX: $SPLUNK_INDEX"'@' .circleci/ci_nozzle_manifest.yml
12+
#copy nozzle binary from shared workspace
13+
cp /tmp/splunk-firehose-nozzle .

‎.github/ISSUE_TEMPLATE/bug_report.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
---
2+
Name: Bug report
3+
About: Report a bug encountered while operating Splunk Firehose Nozzle
4+
Title: "[BUG]"
5+
Labels: ''
6+
Assignees: ''
7+
8+
---
9+
10+
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!
11+
-->
12+
13+
**What happened**
14+
A clear and concise description of what the bug is.
15+
16+
**What you expected to happen**:
17+
A clear and concise description of what you expected to happen.
18+
19+
**How to reproduce it (as minimally and precisely as possible)**:
20+
21+
**Anything else we need to know?**:
22+
Add any other context about the problem here.
23+
24+
**Environment**:
25+
- Cloud Foundry version
26+
- Go version
27+
- Splunk version:
28+
- Others:

‎.github/ISSUE_TEMPLATE/enhancement_request.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
Name: Enhancement Request
3+
About: Suggest an enhancement to the Splunk Firehose Nozzle project
4+
Title: "[ENHANCEMENT]"
5+
Labels: ''
6+
Assignees: ''
7+
8+
---
9+
10+
<<!-- Please only use this template for submitting enhancement requests -->
11+
12+
**What would you like to be added**
13+
A clear and concise description of what you want to happen.
14+
15+
**Why is this needed**:
16+
A clear and concise description of your use case.
17+
18+
**Additional context**
19+
Add any other context about the enhancement request here.

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ tools/dump_app_info/dump_app_info
44
tools/data_gen/data_gen
55

66

7+
78
cache.db
89
*.out
910

0 commit comments

Comments
 (0)