Skip to content

Commit 267c65e

Browse files
nitishtnitisht
authored
Merge branch 'main' into time-parsing
2 parents 7f5c45b + eea4841 commit 267c65e

File tree

8 files changed

+188
-108
lines changed

8 files changed

+188
-108
lines changed

‎README.md

Lines changed: 31 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -2,67 +2,49 @@
22
<picture>
33
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/parseablehq/.github/main/images/logo-dark.png">
44
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/parseablehq/.github/main/images/logo.svg">
5-
<a href="https://www.parseable.io" target="_blank"><img src="https://raw.githubusercontent.com/parseablehq/.github/main/images/logo.svg" alt="Parseable logo" /></a>
5+
<a href="https://www.parseable.com" target="_blank"><img src="https://raw.githubusercontent.com/parseablehq/.github/main/images/logo.svg" alt="Parseable logo" /></a>
66
</picture>
7-
<br>
8-
The <i>versatile</i> log database for search, analysis, and monitoring
97
</h2>
108

119
<div align="center">
1210

1311
[![Docker Pulls](https://img.shields.io/docker/pulls/parseable/parseable?logo=docker&label=Docker%20Pulls)](https://hub.docker.com/r/parseable/parseable)
1412
[![Slack](https://img.shields.io/badge/slack-brightgreen.svg?logo=slack&label=Community&style=flat&color=%2373DC8C&)](https://logg.ing/community)
15-
[![Docs](https://img.shields.io/badge/stable%20docs-parseable.io%2Fdocs-brightgreen?style=flat&color=%2373DC8C&label=Docs)](https://logg.ing/docs)
13+
[![Docs](https://img.shields.io/badge/stable%20docs-parseable.com%2Fdocs-brightgreen?style=flat&color=%2373DC8C&label=Docs)](https://logg.ing/docs)
1614
[![Build](https://img.shields.io/github/checks-status/parseablehq/parseable/main?style=flat&color=%2373DC8C&label=Checks)](https://github.com/parseablehq/parseable/actions)
1715

18-
[Key Concepts](https://www.parseable.io/docs/concepts) | [Features](https://github.com/parseablehq/parseable#rocket-highlights) | [Documentation](https://www.parseable.io/docs) | [Demo](https://demo.parseable.com/login?q=eyJ1c2VybmFtZSI6ImFkbWluIiwicGFzc3dvcmQiOiJhZG1pbiJ9) | [Integrations](https://www.parseable.io/docs/category/integrations) | [FAQ](https://www.parseable.io/docs/faq)
16+
[Key Concepts](https://www.parseable.com/docs/concepts) | [Features](https://github.com/parseablehq/parseable#rocket-highlights) | [Documentation](https://www.parseable.com/docs) | [Demo](https://demo.parseable.com/login?q=eyJ1c2VybmFtZSI6ImFkbWluIiwicGFzc3dvcmQiOiJhZG1pbiJ9) | [FAQ](https://www.parseable.com/docs/faq)
1917

2018
</div>
2119

22-
Parseable is a **cloud native log analytics platform, with a focus on simplicity & resource efficiency**. Parseable is useful for use cases where **complete data ownership, security and privacy are paramount**.
20+
Parseable is a disk **_less_**, cloud native database for logs, observability, security, and compliance. Parseable is built with focus on simplicity & resource efficiency. Parseable is useful for use cases where **complete data ownership, data security and privacy are paramount**.
2321

24-
To experience Parseable UI, checkout [demo.parseable.com ↗︎](https://demo.parseable.com/login?q=eyJ1c2VybmFtZSI6ImFkbWluIiwicGFzc3dvcmQiOiJhZG1pbiJ9). You can also view the [demo video ↗︎](https://www.parseable.com/video.mp4).
22+
To experience Parseable UI, checkout [demo.parseable.com ↗︎](https://demo.parseable.com/login?q=eyJ1c2VybmFtZSI6ImFkbWluIiwicGFzc3dvcmQiOiJhZG1pbiJ9).
2523

26-
## QuickStart :zap:
24+
## Quickstart :zap:
25+
26+
### Run Parseable
2727

2828
<details>
2929
<summary><a href="https://www.parseable.com/docs/docker-quick-start">Docker Image</a></summary>
3030
<p>
3131

32-
You can <a href="https://www.parseable.com/docs/docker-quick-start">get started with Parseable Docker</a> with a simple Docker run and then send data via cURL to understand how you can ingest data to Parseable. Below is the command to run Parseable in local storage mode with Docker.
32+
Get started with Parseable Docker with a single command:
3333

3434
```bash
3535
docker run -p 8000:8000 \
3636
containers.parseable.com/parseable/parseable:latest \
3737
parseable local-store
3838
```
3939

40-
Once this runs successfully, you'll see dashboard at [http://localhost:8000 ↗︎](http://localhost:8000). You can login to the dashboard default credentials `admin`, `admin`.
41-
42-
To ingest data, run the below command. This will send logs to the `demo` stream. You can see the logs in the dashboard.
43-
44-
```bash
45-
curl --location --request POST 'http://localhost:8000/api/v1/ingest' \
46-
--header 'X-P-Stream: demo' \
47-
--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
48-
--header 'Content-Type: application/json' \
49-
--data-raw '[
50-
{
51-
"id": "434a5f5e-2f5f-11ed-a261-0242ac120002",
52-
"datetime": "24/Jun/2022:14:12:15 +0000",
53-
"host": "153.10.110.81"
54-
}
55-
]'
56-
```
57-
5840
</p>
5941
</details>
6042

6143
<details>
6244
<summary><a href="https://www.parseable.com/docs/docker-quick-start">Executable Binary</a></summary>
6345
<p>
6446

65-
You can download and run the Parseable binary on your laptop.
47+
Download and run the Parseable binary on your laptop:
6648

6749
- Linux or MacOS
6850

@@ -76,9 +58,12 @@ curl -fsSL https://logg.ing/install | bash
7658
powershell -c "irm https://logg.ing/install-windows | iex"
7759
```
7860

79-
Once this runs successfully, you'll see dashboard at [http://localhost:8000 ↗︎](http://localhost:8000). You can login to the dashboard default credentials `admin`, `admin`.
61+
</p>
62+
</details>
63+
64+
### Ingestion and query
8065

81-
To ingest data, run the below command. This will send logs to the `demo` stream. You can see the logs in the dashboard.
66+
Once you have Parseable running, ingest data with the below command. This will send logs to the `demo` stream. You can see the logs in the dashboard.
8267

8368
```bash
8469
curl --location --request POST 'http://localhost:8000/api/v1/ingest' \
@@ -94,16 +79,24 @@ curl --location --request POST 'http://localhost:8000/api/v1/ingest' \
9479
]'
9580
```
9681

97-
</p>
98-
</details>
82+
Access the Parseable UI at [http://localhost:8000 ↗︎](http://localhost:8000). You can login to the dashboard default credentials `admin`, `admin`.
83+
84+
## Getting started :bulb:
85+
86+
For quickstart, refer the [quickstart section ↗︎](#quickstart-zap).
87+
88+
This section elaborates available options to run Parseable in production or development environments.
89+
90+
- Distributed Parseable on Kubernetes: [Helm Installation](https://www.parseable.com/docs/server/installation/distributed/setup-distributed-parseable-on-kubernetes-via-helm).
91+
- Distributed Parseable on AWS EC2 / VMs / Linux: [Binary Installation](https://www.parseable.com/docs/server/installation/distributed/setup-systemd-service-for-distributed-parseable-server).
9992

10093
## Features :rocket:
10194

10295
- [High availability & Cluster mode ↗︎](https://www.parseable.com/docs/concepts/distributed-architecture)
10396
- [Local cache ↗︎](https://www.parseable.com/docs/features/tiering)
104-
- [Alerts ↗︎](https://www.parseable.io/docs/alerts)
105-
- [Role based access control ↗︎](https://www.parseable.io/docs/rbac)
106-
- [OAuth2 support ↗︎](https://www.parseable.io/docs/oidc)
97+
- [Alerts ↗︎](https://www.parseable.com/docs/alerts)
98+
- [Role based access control ↗︎](https://www.parseable.com/docs/rbac)
99+
- [OAuth2 support ↗︎](https://www.parseable.com/docs/oidc)
107100
- [LLM integration ↗︎](https://www.parseable.com/docs/integrations/llm-based-sql-generation)
108101
- [OpenTelemetry support ↗︎](https://www.parseable.com/docs/opentelemetry)
109102

@@ -121,14 +114,14 @@ Traditionally, logging has been seen as a text search problem. Log volumes were
121114

122115
But with log data growing exponentially, today's log data challenges involve whole lot more – Data ingestion, storage, and observation, all at scale. We are building Parseable to address these challenges.
123116

124-
## Verify Image Build Provenance Attestation
125-
All the image builds/pushes are attested for build provenance and integrity using the [attest-build-provenance](https://github.com/actions/attest-build-provenance) action. The attestations can be verified by having the latest version of [GitHub CLI](https://github.com/cli/cli/releases/latest) installed in your system. Then, execute the following command:
117+
## Verify images :writing_hand:
118+
119+
Parseable builds are attested for build provenance and integrity using the [attest-build-provenance](https://github.com/actions/attest-build-provenance) action. The attestations can be verified by having the latest version of [GitHub CLI](https://github.com/cli/cli/releases/latest) installed in your system. Then, execute the following command:
126120

127121
```sh
128122
gh attestation verify PATH/TO/YOUR/PARSEABLE/ARTIFACT-BINARY -R parseablehq/parseable
129123
```
130124

131-
132125
## Contributing :trophy:
133126

134127
<a href="https://github.com/parseablehq/parseable/graphs/contributors"><img src="https://contrib.rocks/image?repo=parseablehq/parseable" /></a>

‎helm-releases/parseable-1.6.5.tgz

49.1 KB
Binary file not shown.

‎helm/Chart.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ apiVersion: v2
22
name: parseable
33
description: Helm chart for Parseable Server
44
type: application
5-
version: 1.6.4
5+
version: 1.6.5
66
appVersion: "v1.6.3"
77
maintainers:
88
- name: Parseable Team

‎helm/templates/ingestor-statefulset.yaml

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ spec:
4242
imagePullPolicy: {{ .Values.parseable.image.pullPolicy }}
4343
args:
4444
- /usr/bin/parseable
45-
- {{ .Values.parseable.store }}
45+
- {{ if eq .Values.parseable.store "gcs-store" }}"s3-store"{{ else }}{{ .Values.parseable.store | quote }}{{ end }}
4646
- --ingestor-endpoint=$(HOSTNAME).{{ include "parseable.fullname" . }}-ingestor-headless.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.parseable.highAvailability.ingestor.port }}
4747
env:
4848
{{- range $key, $value := .Values.parseable.highAvailability.ingestor.env }}
@@ -54,7 +54,9 @@ spec:
5454
apiVersion: v1
5555
fieldPath: metadata.name
5656
{{- end }}
57-
{{- range $secret := .Values.parseable.s3ModeSecret }}
57+
58+
{{- if and .Values.parseable.s3ModeSecret .Values.parseable.s3ModeSecret.enabled }}
59+
{{- range $secret := .Values.parseable.s3ModeSecret.secrets }}
5860
{{- range $key := $secret.keys }}
5961
{{- $envPrefix := $secret.prefix | default "" | upper }}
6062
{{- $envKey := $key | upper | replace "." "_" | replace "-" "_" }}
@@ -65,7 +67,24 @@ spec:
6567
key: {{ $key }}
6668
{{- end }}
6769
{{- end }}
68-
{{- range $secret := .Values.parseable.blobModeSecret }}
70+
{{- end }}
71+
72+
{{- if and .Values.parseable.gcsModeSecret .Values.parseable.gcsModeSecret.enabled }}
73+
{{- range $secret := .Values.parseable.gcsModeSecret.secrets }}
74+
{{- range $key := $secret.keys }}
75+
{{- $envPrefix := $secret.prefix | default "" | upper }}
76+
{{- $envKey := $key | upper | replace "." "_" | replace "-" "_" }}
77+
- name: {{ $envPrefix }}{{ $envKey | replace "GCS" "S3"}}
78+
valueFrom:
79+
secretKeyRef:
80+
name: {{ $secret.name }}
81+
key: {{ $key }}
82+
{{- end }}
83+
{{- end }}
84+
{{- end }}
85+
86+
{{- if and .Values.parseable.blobModeSecret .Values.parseable.blobModeSecret.enabled }}
87+
{{- range $secret := .Values.parseable.blobModeSecret.secrets }}
6988
{{- range $key := $secret.keys }}
7089
{{- $envPrefix := $secret.prefix | default "" | upper }}
7190
{{- $envKey := $key | upper | replace "." "_" | replace "-" "_" }}
@@ -76,6 +95,7 @@ spec:
7695
key: {{ $key }}
7796
{{- end }}
7897
{{- end }}
98+
{{- end }}
7999
- name: P_MODE
80100
value: "ingest"
81101
ports:

‎helm/templates/querier-statefulset.yaml

Lines changed: 24 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -43,20 +43,22 @@ spec:
4343
{{- toYaml .Values.parseable.securityContext | nindent 8 }}
4444
image: {{ .Values.parseable.image.repository }}:{{ .Values.parseable.image.tag | default .Chart.AppVersion }}
4545
imagePullPolicy: {{ .Values.parseable.image.pullPolicy }}
46-
args:
47-
- /usr/bin/parseable
48-
- {{ .Values.parseable.store }}
46+
args: ["/usr/bin/parseable", {{ if eq .Values.parseable.store "gcs-store" }}"s3-store"{{ else }}{{ .Values.parseable.store | quote }}{{ end }}]
4947
env:
5048
- name: HOSTNAME
5149
valueFrom:
5250
fieldRef:
5351
apiVersion: v1
5452
fieldPath: metadata.name
53+
- name: P_MODE
54+
value: "query"
5555
{{- range $key, $value := .Values.parseable.env }}
5656
- name: {{ $key }}
5757
value: {{ tpl $value $ | quote }}
5858
{{- end }}
59-
{{- range $secret := .Values.parseable.s3ModeSecret }}
59+
60+
{{- if and .Values.parseable.s3ModeSecret .Values.parseable.s3ModeSecret.enabled }}
61+
{{- range $secret := .Values.parseable.s3ModeSecret.secrets }}
6062
{{- range $key := $secret.keys }}
6163
{{- $envPrefix := $secret.prefix | default "" | upper }}
6264
{{- $envKey := $key | upper | replace "." "_" | replace "-" "_" }}
@@ -67,13 +69,24 @@ spec:
6769
key: {{ $key }}
6870
{{- end }}
6971
{{- end }}
70-
- name: P_MODE
71-
value: "query"
72-
{{- if .Values.parseable.persistence.querier.enabled }}
73-
- name: P_HOT_TIER_DIR
74-
value: "/parseable/hot-tier"
7572
{{- end }}
76-
{{- range $secret := .Values.parseable.blobModeSecret }}
73+
74+
{{- if and .Values.parseable.gcsModeSecret .Values.parseable.gcsModeSecret.enabled }}
75+
{{- range $secret := .Values.parseable.gcsModeSecret.secrets }}
76+
{{- range $key := $secret.keys }}
77+
{{- $envPrefix := $secret.prefix | default "" | upper }}
78+
{{- $envKey := $key | upper | replace "." "_" | replace "-" "_" }}
79+
- name: {{ $envPrefix }}{{ $envKey | replace "GCS" "S3"}}
80+
valueFrom:
81+
secretKeyRef:
82+
name: {{ $secret.name }}
83+
key: {{ $key }}
84+
{{- end }}
85+
{{- end }}
86+
{{- end }}
87+
88+
{{- if and .Values.parseable.blobModeSecret .Values.parseable.blobModeSecret.enabled }}
89+
{{- range $secret := .Values.parseable.blobModeSecret.secrets }}
7790
{{- range $key := $secret.keys }}
7891
{{- $envPrefix := $secret.prefix | default "" | upper }}
7992
{{- $envKey := $key | upper | replace "." "_" | replace "-" "_" }}
@@ -84,6 +97,7 @@ spec:
8497
key: {{ $key }}
8598
{{- end }}
8699
{{- end }}
100+
{{- end }}
87101
ports:
88102
- containerPort: 8000
89103
{{- with .Values.readinessProbe }}

‎helm/templates/standalone-deployment.yaml

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ spec:
3636
imagePullPolicy: {{ .Values.parseable.image.pullPolicy }}
3737
# Uncomment to debug
3838
# command: [ "/bin/sh", "-c", "sleep 1000000" ]
39-
args: ["/usr/bin/parseable", {{ .Values.parseable.store | quote }}]
39+
args: ["/usr/bin/parseable", {{ if eq .Values.parseable.store "gcs-store" }}"s3-store"{{ else }}{{ .Values.parseable.store | quote }}{{ end }}]
4040
env:
4141
- name: HOSTNAME
4242
valueFrom:
@@ -75,6 +75,20 @@ spec:
7575
{{- end }}
7676
{{- end }}
7777

78+
{{- if and .Values.parseable.gcsModeSecret .Values.parseable.gcsModeSecret.enabled }}
79+
{{- range $secret := .Values.parseable.gcsModeSecret.secrets }}
80+
{{- range $key := $secret.keys }}
81+
{{- $envPrefix := $secret.prefix | default "" | upper }}
82+
{{- $envKey := $key | upper | replace "." "_" | replace "-" "_" }}
83+
- name: {{ $envPrefix }}{{ $envKey | replace "GCS" "S3"}}
84+
valueFrom:
85+
secretKeyRef:
86+
name: {{ $secret.name }}
87+
key: {{ $key }}
88+
{{- end }}
89+
{{- end }}
90+
{{- end }}
91+
7892
{{- if and .Values.parseable.blobModeSecret .Values.parseable.blobModeSecret.enabled }}
7993
{{- range $secret := .Values.parseable.blobModeSecret.secrets }}
8094
{{- range $key := $secret.keys }}

‎helm/values.yaml

Lines changed: 21 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,7 @@ parseable:
33
repository: containers.parseable.com/parseable/parseable
44
tag: v1.6.3
55
pullPolicy: Always
6-
## object store can be local-store, s3-store or blob-store.
7-
## local needs to be false if set to object store.
6+
## object store can be local-store, s3-store, blob-store or gcs-store.
87
store: local-store
98
## Set to true if you want to deploy Parseable in a HA mode (multiple ingestors)
109
## Please note that highAvailability is not supported in local mode
@@ -116,6 +115,22 @@ parseable:
116115
- s3.secret.key
117116
- s3.bucket
118117
- s3.region
118+
gcsModeSecret:
119+
enabled: false
120+
secrets:
121+
- name: parseable-env-secret
122+
prefix: P_
123+
keys:
124+
- addr
125+
- username
126+
- password
127+
- staging.dir
128+
- fs.dir
129+
- gcs.url
130+
- gcs.access.key
131+
- gcs.secret.key
132+
- gcs.bucket
133+
- gcs.region
119134
serviceAccount:
120135
create: true
121136
name: "parseable"
@@ -295,8 +310,9 @@ vector:
295310
# See fluent-bit helm documentation to learn more:
296311
# https://github.com/fluent/helm-charts/tree/main/charts/fluent-bit
297312
fluent-bit:
298-
enabled: true
313+
enabled: false
299314
kind: DaemonSet
315+
serverHost: parseable.parseable.svc.cluster.local
300316
replicaCount: 1
301317
image:
302318
repository: parseable/fluent-bit
@@ -382,7 +398,7 @@ fluent-bit:
382398
[OUTPUT]
383399
Name parseable
384400
Match kube.*
385-
Server_Host parseable.parseable.svc.cluster.local
401+
Server_Host {{ .Values.serverHost }}
386402
Username admin
387403
Password admin
388404
Server_Port 80
@@ -392,7 +408,7 @@ fluent-bit:
392408
[OUTPUT]
393409
Name parseable
394410
Match k8s_events
395-
Server_Host parseable.parseable.svc.cluster.local
411+
Server_Host {{ .Values.serverHost }}
396412
Server_Port 80
397413
Username admin
398414
Password admin

0 commit comments

Comments
 (0)