Skip to content

Commit 2ffc522

Browse files
abeatrixabeatrix
authored
Create new base cluster (#1)
1 parent d8f49e4 commit 2ffc522

File tree

431 files changed

+14219
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

431 files changed

+14219
-0
lines changed

‎.gitignore‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
cluster
2+
cluster.yaml
3+
generated-cluster
4+
.env
5+
.envrc
6+
.cert
7+
.key

‎README.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Sourcegraph on Kubernetes with Kustomize
2+
3+
[![sourcegraph: search](https://img.shields.io/badge/sourcegraph-search-brightgreen.svg)](https://sourcegraph.com/github.com/sourcegraph/deploy-sourcegraph) [![master build status](https://badge.buildkite.com/018ed23ed79d7297e7dd109b745597c58d875323fb06e81786.svg?branch=master)](https://buildkite.com/sourcegraph/deploy-sourcegraph)
4+
5+
## **WARNING: All work in this repository is WIP and should not be used until further notice.**
6+
7+
Deploying Sourcegraph into a Kubernetes cluster is for organizations that need highly scalable and
8+
available code search and code intelligence. This repository contains documentation for creating,
9+
updating, and maintaining a Sourcegraph cluster using Kustomize.
10+
11+
For product and [pricing](https://about.sourcegraph.com/pricing/) information, visit
12+
[about.sourcegraph.com](https://about.sourcegraph.com) or [contact
13+
us](https://about.sourcegraph.com/contact/sales) for more information.
14+
15+
- [Installing](https://docs.sourcegraph.com/admin/deploy/kubernetes/kustomize)
16+
- [Configuring](https://docs.sourcegraph.com/admin/deploy/kubernetes/kustomize/configure)
17+
- [Updating](https://docs.sourcegraph.com/admin/updates/kubernetes)
18+
- [Scaling](https://docs.sourcegraph.com/admin/deploy/scale)- general advice on scaling services
19+
- [Resource estimator: instance size](https://docs.sourcegraph.com/admin/deploy/instance-size) - specific resource values for your instance based on your instance size
20+
- [Troubleshooting](https://docs.sourcegraph.com/admin/install/kubernetes/troubleshoot)
21+
- [Enterprise Getting Started Guide](https://docs.sourcegraph.com/adopt/enterprise_getting_started_guide#kubernetes-admin) and [Admin guide](https://docs.sourcegraph.com/admin) - useful guides for Sourcegraph admins
22+
- [Metrics](https://docs.sourcegraph.com/admin/observability/metrics) - guidance of metrics that can be used for monitoring Sourcegraph
23+
24+
## Deploying
25+
26+
🚨 **The `master` branch tracks development. Use the branch of this repository corresponding to the version of Sourcegraph you wish to deploy, e.g. `git checkout v4.5.0`.**
27+
28+
Always refer to the [Sourcegraph Kustomize docs](https://docs.sourcegraph.com/admin/deploy/kubernetes/kustomize) for the latest instructions.
29+
30+
## Upgrading
31+
32+
When upgrading Sourcegraph, please check [upgrading docs](https://docs.sourcegraph.com/admin/updates/kubernetes) to confirm if manual migrations is necessary.
33+
34+
## Contributing
35+
36+
We've made our deployment configurations open source to better serve our customers' needs. If there is anything we can do to make deploying Sourcegraph easier just [open an issue (in sourcegraph/sourcegraph)](https://github.com/sourcegraph/sourcegraph/issues/new?assignees=&labels=deploy-sourcegraph&template=deploy-sourcegraph.md&title=%5Bdeploy-sourcegraph%5D) or a pull request and we will respond promptly!
37+
38+
## Questions & Issues
39+
40+
[Open an issue (in sourcegraph/sourcegraph)](https://github.com/sourcegraph/sourcegraph/issues/new?assignees=&labels=deploy-sourcegraph&template=deploy-sourcegraph.md&title=%5Bdeploy-sourcegraph%5D) or contact us ([email protected]), we are happy to help!
41+
42+
## Licensing
43+
44+
The contents of this repository are open-source licensed. However, it makes reference to non-open-source images and actually running Sourcegraph using this repository falls under Sourcegraph's [enterprise license terms](https://about.sourcegraph.com/pricing/).

‎base/README.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Sourcegraph Kubernetes Base Cluster
2+
3+
The `sourcegraph` directory contains manifests for all services for the Sourcegraph main stack.
4+
5+
The `monitoring` directory contains manifests for all Sourcegraph monitoring services.
6+
7+
## RBAC
8+
9+
In our base cluster, all services run with non-root and non-privileged. Role-Based Access Control (RBAC) resources are also not enabled by default, which results in manual mapping of service addresses through environment variables being the only way to discover services. However, these service addresses have been preconfigured in the base cluster.
10+
11+
To enable automatic service discovery and clean up of temporary cache data, RBAC resources must be created using the `privileged` component or the `enable/service-discovery` component.
12+
13+
### cAdvisor
14+
15+
For the cadvisor to function properly in your deployment, a Kubernetes cluster with role-based access control (RBAC) enabled is required.
16+
17+
If using cluster roles and cluster rolebinding RBAC is not feasible, you may choose to deploy Sourcegraph without cadvisor as it will not work in your cluster without RBAC enabled.
18+
19+
## Deploy Sourcegraph
20+
21+
See the [Sourcegraph Kustomize docs](https://docs.sourcegraph.com/admin/deploy/kubernetes/kustomize) for the latested instructions.

‎base/kustomization.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
resources:
4+
- monitoring
5+
- sourcegraph

‎base/monitoring/README.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Sourcegraph Monitoring Stack
2+
3+
The monitoring stacks include the following services:
4+
5+
- grafana
6+
- node-exporter
7+
- prometheus
8+
- otel-collector
9+
10+
cAdvisor is not included in the monitoring stack by default.
11+
12+
## cAdvisor
13+
14+
cAdvisor must be run as root in a Kubernetes cluster with role-based access control (RBAC) enabled to collect metrics.
15+
16+
You can add the cadvisor component to your overlay to deploy cAdvisor.

‎base/monitoring/cadvisor/README.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# cAdvisor
2+
3+
[cAdvisor](https://github.com/google/cadvisor) provides container users an understanding of the resource usage and performance characteristics of their running containers. It is a running daemon that collects, aggregates, processes, and exports information about running containers.
4+
5+
cAdvisor is part of the default Sourcegraph cluster installation, and deployed as a [Kubernetes DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/). This setup is based on the [official cAdvisor Kubernetes Daemonset configuration](https://github.com/google/cadvisor/tree/master/deploy/kubernetes). We use our own cAdvisor deployment over the built-in metrics exported by Kubernetes because the latter is often outdated and needs to be kept in sync with our [Docker-Compose deployments](https://docs.sourcegraph.com/admin/install/docker-compose). This setup allows us to have standard dashboards across all Sourcegraph deployments.
6+
7+
Note that the `sourcegraph/cadvisor` Docker images come with a set of default flags to help reduce memory usage and load on Prometheus - see [our Dockerfile](https://github.com/sourcegraph/sourcegraph/blob/master/docker-images/cadvisor/Dockerfile) for more details.
8+
9+
## Namespaces
10+
11+
If you are deploying Sourcegraph to a non-default namespace, you'll have to change the namespace specified in
12+
[cadvisor.ClusterRoleBinding.yaml](cadvisor.ClusterRoleBinding.yaml) to the one that you created. You can do this by editing the namespace directly, or by using the [namespaced overlay](../../configure/../overlays/namespaced/README.md).

‎base/monitoring/cadvisor/cadvisor.DaemonSet.yaml‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
apiVersion: apps/v1
2+
kind: DaemonSet
3+
metadata:
4+
annotations:
5+
description: DaemonSet to ensure all nodes run a cAdvisor pod.
6+
seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
7+
labels:
8+
deploy: sourcegraph
9+
sourcegraph-resource-requires: cluster-admin
10+
app.kubernetes.io/component: cadvisor
11+
name: cadvisor
12+
spec:
13+
selector:
14+
matchLabels:
15+
app: cadvisor
16+
template:
17+
metadata:
18+
annotations:
19+
description: Collects and exports container metrics.
20+
prometheus.io/port: "48080"
21+
sourcegraph.prometheus/scrape: "true"
22+
labels:
23+
deploy: sourcegraph
24+
app: cadvisor
25+
spec:
26+
serviceAccountName: cadvisor
27+
containers:
28+
- name: cadvisor
29+
image: index.docker.io/sourcegraph/cadvisor:187572_2022-12-06_cbecc5321c7d@sha256:755748f2f9b00d8f70bd65349e85235585bdf1a663e26198c8eaf91dfd5636e1
30+
args:
31+
# Kubernetes-specific flags below (other flags are baked into the Docker image)
32+
#
33+
# disable container labels to allow whitelisting to reduce noise
34+
- --store_container_labels=false
35+
- --whitelisted_container_labels=io.kubernetes.container.name,io.kubernetes.pod.name,io.kubernetes.pod.namespace,io.kubernetes.pod.uid
36+
# it is safe to uncomment this option if you use docker as your container runtime to reduce noise
37+
# - --docker_only
38+
resources:
39+
requests:
40+
memory: 200Mi
41+
cpu: 150m
42+
limits:
43+
memory: 2000Mi
44+
cpu: 300m
45+
volumeMounts:
46+
- name: rootfs
47+
mountPath: /rootfs
48+
readOnly: true
49+
- name: var-run
50+
mountPath: /var/run
51+
readOnly: true
52+
- name: sys
53+
mountPath: /sys
54+
readOnly: true
55+
- name: docker
56+
mountPath: /var/lib/docker
57+
readOnly: true
58+
- name: disk
59+
mountPath: /dev/disk
60+
readOnly: true
61+
ports:
62+
- name: http
63+
containerPort: 48080
64+
protocol: TCP
65+
securityContext:
66+
privileged: true
67+
automountServiceAccountToken: false
68+
terminationGracePeriodSeconds: 30
69+
volumes:
70+
- name: rootfs
71+
hostPath:
72+
path: /
73+
- name: var-run
74+
hostPath:
75+
path: /var/run
76+
- name: sys
77+
hostPath:
78+
path: /sys
79+
- name: docker
80+
hostPath:
81+
path: /var/lib/docker
82+
- name: disk
83+
hostPath:
84+
path: /dev/disk

‎base/monitoring/cadvisor/cadvisor.ServiceAccount.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: v1
2+
kind: ServiceAccount
3+
metadata:
4+
labels:
5+
app: cadvisor
6+
category: rbac
7+
deploy: sourcegraph
8+
sourcegraph-resource-requires: cluster-admin
9+
app.kubernetes.io/component: cadvisor
10+
name: cadvisor

‎base/monitoring/cadvisor/kustomization.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
resources:
4+
- cadvisor.DaemonSet.yaml
5+
- cadvisor.ServiceAccount.yaml

‎base/monitoring/grafana/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Grafana
2+
3+
[Grafana](https://https://grafana.com/) is an open-source analytics dashboard application.
4+
5+
A Grafana instance is part of the default Sourcegraph cluster installation.
6+
Learn more about Grafana in the [metrics and dashboards guide](https://docs.sourcegraph.com/admin/observability/metrics).

0 commit comments

Comments
 (0)