first working version

now creating merge request and checking if it's merged to create sentry project and post DSNs

add README.md

add requirements

handle projects with subnamespace and already created sentry projects

do not handle project if it's not in a team namespace

fully functionnal

add makefile and fix code qa

overall code refactoring, optimization and logging

this code is fast enough to be scheduled as a cron instead of
sleeping for 600s

logic:
- we get all MRs for our current user at once
- we run on every group
- we ensure the group exists on sentry
- we iterate on group's projects natively
- we base ourselves on MR counts of each project to quicky guess what needs to be done
- we keep stats and print them at the end of the run

switch action detection from mr count to sentryclirc presence

add run target to makefile

Update .sentryclirc

Udpate .sentryclirc

fix some logic for real deployment, add sentry to our project ;)

handle and report exceptions on mr proposal errors

handle subgroups thanks to @QN

do not create a sentry team per subgroup

try to created sentry teams only once

add Dockerfile and .gitlab-ci.yml for docker image

fix TypeError: 'NoneType' object is not subscriptable

due to:
2020-11-23 09:20:23 root INFO: creating sentry project team-emailing / Banner_Display_Api
{'name': 'Banner_Display_Api', 'slug': 'Banner-Display-Api'}
{'slug': ['This value does not match the required pattern.']}

add helm chart

handle and log create or get sentry project errors

helm/values-production: fix SENTRY_TOKEN env var reference

makefile: add upgrade task

construct sentry project name from path_with_namespace

to include sub groups in sentry project names

dump deployment image tag

apply code QA

makefile: add build, push steps

add activeDeadlineSeconds to 600

fix activeDeadlineSeconds in the yaml

switch to restartPolicy = OnFailure

optimize run time: check only repos that were modified < 24h

switch to version tag 0.5

make tag and image building dynamic

make gitlab2sentry respectful of project default branch

add rate limiting to new created keys

upgrade helm-charts

Add renovate.json

Update renovate.json

Update dependency sentry-sdk to v0.19.5

Update makefile and gitignore

Remove main.py and setup_ratelimiting.py

Add pyyaml to requirements.txt

Implement configuration yaml file

Implement Gitlab2Sentry module

Add custom exceptions

Implement resources

Add run py as entrypoint

Import all providers

Implement GitlabProvider

Implement SetnryProvider

Add custom exceptions

Implement SentryAPIClient

Add type hints and custom exception handling

Apply reformats

Apply mypy type hints rules

Update config.yaml

Refactor Gitlab2Sentry to use GraphQL for gitlab integration

Add gitlab GraphQL query

Implement GraphQLClient for GitlabProvider

Refactor SentryProvider

Update requirements.txt

Implement setup.py

Fix logging on gitlab2sentry

Add exceptions for MR creation

Add exceptions to setnry provider

Rename config to g2s

Fix update flow after testing for resetly

Remove test case

Catch error upon configuration initialization

Make pagination length configurable

Fix logging in sentry_provider

Apply reformats

Fix MRs error on on check mr states

Set page length to 100

Fix returning stats and query performance

Initialize data type for statistics

Make query more re-usable

Update readme.md

Update logging

Set mentions access_level for MRs

Add configuration for access_level

Add default Mentions if no specific mentions given

Add default env

Add time limit to reduce query time

Add created_at to G2SProject instances

Add creation limit to resources

Break query if the time limit is reached

Update readme

Support multiple or one project update

Refactor dockerfile

TEMP log env vars to debug 401 error

Fix helm files

Revert temp logging

Add secrets into helm files

Update upgrade command

Update helm templates

Skip project if closed dsn MR found

Remove path_with_namespace variable from Gitlab2Sentry

Set counters instead of list logging

Replace path_with_namespace with full_path in gitlabprovider

Update logging in SentryProvider

Fix full path error

Fix duplicated attribute

Move 3 files

Update dockerfile

Update helm file

Update readme

Update makefile

Remove sensitive data

Update readme.md

Sync commits

Add tests

Split requirements files

Author: Solvik

.gitignore

@@ -0,0 +1,3 @@
+.env
+/venv/
+__pycache__/

.mypy.ini

@@ -0,0 +1,6 @@
+[mypy]
+python_version = 3.7
+mypy_path = gitlab2sentry
+
+[mypy-pytest]
+ignore_missing_imports = True

Dockerfile

@@ -0,0 +1,12 @@
+FROM python:3.9-slim
+
+WORKDIR /usr/src/app
+
+COPY requirements.txt requirements.txt
+RUN pip3 install -r requirements.txt
+
+COPY gitlab2sentry/ gitlab2sentry/
+COPY run.py run.py
+COPY g2s.yaml g2s.yaml
+
+CMD ["python3", "run.py"]

Makefile

@@ -0,0 +1,25 @@
+# project name
+PRJ	?= gitlab2sentry
+REG ?= your-registry
+NS	?= your-namespace
+IMG	?= $(REG)/$(NS)/$(PRJ)
+TAG	?= $(shell git describe --tags)
+
+build:
+	docker build -t $(IMG):$(TAG) .
+
+push: build
+	docker push $(IMG):$(TAG)
+
+qa:
+	isort --profile black . && black . && flake8
+
+mypy:
+	mypy gitlab2sentry/ --config-file .mypy.ini --ignore-missing-imports
+
+run:
+	# needed env: GITLAB_TOKEN + SENTRY_TOKEN
+	python3 gitlab2sentry/run.py
+
+upgrade: push
+	helm secrets -d vault -n $(NS) upgrade -f helm/values-production.yaml --set cronjob.imageTag=$(TAG) gitlab2sentry ./helm

README.md

@@ -0,0 +1,112 @@
+# gitlab2sentry
+
+This project aims to automate Sentry project creation. Moreover it aims to create a sentry project for every gitlab team's project.
+
+The script aims to add sentry for every new project you have created.
+
+
+## Two-Steps process
+
+1. After creating your new project, in your gitlab, ```gitlab2sentry``` will create the first Pull Request. This pull request will contain the creation of the ```.sentryclirc``` file. Although, it will not contain the ```DSN``` for this project and no project will be created to ```sentry```
+
+2. Once the user has merged the first Pull request, ```gitlab2sentry``` will create the second Pull Request. This pull request will update the newly created ```.sentryclirc``` file with the ```DSN``` of the sentry project. Moreover, after the merge of the first Pull Request ```gitlabsentry``` will create a new ```sentry project```, update its rate limit and save the ```DSN``` inside ```.sentryclirc```. Once the user has merged the second pull request everything will be set up.
+
+**NOTE**: ```Gitlab2Sentry``` looks only for group projects and searches for PRs having specific keyword inside (check "Configuration" section)
+
+
+## Run locally
+
+You can install all requirements for this project with
+
+```
+python3 -m venv venv
+pip3 install -r requirements.txt
+source venv/bin/ac
+```
+
+After the installation of all requirements you have to:
+
+```
+export SENTRY_URL=<your sentry's url>
+export SENTRY_TOKEN=<your sentry token>
+export SENTRY_ENV=<your environment - default production>
+export GITLAB_TOKEN=<your gitlab token>
+export GITLAB_URL=<your gitlab url>
+python3 run.py
+```
+
+## Deployment
+
+We prefer to deploy and manage ```gitlab2sentry``` with ```helm```. Inside ```helm/``` folder you can find an example deployment.
+
+You can upgrade your deployment with:
+
+```
+make upgrade
+```
+
+## Configuration
+
+```Gitlab2Sentry``` requires some configuration in 3 specific files.
+
+1. First of all you have to configure the ```g2s.yaml``` file where everything is configured for the ```gitlab2sentry``` service. Here you can find a description for every field:
+
+```
+sentry:
+  # Sentry configuration.
+  slug: <your sentry organization>
+gitlab:
+  # DSN PR configuration.
+  dsn_mr:
+    # Default values for dsn (2nd) pull request configuration.
+    content: |
+      ## File generated by gitlab2sentry
+      [defaults]
+      url = {sentry_url}
+      dsn = {dsn}
+    description: <your description in the DSN PR>
+    branch_name: <default dsn branch name>
+    title: <default title of the dsn mr>
+  # Sentryclirc PR configuration.
+  sentryclirc_mr:
+    # Default values for .sentryclirc (1st) pull request configuration.
+    content: |
+      ## File generated by gitlab2sentry
+      [defaults]
+      url = {sentry_url}
+    description: <your description in the sentryclirc MR>
+    branch_name: <your sentryclirc branch_name
+    filepath: .sentryclirc
+    commit_message: <commit message>
+    title: <default title of the sentryclirc mr>
+  # Gitlab configuration.
+  config:
+    author:
+      name: <name>
+      email: <email>
+    graphql_suffix: <endpoint of gitlab graphql>
+    # Mention some specific people.
+    mentions:
+      - "@<mention>
+      - # everyone you want to include on those MRs
+    # If no "mentions" field is included, you have to define
+    # The access_level of those who will be mentioned in your PR.
+    mentions_access_level: 40
+    # This keyword must be included in your Pull Request title.
+    # Gitlab2sentry will search with this keyword for PRs
+    keyword: sentry
+    # How old has to be the projects checked from gitlab2sentry
+    creation_days_limit: 60
+    # Do you want to remove the source branch after merge?
+    remove_source: true
+    # Gitlab2Sentry searches only for group projects.
+    group_identifier: <define a keyword for the groups you want to include>
+    # Configure the aiohttp timeout for the GraphQL API of gitlab
+    graphql_aiohttp_timeout: 40
+    # How many items per page will have the response of the GraphQL API.
+    graphql_page_length: 100
+```
+
+2. If you want to follow the ```helm``` deployment process you will have to fill your details into the ```helm/values-production.yaml``` and ```helm/Chart.yaml```.
+
+3. You can update ```REG ?= your-registry``` and ```NS	?= your-namespace``` values inside ```Makefile```.

dev-requirements.txt

@@ -0,0 +1,12 @@
+-r requirements.txt
+
+black
+flake8
+isort
+mock
+mypy
+pytest
+pytest-mock
+types-python-slugify
+types-PyYAML
+types-requests

g2s.yaml

@@ -0,0 +1,48 @@
+sentry:
+  # Sentry configuration.
+  slug: your-sentry-slug
+gitlab:
+  # DSN MR configuration.
+  dsn_mr:
+    content: |
+      ## File generated by gitlab2sentry
+      [defaults]
+      url = {sentry_url}
+      dsn = {dsn}
+    description: |
+      {mentions} Congrats, your Sentry project has been
+      created, merge this
+      to finalize your Sentry integration of
+      {name_with_namespace} :clap: :cookie:
+    branch_name: auto_add_sentry_dsn
+    title: "[gitlab2sentry] Merge me to add your sentry DSN to {project_name}"
+  # Sentryclirc MR configuration.
+  sentryclirc_mr:
+    content: |
+      ## File generated by gitlab2sentry
+      [defaults]
+      url = {sentry_url}
+    description: |
+      {mentions} Merge this and it will automatically
+      create a Sentry project \n
+      for {name_with_namespace} :cookie:
+    branch_name: auto_add_sentry
+    filepath: .sentryclirc
+    commit_message: Update .sentryclirc
+    title: "[gitlab2sentry] Merge me to add sentry to {project_name} or close me"
+  # Gitlab configuration.
+  config:
+    author:
+      name: gitlab2sentry
+      email: your-author-email
+    graphql_suffix: api/graphql
+    # mentions:
+    #   - "@all"
+    # If mentions exist then access_level is not used
+    mentions_access_level: 40
+    creation_days_limit: 60
+    keyword: sentry
+    remove_source: true
+    group_identifier: your-group-identifier
+    graphql_aiohttp_timeout: 40
+    graphql_page_length: 100