Stages not created AND automatic stages created #35
Description
This ticket is most likely a "misunderstanding" ticket from a newbie but showing my working / thinking here so it may inform further enhancements / docs.
I've onboarded project1 and I automatically see 3 stages: dev, hardening and production. I didn't apply these stages and they don't exist in my Git repo so the operator is creating them automatically. To my (extremely limited) understanding of GitOps, this is against the paradigm as my repo should be the source of truth and thus I now have stages that do not appear in my repo.
I believe this is done because creating a keptn project without stages is not currently possible.
Perhaps I'm forced down this workflow because I'm using GitHub.com and modifying one file at a time rather than committing everything at once. Even so, this is, IMO, how most beginners would probably approach things.
Nevertheless, I create a .keptn/stage.yaml file (side note, is this name hardcoded? I'd like to call this stages.yaml instead):
---
apiVersion: "keptn.sh/v1"
kind: "KeptnStage"
metadata:
name: "foostage"
spec:
project: "project1"
and my stage isn't created. In my Git upstream I see a shipyard.yaml file has been created for me so perhaps that is overriding the gitops operator:
apiVersion: "spec.keptn.sh/0.2.0"
kind: "Shipyard"
metadata:
name: "podtato-head"
spec:
stages:
- name: "dev"
sequences:
- name: "dummy"
tasks:
- name: "dummy"
- name: "hardening"
sequences:
- name: "dummy"
tasks:
- name: "dummy"
- name: "production"
sequences:
- name: "dummy"
tasks:
- name: "dummy"
So I decide to delete the shipyard.yaml file from the Git upstream, which is OK right, since gitops will see take the repo as the source of truth and regenerate the shipyard file.
Other Issues
I'm (seemingly randomly) getting the below even though project1 does exist:
{
"nextPageKey": "0",
"projects": [{
"creationDate": "1647649950814030400",
"gitRemoteURI": "https://github.com/agardnerIT/keptn-gitops-project1",
"gitUser": "agardnerit",
"projectName": "project1",
"shipyard": "apiVersion: \"spec.keptn.sh/0.2.0\"\nkind: \"Shipyard\"\nmetadata:\n name: \"podtato-head\"\nspec:\n stages:\n - name: \"dev\"\n sequences:\n - name: \"dummy\"\n tasks:\n - name: \"dummy\"\n - name: \"hardening\"\n sequences:\n - name: \"dummy\"\n tasks:\n - name: \"dummy\"\n - name: \"production\"\n sequences:\n - name: \"dummy\"\n tasks:\n - name: \"dummy\"\n\n",
"shipyardVersion": "spec.keptn.sh/0.2.0",
"stages": [{
"services": [],
"stageName": "dev"
}, {
"services": [],
"stageName": "hardening"
}, {
"services": [],
"stageName": "production"
}]
}]
}
INFO Reconciling Project {"Request.Namespace": "keptn", "Request.Name": "project1"}
1.6476532207766266e+09 ERROR Could not update status of project project1 {"Request.Namespace": "keptn", "Request.Name": "project1", "error": "keptnprojects.keptn.sh \"project1\" not found"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:114
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:311
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:227
1.6476532207767227e+09 ERROR Finished Reconciling KeptnProject with error: %w {"Request.Namespace": "keptn", "Request.Name": "project1", "error": "keptnprojects.keptn.sh \"project1\" not found"}
github.com/keptn-sandbox/keptn-gitops-operator/keptn-operator/controllers/keptnprojectcontroller.(*KeptnProjectReconciler).Reconcile
/workspace/controllers/keptnprojectcontroller/keptnproject_controller.go:153
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:114
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:311
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:227
1.6476532207767754e+09 ERROR controller.keptnproject Reconciler error {"reconciler group": "keptn.sh", "reconciler kind": "KeptnProject", "name": "project1", "namespace": "keptn", "error": "keptnprojects.keptn.sh \"project1\" not found"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:227
This thread may lead to Keptn RFEs but the workflow that would work for me (and IMHO is the "most logical" esp. to a new user is):
1. Install operators
2. Create `instance.yaml`
3. Create `project.yaml` (this is an empty project without stages)
4. Create `stages.yaml` (creates an empty stage without sequences)
5. Create `sequences.yaml` with one or more sequences and tasks
6. Modify `stages.yaml` to link sequences
Up to step 4 would be the equivalent of the existing:
keptn create project project2 --shipyard=shipyard.yaml
Where shipyard.yaml:
apiVersion: "spec.keptn.sh/0.2.0"
kind: "Shipyard"
metadata:
name: "project2-shipyard"
spec:
stages:
- name: "stage1"