Docs: Update README.md

This patch updates the README.md file to be a bit more detailed in how
we can get a working deployment with Ceph as a backend for Cinder and
Glance.

Author: Akrog

README.md

@@ -1,149 +1,239 @@
-# cinder-operator
-// TODO(user): Add simple overview of use/purpose
-
-## Description
-// TODO(user): An in-depth paragraph about your project and overview of use
+# CINDER-OPERATOR
 
-## Getting Started
-You’ll need a Kubernetes cluster to run against.  Our recommendation for the time being is to use
-[OpenShift Local](https://access.redhat.com/documentation/en-us/red_hat_openshift_local/2.2/html/getting_started_guide/installation_gsg) (formerly known as CRC / Code Ready Containers).  
-We have [companion development tools](https://github.com/openstack-k8s-operators/install_yamls/blob/master/devsetup/README.md) available that will install OpenShift Local for you.
+The cinder-operator is an OpenShift Operator built using the Operator Framework
+for Go. The Operator provides a way to easily install and manage an OpenStack
+Cinder installation on OpenShift. This Operator was developed using RDO
+containers for openStack.
 
-### Running on the cluster
-1. Install Instances of Custom Resources:
+## Getting started
+
+**NOTES:**
+
+- *The project is in a rapid development phase and not yet intended for
+  production consumption, so instructions are meant for developers.*
+
+- *If possible don't run things in your own machine to avoid the risk of
+  affecting the development of your other projects.*
+
+Here we'll explain how to get a functiona OpenShift deployment running inside a
+VM that is running MariaDB, RabbitMQ, KeyStone, Glance, and Cinder services
+against a Ceph backend.
+
+There are 4 steps:
+
+- [Install prerequisites](#prerequisites)
+- [Deploy an OpenShift cluster](#openshift-cluster)
+- [Prepare Storage](#storage)
+- [Deploy OpenStack](#deploy)
+
+### Prerequisites
+
+There are some tools that will be required through this process, so the first
+thing we do is install them:
 
 ```sh
-kubectl apply -f config/samples/
+sudo dnf install -y git wget make ansible-core python-pip podman gcc
 ```
 
-2. Build and push your image to the location specified by `IMG`:
-	
+We'll also need this repository as well as `install_yamls`:
+
 ```sh
-make docker-build docker-push IMG=<some-registry>/cinder-operator:tag
+cd ~
+git clone https://github.com/openstack-k8s-operators/install_yamls.git
+git clone https://github.com/openstack-k8s-operators/cinder-operator.git
 ```
-	
-3. Deploy the controller to the cluster with the image specified by `IMG`:
+
+### OpenShift cluster
+
+There are many ways get an OpenShift cluster, and our recommendation for the
+time being is to use [OpenShift Local](https://access.redhat.com/documentation/en-us/red_hat_openshift_local/2.5/html/getting_started_guide/index)
+(formerly known as CRC / Code Ready Containers).
+
+To help with the deployment we have [companion development tools](https://github.com/openstack-k8s-operators/install_yamls/blob/master/devsetup)
+available that will install OpenShift Local for you and will also help with
+later steps.
+
+Running OpenShift requires a considerable amount of resources, even more when
+running all the operators and services required for an OpenStack deployment,
+so make sure that you have enough resources in the machine to run everything.
+
+You will need at least 5 CPUS and 16GB of RAM, preferably more, just for the
+local OpenShift VM.
+
+**You will also need to get your [pull-secrets from Red Hat](
+https://cloud.redhat.com/openshift/create/local) and store it in the machine,
+for example on your home directory as `pull-secret`.**
 
 ```sh
-make deploy IMG=<some-registry>/cinder-operator:tag
+cd ~/install_yamls/devsetup
+PULL_SECRET=~/pull-secret CPUS=6 MEMORY=20480 make download_tools crc
 ```
 
-### Uninstall CRDs
-To delete the CRDs from the cluster:
+This will take a while, but once it has completed you'll have an OpenShift
+cluster ready.
+
+Now you need to set the right environmental variables for the OCP cluster, and
+you may want to logging to the cluster manually (although the previous step
+already logs in at the end):
 
 ```sh
-make uninstall
+eval $(crc oc-env)
 ```
 
-### Undeploy controller
-UnDeploy the controller to the cluster:
+**NOTE**: When CRC finishes the deployment the `oc` client is logged in, but
+the token will eventually expire, in that case we can login again with
+`oc login -u kubeadmin -p 12345678 https://api.crc.testing:6443`, or use the
+[helper functions](CONTRIBUTING.md#helpful-scripts).
+
+Let's now get the cluster version confirming we have access to it:
 
 ```sh
-make undeploy
+oc get clusterversion
 ```
 
-### Configure Cinder with Ceph backend
+If you are running OCP on a different machine you'll need additional steps to
+[access its dashboard from an external system](https://github.com/openstack-k8s-operators/install_yamls/tree/master/devsetup#access-ocp-from-external-systems).
+
+### Storage
 
-The Cinder spec API can be used to configure and customize the Ceph backend. In
-particular, the `customServiceConfig` parameter should be used, for each
-defined volume, to override the `enabled_backends` parameter, which must exist
-in `cinder.conf` to make the `cinderVolume` pod run. The global `cephBackend`
-parameter is used to specify the Ceph client-related "key/value" pairs required
-to connect the service with an external Ceph cluster. Multiple external Ceph
-clusters are not supported at the moment. The following represents an example
-of the Cinder object that can be used to trigger the Cinder service deployment,
-and enable the Cinder backend that points to an external Ceph cluster.
+There are 2 kinds of storage we'll need: One for the pods to run, for example
+for the MariaDB database files, and another for the OpenStack services to use
+for the VMs.
 
+To create the pod storage we run:
+
+```sh
+cd ~/install_yamls
+make crc_storage
 ```
-apiVersion: cinder.openstack.org/v1beta1
-kind: Cinder
-metadata:
-  name: cinder
-  namespace: openstack
-spec:
-  serviceUser: cinder
-  databaseInstance: openstack
-  databaseUser: cinder
-  cinderAPI:
-    replicas: 1
-    containerImage: quay.io/tripleowallabycentos9/openstack-cinder-api:current-tripleo
-  cinderScheduler:
-    replicas: 1
-    containerImage: quay.io/tripleowallabycentos9/openstack-cinder-scheduler:current-tripleo
-  cinderBackup:
-    replicas: 1
-    containerImage: quay.io/tripleowallabycentos9/openstack-cinder-backup:current-tripleo
-  secret: cinder-secret
-  cinderVolumes:
-    volume1:
-      containerImage: quay.io/tripleowallabycentos9/openstack-cinder-volume:current-tripleo
-      replicas: 1
-      customServiceConfig: |
-        [DEFAULT]
-        enabled_backends=ceph
-  cephBackend:
-    cephFsid: <CephClusterFSID>
-    cephMons: <CephMons>
-    cephClientKey: <cephClientKey>
-    cephUser: openstack
-    cephPools:
-      cinder:
-        name: volumes
-      nova:
-        name: vms
-      glance:
-        name: images
-      cinder_backup:
-        name: backup
-      extra_pool1:
-        name: ceph_ssd_tier
-      extra_pool2:
-        name: ceph_nvme_tier
-      extra_pool3:
-        name: ceph_hdd_tier
+
+As for the storage for the OpenStack services, at the time of this writing only
+NFS and Ceph are supported.
+
+For simplicity's sake we'll use a *toy* Ceph cluster that runs in a single
+local container using a simple script provided by this project. Beware that
+this script overrides things under `/etc/ceph`:
+
+**NOTE**: This step must be run after the OpenShift VM is running because it
+binds to an IP address created by it.
+
+```sh
+~/cinder-operator/hack/dev/create-ceph.sh
 ```
 
-When the service is up and running, it's possible to interact with the cinder
-API and create the Ceph `cinder type` backend which is associated with the Ceph
-tier specified in the config file.
+Using an external Ceph cluster is also possible, but out of the scope of this
+document, and the manifest we'll use have been tailor made for this specific
+*toy* Ceph cluster.
 
+### Deploy
 
-## Contributing
-// TODO(user): Add detailed information on how you would like others to contribute to this project
+Deploying the podified OpenStack control plane is a 2 step process. First
+deploying the operators, and then telling the openstack-operator how we want
+our OpenStack deployment to look like.
 
-### How it works
-This project aims to follow the Kubernetes [Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/)
+Deploying the openstack operator:
 
-It uses [Controllers](https://kubernetes.io/docs/concepts/architecture/controller/) 
-which provides a reconcile function responsible for synchronizing resources untile the desired state is reached on the cluster 
+```sh
+cd ~/install_yamls
+make openstack
+```
 
-### Test It Out
-1. Install the CRDs into the cluster:
+Once all the operator ready we'll see the pod with:
 
 ```sh
-make install
+oc get pod -l control-plane=controller-manager
 ```
 
-2. Run your controller (this will run in the foreground, so switch to a new terminal if you want to leave it running):
+And now we can tell this operator to deploy RabbitMQ, MariaDB, Keystone, Glance
+and Cinder using the Ceph *toy* cluster:
 
 ```sh
-make run
+export OPENSTACK_CR=`realpath ~/cinder-operator/hack/dev/openstack-ceph.yaml`
+cd ~/install_yamls
+make openstack_deploy
 ```
 
-**NOTE:** You can also run this in one step by running: `make install run`
+After a bit we can see the 5 operators are running:
+
+```sh
+oc get pods -l control-plane=controller-manager
+```
 
-### Modifying the API definitions
-If you are editing the API definitions, generate the manifests such as CRs or CRDs using:
+And a while later the services will also appear:
 
 ```sh
-make manifests
+oc get pods -l app=mariadb
+oc get pods -l app.kubernetes.io/component=rabbitmq
+oc get pods -l service=keystone
+oc get pods -l service=glance
+oc get pods -l service=cinder
 ```
 
-**NOTE:** Run `make --help` for more information on all potential `make` targets
+### Configure Clients
 
-More information can be found via the [Kubebuilder Documentation](https://book.kubebuilder.io/introduction.html)
+Now that we have the OpenStack services running we'll want to setup the
+different OpenStack clients.
 
-## License
+For convenience this project has a simple script that does it for us:
+
+```sh
+source ~/cinder-operator/hack/dev/osp-clients-cfg.sh
+```
+
+We can now see available endpoints and services to confirm that the clients and
+the Keystone service work as expected:
+
+```sh
+openstack service list
+openstack endpoint list
+```
+
+Upload a glance image:
+
+```sh
+cd
+wget http://download.cirros-cloud.net/0.5.2/cirros-0.5.2-x86_64-disk.img -O cirros.img
+openstack image create cirros --container-format=bare --disk-format=qcow2 < cirros.img
+openstack image list
+```
+
+And create a cinder volume:
+
+```sh
+openstack volume create --size 1 myvolume
+```
+
+## Cleanup
+
+To delete the deployed OpenStack we can do:
+
+```sh
+cd ~/install_yamls
+make openstack_deploy_cleanup
+```
+
+Once we've done this we need to recreate the PVs that we created at the start,
+since some of them will be in failed state:
+
+```sh
+make crc_storage_cleanup crc_storage
+```
+
+We can now remove the openstack-operator as well:
+
+```sh
+make openstack_cleanup
+```
+
+# ADDITIONAL INFORMATION
+
+**NOTE:** Run `make --help` for more information on all potential `make`
+targets.
+
+More information about the Makefile can be found via the [Kubebuilder
+Documentation]( https://book.kubebuilder.io/introduction.html).
+
+# LICENSE
 
 Copyright 2022.
 
@@ -158,4 +248,3 @@ distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-