Updates

enschilling · enschilling · commit 37a5645acab9 · 2024-10-06T21:13:19.000-07:00
Pending additional formatting / screenshots.
diff --git a/nvidia-nemo-oke/readme.md b/nvidia-nemo-oke/readme.md
@@ -149,7 +149,315 @@
 
 ## Task 3: Deploy the Oracle Database 23ai pod
 
-1. 
+1. Before creating the database, you'll need to create role-based access control (RBAC) for the node. Create a file called **node-rbac.yaml** and paste the following:
+
+    ```
+    <copy>
+    ---
+    apiVersion: rbac.authorization.k8s.io/v1
+    kind: ClusterRole
+    metadata:
+    name: oracle-database-operator-manager-role-node
+    rules:
+    - apiGroups:
+    - ""
+    resources:
+    - nodes
+    verbs:
+    - list
+    - watch
+    ---
+    apiVersion: rbac.authorization.k8s.io/v1
+    kind: ClusterRoleBinding
+    metadata:
+    name: oracle-database-operator-manager-role-node-cluster-role-binding
+    roleRef:
+    apiGroup: rbac.authorization.k8s.io
+    kind: ClusterRole
+    name: oracle-database-operator-manager-role-node
+    subjects:
+    - kind: ServiceAccount
+    name: default
+    namespace: oracle-database-operator-system
+    ---
+    </copy>
+    ```
+
+2. Create a file called **db-admin-secret.yaml** that will be used to set the DB password upon deployment. Paste the follwing:
+
+    ```
+    <copy>
+    apiVersion: v1
+    kind: Secret
+    metadata:
+    name: freedb-admin-secret
+    namespace: oracle23ai
+    type: Opaque
+    stringData:
+    oracle_pwd:  YOURPASSWORDHERE
+    </copy>
+    ```
+
+    >Note: Be sure to replace the **YOURPASSWORDHERE** above with a value of your own choosing. At least 15 characters, 2 upper case, 2 lower case, 2 numbers, and 2 special characters.
+
+3. Create a file called **db23ai-instance.yaml** and paste the following:
+
+    ```
+    <copy>
+    apiVersion: database.oracle.com/v1alpha1
+    kind: SingleInstanceDatabase
+    metadata:
+    name: nemo-23ai
+    namespace: oracle23ai
+    spec:
+    sid: FREE
+    edition: free
+    adminPassword:
+        secretName: freedb-admin-secret
+
+    image:
+        pullFrom: container-registry.oracle.com/database/free:latest
+        prebuiltDB: true
+
+    persistence:
+        size: 50Gi
+        storageClass: "oci-bv"
+        accessMode: "ReadWriteOnce"
+
+    replicas: 1
+    ---
+    </copy>
+    ```
+
+4. Apply the manifests using the following command; this creates the RBAC, the password, and the DB pod.
+
+    ```bash
+    <copy>
+    kubectl apply -n oracle23ai -f node-rbac.yaml,db-admin-secret.yaml,db23ai-instance.yaml
+    </copy>
+    ```
+
+5. After the command completes, it may take 3-5 minutes for the DB instance to come online. You can check the status with the following command. Do not proceed until the status is **Healthy**
+
+    ```bash
+    <copy>
+    kubectl get singleinstancedatabase -n oracle23ai
+    </copy>
+    ```
+
+    Output:
+    ```bash
+    kubectl get singleinstancedatabase -n oracle23ai
+    NAME        EDITION   STATUS    ROLE      VERSION        CONNECT STR              TCPS CONNECT STR   OEM EXPRESS URL
+    nemo-23ai   Free      Healthy   PRIMARY   23.4.0.24.05   10.0.10.246:31452/FREE   Unavailable        Unavailable
+    ```
+
+6. Run the following command to gather details about the DB instance and set them to environment variables.
+
+    ```bash
+    <copy>
+    export ORA_PASS=$(kubectl get secret/freedb-admin-secret -n oracle23ai -o jsonpath='{.data.oracle_pwd}' | base64 -d)
+    export ORACLE_SID=$(kubectl get singleinstancedatabase -n oracle23ai -o 'jsonpath={.items[0].metadata.name}')
+    export ORA_POD=$(kubectl get pods -n oracle23ai -o jsonpath='{.items[0].metadata.name}')
+    export ORA_CONN=$(kubectl get singleinstancedatabase ${ORACLE_SID} -n oracle23ai -o "jsonpath={.status.connectString}")
+    </copy>
+    ```
+
+    >Note: If you leave Cloud Shell and return later, you'll need to run the above commands again if you wish to connect to the DB instance directly. That said, after this section, all DB access should be done via Jupyter Notebooks.
+
+7. Connect to the DB instance.
+
+    ```bash
+    <copy>
+    kubectl exec -it pods/${ORA_POD} -n oracle23ai -- sqlplus sys/${ORA_PASS}@${ORACLE_SID} as sysdba
+    </copy>
+    ```
+
+8. Create a vector DB user that will enable your Python code to access the vector data store.
+
+    ```bash
+    create user c##vector identified by <enter a password here>;
+    grant create session, db_developer_role, unlimited tablespace to c##vector container=ALL;
+    ```
+
+    >Note: You will need to run these commands one at at a time. **Don't forget** to specify your own password in the first command. *<enter password here>* making sure to remove the <> brackets.
+
+9. Type *exit* to leave the container. 
+
+## Task 4: Prepare the NeMo deployment
+**TODO:** Clean up this section.
+
+23.	Now to prep for the NeMo deployment. Create a new Kubernetes namespace.
+
+kubectl create ns embedding-nim
+
+24.	Add your NGC API Key to an environment variable.
+
+export NGC_API_KEY=<your api key here>
+
+25.	Confirm that your key gets you access to the NVCR container registry:
+
+echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
+
+You should get Login Succeeded
+
+26.	Create a docker-registry secret in Kubernetes. The kubelet will use this secret to download the container images needed to run pods.
+
+kubectl -n embedding-nim create secret docker-registry registry-secret --docker-server=nvcr.io --docker-username='$oauthtoken' --docker-password=$NGC_API_KEY
+
+27.	Create a secret for your NGC API KEY that will be passed to your pod via environment variable later.
+
+kubectl -n embedding-nim create secret generic ngc-api-key --from-literal=ngc-api-key=”$NGC_API_KEY”
+28.	You can check the value with the following command.
+
+kubectl -n embedding-nim get secret/ngc-api-key -o jsonpath='{.data.ngc-api-key}' | base64 -d
+
+29.	Next, you’ll create three separate files to deploy the NeMo retriever microservices.
+    a.	llama3-8b-instruct.yaml
+
+        ```
+        <copy>
+        apiVersion: v1
+        kind: Pod
+        metadata:
+        name: nim-llama3-8b-instruct
+        labels:
+            name: nim-llama3-8b-instruct
+            
+        spec:
+        containers:
+        - name: nim-llama3-8b-instruct
+            image: nvcr.io/nim/meta/llama3-8b-instruct:latest
+            securityContext:
+            privileged: true
+            env:
+            - name: NGC_API_KEY
+                valueFrom:
+                secretKeyRef:
+                    name: ngc-api-key
+                    key: ngc-api-key
+            resources:
+            limits:
+                nvidia.com/gpu: 1
+            imagePullPolicy: Always
+        
+        hostNetwork: true
+
+        imagePullSecrets:
+        - name: registry-secret
+        </copy>
+        ```
+
+    b.	nv-embedqa-e5-v5.yaml
+
+        ```
+        <copy>
+        apiVersion: v1
+        kind: Pod
+        metadata:
+        name: nim-nv-embedqa-e5-v5
+        labels:
+            name: nim-nv-embedqa-e5-v5
+            
+        spec:
+        containers:
+        - name: nim-nv-embedqa-e5-v5
+            image: nvcr.io/nim/nvidia/nv-embedqa-e5-v5:1.0.1
+            securityContext:
+            privileged: true
+            env:
+            - name: NGC_API_KEY
+                valueFrom:
+                secretKeyRef:
+                    name: ngc-api-key
+                    key: ngc-api-key
+            resources:
+            limits:
+                nvidia.com/gpu: 1
+            imagePullPolicy: Always
+        
+        hostNetwork: true
+
+        imagePullSecrets:
+        - name: registry-secret
+        </copy>
+        ```
+
+    c.	nv-rerankqa-mistral-4b-v3.yaml
+
+        ```
+        <copy>
+        apiVersion: v1
+        kind: Pod
+        metadata:
+        name: nim-nv-ererankqa-mistral-4b-v3
+        labels:
+            name: nim-nv-ererankqa-mistral-4b-v3
+            
+        spec:
+        containers:
+        - name: nim-nv-ererankqa-mistral-4b-v3
+            image: nvcr.io/nim/nvidia/nv-rerankqa-mistral-4b-v3:1.0.1
+            securityContext:
+            privileged: true
+            env:
+            - name: NGC_API_KEY
+                valueFrom:
+                secretKeyRef:
+                    name: ngc-api-key
+                    key: ngc-api-key
+            resources:
+            limits:
+                nvidia.com/gpu: 1
+            imagePullPolicy: Always
+        
+        hostNetwork: true
+
+        imagePullSecrets:
+        - name: registry-secret
+        </copy>
+        ```
+
+30.	Apply the 3 manifest files to your Kubernetes cluster.
+
+    ```bash
+    <copy>
+    kubectl -n embedding-nim apply -f llama3-8b-instruct.yaml,nv-embedqa-e5-v5.yaml,nv-rerankqa-mistral-4b-v3.yaml
+    </copy>
+    ```
+
+31.	View the pods to ensure they are all running.
+
+    ```bash
+    <copy>
+    kubectl -n embedding-nim get pods -o wide
+    </copy>
+    ```
+
+    Output:
+
+    ```bash
+    NAME				            READY	STATUS	   RESTARTS	AGE	IP		    NODE
+    nim-llama3-8b-instruct		    1/1	    Running	   0		3m	10.0.10.7	10.0.10.7
+    nim-nv-embedqa-e5-v5		    1/1	    Running	   0		3m	10.0.10.11	10.0.10.11
+    nim-nv-rerankqa-mistral-4b-v3	1/1	    Running	   0		3m	10.0.10.18	10.0.10.18
+    ```
+
+
+32.	Now that everything is up and running, you can return to your Jupyter-hub web page and launch a new notebook.
+
+33.	Within the notebood, install the oracledb libraries
+
+pip install oracledb
+
+
+
+
+
+
+
+
+
 
 
 
