Merge branch 'develop' into develop-bug2968457-bug29591809

Author: jshum2479

docs-source/content/userguide/managing-domains/fmw-infra/_index.md

@@ -7,16 +7,16 @@ pre = "<b> </b>"
 
 Starting with release 2.2, the operator supports FMW Infrastructure domains.
 This means domains that are created with the FMW Infrastructure installer rather than the WebLogic
-installer.  These domains contain the Java Required Files (JRF) feature and are
+Server installer.  These domains contain the Java Required Files (JRF) feature and are
 the pre-requisite for "upper stack" products like Oracle SOA Suite, for example.
 These domains also require a database and the use of the Repository
 Creation Utility (RCU).
 
 This section provides details about the special considerations for running
 FMW Infrastructure domains with the operator.  Other than those considerations
-listed here, FMW Infrastructure domains work in the same way as WebLogic domains.
+listed here, FMW Infrastructure domains work in the same way as WebLogic Server domains.
 That is, the remainder of the documentation in this site applies equally to FMW
-Infrastructure domains and WebLogic domains.
+Infrastructure domains and WebLogic Server domains.
 
 FMW Infrastructure domains are supported using both the "domain on a persistent volume"
 and the "domain in a Docker image" [models]({{< relref "/userguide/managing-domains/choosing-a-model/_index.md" >}}).
@@ -49,12 +49,16 @@ A [sample](https://github.com/oracle/docker-images/tree/master/OracleFMWInfrastr
 is provided in the Oracle GitHub account that demonstrates how to create a Docker image
 to run FMW Infrastructure.  
 
+Please consult the [README](https://github.com/oracle/docker-images/blob/master/OracleFMWInfrastructure/dockerfiles/12.2.1.3/README.md) file associated with this sample for important prerequisite steps,
+such as building or pulling the Server JRE Docker image and downloading the Fusion Middleware
+Infrastructure installer binary.
+
 After cloning the repository and downloading the installer from Oracle Technology Network
 or e-delivery, you create your image by running the provided script:
 
 ```bash
 cd docker-images/OracleFMWInfrastructure/dockerfiles
-./buildDockerImage.sh -v 12.2.1.3 -g
+./buildDockerImage.sh -v 12.2.1.3 -s
 ```
 
 The image produced will be named `oracle/fmw-infrastructure:12.2.1.3`.
@@ -68,7 +72,7 @@ by running the provided script:
 
 ```bash
 cd docker-images/OracleFMWInfrastructure/samples/12213-patch-fmw-for-k8s
-./buildDockerImage.sh
+./build.sh
 ```
 
 This will produce an image named `oracle/fmw-infrastructure:12213-update-k8s`.
@@ -160,8 +164,10 @@ spec:
 ```
 
 Notice that you can pass in environment variables to set the SID, the name of the PDB, and
-so on.  The documentation describes the other variables that are available.  You should
-also create a service to make the database available within the Kubernetes cluster with
+so on.  The documentation describes the other variables that are available.  The `sys` password
+defaults to `Oradoc_db1`.  Follow the instructions in the documentation to reset this password.
+
+You should also create a service to make the database available within the Kubernetes cluster with
 a well known name.  Here is an example:
 
 ```yaml
@@ -171,7 +177,6 @@ metadata:
   name: oracle-db
   namespace: default
 spec:
-  clusterIP: 10.97.236.215
   ports:
   - name: tns
     port: 1521
@@ -185,7 +190,7 @@ spec:
 ```
 
 In the example above, the database would be visible in the cluster using the address
-`oracle-db.default.svc.cluster.local:1521/devpdc.k8s`.
+`oracle-db.default.svc.cluster.local:1521/devpdb.k8s`.
 
 When you run the database in the Kubernetes cluster, you will probably want to also
 run RCU from a pod inside your network, though this
@@ -249,13 +254,13 @@ image that you built earlier as a "service" pod to run RCU.  To do this, start u
 pod using that image as follows:
 
 ```bash
-kubectl run rcu -ti --image oracle/fmw-infrastructure:12.2.1.3 -- sleep 100000
+kubectl run rcu --generator=run-pod/v1 --image oracle/fmw-infrastructure:12213-update-k8s -- sleep infinity
 
 ```
 
 This will create a Kubernetes deployment called `rcu` containing a pod running a container
-created from the `oracle/fmw-infrastructure:12.2.1.3` image which will just run
-`sleep 100000`, which essentially creates a pod that we can "exec" into and use to run whatever
+created from the `oracle/fmw-infrastructure:12213-update-k8s` image which will just run
+`sleep infinity`, which essentially creates a pod that we can "exec" into and use to run whatever
 commands we need to run.
 
 To get inside this container and run commands, use this command:
@@ -267,11 +272,11 @@ kubectl exec -ti rcu /bin/bash
 When you are finished with this pod, you can remove it with this command:
 
 ```bash
-kubectl delete deploy rcu
+kubectl delete pod rcu
 ```
 
 {{% notice note %}}
-You can use the same approach to get a temporary "service" pod to run other utilities
+You can use the same approach to get a temporary pod to run other utilities
 like WLST.
 {{% /notice %}}
 
@@ -287,7 +292,7 @@ for the regular schema users:
   -silent \
   -createRepository \
   -databaseType ORACLE \
-  -connectString oracle-db:1521/devpdb.k8s \
+  -connectString oracle-db.default:1521/devpdb.k8s \
   -dbUser sys \
   -dbRole sysdba \
   -useSamePasswordForAllSchemaUsers true \
@@ -305,7 +310,7 @@ for the regular schema users:
 You need to make sure that you maintain the association between the database schemas and the
 matching domain just like you did in a non-Kubernetes environment.  There is no specific
 functionality provided to help with this.  We recommend that you consider making the RCU
-prefix the same as your `domainUID` to help maintain this association.
+prefix (value of `schemaPrefix` argument) the same as your `domainUID` to help maintain this association.
 
 ##### Dropping schemas
 
@@ -316,7 +321,7 @@ If you want to drop the schema, you can use a command like this:
   -silent \
   -dropRepository \
   -databaseType ORACLE \
-  -connectString oracle-db:1521/devpdb.k8s \
+  -connectString oracle-db.default:1521/devpdb.k8s \
   -dbUser sys \
   -dbRole sysdba \
   -selectDependentsForComponents true \
@@ -340,8 +345,9 @@ When you create your domain using the sample provided below, it will obtain the
 from this secret.
 
 A [sample](/weblogic-kubernetes-operator/blob/master/kubernetes/samples/scripts/create-rcu-credentials/README.md)
-is provided that demonstrates how to create the secret.
-
+is provided that demonstrates how to create the secret.  The schema owner user name required will be the
+`schemaPrefix` value followed by an underscore and a component name, such as `FMW1_STB`.  The schema owner
+password will be the password you provided for regular schema users during RCU creation.
 
 #### Creating a FMW Infrastructure domain
 

integration-tests/JRFUSECASES.MD

@@ -0,0 +1,61 @@
+# Use Cases for FMW Infra Domain in the Operator
+
+Java integration tests cover the following use cases:
+
+## Basic test Configuration & Use Cases
+
+|  |  |
+| --- | --- |
+| Operator Configuration | operator1 deployed in `weblogic-operator` namespace and manages domains in `default` and `jrfdomains` namespaces |
+| Domain Configuration | Domain on PV using WLST, Traefik load balancer |
+
+**Basic Use Cases**
+
+1. Create operator `weblogic-operator` which manages `default` and `jrfdomains` namespaces, verify it's deployed successfully, pods created, operator ready and verify external REST service, if configured.
+2. Create domain `jrfdomain` in `jrfdomains` namespace and verify the pods, services are created and servers are in ready state.
+3. Verify the admin external service by accessing the admin REST endpoint with `nodeport` in URL.
+4. Verify exec into the admin pod and deploying webapp using the admin port with WLST.
+5. Verify web app load balancing by accessing the webapp using `loadBalancerWebPort`.
+
+**Advanced Use Cases**
+
+6. Verify domain life cycle (destroy and create) should not have any impact on operator managing the domain and web app load balancing and admin external service.
+7. Cluster scale up/down using operator REST endpoint, webapp load balancing should adjust accordingly.
+8. Operator life cycle (destroy and create) should not impact the running domain.
+
+Also the below use cases are covered for each test:
+
+9. Verify the liveness probe by killing managed server 1 process 3 times to kick pod auto-restart.
+10. Shutdown the domain by changing domain `serverStartPolicy` to `NEVER`.
+
+
+## Full test Configuration & Use Cases 
+
+Additional Operator Configuration:
+
+|  |  |
+| --- | --- |
+| Operator Configuration | `operator2` deployed in `weblogic-operator2` namespace and manages domains in `test2` namespace |
+
+Basic Use Cases described above are verified in all the domain configurations. Also the below use cases are covered:
+
+| Domain | Use Case |
+| --- | --- |
+| Domain on PV using WLST |  as above in basic use cases |
+| Domain with ADMIN_ONLY | verify only admin server is started and managed servers are not started. Shutdown domain by deleting domain CRD. Create domain on existing PV dir, pv is already populated by a shutdown domain. |
+| Domain with situational config | create domain with listen address not set for admin server and t3 channel/NAP and incorrect file for admin server log location. Introspector should override these with sit-config automatically. Also, with some junk value for t3 channel public address and using custom situational config override replace with valid public address using secret. Also, on Jenkins this domain uses NFS instead of HOSTPATH PV storage |	
+| Two domains managed by two operators | verify scaling and restart of one domain doesn't impact another domain. Delete domain resources using delete script from samples. |			
+| Two domains in the same namespace managed by one operator | create two FMW Infra Domains in the same namespace managed by one operator. Verify scaling and restart of one domain doesn't impact another domain. |
+| Two domains in the different namespaces managed by one operator | create two FMW Infra Domains in the different namespaces managed by one operator. Domain1 uses VOYAGER load balancer. Domain2 uses TRAEFIK load balancer. Verify scaling and restart of one domain doesn't impact another domain. |
+| Domain with Recycle policy | create domain with pvReclaimPolicy="Recycle" and using Configured cluster. Verify that the PV is deleted once the domain and PVC are deleted |
+| Domain with default sample values | create domain using mostly default values for inputs |	
+
+Test cases cover the bugs found:
+
+| Domain | Use Case |
+| --- | --- | 
+| Domain on PV using WLST | cluster rolling restart with restartVersion and maxUnavailable set to 2, verify the not-ready server counts can not exceed maxUnavailable value |
+| Domain on PV using WLST | in create domain input file, set exposeAdminNodePort to false and exposeAdminT3Channel to true, verify the managed server pods are created |
+| Domain on PV using WLST | in create domain input file, set createDomainScriptsMountPath to non-default value, verify the create domain sample script works |
+| Domain on PV using WLST | in createFMWDomain.py file, set administration_port_enabled to true, verify the admin sever pod is running and ready and all managed server pods are created |
+| Domain on PV using WLST | in create domain input file, set exposeAdminT3Channel to true, verify the admin t3 channel is exposed |

integration-tests/README.md

@@ -16,6 +16,8 @@ Shared cluster runs only Quick test use cases, Jenkins runs both Quick and Full
 
 Use Cases covered in integration tests for the operator is available [here](USECASES.MD)
 
+JRF Use Cases are covered [here](JRFUSECASES.MD)
+
 # Directory Configuration and Structure
 
 Directory structure of source code:
@@ -226,8 +228,20 @@ SHARED_CLUSTER=true:
 | IMAGE_NAME_OPERATOR | Docker image name for operator. Default is weblogic-kubernetes-operator |
 | IMAGE_PULL_POLICY_OPERATOR | Default 'Never'. |
 | IMAGE_PULL_SECRET_OPERATOR | Default ''. |
- | IMAGE_PULL_SECRET_WEBLOGIC | Default ''.
+| IMAGE_PULL_SECRET_WEBLOGIC | Default ''. |
 
+The below env variables are required for SHARED_CLUSTER=true:
+
+| Variable | Description |
+| --- | --- |
+| REPO_REGISTRY | OCIR Server to push/pull the Operator image |
+| REPO_USERNAME | OCIR Username |
+| REPO_PASSWORD | OCIR token |
+| REPO_EMAIL | OCIR email |
+| DOCKER_USERNAME | Docker username to pull the Weblogic image  |
+| DOCKER_PASSWORD | Docker password |
+| DOCKER_EMAIL | Docker email |
+| K8S_NODEPORT_HOST | DNS name of a Kubernetes worker node. |
 
 Successful run will have the output like below:
 ```
@@ -282,6 +296,35 @@ Failed run will have the output like
 ```
 JUnit test results can be seen at "integration-tests/target/failsafe-reports/TEST-oracle.kubernetes.operator.ITOperator.xml". This file shows how much time each test case took to run and the failed test results if any.
 
+# How to run JRF domain In Operator related tests
+* Setup docker access to FMW Infrastructure 12c Image and Oracle Database 12c Image
+
+ Method 1 
+  - Setup a personal account on hub.docker.com
+  - Then sign in to hub.docker.com and signup for access to Oracle Database 12c Images via https://hub.docker.com/_/oracle-database-enterprise-edition
+  - Then export the following before running the tests: 
+	```
+	export DOCKER_USERNAME=<docker_username>
+	export DOCKER_PASSWORD=<docker_password>
+	export DOCKER_EMAIL=<docker_email>
+	```
+  - Setup an account in phx.ocir.io
+  - Then sign in to phx.ocir.io to get access to FMW Infrastructure 12c Image: **_phx.ocir.io/weblogick8s/oracle/fmw-infrastructure:12.2.1.3_**
+  - export the following before running the tests:
+    ```
+    export REPO_USERNAME=<ocir_username>
+    export REPO_PASSWORD=<ocir_password>
+    export REPO_EMAIL=<ocir_email>
+    ```
+ 
+ Method 2 
+ - Make sure the FMW Infrastructure image i.e. **_phx.ocir.io/weblogick8s/oracle/fmw-infrastructure:12.2.1.3_** and the Oracle database image i.e. **_store/oracle/database-enterprise:12.2.0.1_** already exist locally in a docker repository the k8s cluster can access
+		
+* Command to run the tests: 
+```
+mvn clean verify -P jrf-integration-tests 2>&1 | tee log.txt
+```
+
 # How to run a single test
 
 mvn -Dit.test="ITOperator#testDomainOnPVUsingWLST" -DfailIfNoTests=false integration-test -P java-integration-tests

integration-tests/USECASES.MD

@@ -2,7 +2,7 @@
 
 Java integration tests cover the below use cases:
 
-## Quick test Configuration & Use Cases - 
+## Quick test Configuration & Use Cases -
 
 |  |  |
 | --- | --- |
@@ -101,4 +101,23 @@ Configuration Overrides Usecases
 | Sticky Session | Use Case |
 | --- | --- |
 | Server affinity | Use a web application deployed on Weblogic cluster to track HTTP session. Test server-affinity by sending two HTTP requests to Weblogic and verify that all requests are directed to same Weblogic server |
-| Session state isolation | Verify that values saved in a client session state are not visible to another client |
+| Session state isolation | Verify that values saved in a client session state are not visible to another client |
+
+| Monitoring Exporter | Use Case |
+| --- | --- |
+| Check Metrics via Prometheus | build, deploy webapp for Monitoring Exporter, start Prometheus and verify the metrics was produced by using Prometheus APIs |
+| Replace Configuration via exporter console| Verify that configuration for monitoring exporter can be replaced during runtime, check applied metrics via Prometheus APIs|
+| Append Configuration via exporter console| Verify that configuration for monitoring exporter can be appended during runtime, check applied metrics via Prometheus APIs|
+| Append Configuration with varios combinations of attributes via exporter console| Append monitoring exporter configuration [a] to new config [a,b] and verify it was applied |
+| Replace Configuration with only one attribute as array via exporter console| Replace monitoring exporter configuration [a,b,c] attributes with new config [a] and verify it was applied |
+| Replace Configuration with empty config file via exporter console| Replace monitoring exporter configuration with empty config file, verify it was applied |
+| Replace/Append Configuration with  config file written in non yml format via exporter console| Try to replace/append monitoring exporter configuration with config file written in non yml format, verify configuration has not changed |
+| Replace/Append Configuration with corrupted yml file via exporter console| Try to replace/append monitoring exporter configuration with config file written in corrupted yml format, verify configuration has not changed |
+| Replace/Append Configuration with dublicated values in the config file via exporter console| Try to replace/append monitoring exporter configuration with dublicated values in the config file, verify configuration has not changed |
+| Replace/Append Configuration with invalid credentials via exporter console| Try to replace/append monitoring exporter configuration with varios combos for invalid credentials, verify configuration has not changed and `401 Unauthorized` exception was thrown |
+
+| Logging with Elastic Stack | Use Case |
+| --- | --- |
+| Search log level | Use Elasticsearch Count API to query logs of level=INFO and verify that total number of logs for level=INFO is not zero and failed count is zero |
+| Search Operator log | Use Elasticsearch Search APIs to query Operator log info and verify that log hits for type=weblogic-operator are not empty |
+| Search Weblogic log | Use Elasticsearch Search APIs to query Weblogic log info and verify that log hits for Weblogic servers are not empty |

integration-tests/pom.xml

@@ -113,6 +113,11 @@
 			<artifactId>jaxb-api</artifactId>
 			<version>2.3.1</version>
 		</dependency>
+		<dependency>
+			<groupId>net.sourceforge.htmlunit</groupId>
+			<artifactId>htmlunit</artifactId>
+			<version>2.32</version>
+		</dependency>
 	</dependencies>
 
 	<build>