Merge remote-tracking branch 'origin/main' into release/4.0

Author: rjeberhard

common/src/main/java/oracle/kubernetes/common/CommonConstants.java

@@ -10,6 +10,7 @@ private CommonConstants() {
   }
 
   public static final String COMPATIBILITY_MODE = "compat-";
+  public static final String WLS_SHARED = "wls-shared-";
   public static final String API_VERSION_V9 = "weblogic.oracle/v9";
   public static final String API_VERSION_V8 = "weblogic.oracle/v8";
 

common/src/main/java/oracle/kubernetes/common/utils/SchemaConversionUtils.java

@@ -589,14 +589,14 @@ private void removeNamedItems(Map<String, Object> serverPod, String kind, Predic
   private void addInitContainersVolumeAndMountsToServerPod(Map<String, Object> serverPod, List<Object> auxiliaryImages,
                                                                          List<Object> auxiliaryImageVolumes) {
     addEmptyDirVolume(serverPod, auxiliaryImageVolumes);
-    List<Object> initContainers = Optional.ofNullable((List<Object>) serverPod.get(INIT_CONTAINERS))
-        .orElse(new ArrayList<>());
+    List<Object> initContainers =  new ArrayList<>();
     for (Object auxiliaryImage : auxiliaryImages) {
       initContainers.add(
           createInitContainerForAuxiliaryImage((Map<String, Object>) auxiliaryImage, containerIndex.get(),
               auxiliaryImageVolumes));
       containerIndex.addAndGet(1);
     }
+    initContainers.addAll(Optional.ofNullable((List<Object>) serverPod.get(INIT_CONTAINERS)).orElse(new ArrayList<>()));
     serverPod.put(INIT_CONTAINERS, initContainers);
     auxiliaryImages.forEach(ai -> addVolumeMount(serverPod, (Map<String, Object>)ai, auxiliaryImageVolumes));
     addAuxiliaryImageEnv(auxiliaryImages, serverPod, auxiliaryImageVolumes);

documentation/4.0/content/faq/resource-settings.md

@@ -40,10 +40,12 @@ You can tune container memory and CPU usage
 by configuring Kubernetes resource requests and limits,
 and you can tune a WebLogic JVM heap usage
 using the `USER_MEM_ARGS` environment variable in your Domain YAML file.
-The introspector job pod uses the same CPU and memory settings as the
+By default, the introspector job pod uses the same CPU and memory settings as the
 domain's WebLogic Administration Server pod. Similarly, the operator created init containers in the
 introspector job pod for the [Auxiliary Images]({{< relref "/managing-domains/model-in-image/auxiliary-images" >}})
 based domains use the same CPU and memory settings as the domain's WebLogic Administration Server pod.
+Beginning with operator version 4.0.5, you can override the settings of the introspector job pod using
+the `domain.spec.introspector.serverPod` element.
 A resource request sets the minimum amount of a resource that a container requires.
 A resource limit is the maximum amount of a resource a container is given
 and prevents a container from using more than its share of a resource.
@@ -61,6 +63,8 @@ and you can override the setting for individual WebLogic Server instances using
 setting for member servers of a cluster using the `cluster.spec.serverPod` element.
 Note that the introspector job pod uses the same settings
 as the WebLogic Administration Server pod.
+Beginning with operator version 4.0.5, you can override the settings of the introspector job pod
+using the `domain.spec.introspector.serverPod` element.
 
 Values set in the `.serverPod` stanzas for a more specific type of pod, override
 the same values if they are also set for a more general type of pod, and inherit

documentation/4.0/content/managing-domains/model-in-image/auxiliary-images.md

@@ -284,13 +284,13 @@ to use Docker to build its auxiliary image:
 ### Automated upgrade of the `weblogic.oracle/v8` schema auxiliary images configuration
 
 {{% notice note %}}
-The automated upgrade described in this section converts `weblogic.oracle/v8` schema auxiliary image configuration into low-level Kubernetes schema, for example, init-containers and volumes.
+The automated upgrade described in this section converts `weblogic.oracle/v8` schema auxiliary image configuration into low-level Kubernetes schema, for example, init containers and volumes.
 Instead of relying on the generated low-level schema, Oracle recommends using a simplified `weblogic.oracle/v9` schema configuration for auxiliary images, as documented in the [Configuration](#configuration) section.
 {{% /notice %}}
 
 In operator version 4.0, we have enhanced auxiliary images to improve ease of use; also, its configuration has changed from operator 3.x releases.
 
-Operator 4.0 provides a seamless upgrade of Domains with `weblogic.oracle/v8` schema auxiliary images configuration. When you create a Domain with auxiliary images using `weblogic.oracle/v8` schema in a namespace managed by the 4.0 operator, the [WebLogic Domain resource conversion webhook]({{< relref "/managing-operators/conversion-webhook.md" >}}) performs an automated upgrade of the domain resource to the `weblogic.oracle/v9` schema. The conversion webhook runtime converts the `weblogic.oracle/v8` auxiliary image configuration to the equivalent configuration using init-containers, volume and volume mounts under the `serverPod` spec in `weblogic.oracle/v9`. Similarly, when [upgrading the operator]({{< relref "/managing-operators/installation#upgrade-the-operator" >}}), Domains with `weblogic.oracle/v8` schema auxiliary images are seamlessly upgraded.
+Operator 4.0 provides a seamless upgrade of Domains with `weblogic.oracle/v8` schema auxiliary images configuration. When you create a Domain with auxiliary images using `weblogic.oracle/v8` schema in a namespace managed by the 4.0 operator, the [WebLogic Domain resource conversion webhook]({{< relref "/managing-operators/conversion-webhook.md" >}}) performs an automated upgrade of the domain resource to the `weblogic.oracle/v9` schema. The conversion webhook runtime converts the `weblogic.oracle/v8` auxiliary image configuration to the equivalent configuration using init containers, volume and volume mounts under the `serverPod` spec in `weblogic.oracle/v9`. Similarly, when [upgrading the operator]({{< relref "/managing-operators/installation#upgrade-the-operator" >}}), Domains with `weblogic.oracle/v8` schema auxiliary images are seamlessly upgraded.
 
 The following is a sample `weblogic.oracle/v8` schema auxiliary image configuration in operator 3.x and
 the equivalent `weblogic.oracle/v9` schema configuration generated by the conversion webhook in operator 4.0.
@@ -344,6 +344,10 @@ spec:
       name: compat-ai-vol-auxiliaryimagevolume1
 ```
 
+{{% notice note %}}
+The conversion webhook runtime creates init containers with names prefixed with `compat-` when converting the auxiliary image configuration of the `weblogic.oracle/v8` schema. The operator generates only init containers with names starting with either `compat-` or `wls-shared-` in the introspector job pod. To alter the generated init container's name, the new name must start with either `compat-` or `wls-shared-`.
+{{% /notice %}}
+
 ### Domain upgrade tool to manually upgrade the `weblogic.oracle/v8` schema domain resource
 
 To manually upgrade the domain resource from the `weblogic.oracle/v8` schema to the `weblogic.oracle/v9` schema, see [Upgrade the `weblogic.oracle/v8` schema domain resource manually]({{< relref "/managing-domains/upgrade-domain-resource#upgrade-the-weblogicoraclev8-schema-domain-resource-manually" >}}).

documentation/4.0/content/managing-domains/upgrade-domain-resource/_index.md

@@ -14,7 +14,7 @@ The Domain CustomResourceDefinition in operator version 4.0 has changed signific
 ### Automated upgrade of `weblogic.oracle/v8` schema domain resource
 
 {{% notice note %}}
-The automated upgrade described in this section converts `weblogic.oracle/v8` schema auxiliary image configuration into low-level Kubernetes schema, for example, init-containers and volumes.
+The automated upgrade described in this section converts `weblogic.oracle/v8` schema auxiliary image configuration into low-level Kubernetes schema, for example, init containers and volumes.
 Instead of relying on the generated low-level schema, Oracle recommends using a simplified `weblogic.oracle/v9` schema configuration for auxiliary images, as documented in the
 Auxiliary Images [Configuration]({{<relref "/managing-domains/model-in-image/auxiliary-images#configuration" >}}) section.
 {{% /notice %}}
@@ -24,7 +24,7 @@ The 4.0 operator provides a seamless upgrade of the Domain resources with the `w
 ### Upgrade the `weblogic.oracle/v8` schema domain resource manually
 
 {{% notice note %}}
-The manual upgrade tooling described in this section converts `weblogic.oracle/v8` schema auxiliary image configuration into low-level Kubernetes schema, for example, init-containers and volumes.
+The manual upgrade tooling described in this section converts `weblogic.oracle/v8` schema auxiliary image configuration into low-level Kubernetes schema, for example, init containers and volumes.
 Instead of relying on the generated low-level schema, Oracle recommends using a simplified `weblogic.oracle/v9` schema configuration for auxiliary images, as documented in the
 Auxiliary Images [Configuration]({{<relref "/managing-domains/model-in-image/auxiliary-images#configuration" >}}) section.
 {{% /notice %}}
@@ -69,3 +69,7 @@ In the previous example, the tool writes the upgraded file to the `/tmp` directo
 $ ls -ltr /tmp/domain-v9.yaml
 -rw-r----- 1 user dba 2818 Apr 18 23:11 /tmp/domain-v9.yaml
 ```
+
+{{% notice note %}}
+The manual upgrade tooling creates init containers with names prefixed with `compat-` when converting the auxiliary image configuration of the `weblogic.oracle/v8` schema. The operator generates only init containers with names starting with either `compat-` or `wls-shared-` in the introspector job pod. To alter the generated init container's name, the new name must start with either `compat-` or `wls-shared-`.
+{{% /notice %}}

integration-tests/src/test/java/oracle/weblogic/kubernetes/ItMiiUpdateDomainConfig.java

@@ -359,23 +359,26 @@ void testMiiCheckSystemResources() {
   }
 
   /**
-   * Start a WebLogic domain using model-in-image with JMS/JDBC SystemResources.
-   * Create a configmap to delete JMS/JDBC SystemResources.
+   * Start a WebLogic domain using model-in-image.
+   * Create 1 configmap with 2 models files, one of them to add JMS/JDBC SystemResources
+   * and another one to delete JMS/JDBC SystemResources
    * Patch the domain resource with the configmap.
    * Update the restart version of the domain resource.
    * Verify rolling restart of the domain by comparing PodCreationTimestamp
    * for all the server pods before and after rolling restart.
    * Verify SystemResources are deleted from the domain.
    */
   @Test
-  @Order(4)
+  @Order(5)
   @DisplayName("Delete SystemResources from the domain")
   void testMiiDeleteSystemResources() {
 
     String configMapName = "deletesysrescm";
+    final String modelFileAdd =  "model.add.sysresources.yaml";
+    final String modelFileDelete = "model.delete.sysresources.yaml";
+    List<String> modelFiles = Arrays.asList(MODEL_DIR + "/" + modelFileAdd, MODEL_DIR + "/" + modelFileDelete);
     createConfigMapAndVerify(
-        configMapName, domainUid, domainNamespace,
-        Arrays.asList(MODEL_DIR + "/model.delete.sysresourcesbyconfigmap.yaml"));
+        configMapName, domainUid, domainNamespace, modelFiles);
 
     LinkedHashMap<String, OffsetDateTime> pods = new LinkedHashMap<>();
     // get the creation time of the admin server pod before patching
@@ -433,7 +436,7 @@ void testMiiDeleteSystemResources() {
    * Verify JMS Server logs are written on PV.
    */
   @Test
-  @Order(5)
+  @Order(6)
   @DisplayName("Add new JDBC/JMS SystemResources to the domain")
   void testMiiAddSystemResources() {
 
@@ -505,7 +508,7 @@ void testMiiAddSystemResources() {
    * Verify servers from the new cluster are running.
    */
   @Test
-  @Order(6)
+  @Order(7)
   @DisplayName("Add a dynamic cluster to domain with non-zero replica count")
   void testMiiAddDynamicCluster() {
 
@@ -579,7 +582,7 @@ void testMiiAddDynamicCluster() {
    * Check the validity of new credentials by accessing WebLogic RESTful Service
    */
   @Test
-  @Order(7)
+  @Order(8)
   @DisplayName("Change the WebLogic Admin credential of the domain")
   void testMiiUpdateWebLogicCredential() {
     verifyUpdateWebLogicCredential(adminSvcExtHost, domainNamespace, domainUid, adminServerPodName,
@@ -601,7 +604,7 @@ void testMiiUpdateWebLogicCredential() {
    * Make sure JMS Connections and messages are distributed across 4 servers.
    */
   @Test
-  @Order(8)
+  @Order(9)
   @DisplayName("Test modification to Dynamic cluster size parameters")
   void testMiiUpdateDynamicClusterSize() {
 
@@ -745,6 +748,70 @@ private void buildClientOnPod() {
     assertEquals(0, result.exitValue(), "Client compilation fails");
   }
 
+  /**
+   * Start a WebLogic domain using model-in-image with JMS/JDBC SystemResources.
+   * Create a empty configmap to delete JMS/JDBC SystemResources
+   * Patch the domain resource with the configmap.
+   * Update the restart version of the domain resource.
+   * Verify rolling restart of the domain by comparing PodCreationTimestamp
+   * for all the server pods before and after rolling restart.
+   * Verify SystemResources are deleted from the domain.
+   */
+  @Test
+  @Order(4)
+  @DisplayName("Delete SystemResources from the domain")
+  void testMiiDeleteSystemResourcesByEmptyConfigMap() {
+
+    String configMapName = "deletesysrescm";
+    createConfigMapAndVerify(
+        configMapName, domainUid, domainNamespace,
+        Arrays.asList(MODEL_DIR + "/model.delete.sysresourcesbyconfigmap.yaml"));
+
+    LinkedHashMap<String, OffsetDateTime> pods = new LinkedHashMap<>();
+    // get the creation time of the admin server pod before patching
+    OffsetDateTime adminPodCreationTime = getPodCreationTime(domainNamespace,adminServerPodName);
+    pods.put(adminServerPodName, adminPodCreationTime);
+    // get the creation time of the managed server pods before patching
+    for (int i = 1; i <= replicaCount; i++) {
+      pods.put(managedServerPrefix + i, getPodCreationTime(domainNamespace, managedServerPrefix + i));
+    }
+
+    StringBuffer patchStr = null;
+    patchStr = new StringBuffer("[{");
+    patchStr.append("\"op\": \"replace\",")
+        .append(" \"path\": \"/spec/configuration/model/configMap\",")
+        .append(" \"value\":  \"" + configMapName + "\"")
+        .append(" }]");
+    logger.log(Level.INFO, "Configmap patch string: {0}", patchStr);
+
+    patch = new V1Patch(new String(patchStr));
+    boolean cmPatched = assertDoesNotThrow(() ->
+            patchDomainCustomResource(domainUid, domainNamespace, patch, "application/json-patch+json"),
+        "patchDomainCustomResource(configMap)  failed ");
+    assertTrue(cmPatched, "patchDomainCustomResource(configMap) failed");
+
+    String newRestartVersion = patchDomainResourceWithNewRestartVersion(domainUid, domainNamespace);
+    logger.log(Level.INFO, "New restart version is {0}", newRestartVersion);
+
+    assertTrue(verifyRollingRestartOccurred(pods, 1, domainNamespace),
+                "Rolling restart failed");
+
+    // Even if pods are created, need the service to created
+    for (int i = 1; i <= replicaCount; i++) {
+      logger.info("Check managed server service {0} created in namespace {1}",
+          managedServerPrefix + i, domainNamespace);
+      checkServiceExists(managedServerPrefix + i, domainNamespace);
+    }
+
+    int adminServiceNodePort
+        = getServiceNodePort(domainNamespace, getExternalServicePodName(adminServerPodName), "default");
+    assertNotEquals(-1, adminServiceNodePort, "admin server default node port is not valid");
+    verifySystemResourceConfiguration(adminSvcExtHost, adminServiceNodePort,
+                                          "JDBCSystemResources", "TestDataSource", "404");
+    verifySystemResourceConfiguration(adminSvcExtHost, adminServiceNodePort,
+                                          "JMSSystemResources", "TestClusterJmsModule", "404");
+  }
+
   // Run standalone JMS Client in the pod using wlthint3client.jar in classpath.
   // The client sends 300 messsage to a Uniform Distributed Queue.
   // Make sure that each destination get excatly 150 messages each.

integration-tests/src/test/resources/wdt-models/model.add.sysresources.yaml

@@ -0,0 +1,57 @@
+# Copyright (c) 2023 Oracle and/or its affiliates.
+# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.
+resources:
+    FileStore:
+        TestClusterFileStore:
+           Target: 'cluster-1'
+    JMSServer:
+        TestClusterJmsServer:
+            ProductionPausedAtStartup: false
+            ConsumptionPausedAtStartup: false
+            Target: 'cluster-1'
+            PersistentStore: 'TestClusterFileStore'
+            InsertionPausedAtStartup: false
+            MessageCompressionOptions: GZIP_DEFAULT_COMPRESSION
+            Notes: '@@ENV:CUSTOM_ENV@@'
+
+    JMSSystemResource:
+        TestClusterJmsModule:
+            Target: 'cluster-1'
+            SubDeployment:
+                TestClusterSubDeployment:
+                    Target: TestClusterJmsServer
+            JmsResource:
+              UniformDistributedQueue:
+                testUniformQueue:
+                   SubDeploymentName: TestClusterSubDeployment
+                   JNDIName: jms/testUniformQueue
+              UniformDistributedTopic:
+                testUniformTopic:
+                   SubDeploymentName: TestClusterSubDeployment
+                   ForwardingPolicy: Partitioned
+                   JNDIName: jms/testUniformTopic
+
+    JDBCSystemResource:
+        TestDataSource:
+            Target: 'cluster-1'
+            JdbcResource:
+                JDBCConnectionPoolParams:
+                    InitialCapacity: 0
+                    MinCapacity: 0
+                    MaxCapacity: 15
+                JDBCDataSourceParams:
+                    GlobalTransactionsProtocol: OnePhaseCommit
+                    RowPrefetchSize: 200
+                    JNDIName: jdbc/TestDataSource
+                JDBCDriverParams:
+                    URL: '@@SECRET:@@ENV:DOMAIN_UID@@-db-secret:url@@'
+                    PasswordEncrypted: '@@SECRET:@@ENV:DOMAIN_UID@@-db-secret:password@@'
+                    DriverName: oracle.jdbc.OracleDriver
+                    Properties:
+                        user:
+                            Value: '@@SECRET:@@ENV:DOMAIN_UID@@-db-secret:username@@'
+
+    WLDFSystemResource:
+        'TestWldfModule':
+            Description: WLDF Module for Mii domain
+            Target: 'cluster-1'