Merge branch 'develop' into restartversiontests

Author: sankarpn

docs-source/content/developerguide/branching.md

@@ -11,7 +11,7 @@ The `develop` branch is protected and contains source for the latest completed f
 
 Because we want to balance separating destabilizing work into feature branches against the possibility of later difficult merges, we encourage developers working on features to pull out any necessary refactoring or improvements that are general purpose into their own shorter-lived branches and create pull requests to `develop` when these smaller work items are completed.
 
-All commits to `develop` must pass the [integration test suite]({{< relref "/developerguide/integration-tests.md" >}}).  Please run these tests locally before submitting a pull request.  Additionally, each push to a branch in our GitHub repository triggers a run of a subset of the integration tests with the results visible [here](https://app.wercker.com/Oracle/weblogic-kubernetes-operator/runs).
+All commits to `develop` must pass the [integration test suite]({{< relref "/developerguide/integration-tests.md" >}}).  Please run these tests locally before submitting a pull request.  Additionally, each push to a branch in our GitHub repository triggers a run of a subset of the integration tests with the results visible [here](http://build.weblogick8s.org:8080/job/weblogic-kubernetes-operator-quicktest/).
 
 Please submit pull requests to the `develop` branch unless you are collaborating on a feature and have another target branch.  Please see details on the Oracle Contributor Agreement (OCA) and guidelines for pull requests on the [README]({{< relref "/_index.md#contributing-to-the-oracle-weblogic-server-kubernetes-operator-repository" >}}).
 

docs-source/content/userguide/managing-domains/configoverrides/_index.md

@@ -87,13 +87,15 @@ Typical attributes for overrides include:
 * Server and domain log locations
 * Node Manager related configuration
 * Changing any existing MBean name
+* Adding or removing a module (for example, a JDBC module)
 
 **Specifically, do not use custom overrides for:**
 
 * Adding or removing:
   * Servers
   * Clusters
   * Network Access Points (custom channels)
+  * Modules
 * Changing any of the following:
   * Dynamic cluster size
   * Default, SSL, and Admin channel `Enabled`, listen address, and port
@@ -122,7 +124,7 @@ The operator requires a different file name format for override templates than W
 | JDBC module     |  `jdbc-MODULENAME.xml`  |
 | Diagnostics module     |  `diagnostics-MODULENAME.xml`  |
 
-A `MODULENAME` must correspond to the MBean name of a system resource defined in your original `config.xml` file.
+A `MODULENAME` must correspond to the MBean name of a system resource defined in your original `config.xml` file. It's not possible to add a new module by using overrides. If you need your overrides to set up a new module, then have your original configuration specify 'skeleton' modules that can be overridden.
 
 #### Override template schemas
 
@@ -184,24 +186,28 @@ The secret macro `SECRETNAME` field must reference the name of a Kubernetes secr
 
 #### Override template syntax special requirements
 
-**Check each item below to ensure custom situational configuration takes effect:**
+**Check each item below for best practices and to ensure custom situational configuration takes effect:**
 
 * Reference the name of the current bean and each parent bean in any hierarchy you override.
   * Note that the `combine-mode` verbs (`add` and `replace`) should be omitted for beans that are already defined in your original domain home configuration.
-     * See [Override template samples](#override-template-samples) for examples.
+       * See [Override template samples](#override-template-samples) for examples.
 * Use situational config `replace` and `add` verbs as follows:
   * If you are adding a new bean that doesn't already exist in your original domain home `config.xml`, then specify `add` on the MBean itself and on each attribute within the bean.
        * See the `server-debug` stanza in [Override template samples](#override-template-samples) for an example.
   * If you are adding a new attribute to an existing bean in the domain home `config.xml`, then the attribute needs an `add` verb.
        * See the `max-message-size` stanza in [Override template samples](#override-template-samples) for an example.
   * If you are changing the value of an existing attribute within a domain home `config.xml`, then the attribute needs a `replace` verb.
-    * See the `public-address` stanza in  [Override template samples](#override-template-samples) for an example.
+       * See the `public-address` stanza in  [Override template samples](#override-template-samples) for an example.
 * When overriding `config.xml`:
   * The XML namespace (`xmlns:` in the XML) must be exactly as specified in [Override template schemas](#override-template-schemas).
        * For example, use `d:` to reference `config.xml` beans and attributes, `f:` for `add` and `replace` `domain-fragment` verbs, and `s:` to reference the situational configuration schema.
   * Avoid specifying the domain name stanza, as this may cause some overrides to be ignored (for example, server-template scoped overrides).
 * When overriding modules:
   * It is a best practice to use XML namespace abbreviations `jms:`, `jdbc:`, and `wldf:` respectively for JMS, JDBC, and WLDF (diagnostics) module override files.
+  * A module must already exist in your original configuration if you want to override it; it's not possible to add a new module by using overrides. If you need your overrides to set up a new module, then have your original configuration specify 'skeleton' modules that can be overridden.
+  * See [Overriding a data source module](#overriding-a-data-source-module) for best practice advice. Note that similar advice applies generally to other module types.
+* Consider having your original configuration reference invalid user names, passwords, and URLs:
+  * If your original (non-overridden) configuration references non-working user names, passwords, and URLS, then this helps guard against accidentally deploying a working configuration that's invalid for the intended environment. For example, if your base configuration references a working QA database, and there is some mistake in setting up overrides, then it's possible the running servers will connect to the QA database when you deploy to your production environment.
 
 #### Override template samples
 
@@ -238,8 +244,13 @@ The following `config.xml` override file demonstrates:
 
 #### Overriding a data source module
 
-The following `jdbc-testDS.xml` override template demonstrates setting the URL, user name, and password-encrypted fields of a JDBC module named `testDS` via `secret macros`.  The generated situational configuration that replaces the macros with secret values will be located in the `DOMAIN_HOME/optconfig/jdbc` directory.   The `password-encrypted` field will be populated with an encrypted value because it uses a secret macro with an `:encrypt` suffix.  The secret is named `dbsecret` and contains three keys: `url`, `username`, and `password`.
+The following `jdbc-testDS.xml` override template demonstrates setting the URL, user name, and password-encrypted fields of a JDBC module named `testDS` by using `secret macros`.  The generated situational configuration that replaces the macros with secret values will be located in the `DOMAIN_HOME/optconfig/jdbc` directory.   The `password-encrypted` field will be populated with an encrypted value because it uses a secret macro with an `:encrypt` suffix.  The secret is named `dbsecret` and contains three keys: `url`, `username`, and `password`.
 
+Best practices for data source modules and their overrides:
+
+* A data source module must already exist in your original configuration if you want to override it; it's not possible to add a new module by using overrides. If you need your overrides to set up a new module, then have your original configuration specify 'skeleton' modules that can be overridden. See the next two bulleted items for the typical contents of a skeleton data source module.
+* Set your original (non-overridden) URL, username, and password to invalid values. This helps prevent accidentally starting a server without overrides, and then having the data source successfully connect to a database that's wrong for the current environment. For example, if these attributes are set to reference a QA database in your original configuration, then a mistake configuring overrides in your production Kubernetes deployment could cause your production applications to use your QA database.
+* Set your original (non-overridden) `JDBCConnectionPoolParams` `MinCapacity` and `InitialCapacity` to `0`, and set your original `DriverName` to a reference an existing JDBC Driver. This ensures that you can still successfully boot a server even when you have configured invalid URL/username/password values, your database isn't running, and/or you haven't specified your overrides yet.
 
 ```
 <?xml version='1.0' encoding='UTF-8'?>
@@ -282,7 +293,7 @@ The following `jdbc-testDS.xml` override template demonstrates setting the URL,
     kubectl -n MYNAMESPACE label cm MYCMNAME weblogic.domainUID=DOMAIN_UID
     ```
 * Create any Kubernetes secrets referenced by a template 'secret macro'.
-  * Secrets can have multiple keys (files) that can hold either cleartext or base64 values. We recommend that you use base64 values for passwords via `Opaque` type secrets in their `data` field, so that they can't be easily read at a casual glance. For more information, see https://kubernetes.io/docs/concepts/configuration/secret/.
+  * Secrets can have multiple keys (files) that can hold either cleartext or base64 values. We recommend that you use base64 values for passwords by using `Opaque` type secrets in their `data` field, so that they can't be easily read at a casual glance. For more information, see https://kubernetes.io/docs/concepts/configuration/secret/.
   * Secrets must be in the same Kubernetes namespace as the domain.
   * If a secret is going to be used by a single `DOMAIN_UID`, then we recommend adding the `weblogic.domainUID=<mydomainuid>` label to help track the resource.
   * For example:
@@ -382,7 +393,7 @@ Incorrectly formatted override files may be accepted without warnings or errors
 
 * When a domain is first deployed, or is restarted, the operator runtime creates an introspector Kubernetes job named `DOMAIN_UID-introspect-domain-job`.
 * The introspector job's pod:
-  * Mounts the Kubernetes configuration map and secrets specified via the operator domain resource `configOverrides`, `webLogicCredentialsSecret`, and `configOverrideSecrets` fields.
+  * Mounts the Kubernetes configuration map and secrets specified by using the operator domain resource `configOverrides`, `webLogicCredentialsSecret`, and `configOverrideSecrets` fields.
   * Reads the mounted situational configuration templates from the configuration map and expands them to create the actual situational configuration files for the domain:
       * It expands some fixed replaceable values (for example, `${env:DOMAIN_UID}`).
       * It expands referenced secrets by reading the value from the corresponding mounted secret file (for example, `${secret:mysecret.mykey}`).

docs-source/themes/hugo-theme-learn/README.md

@@ -4,7 +4,8 @@ This repository contains a theme for [Hugo](https://gohugo.io/), based on great
 
 Visit the [theme documentation](https://learn.netlify.com/en/) to see what is going on. It is actually built with this theme.
 
-[![wercker status](https://app.wercker.com/status/233466a2be73fcea400e7dc02ef6adf9/s/master "wercker status")](https://app.wercker.com/project/byKey/233466a2be73fcea400e7dc02ef6adf9)
+
+[![Build Status](http://build.weblogick8s.org:8080/buildStatus/icon?job=weblogic-kubernetes-operator-quicktest)](http://build.weblogick8s.org:8080/job/weblogic-kubernetes-operator-quicktest/)
 [![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fmatcornic%2Fhugo-theme-learn.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Fmatcornic%2Fhugo-theme-learn?ref=badge_shield)
 
 ## Main features

integration-tests/README.md

@@ -4,13 +4,13 @@ This documentation describes the functional use cases that are covered in integr
 
 # Environments
 
-The tests currently run in three modes: "Wercker", "Jenkins", and "standalone" Oracle Linux, where the mode is controlled by the `WERCKER` and `JENKINS` environment variables described below. The default is "standalone".
+The tests currently run in three modes: "shared cluster", "Jenkins", and "standalone" Oracle Linux, where the mode is controlled by the `SHARED_CLUSTER` and `JENKINS` environment variables described below. The default is "standalone".
 
 * "Standalone" Oracle Linux, i.e, run the tests manually with the `mvn` command.
-* Wercker - https://app.wercker.com/Oracle/weblogic-kubernetes-operator/runs - `integration-test-java` is the pipeline name.
+* Shared cluster - http://build.weblogick8s.org:8080/job/weblogic-kubernetes-operator-quicktest/
 * Jenkins - http://wls-jenkins.us.oracle.com/view/weblogic-operator/job/weblogic-kubernetes-operator-javatest/ - Jenkins Run is restricted to Oracle Internal development process.
 
-Wercker runs only Quick test use cases, Jenkins runs both Quick and Full test use cases.
+Shared cluster runs only Quick test use cases, Jenkins runs both Quick and Full test use cases.
 
 # Use Cases
 
@@ -41,7 +41,7 @@ Defaults for `RESULT_ROOT` & `PV_ROOT`:
 | --- | --- | --- | --- |
 | stand-alone	| /scratch/$USER/wl_k8s_test_results |	/scratch/$USER/wl_k8s_test_results	| test defaults |
 | Jenkins	| /scratch/k8s_dir |	/scratch/k8s_dir	 | jenkins configuration |
-| Wercker	| /pipeline/output/k8s_dir	| /scratch	| wercker.yml |
+| Shared cluster	| /pipeline/output/k8s_dir	| /scratch	|  |
 
 
 'Physical' subdirectories created by test:
@@ -158,7 +158,7 @@ When the integration test class ITOperator is executed, staticPrepare() method i
 
 staticPrepare() - initializes the application properties from OperatorIT.properties and creates resultRoot, pvRoot, userprojectsDir directories by calling initialize() method from the base class BaseTest.
 
-staticUnPrepare() - releases the cluster lease on wercker env.
+staticUnPrepare() - releases the cluster lease.
 
 test methods -  testDomainOnPVUsingWLST, testDomainOnPVUsingWDT, testTwoDomainsManagedByTwoOperators, testCreateDomainWithStartPolicyAdminOnly, testCreateDomainPVReclaimPolicyRecycle, testCreateDomainWithDefaultValuesInSampleInputs, testAutoAndCustomSitConfigOverrides, testOperatorRESTIdentityBackwardCompatibility, testOperatorRESTUsingCertificateChain
 
@@ -211,14 +211,14 @@ The tests accepts optional env var overrides:
 | QUICKTEST  | When set to "true", limits testing to a subset of the tests. |
 | LB_TYPE    | The default value is "TRAEFIK". Set to "VOYAGER" if you want to use it as LB. |
 | INGRESSPERDOMAIN  | The defult value is true. If you want to test creating TRAEFIK LB by kubectl yaml for multiple domains, set it to false. |
-| WERCKER    | Set to true if invoking from Wercker, set to false or "" if running stand-alone or from Jenkins. Default is "". |
-| JENKINS    | Set to true if invoking from Jenkins, set to false or "" if running stand-alone or from Wercker. Default is "". |
+| SHARED_CLUSTER    | Set to true if invoking on shared cluster, set to false or "" if running stand-alone or from Jenkins. Default is "". |
+| JENKINS    | Set to true if invoking from Jenkins, set to false or "" if running stand-alone or on shared cluster. Default is "". |
 | K8S_NODEPORT_HOST | DNS name of a Kubernetes worker node. Default is the local host's hostname. |
 | BRANCH_NAME  | Git branch name.   Default is determined by calling 'git branch'. |
 | LEASE_ID   |   Set to a unique value to (A) periodically renew a lease on the k8s cluster that indicates that no other test run should attempt to use the cluster, and (B) delete this lease when the test completes. |
 
 The following additional overrides are currently only used when
-WERCKER=true:
+SHARED_CLUSTER=true:
 
 | Variable | Description |
 | --- | --- |
@@ -305,8 +305,6 @@ $RESULT_ROOT/acceptance_test_tmp is archived under $RESULT_ROOT/acceptance_test_
 
 $PV_ROOT/acceptance_test_pv is archived under $PV_ROOT/acceptance_test_pv_archive
 
-On Wercker, these logs can be downloaded by clicking "Download artifact" on cleanup and store step.
-
 # How to add a new test
 
 Add a new JUnit test under integration-tests/src/test/java/oracle/kubernetes/operator.

integration-tests/src/test/java/oracle/kubernetes/operator/BaseTest.java

@@ -127,7 +127,7 @@ public static void initialize(String appPropsFile) throws Exception {
     }
 
     // for manual/local run, do cleanup
-    if (System.getenv("WERCKER") == null && System.getenv("JENKINS") == null) {
+    if (System.getenv("SHARED_CLUSTER") == null && System.getenv("JENKINS") == null) {
 
       // delete k8s artifacts created if any, delete PV directories
       ExecResult clnResult = cleanup();
@@ -176,7 +176,7 @@ public static void initialize(String appPropsFile) throws Exception {
     logger.info("Adding file handler, logging to file at " + resultDir + "/java_test_suite.out");
 
     // for manual/local run, create file handler, create PVROOT
-    if (System.getenv("WERCKER") == null && System.getenv("JENKINS") == null) {
+    if (System.getenv("SHARED_CLUSTER") == null && System.getenv("JENKINS") == null) {
       logger.info("Creating PVROOT " + pvRoot);
       Files.createDirectories(Paths.get(pvRoot));
       ExecResult result = ExecCommand.exec("chmod 777 " + pvRoot);
@@ -518,7 +518,7 @@ public static ExecResult cleanup() throws Exception {
   protected void logTestBegin(String testName) throws Exception {
     logger.info("+++++++++++++++++++++++++++++++++---------------------------------+");
     logger.info("BEGIN " + testName);
-    // renew lease at the beginning for every test method, leaseId is set only for Wercker
+    // renew lease at the beginning for every test method, leaseId is set only for shared cluster
     TestUtils.renewK8sClusterLease(getProjectRoot(), getLeaseId());
   }
 
@@ -636,7 +636,7 @@ private void callWebAppAndVerifyScaling(Domain domain, int replicas) throws Exce
    *
    * @param iTClassName - IT class name to be used in the archive file name
    * @throws Exception when errors while running statedump.sh or cleanup.sh scripts or while
-   *     renewing the lease for wercker run
+   *     renewing the lease for shared cluster run
    */
   public static void tearDown(String iTClassName) throws Exception {
     logger.log(

integration-tests/src/test/java/oracle/kubernetes/operator/utils/Domain.java

@@ -1354,9 +1354,15 @@ private void initialize(Map<String, Object> inputDomainMap) throws Exception {
     }
 
     if (domainMap.containsKey("domainHomeImageBuildPath")) {
-      domainHomeImageBuildPath = ((String) domainMap.get("domainHomeImageBuildPath")).trim();
+      domainHomeImageBuildPath =
+          BaseTest.getResultDir()
+              + "/"
+              + ((String) domainMap.get("domainHomeImageBuildPath")).trim();
       domainMap.put(
-          "domainHomeImageBuildPath", BaseTest.getResultDir() + "/" + domainHomeImageBuildPath);
+          "domainHomeImageBuildPath",
+          BaseTest.getResultDir()
+              + "/"
+              + ((String) domainMap.get("domainHomeImageBuildPath")).trim());
     }
     if (System.getenv("IMAGE_PULL_SECRET_WEBLOGIC") != null) {
       domainMap.put("imagePullSecretName", System.getenv("IMAGE_PULL_SECRET_WEBLOGIC"));

integration-tests/src/test/java/oracle/kubernetes/operator/utils/Operator.java

@@ -378,7 +378,13 @@ private void generateInputYaml() throws Exception {
         sb.append(operatorNS);
         break;
     }
-    sb.append(" DNS:");
+    // here we are assuming that if the "host name" starts with a digit, then it is actually
+    // an IP address, and so we need to use the "IP" prefix in the SANS.
+    if (Character.isDigit(TestUtils.getHostName().charAt(0))) {
+      sb.append(" IP:");
+    } else {
+      sb.append(" DNS:");
+    }
     sb.append(TestUtils.getHostName());
     sb.append(" >> ");
     sb.append(generatedInputYamlFile);
@@ -476,7 +482,7 @@ private void initialize(
     }
 
     // customize the inputs yaml file to use our pre-built docker image
-    // IMAGE_NAME_OPERATOR & IMAGE_TAG_OPERATOR variables are used for wercker
+    // IMAGE_NAME_OPERATOR & IMAGE_TAG_OPERATOR variables are used for shared cluster
     if (System.getenv("IMAGE_NAME_OPERATOR") != null
         && System.getenv("IMAGE_TAG_OPERATOR") != null) {
       operatorMap.put(