Merge branch 'master' into feature/integration-tests

Author: rjeberhard

README.md

@@ -52,15 +52,15 @@ The operator can configure services to expose WebLogic applications and features
 *	How will users authenticate?
 * Is the network channel encrypted?
 
-While it is natural to expose web applications outside the cluster, exposing administrative features like the administration console and a T3 channel for WLST should be given more careful consideration.  There are alternative options that should be weighed.  For example, Kubernetes provides the ability to securely access a shell running in a container in a pod in the cluster.  WLST could be executed from such an environment, meaning the T3 communications are entirely within the Kubernetes cluster and therefore more secure.
+While it is natural to expose web applications outside the cluster, exposing administrative features like the Administration Console and a T3 channel for WLST should be given more careful consideration.  There are alternative options that should be weighed.  For example, Kubernetes provides the ability to securely access a shell running in a container in a pod in the cluster.  WLST could be executed from such an environment, meaning the T3 communications are entirely within the Kubernetes cluster and therefore more secure.
 
 Oracle recommends careful consideration before deciding to expose any administrative interfaces externally.
 
 # Requirements
 
 The Oracle WebLogic Server Kubernetes Operator has the following requirements:
 
-*	Kubernetes 1.7.5+, 1.8.0+ or 1.9.0+ (check with `kubectl version`)
+*	Kubernetes 1.7.5+, 1.8.0+ (check with `kubectl version`).  Note that Kubernetes 1.9.x is not supported yet.
 *	Flannel networking v0.9.1-amd64 (check with `docker images | grep flannel`)
 *	Docker 17.03.1.ce (check with `docker version`)
 *	Oracle WebLogic Server 12.2.1.3.0
@@ -73,10 +73,10 @@ The following features are not certified or supported in the Technology Preview
 
 *	Whole Server Migration
 *	Consensus Leasing
-*	Node Manager (although it is used internally for the liveness probe and to start WebLogic servers)
+*	Node Manager (although it is used internally for the liveness probe and to start WebLogic Server instances)
 *	Dynamic domains (the current certification only covers configured clusters, certification of dynamic clusters is planned at a future date)
 *	Multicast
-*	If using a `hostPath` persistent volume, then it must have read/write/many permissions for all container/pods in the WebLogic deployment
+*	If using a `hostPath` persistent volume, then it must have read/write/many permissions for all container/pods in the WebLogic Server deployment
 *	Multitenancy
 *	Production redeployment
 
@@ -97,11 +97,11 @@ Documentation for APIs is provided here:
 If you would rather see the developers demonstrating the operator rather than reading the documentation, then here are your videos:
 
 * [Installing the operator](https://youtu.be/B5UmY2xAJnk) includes the installation and also shows using the operator's REST API.
-* [Creating a WebLogic domain with the operator](https://youtu.be/Ey7o8ldKv9Y) shows creation of two WebLogic domains including accessing the administration console and looking at the various resources created in Kubernetes - services, Ingresses, pods, load balancers, etc.
+* [Creating a WebLogic domain with the operator](https://youtu.be/Ey7o8ldKv9Y) shows creation of two WebLogic domains including accessing the Administration Console and looking at the various resources created in Kubernetes - services, Ingresses, pods, load balancers, etc.
 * [Deploying a web application, scaling a WebLogic cluster with the operator and verifying load balancing](https://youtu.be/hx4OPhNFNDM)
 * [Using WLST against a domain running in Kubernetes](https://youtu.be/eY-KXEk8rI4) shows how to create a data source for an Oracle database that is also running in Kubernetes.
 * [Scaling a WebLogic cluster with WLDF](https://youtu.be/Q8iZi2e9HvU)
-* watch this space, more to come!
+* Watch this space, more to come!
 
 Like what you see?  Read on for all the nitty-gritty details...
 
@@ -157,10 +157,10 @@ Please refer to [Scaling a WebLogic cluster](site/scaling.md) for more informati
 
 Please refer to [Shutting down a domain](site/shutdown-domain.md) for information about how to shut down a domain running in Kubernetes.
 
-## Load balancing with the Traefik ingress controller
+## Load balancing with the Traefik Ingress controller
 
 The initial Technology Preview release of the operator supports only the Traefik load balancer/Ingress controller.  Support for other load balancers is planned in the future.
-Please refer to [Load balancing with the Traefik ingress controller](site/traefik.md) for information about current capabilities.
+Please refer to [Load balancing with Traefik](site/traefik.md) for information about current capabilities.
 
 [comment]: # (Exporting operator logs to ELK.  The operator provides an option to export its log files to the ELK stack. Please refer to [ELK integration]site/elk.md for information about this capability.)
 
@@ -169,11 +169,11 @@ Please refer to [Load balancing with the Traefik ingress controller](site/traefi
 To permanently remove a domain from a Kubernetes cluster, first shut down the domain using the instructions provided above in the section titled “Shutting down a domain”, then remove the persistent volume claim and the persistent volume using these commands:
 
 ```
-kubectl delete pvc PVC-NAME
+kubectl delete pvc PVC-NAME -n NAMESPACE
 kubectl delete pv PV-NAME
 ```
 
-Find the names of the persistent volume claim and the persistent volume in the domain custom resource YAML file, or if it is not available, check for the `domainUID` in the metadata on the persistent volumes.
+Find the names of the persistent volume claim (represented above as `PVC-NAME`) and the persistent volume (represented as `PV-NAME`) in the domain custom resource YAML file, or if it is not available, check for the `domainUID` in the metadata on the persistent volumes. Replace `NAMESPACE` with the namespace that the operator is running in.
 
 To permanently delete the actual domain configuration, delete the physical volume using the appropriate tools.  For example, if the persistent volume used the `HostPath provider`, then delete the corresponding directory on the Kubernetes master.
 

kubernetes/create-domain-job-inputs.yaml

@@ -52,6 +52,10 @@ persistenceVolumeClaimName: pv001-claim
 # if it is not already
 persistenceVolumeName: pv001
 
+# Boolean indiciating whether to erase/overwrite an existing data found in the persistent volume
+# WARNING: if you set this to true, then ALL data found in the persistent volume will be deleted! 
+replaceExistingDomain: false
+
 # Boolean indicating if production mode is enabled for the domain
 productionModeEnabled: true
 

kubernetes/create-domain-job.sh

@@ -273,7 +273,7 @@ function initialize {
   validateInputParamsSpecified adminPort adminServerName createDomainScript domainName domainUid clusterName managedServerCount managedServerStartCount managedServerNameBase
   validateInputParamsSpecified managedServerPort persistencePath persistenceSize persistenceVolumeClaimName persistenceVolumeName
   validateInputParamsSpecified productionModeEnabled secretsMountPath secretName t3ChannelPort exposeAdminT3Channel adminNodePort exposeAdminNodePort
-  validateInputParamsSpecified namespace loadBalancer loadBalancerWebPort loadBalancerAdminPort loadBalancer
+  validateInputParamsSpecified namespace loadBalancer loadBalancerWebPort loadBalancerAdminPort loadBalancer replaceExistingDomain
   validateStorageClass
   validateLoadBalancer
   validateImagePullSecretName
@@ -349,6 +349,7 @@ function createYamlFiles {
   sed -i -e "s:%T3_CHANNEL_PORT%:${t3ChannelPort}:g" ${jobOutput}
   sed -i -e "s:%T3_PUBLIC_ADDRESS%:${t3PublicAddress}:g" ${jobOutput}
   sed -i -e "s:%CLUSTER_NAME%:${clusterName}:g" ${jobOutput}
+  sed -i -e "s:%REPLACE_EXISTING_DOMAIN%:${replaceExistingDomain}:g" ${jobOutput}
 
   # Generate the yaml to create the domain custom resource
   echo Generating ${dcrOutput}

kubernetes/internal/domain-job-template.yaml

@@ -60,10 +60,16 @@ data:
       fail "The domain secret %SECRETS_MOUNT_PATH%/password was not found"
     fi
 
-    # Do not proceed if the domain already exists
+    # Check if the domain already exists
     domainFolder=${SHARED_PATH}/domain/%DOMAIN_NAME%
     if [ -d ${domainFolder} ]; then
-      fail "The create domain job will not overwrite an existing domain. The domain folder ${domainFolder} already exists"
+      # check if user asked to replace existing data
+      if [ "%REPLACE_EXISTING_DOMAIN%" = "true" ]; then
+        echo "As requested, deleting all data in the peristent volume to make way for new domain!"
+        rm -rf ${SHARED_PATH}/*
+      else
+        fail "The create domain job will not overwrite an existing domain unless you set the parameter 'replaceExistingDomain' to 'true'. The domain folder ${domainFolder} already exists"
+      fi
     fi
 
     # Create the base folders
@@ -177,8 +183,8 @@ data:
     sleep 15
     echo "Finished waiting for the nodemanager to start"
 
-    echo "Update JVM arguments" 
-    echo "Arguments=\${USER_MEM_ARGS} -XX\:+UnlockExperimentalVMOptions -XX\:+UseCGroupMemoryLimitForHeap \${JAVA_OPTIONS}" >> ${startProp} 
+    echo "Update JVM arguments"
+    echo "Arguments=\${USER_MEM_ARGS} -XX\:+UnlockExperimentalVMOptions -XX\:+UseCGroupMemoryLimitForHeap \${JAVA_OPTIONS}" >> ${startProp}
 
     echo "Start the server"
     wlst.sh -skipWLSModuleScanning ${pyFile}
@@ -207,7 +213,7 @@ data:
       checkFileExists ${pyFile}
 
       # Create a liveness probe script. It checks a WL server state file maintained by the node manager.
-      # The script and 'EOF' on the following lines must not be indented! 
+      # The script and 'EOF' on the following lines must not be indented!
 
       scriptFile=${nmdir}/livenessProbe.sh
 
@@ -317,7 +323,7 @@ data:
     done
 
     echo "Successfully Completed"
-    
+
   create-domain.py: |-
     # This python script is used to create a WebLogic domain
 

kubernetes/internal/generate-security-policy.sh

@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/usr/bin/env bash
 # Copyright 2017, 2018, Oracle Corporation and/or its affiliates.  All rights reserved.
 # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
 
@@ -44,10 +44,10 @@ else
     shift
 fi
 
-if [ "$1" == "-o" ] ; then
+if [ "$1" = "-o" ] ; then
         shift
         SCRIPT="$1"
-elif [[ "$1" = --output=* ]] ; then
+elif [[ "$1" = "--output="* ]] ; then
         SCRIPT=`echo "$1" | cut -d \= -f 2`
 else
         SCRIPT=$SCRIPT_DEFAULT

site/architecture.md

@@ -9,9 +9,9 @@ The operator is packaged in a Docker image `container-registry.oracle.com/middle
 
 Scripts are provided to deploy the operator to a Kubernetes cluster.  These scripts also provide options to install and configure a load balancer and ELK integration.
 
-The operator registers a Kubernetes custom resource definition called `domain.weblogic.oracle` (shortname `domain`, plural `domains`).  
+The operator registers a Kubernetes custom resource definition called `domain.weblogic.oracle` (shortname `domain`, plural `domains`).
 
-The diagram below shows the general layout of high level components, including optional components, in a Kubernetes cluster that is hosting WebLogic domains and the operator:
+The diagram below shows the general layout of high-level components, including optional components, in a Kubernetes cluster that is hosting WebLogic domains and the operator:
 
 ![High level architecture](images/high-level-architecture.png)
 
@@ -32,11 +32,11 @@ This diagram shows the following details:
 
 *	A persistent volume is created using one of the available providers.  The chosen provider must support “Read Write Many” access mode.  A persistent volume claim is created to claim space in that persistent volume.  Both the persistent volume and the persistent volume claim are labeled with `weblogic.domainUID` and these labels allow the operator to find the correct volume for a particular domain.  There must be a different persistent volume for each domain.  The shared state on the persistent volume include the “domain” directory, the “applications” directory, a directory for storing logs and a directory for any file-based persistence stores.
 
-*	A pod is created for the WebLogic Administration Server.  This pod is labeled with `weblogic.domainUID`, `weblogic.serverName` and `weblogic.domainName`.  One container runs in this pod.  WebLogic Node Manager and Administration Server processes are run inside this container.  The Node Manager process is used as an internal implementation detail for the liveness probe, for patching and to provide monitoring and control capabilities to the administration console.  It is not intended to be used for other purposes, and it may be removed in some future release.  
+*	A pod is created for the WebLogic Administration Server.  This pod is labeled with `weblogic.domainUID`, `weblogic.serverName` and `weblogic.domainName`.  One container runs in this pod.  WebLogic Node Manager and Administration Server processes are run inside this container.  The Node Manager process is used as an internal implementation detail for the liveness probe, for patching and to provide monitoring and control capabilities to the Administration Console.  It is not intended to be used for other purposes, and it may be removed in some future release.
 *	A `ClusterIP` type service is created for the Administration Server pod.  This service provides a stable, well-known network (DNS) name for the Administration Server.  This name is derived from the `domainUID` and the Administration Server name, and it is known before starting up any pod.  The Administration Server `ListenAddress` is set to this well-known name.  `ClusterIP` type services are only visible inside the Kubernetes cluster.  They are used to provide the well-known names that all of the servers in a domain use to communicate with each other.  This service is labeled with `weblogic.domainUID` and `weblogic.domainName`.
 *	A `NodePort` type service is created for the Administration Server pod.  This service provides HTTP access to the Administration Server to clients that are outside the Kubernetes cluster.  This service is intended to be used to access the WebLogic Server Administration Console only.  This service is labeled with `weblogic.domainUID` and `weblogic.domainName`.
 * If requested when configuring the domain, a second `NodePort` type service is created for the Administration Server pod.  This second service is used to expose a WebLogic channel for the T3 protocol.  This service provides T3 access to the Administration Server to clients that are outside the Kubernetes cluster.  This service is intended to be used for WLST connections to the Administration Server.  This service is labeled with `weblogic.domainUID` and `weblogic.domainName`.
-*	A pod is created for each WebLogic Managed Server.  These pods are labeled with `weblogic.domainUID`, `weblogic.serverName` and `weblogic.domainName`.  One container runs in each pod.  WebLogic Node Manager and Managed Server processes are run inside each of these containers.  The Node Manager process is used as an internal implementation detail for the liveness probe.  It is not intended to be used for other purposes, and it may be removed in some future release.  
+*	A pod is created for each WebLogic Managed Server.  These pods are labeled with `weblogic.domainUID`, `weblogic.serverName` and `weblogic.domainName`.  One container runs in each pod.  WebLogic Node Manager and Managed Server processes are run inside each of these containers.  The Node Manager process is used as an internal implementation detail for the liveness probe.  It is not intended to be used for other purposes, and it may be removed in some future release.
 *	A `NodePort` type service is created for each Managed Server pod that contains a Managed Server that is not part of a WebLogic cluster.  These services provide HTTP access to the Managed Servers to clients that are outside the Kubernetes cluster.  These services are intended to be used to access applications running on the Managed Servers.  These services are labeled with `weblogic.domainUID` and `weblogic.domainName`.
 *	An Ingress is created for each WebLogic cluster.  This Ingress provides load balanced HTTP access to all Managed Servers in that WebLogic cluster.  The operator updates the Ingress every time a Managed Server in the WebLogic cluster becomes “ready” or ceases to be able to service requests, such that the Ingress always points to just those Managed Servers that are able to handle user requests.  The Ingress is labeled with `weblogic.domainUID`, `weblogic.clusterName` and `weblogic.domainName`.  The Ingress is also annotated with a class which is used to match Ingresses to the correct instances of the load balancer.  In the Technology Preview release, there is one instance of the load balancer running for each WebLogic cluster, and the load balancers are configured with the root URL path (“/”).  More flexible load balancer configuration is planned for a future release.
 *	If the ELK integration was requested when configuring the operator, there will also be another pod that runs logstash in a container.  This pod will publish the logs from all WebLogic Server instances in the domain into ElasticSearch.  There is one logstash per domain, but only one ElasticSearch and one Kibana for the entire Kubernetes cluster.
@@ -59,7 +59,7 @@ The operator expects (and requires) that all state be stored outside of the Dock
 
 It is worth providing some background on why this approach was adopted, in addition to the fact that this separation is consistent with other existing operators (for other products) and the Kubernetes “cattle, not pets” philosophy when it comes to containers.
 
-The external state approach allows the operator to treat the Docker images as essentially immutable, read-only, binary images.  This means that the image needs to be pulled only once, and that many domains can share the same image.  This helps to minimize the amount of bandwidth and storage needed for WebLogic Server Docker images.  
+The external state approach allows the operator to treat the Docker images as essentially immutable, read-only, binary images.  This means that the image needs to be pulled only once, and that many domains can share the same image.  This helps to minimize the amount of bandwidth and storage needed for WebLogic Server Docker images.
 
 This approach also eliminates the need to manage any state created in a running container, because all of the state that needs to be preserved is written into either the persistent volume or a database back end. The containers and pods are completely throwaway and can be replaced with new containers and pods as necessary.  This makes handling failures and rolling restarts much simpler because there is no need to preserve any state inside a running container.