Merge pull request #52 from ibm-messaging/v2

Merge v2 feature branch

Author: arthurbarr

server/listener.mqsc renamed to 10-listener.mqsc

@@ -1,4 +1,4 @@
-* © Copyright IBM Corporation 2015, 2016
+* © Copyright IBM Corporation 2015, 2017
 *
 *
 * Licensed under the Apache License, Version 2.0 (the "License");

CHANGELOG.md

@@ -1,5 +1,17 @@
 # Change log
 
+## 2.0.0 (2017-03-11)
+### Action required
+* Ensure that you use the `REPLACE` keyword in all of your `DEFINE` MQSC statements.  With this change, any supplied MQSC files are run *every* time the queue manager runs.  This allows you to update the MQSC file, re-build the image, and then have the changes applied when you start a container based on that new image.
+* Code has been re-structured to use git branches for older versions of MQ.
+
+### Other notable changes
+* Updated to MQ V9.0.1, adding the web console on port 9443.
+* Updated base image to Ubuntu 16.04
+* Set version number in command prompt dynamically
+* NFS and Bluemix Volume support added. (See: `setup-var-mqm.sh`).  Note that it is now recommended to mount volumes into `/mnt/mqm` instead of `/var/mqm`.
+* Added MQ Developer Defaults, to provide better defaults for security, as well as queues and topics useful for development
+
 ## 1.0.2 (2016-11-02)
 * Add MQ V9
 * Don't apply CMDLEVEL unless specifically requested

server/Dockerfile renamed to Dockerfile

@@ -1,4 +1,4 @@
-# © Copyright IBM Corporation 2015, 2016
+# © Copyright IBM Corporation 2015, 2017
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -12,19 +12,17 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM ubuntu:14.04
+FROM ubuntu:16.04
 
-MAINTAINER Arthur Barr <[email protected]>
+LABEL maintainer "Arthur Barr <[email protected]>"
 
 # The URL to download the MQ installer from in tar.gz format
-ARG MQ_URL=http://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev90_linux_x86-64.tar.gz
+ARG MQ_URL=http://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev901_linux_x86-64.tar.gz
 
 # The MQ packages to install
-ARG MQ_PACKAGES="MQSeriesRuntime-*.rpm MQSeriesServer-*.rpm MQSeriesMsg*.rpm MQSeriesJava*.rpm MQSeriesJRE*.rpm MQSeriesGSKit*.rpm"
+ARG MQ_PACKAGES="MQSeriesRuntime-*.rpm MQSeriesServer-*.rpm MQSeriesMsg*.rpm MQSeriesJava*.rpm MQSeriesJRE*.rpm MQSeriesGSKit*.rpm MQSeriesWeb*.rpm"
 
 RUN export DEBIAN_FRONTEND=noninteractive \
-  # Optional: Update the command prompt
-  && echo "mq:9.0" > /etc/debian_chroot \
   # Install additional packages required by MQ, this install process and the runtime scripts
   && apt-get update -y \
   && apt-get install -y --no-install-recommends \
@@ -37,6 +35,7 @@ RUN export DEBIAN_FRONTEND=noninteractive \
     gawk \
     grep \
     libc-bin \
+    lsb-release \
     mount \
     passwd \
     procps \
@@ -51,7 +50,7 @@ RUN export DEBIAN_FRONTEND=noninteractive \
   && tar -zxvf ./*.tar.gz \
   # Recommended: Create the mqm user ID with a fixed UID and group, so that the file permissions work between different images
   && groupadd --gid 1000 mqm \
-  && useradd --uid 1000 --gid mqm --home-dir /var/mqm mqm \
+  && useradd --uid 1000 --gid mqm mqm \
   && usermod -G mqm root \
   && cd /tmp/mq/MQServer \
   # Accept the MQ license
@@ -62,17 +61,27 @@ RUN export DEBIAN_FRONTEND=noninteractive \
   && /opt/mqm/bin/setmqinst -p /opt/mqm -i \
   # Clean up all the downloaded files
   && rm -rf /tmp/mq \
-  && rm -rf /var/lib/apt/lists/*
+  && rm -rf /var/lib/apt/lists/* \
+  # Optional: Update the command prompt with the MQ version
+  && echo "mq:$(dspmqver -b -f 2)" > /etc/debian_chroot \
+  && rm -rf /var/mqm \
+  # Optional: Set these values for the Bluemix Vulnerability Report
+  && sed -i 's/PASS_MAX_DAYS\t99999/PASS_MAX_DAYS\t90/' /etc/login.defs \
+  && sed -i 's/PASS_MIN_DAYS\t0/PASS_MIN_DAYS\t1/' /etc/login.defs \
+  && sed -i 's/password\t\[success=1 default=ignore\]\tpam_unix\.so obscure sha512/password\t[success=1 default=ignore]\tpam_unix.so obscure sha512 minlen=8/' /etc/pam.d/common-password
 
 COPY *.sh /usr/local/bin/
 COPY *.mqsc /etc/mqm/
+COPY admin.json /etc/mqm/
+
+COPY mq-dev-config /etc/mqm/mq-dev-config
 
 RUN chmod +x /usr/local/bin/*.sh
 
 # Always use port 1414 (the Docker administrator can re-map ports at runtime)
-EXPOSE 1414
+# Expose port 9443 for the web console
+EXPOSE 1414 9443
 
-# Always put the MQ data directory in a Docker volume
-VOLUME /var/mqm
+ENV LANG=en_US.UTF-8
 
 ENTRYPOINT ["mq.sh"]

README.md

@@ -1,12 +1,34 @@
+# Contents
+* [Overview](#overview)
+* [Docker Hub](#docker-hub)
+* [Preparing your Docker host](#preparing-your-docker-host)
+* [Build](#build)
+* [Usage](#usage)
+    * [Running with the default configuration](#running-with-the-default-configuration)
+    * [Running on Bluemix with volumes](#running-on-bluemix-with-volumes)
+    * [Customizing the queue manager configuration](#customizing-the-queue-manager-configuration)
+    * [Running MQ commands](#running-mq-commands)
+    * [Installed components](#installed-components)
+    * [MQ developer defaults](#mq-developer-defaults)
+    * [Customizing MQ developer defaults](#customizing-mq-developer-defaults)
+    * [Web console](#web-console)
+    * [List of all environment variables supported by this image](#list-of-all-environment-variables supported by this image)
+* [Troubleshooting](#troubleshooting)
+    * [Container command not found or does not exist](#container-command-not-found-or-does-not-exist)
+    * [AMQ7017: Log not available](#amq7017-log-not-available)
+* [Issues and contributions](#issues-and-contributions)
+* [License](#license)
+
 # Overview
 
 Run [IBM® MQ](http://www-03.ibm.com/software/products/en/ibm-mq) in a Docker container.  By default, the supplied Dockerfile runs [IBM MQ for Developers](http://www-03.ibm.com/software/products/en/ibm-mq-advanced-for-developers), but also works for IBM MQ.  The source can be found on the [ibm-messaging GitHub](http://github.com/ibm-messaging/mq-docker).  There's also a short [demo video](https://www.youtube.com/watch?v=BoomAVqk0cI) available.
 
 # Docker Hub
 The image is available on Docker Hub as [`ibmcom/mq`](https://hub.docker.com/r/ibmcom/mq/) with the following tags:
 
-  * `9`, `latest` ([Dockerfile](https://github.com/ibm-messaging/mq-docker/blob/master/server/Dockerfile))
-  * `8` ([Dockerfile](https://github.com/ibm-messaging/mq-docker/blob/master/server/Dockerfile-mq8))
+  * `cd`, `9-cd`, `9`, `latest` ([Dockerfile](https://github.com/ibm-messaging/mq-docker/blob/master/server/Dockerfile))
+  * `lts`, `9-lts` ([Dockerfile](https://github.com/ibm-messaging/mq-docker/blob/mq-9-lts/Dockerfile))
+  * `8` ([Dockerfile](https://github.com/ibm-messaging/mq-docker/blob/mq-8/Dockerfile))
 
 # Preparing your Docker host
 You need to make sure that you either have a Linux kernel version of V3.16, or else you need to add the [`--ipc host`](http://docs.docker.com/reference/run/#ipc-settings) option when you run an MQ container.  The reason for this is that IBM MQ uses shared memory, and on Linux kernels prior to V3.16, containers are usually limited to 32 MB of shared memory.  In a [change](https://git.kernel.org/cgit/linux/kernel/git/mhocko/mm.git/commit/include/uapi/linux/shm.h?id=060028bac94bf60a65415d1d55a359c3a17d5c31
@@ -16,13 +38,7 @@ You need to make sure that you either have a Linux kernel version of V3.16, or e
 After extracting the code from this repository, you can build an image with the latest version of MQ using the following command:
 
 ```
-sudo docker build --tag mq ./server/
-```
-
-To build alternative versions, you can use commands similar to the following:
-
-```
-sudo docker build --tag mq:8 --file ./server/Dockerfile-mq8 ./server/
+sudo docker build --tag mq .
 ```
 
 # Usage
@@ -31,27 +47,39 @@ In order to use the image, it is necessary to accept the terms of the IBM MQ lic
 This image is primarily intended to be used as an example base image for your own MQ images.
 
 ## Running with the default configuration
-You can run a queue manager with the default configuration and a listener on port 1414 using the following command.  Note that the default configuration is locked-down from a security perspective, so you will need to customize the configuration in order to effectively use the queue manager.  For example, the following command creates and starts a queue manager called `QM1`, and maps port 1414 on the host to the MQ listener on port 1414 inside the container:
+You can run a queue manager with the default configuration and a listener on port 1414 using the following command.  Note that the default configuration is locked-down from a security perspective, so you will need to customize the configuration in order to effectively use the queue manager.  For example, the following command creates and starts a queue manager called `QM1`, and maps port 1414 on the host to the MQ listener on port 1414 inside the container, as well as port 9443 on the host to the web console on port 9443 inside the container:
 
 ```
 sudo docker run \
   --env LICENSE=accept \
   --env MQ_QMGR_NAME=QM1 \
-  --volume /var/example:/var/mqm \
+  --volume /var/example:/mnt/mqm \
   --publish 1414:1414 \
+  --publish 9443:9443 \
   --detach \
   mq
 ```
 
 Note that in this example, the name "mq" is the image tag you used in the previous build step.
 
-Also note that the filesystem for the mounted volume directory (`/var/example` in the above example) must be [supported](http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_8.0.0/com.ibm.mq.pla.doc/q005820_.htm?lang=en).
+Also note that the filesystem for the mounted volume directory (`/var/example` in the above example) must be [supported](http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.pla.doc/q005820_.htm?lang=en).
+
+## Running on IBM Bluemix with volumes
+If you wish to run a queue manager with default configuration and a listener on port 1414, but using an IBM Bluemix volume to store your data you will need to mount the volume in a different directory than `/var/mqm`. When using a volume in Bluemix, special actions need to be taken in order to mount the IBM MQ data directory with the correct permissions on the volume. These actions are performed in the `setup-var-mqm.sh` script. The script is configured to look for a directory called `/mnt/mqm`, if it finds this then it will perform the special actions to create the IBM MQ data directory. When using mounting a volume to a Bluemix container you should mount the volume to the `/mnt/mqm` directory:
+
+```
+bx ic run \
+  --env LICENSE=accept \
+  --env MQ_QMGR_NAME=QM1 \
+  --volume /var/example:/mnt/mqm \
+  mq
+```
 
 ## Customizing the queue manager configuration
 You can customize the configuration in several ways:
 
-1. By creating your own image and adding your an MQSC file called `/etc/mqm/config.mqsc`.  This file will be run when your queue manager is created.
-2. By using [remote MQ administration](http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_8.0.0/com.ibm.mq.adm.doc/q021090_.htm).  Note that this will require additional configuration as remote administration is not enabled by default.
+1. By creating your own image and adding your own MQSC file into the `/etc/mqm` directory on the image.  This file will be run when your queue manager is created.
+2. By using [remote MQ administration](http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.adm.doc/q021090_.htm).  Note that this will require additional configuration as remote administration is not enabled by default.
 
 Note that a listener is always created on port 1414 inside the container.  This port can be mapped to any port on the Docker host.
 
@@ -61,13 +89,13 @@ The following is an *example* `Dockerfile` for creating your own pre-configured
 FROM mq
 RUN useradd alice -G mqm && \
     echo alice:passw0rd | chpasswd
-COPY config.mqsc /etc/mqm/
+COPY 20-config.mqsc /etc/mqm/
 ```
 
-Here is an example corresponding `config.mqsc` script from the [mqdev blog](https://www.ibm.com/developerworks/community/blogs/messaging/entry/getting_going_without_turning_off_mq_security?lang=en), which allows users with passwords to connect on the `PASSWORD.SVRCONN` channel:
+Here is an example corresponding `20-config.mqsc` script from the [mqdev blog](https://www.ibm.com/developerworks/community/blogs/messaging/entry/getting_going_without_turning_off_mq_security?lang=en), which allows users with passwords to connect on the `PASSWORD.SVRCONN` channel:
 
 ```
-DEFINE CHANNEL(PASSWORD.SVRCONN) CHLTYPE(SVRCONN)
+DEFINE CHANNEL(PASSWORD.SVRCONN) CHLTYPE(SVRCONN) REPLACE
 SET CHLAUTH(PASSWORD.SVRCONN) TYPE(BLOCKUSER) USERLIST('nobody') DESCR('Allow privileged users on this channel')
 SET CHLAUTH('*') TYPE(ADDRESSMAP) ADDRESS('*') USERSRC(NOACCESS) DESCR('BackStop rule')
 SET CHLAUTH(PASSWORD.SVRCONN) TYPE(ADDRESSMAP) ADDRESS('*') USERSRC(CHANNEL) CHCKCLNT(REQUIRED)
@@ -93,6 +121,80 @@ Using this technique, you can have full control over all aspects of the MQ insta
 
 This image includes the core MQ server, Java, language packs, and GSKit.  Other features (except the client) are not currently supported running in Docker.  See the [MQ documentation](http://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.ins.doc/q008350_.htm) for details of which RPMs to choose.
 
+## MQ Developer Defaults
+
+This image includes the MQ Developer defaults scripts which are automatically ran during Queue Manager startup. This configures your Queue Manager with a set of default objects that you can use to quickly get started developing with IBM MQ. If you do not want the default objects to be created you can set the `MQ_DEV` environment variable to `false`.
+
+#### Users
+**Userid:**   admin
+**Groups:**   mqm
+**Password:** passw0rd
+
+**Userid:**   app
+**Groups:**   mqclient
+**Password:**
+
+#### Queues
+* DEV.QUEUE.1
+* DEV.QUEUE.2
+* DEV.QUEUE.3
+* DEV.DEAD.LETTER.QUEUE - Set as the Queue Manager's Dead Letter Queue.
+
+#### Channels
+* DEV.ADMIN.SVRCONN - Set to only allow the `admin` user to connect into it and a Userid + Password must be supplied.
+* DEV.APP.SVRCONN - Does not allow Administrator users to connect.
+
+#### Listener
+* DEV.LISTENER.TCP - Listening on Port 1414.
+
+#### Topic
+DEV.BASE.TOPIC - With a topic string of `dev/`.
+
+#### Authentication information
+* DEV.AUTHINFO - Set to use OS as the user repository and adopt supplied users for authorization checks
+
+#### Authority records
+* Users in `mqclient` group have been given access connect to all Queues and topics starting with `DEV.**` and have `put` `get` `pub` and `sub` permissions.
+
+## Customizing MQ Developer Defaults
+
+The MQ Developer Defaults supports some customization options, these are all controlled using environment variables:
+
+* **MQ_DEV** - Set this to `false` to stop the Default objects being created.
+* **MQ_ADMIN_PASSWORD** - Changes the password of the `admin` user. Must be at least 8 characters long.
+* **MQ_APP_PASSWORD** - Changes the password of the app user. If set, this will cause the `DEV.APP.SVRCONN` channel to become secured and only allow connections that supply a valid userid and password. Must be at least 8 characters long.
+* **MQ_TLS_KEYSTORE** - Allows you to supply the location of a PKCS#12 keystore containing a single certificate which you want to use in both the web console and the queue manager. Requires `MQ_TLS_PASSPHRASE`. When enabled the channels created will be secured using the `TLS_RSA_WITH_AES_256_GCM_SHA384` CipherSpec. *Note*: you will need to make the keystore available inside your container, this can be done by mounting a volume to your container.
+* **MQ_TLS_PASSPHRASE** - Passphrase for the keystore referenced in `MQ_TLS_KEYSTORE`.
+
+## Web Console
+
+By default the image will start the IBM MQ Web Console that allows you to administer your Queue Manager running on your container. When the web console has been started, you can access it by opening a web browser and navigating to https://<Container IP>:9443/ibmmq/console. Where <Container IP> is replaced by the IP address of your running container.
+
+When you navigate to this page you may be presented with a security exception warning. This happens because, by default, the web console creates a self-signed certificate to use for the HTTPS operations. This certificate is not trusted by your browser and has an incorrect distinguished name.
+
+If you chose to accept the security warning, you will be presented with the login menu for the IBM MQ Web Console. The default login for the console is:
+
+* **User:** admin
+* **Password:** passw0rd
+
+If you wish to change the password for the admin user, this can be done using the `MQ_ADMIN_PASSWORD` environment variable. If you supply a PKCS#12 keystore using the `MQ_TLS_KEYSTORE` environment variable, then the web console will be configured to use the certificate inside the keystore for HTTPS operations.
+
+If you do not wish the web console to run, you can disable it by setting the environment variable `MQ_DISABLE_WEB_CONSOLE` to `true`.
+
+## List of all Environment variables supported by this image
+
+* **LICENSE** - Set this to `accept` to agree to the MQ Advanced for Developers license. If you wish to see the license you can set this to `view`.
+* **LANG** - Set this to the language you would like the license to be printed in.
+* **MQ_QMGR_NAME** - Set this to the name you want your Queue Manager to be created with.
+* **MQ_QMGR_CMDLEVEL** - Set this to the `CMDLEVEL` you wish your Queue Manager to be started with.
+* **MQ_DEV** - Set this to `false` to stop the Default objects being created.
+* **MQ_ADMIN_PASSWORD** - Changes the password of the `admin` user. Must be at least 8 characters long.
+* **MQ_APP_PASSWORD** - Changes the password of the app user. If set, this will cause the `DEV.APP.SVRCONN` channel to become secured and only allow connections that supply a valid userid and password. Must be at least 8 characters long.
+* **MQ_TLS_KEYSTORE** - Allows you to supply the location of a PKCS#12 keystore containing a single certificate which you want to use in both the web console and the queue manager. Requires `MQ_TLS_PASSPHRASE`. When enabled the channels created will be secured using the `TLS_RSA_WITH_AES_256_GCM_SHA384` CipherSpec. *Note*: you will need to make the keystore available inside your container, this can be done by mounting a volume to your container.
+* **MQ_TLS_PASSPHRASE** - Passphrase for the keystore referenced in `MQ_TLS_KEYSTORE`.
+* **MQ_DISABLE_WEB_CONSOLE** - Set this to `true` if you want to disable the Web Console from being started.
+
+
 # Troubleshooting
 
 ## Container command not found or does not exist

admin.json

@@ -0,0 +1 @@
+{"version":0.1,"tabs":[{"title":"IBM MQ Container","numColumns":2,"model":{"title":"","rows":[{"columns":[{"widgets":[{"type":"channel","config":{"selectedQM":"<QM>","showSysObjs":false,"sizex":1,"sizey":1,"subType":"all"},"title":"Channels on <QM>","titleTemplateUrl":"adf/templates/widget-title.html","gridsterrow":0,"gridstercol":1},{"type":"topic","config":{"selectedQM":"<QM>","showSysObjs":false,"sizex":1,"sizey":1},"title":"Topics on <QM>","titleTemplateUrl":"adf/templates/widget-title.html","gridsterrow":1,"gridstercol":1},{"type":"queue","config":{"selectedQM":"<QM>","showSysObjs":false,"sizex":1,"sizey":1,"subType":"all"},"title":"Queues on <QM>","titleTemplateUrl":"adf/templates/widget-title.html","gridsterrow":1,"gridstercol":0},{"type":"queuemanager","gridstercol":0,"gridsterrow":0,"config":{"type":"local","sizex":1,"sizey":1,"customTitle":"Queue Manager"},"title":"Queue Manager","titleTemplateUrl":"adf/templates/widget-title.html"}]}]}],"titleTemplateUrl":"adf/templates/dashboard-title.html"},"isMobile":false}]}