Docker syslog driver to local facility


Scenario

You have Docker installed and want to send your Docker logs (default is json-file) to a local syslog facility (i.e. local6).

You can use the --log-driver=VALUE with the docker run command to configure the container’s logging driver or you can set the parameter globally in the docker daemon configuration file. This is useful when using container orchestration such as Kubernetes or Apache Mesos.

Step #1

Edit the docker configuration file (/etc/sysconfig/docker on RHEL/CentOS based systems).

# vi /etc/sysconfig/docker

Add the log driver parameter (--log-driver=syslog --log-opt syslog-facility=local6 --log-level=warn) to the OPTIONS line.

# /etc/sysconfig/docker

# Modify these options if you want to change the way the docker daemon runs
OPTIONS='--selinux-enabled --log-driver=syslog --log-opt syslog-facility=local6 --log-level=warn'

DOCKER_CERT_PATH=/etc/docker

# If you want to add your own registry to be used for docker search and docker
# pull use the ADD_REGISTRY option to list a set of registries, each prepended
# with --add-registry flag. The first registry added will be the first registry
# searched.
ADD_REGISTRY=''

# If you want to block registries from being used, uncomment the BLOCK_REGISTRY
# option and give it a set of registries, each prepended with --block-registry
# flag. For example adding docker.io will stop users from downloading images
# from docker.io
# BLOCK_REGISTRY='--block-registry'

# If you have a registry secured with https but do not have proper certs
# distributed, you can tell docker to not look for full authorization by
## adding the registry to the INSECURE_REGISTRY line and uncommenting it.
INSECURE_REGISTRY=''

# On an SELinux system, if you remove the --selinux-enabled option, you
# also need to turn on the docker_transition_unconfined boolean.
# setsebool -P docker_transition_unconfined 1

# Location used for temporary files, such as those created by
# docker load and build operations. Default is /var/lib/docker/tmp
# Can be overriden by setting the following environment variable.
# DOCKER_TMPDIR=/var/tmp

# Controls the /etc/cron.daily/docker-logrotate cron job status.
# To disable, uncomment the line below.
# LOGROTATE=false

Restart the docker daemon.

# systemctl restart docker

Step #2

Configure the syslog daemon to listen on local6 and write logs to specified location.

Create a new file in /etc/rsyslog.d called docker.conf.

# vi /etc/rsyslog.d/docker.conf

Add the following line to the /etc/rsyslog.d/docker.conf file.

local6.*    -/var/log/docker/docker.log

Make sure that /var/log/docker exists.

# mkdir /var/log/docker

Restart the rsyslog daemon

# systemctl restart rsyslog

Start a Docker container or two. You will be now able to view all Docker logs in /var/log/docker/docker.log.

For additional information on configuring Docker's logging drivers, please visit https://docs.docker.com/engine/admin/logging/overview.

Inserting certificates into Java keystore via Dockerfile


Scenario

You have a Root CA and Issuing CA certificate that you need to import into the Java keystore of a Docker image to allow your application to make trusted calls to another secured site signed by your Issuing CA.


Create the below Dockfile to install Java, copy your certificates from your host system (relative path is ./certs) to the Docker image and use the keytool command to import the certificates into the the default Java keystore ($JAVA_HOME/lib/security/cacerts).

You will obviously want to customize this to suit your needs adding your Java application server (i.e. Apache Tomcat, Wildfly, etc.) and copy your code into the Docker image via the Dockerfile as well.

Dockerfile

FROM registry.access.redhat.com/rhel7.1

ENV JAVA_HOME=/usr/lib/jvm/jre

COPY ./certs/My_Root_CA.cer /etc/ssl/certs/
COPY ./certs/My_Issuing_CA.cer /etc/ssl/certs/


RUN yum clean all && \
    yum --releasever=7.1 swap -y -- remove systemd-container\* -- install systemd systemd-libs && \
    yum update -y --releasever=7.1 && \
    yum install java-1.8.0-openjdk --releasever=7.1 -y && \
    yum clean all && \
    $JAVA_HOME/bin/keytool -storepasswd -new mysecretpassword -keystore $JAVA_HOME/lib/security/cacerts -storepass changeit && \
    echo "yes" | $JAVA_HOME/bin/keytool -import -trustcacerts -file /etc/ssl/certs/My_Root_CA.cer -alias my-root-ca -keystore $JAVA_HOME/lib/security/cacerts -storepass mysecretpassword && \
    echo "yes" | $JAVA_HOME/bin/keytool -import -trustcacerts -file /etc/ssl/certs/My_Issuing_CA.cer -alias my-issuing-ca -keystore $JAVA_HOME/lib/security/cacerts -storepass mysecretpassword && \
    rm -f /etc/ssl/certs/My_Root_CA.cer &&\
    rm -f /etc/ssl/certs/My_Issuing_CA.cer

Command

Next you will need to build your docker image using the docker build command.

docker build -t myprivatedockerregistry:5000/rhel7.1-with-my-certs .

Your containerized application will now trust certificates signed by your Issuing CA.

Experiences patching a RHEL 7.1 base Docker image


Scenario

You have Docker installed on a host system and you want to deploy a patched Red Hat Enterprise Linux base Docker image. I have presented two options and have shown several challenges that I had to overcome.


Option #1

Start an interactive shell into a RHEL 7.1 docker container.

# docker run -it registry.access.redhat.com/rhel7.1 bash

Run yum update inside the container.

 [root@0e439b6c0ec6 /]# yum update -y
Loaded plugins: product-id, subscription-manager

https://mysatelliteserver/pulp/repos/myorganization/myenvironment/mycontentview/content/dist/rhel/server/7/7Server/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.


 One of the configured repositories failed (Red Hat Enterprise Linux 7 Server (RPMs)),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Disable the repository, so yum won't use it by default. Yum will then
        just ignore the repository until you permanently enable it again or use
        --enablerepo for temporary usage:

            yum-config-manager --disable rhel-7-server-rpms

     4. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=rhel-7-server-rpms.skip_if_unavailable=true

failure: repodata/repomd.xml from rhel-7-server-rpms: [Errno 256] No more mirrors to try.

https://mysatelliteserver/pulp/repos/myorganization/myenvironment/mycontentview/content/dist/rhel/server/7/7Server/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found

Ugh! We hit an error. If you encounter the same error, then you need to specify the Red Hat release version using the "--releasever" yum parameter. See below.

[root@0e439b6c0ec6 /]# yum update -y --releasever=7.1
 Loaded plugins: product-id, subscription-manager
Resolving Dependencies
--> Running transaction check
---> Package bash.x86_64 0:4.2.46-12.el7 will be updated
 ...
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
Importing GPG key 0xFD431D51:
 Userid     : "Red Hat, Inc. (release key 2) <security@redhat.com>"
 Fingerprint: 567e 347a d004 4ade 55ba 8a5f 199e 2f91 fd43 1d51
 Package    : redhat-release-server-7.1-1.el7.x86_64 (@koji-override-1/7.0)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
Importing GPG key 0x2FA658E0:
 Userid     : "Red Hat, Inc. (auxiliary key) <security@redhat.com>"
 Fingerprint: 43a6 e49c 4a38 f4be 9abf 2a53 4568 9c88 2fa6 58e0
 Package    : redhat-release-server-7.1-1.el7.x86_64 (@koji-override-1/7.0)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
Running transaction check
Running transaction test


Transaction check error:
  file /usr/lib64/libsystemd-daemon.so.0 from install of systemd-libs-219-19.el7.x86_64 conflicts with file from package systemd-container-libs-208.20-6.el7.x86_64
  file /usr/lib64/libsystemd-id128.so.0 from install of systemd-libs-219-19.el7.x86_64 conflicts with file from package systemd-container-libs-208.20-6.el7.x86_64
  file /usr/lib64/libsystemd-journal.so.0 from install of systemd-libs-219-19.el7.x86_64 conflicts with file from package systemd-container-libs-208.20-6.el7.x86_64
  file /usr/lib64/libsystemd-login.so.0 from install of systemd-libs-219-19.el7.x86_64 conflicts with file from package systemd-container-libs-208.20-6.el7.x86_64
  file /usr/lib64/libudev.so.1 from install of systemd-libs-219-19.el7.x86_64 conflicts with file from package systemd-container-libs-208.20-6.el7.x86_64
  file /usr/lib64/security/pam_systemd.so from install of systemd-libs-219-19.el7.x86_64 conflicts with file from package systemd-container-libs-208.20-6.el7.x86_64

Error Summary
-------------

Again, we hit another error. This error is a known Red Hat Bug (1284056). The good news is that there is a work around.

[root@0e439b6c0ec6 /]# yum --releasever=7.1 swap -y -- remove systemd-container\* -- install systemd systemd-libs
 Loaded plugins: product-id, subscription-manager
Resolving Dependencies
--> Running transaction check
---> Package systemd.x86_64 0:219-19.el7 will be installed
...
 Complete!
 [root@0e439b6c0ec6 /]# yum update -y --releasever=7.1
 Loaded plugins: product-id, subscription-manager
Resolving Dependencies
--> Running transaction check
---> Package bash.x86_64 0:4.2.46-12.el7 will be updated
...
 Complete!

Bingo! A completely patched RHEL 7.1 Docker image. The final step is to commit the changes. Exit your running container by typing "exit" and then run a docker commit command.

# docker commit 0e439b6c0ec6 myprivatedockerregistry:5000/rhel7.1-patched

Golden! And the security department is happy!!


Option #2

Now for the easy way. Create a Dockerfile with the below contents.

FROM registry.access.redhat.com/rhel7.1

RUN yum clean all && \
    yum --releasever=7.1 swap -y -- remove systemd-container\* -- install systemd systemd-libs && \
    yum update -y --releasever=7.1 && \
    yum clean all

Run the docker build command and you're done!

# docker build -t myprivatedockerregistry:5000/rhel7.1-patched .