Blog

Setting environment variables in php-fpm when using Docker links

Update: Since 7/May/2014 docker v0.11.1 was released with a feature called “Link hostnames“. It is best to use them instead of environment variables and skip reading this blog post.

If you use Docker with php FastCGI Process Manager (php-fpm) then you may find that container linking is a bit tricky to get working correctly. This post gives one solution.

In PHP a common use-case is that you might have one Docker container serving your PHP webapp with nginx and php-fpm and a 2nd container running your database. If you link the two using the --link externalname:internalname docker run parameter. Docker creates environment variables containing the IP addresses and ports of the linked containers. The environment variables Docker creates look like this:

DB_PORT_6379_TCP_PROTO=tcp
DB_PORT_6379_TCP_ADDR=172.17.0.8
DB_PORT_6379_TCP_PORT=6379

and we want to use them in our PHP code using getenv (e.g. getenv('DB_PORT_27017_TCP_ADDR')) , maybe then your DB connection config might look something like this:

 'db'=>['connectionString' =>'mysql:host='.getenv('DB_PORT_3306_TCP_ADDR').';port='.getenv('DB_PORT_3306_TCP_PORT').';dbname=mydb'],

This works fine on the CLI, but when running via php-fpm, the environment variables will be ignored. It turns out that you have to explicitly set the ENV vars in the php-fpm.conf. Here’s an example, but the last 2 lines are the ones we need:

[global]
pid = /var/run/php5-fpm.pid
error_log = /var/log/php5-fpm.log

[www]
user = www-data
group = www-data
listen = /var/run/php5-fpm.sock
pm = dynamic
pm.max_children = 5
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
chdir = /
env[DB_PORT_3306_TCP_ADDR] = 172.17.0.2
env[DB_PORT_3306_TCP_PORT] = 3306

Ok, so now the question is how do you put these environment variables in the fpm config file automatically?

It took me 3 attempts before I was happyish. All attempts rely on using supervisord in my webapp container.

The first attempt used supervisord to run a little php cli script before the php5-fpm service. The php script adds the needed env vars to the config.

This is the end portion of the Dockerfile that builds the PHP webapp container showing we add a custom supervisord config and use supervisor as our ENTRYPOINT:

...
RUN apt-get install --assume-yes supervisor

# Put the supervisord configs file into place
ADD ./containerconfig/supervisord.conf /etc/supervisor/supervisord.conf
RUN mkdir -p /var/log/supervisor

# The whole container runs as if it was just the supervisord executable
ENTRYPOINT ["/usr/bin/supervisord"]

# We use CMD to supply a default argument to the entrypoint command (if none is specified)
# NOTE1: CMD in a Dockerfile is completely replaced by what we provide on the commandline.
# NOTE2: An image imported via docker pull (or docker import) won't know what command to run. Any image will lose all of its associated metadata on export
CMD ["--nodaemon"]

This is the supervisord config that we add to the container:

; This will run the webserver software - nginx and php5-fpm
; This will also run a SSH daemon for debugging
; It will also update php-fpm with any environment variables created by Docker

[supervisord]
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
childlogdir=/var/log/supervisor             ; ('AUTO' child log dir, default $TEMP)

[program:sshd]
command=/usr/sbin/sshd -D
priority=900
autorestart=true

[program:nginx]
command=/usr/sbin/nginx -g "daemon off;"
priority=990
username=www-data
autorestart=true

; Run this script once before starting php5-fpm - it sets environment variables
[program:pre-php5-fpm]
command=/opt/setuplinks.php
priority=998
autostart=true
startretries=0
exitcodes=0
nodaemon=true
stdout_logfile=/var/log/supervisor/%(program_name)s.log
stderr_logfile=/var/log/supervisor/%(program_name)s.log

[program:php5-fpm]
command=/usr/sbin/php5-fpm --nodaemonize
priority=999
username=www-data
autorestart=true

Notice we have a pre-php5-fpm program that runs a script before the php5-fpm service runs (it has a higher priority value).

Here is the /opt/setuplinks.php script:

#!/usr/bin/php
$envVal ) {

    if (stristr($envName, '_PORT_') !== false) {
        $fileText = file_get_contents($confFile);

        # Either Add or Reset the variable
        if (strstr($fileText, $envName) !== false) {
            `sed -i "s/^env\[$envName.*/env[$envName] = $envVal/g" $confFile`;
            echo "MODIFIED $envName\n";
        } else {
            `echo "env[$envName] = $envVal" >>$confFile`;
            echo "ADDED    $envName\n";
        }
    }
}

// Log something - Helps use know this script has run because out will be in the supervisord log
echo "DONE\n";

Unfortunately, the above script doesn’t work because it does not finish before supervisord starts fpm. I need to be sure the environment variables are added to the fpm conf before fpm is started. In my 2nd attempt I solved this issue using a wrapper script around fpm:

; This script sets environment variables before starting php5-fpm with --nodaemonize
[program:php5-fpm]
command=/opt/startFPMWithDockerEnvs.sh
priority=999
stdout_logfile=/var/log/supervisor/%(program_name)s.log
stderr_logfile=/var/log/supervisor/%(program_name)s.log
autorestart=true

I prefer not to use PHP for the wrapper script so this is the bash version that does the same thing (thanks to Robert Gründler):

#!/bin/bash

# Function to update the fpm configuration to make the service environment variables available
function setEnvironmentVariable() {

    if [ -z "$2" ]; then
        echo "Environment variable '$1' not set."
        return
    fi

    # Check whether variable already exists
    if grep -q $1 /etc/php5/fpm/pool.d/www.conf; then
        # Reset variable
        sed -i "s/^env\[$1.*/env[$1] = $2/g" /etc/php5/fpm/pool.d/www.conf
    else
        # Add variable
        echo "env[$1] = $2" >> /etc/php5/fpm/pool.d/www.conf
    fi
}

# Grep for variables that look like docker set them (_PORT_)
for _curVar in `env | grep _PORT_ | awk -F = '{print $1}'`;do
    # awk has split them by the equals sign
    # Pass the name and value to our function
    setEnvironmentVariable ${_curVar} ${!_curVar}
done

# Log something to the supervisord log so we know this script as run
echo "DONE"

# Now start php-fpm
/usr/sbin/php5-fpm --nodaemonize

This worked but I did not like that the php5-fpm service is now not managed directly by supervisord. The 3rd and final attempt defines the php5-fpm program with autostart=false. Then the prep script calls supervisorctl start php5-fpm when it has done it’s stuff to get supervisord to start php5-fpm. This way I know that the prep script will finish before the other one starts.

This is the relevant portion of the supervisord conf:

; Run this script once before starting php5-fpm - it adds environment variables to the fpm config
[program:pre-php5-fpm]
command=/opt/startFPMWithDockerEnvs.php
priority=998
autostart=true
startretries=0
exitcodes=0
nodaemon=true
stdout_logfile=/var/log/supervisor/%(program_name)s.log
stderr_logfile=/var/log/supervisor/%(program_name)s.log

; Note: autostart=false. Once started keep in foreground via --nodaemonize
[program:php5-fpm]
command=/usr/sbin/php5-fpm --nodaemonize
priority=999
stdout_logfile=/var/log/supervisor/%(program_name)s.log
stderr_logfile=/var/log/supervisor/%(program_name)s.log
autostart=false        ; Don't start this automatically. Leave it to the 'pre-php5-fpm' program to start it.
autorestart=true
username=www-data

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; This next stuff is needed to be able to use supervisord from the command line.

[unix_http_server]
file=%(here)s/supervisor.sock

[supervisorctl]
serverurl=unix://%(here)s/supervisor.sock

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

This is the new ending to the prep script:

`supervisorctl -c /etc/supervisor/supervisord_nonlive.conf start php5-fpm`;
echo "DONE\n";

Now we can see from the logs that everything is starting in order:

2014-03-13 15:57:55,830 INFO supervisord started with pid 1
2014-03-13 15:57:56,834 INFO spawned: 'sshd' with pid 8
2014-03-13 15:57:56,835 INFO spawned: 'nginx' with pid 9
2014-03-13 15:57:56,836 INFO spawned: 'pre-php5-fpm' with pid 10
2014-03-13 15:57:56,997 INFO spawned: 'php5-fpm' with pid 37
2014-03-13 15:57:58,033 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-03-13 15:57:58,033 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-03-13 15:57:58,034 INFO success: pre-php5-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-03-13 15:57:58,034 INFO success: php5-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-03-13 15:57:58,058 INFO exited: pre-php5-fpm (exit status 0; expected)

Job Done! (Thanks goes to Brian Lalor for help on the docker mailing list)

Warning: Debugging is made extra hard because usually you would SSH into a container to debug it. However the problem is that --link environment variables don’t appear in the SSH session. The environment variables exist, its just they aren’t visible to the SSH session. For more info see this github issue: https://github.com/dotcloud/docker/issues/2569.

Tags > ,

8 Comments

  • Michael Zedeler

    You can see the environment variables by using the /proc directory like so:

    cat /proc/1/environ | tr -s ’00’ ‘\n’

    Reply
    • Tom

      Nice tip – Thanks!

      Reply
  • Chris

    > Update: Since 7/May/2014 docker v0.11.1 was released with a feature called “Link hostnames“. It is best to use them instead of environment variables and skip reading this blog post.

    Could you expand on this? How does link hostnames get around the requirement by php-fpm for env variables to be set in it’s config?

    Reply
    • Tom

      As an example, Lets say you need to connect to a database in PHP and the container was started with `–link mysqlcontainer:db`,

      Before Docker v0.11 (link hostnames), you would need to do:

      $dbhost = getenv(‘DB_PORT_3306_TCP_ADDR’);

      But since Docker v0.11, you just do:

      $dbhost = ‘db’;

      This works because there is now a entry in /etc/hosts in the container that maps the ‘db’ hostname to a IP address and Docker handles the networking for us.

      Reply
      • Chris

        Thanks for the reply Tom! I guess I was wondering at the username/password portion to avoid hard coding.

        Would you just have some default env vars like docker/docker which can be overridden in production for more secure username/passwords?

        Reply
        • Tom

          Hi Chris, it’s true that Docker link hostnames don’t help you inject username/password into the PHP code of your container. If you pass these in via `docker run -e “username=blah”` then the hacks in this blog post are still relevant.

          But yes, what you suggested is another way, configure dummy username/password in the container, and then override them in production. One way to override them is to ensure the folder where your PHP config file lives is exposed via a Docker Volume so you can change the values after the container starts, from the host, or from another docker container that uses `–volumes-from=””` to get access to our PHP config files.

          I think the really cool kids are getting containers to auto-discover things like usernames themselves based on what environment they are told they are in. I think etcd is the core technology for this but I haven’t used it myself yet…

          Reply
  • Dan

    Thank you for this very informative post. It was exactly what I was looking for. I fixed up the run script a little bit because some of the env values had colons. Also the script would not add some of the shortended variable names. This is probably because I am running Docker 1.1.

    #!/bin/bash

    CONF_FILE_LOCATION=”/etc/php5/fpm/pool.d/www.conf”

    # Function to update the fpm configuration to make the service environment variables available
    function setEnvironmentVariable() {

    if [ -z “$2” ]; then
    echo “Environment variable ‘$1’ not set.”
    return
    fi

    # Check whether variable already exists
    if grep -q “env\[$1\]” “$CONF_FILE_LOCATION”; then
    # Reset variable
    sed -i ‘s/^env\[$1].*/c\env[$1] = “$2″/g’ “$CONF_FILE_LOCATION”
    else
    # Add variable
    echo “env[$1] = $2” >> “$CONF_FILE_LOCATION”
    fi
    }

    # Grep for variables that look like docker set them (_PORT_)
    for _curVar in `env | grep _PORT | awk -F = ‘{print $1}’`;do
    # awk has split them by the equals sign
    # Pass the name and value to our function
    setEnvironmentVariable ${_curVar} ${!_curVar}
    done

    # Log something to the supervisord log so we know this script as run
    echo “DONE”

    # Now start php-fpm
    /usr/sbin/php5-fpm -c /etc/php5/fpm

    Reply
  • Lefthandedsquid

    You can set up the environment variables in php-fpm’s config using something like

    env[SOME_VAR]=$SOME_VAR

    If $SOME_VAR is defined by the calling process then it will pass that on to php-fpm

    Reply

Post A Reply to Tom Cancel Reply