{"id":165,"date":"2014-03-13T00:28:04","date_gmt":"2014-03-13T00:28:04","guid":{"rendered":"https:\/\/www.peopleperhour.com\/engineering\/?p=165"},"modified":"2014-05-13T10:09:34","modified_gmt":"2014-05-13T10:09:34","slug":"setting-environment-variables-in-php-fpm-when-using-docker-links","status":"publish","type":"post","link":"https:\/\/www.peopleperhour.com\/engineering\/2014\/03\/13\/setting-environment-variables-in-php-fpm-when-using-docker-links\/","title":{"rendered":"Setting environment variables in php-fpm when using Docker links"},"content":{"rendered":"

Update:<\/strong> Since 7\/May\/2014 docker v0.11.1 was released<\/a> with a feature called “Link hostnames<\/em>“. It is best to use them instead of environment variables and skip reading this blog post.<\/p>\n

If you use Docker with php FastCGI Process Manager (php-fpm<\/a>) then you may find that container linking is a bit tricky to get working correctly. This post gives one solution.<\/p>\n

In PHP a common use-case is that you might have one Docker container serving your PHP webapp with nginx and php-fpm and a 2nd container running your database. If you link the two<\/a> using the --link externalname:internalname<\/code> docker run parameter. Docker creates environment variables containing the IP addresses and ports of the linked containers. The environment variables Docker creates look like this: <\/p>\n

\r\nDB_PORT_6379_TCP_PROTO=tcp\r\nDB_PORT_6379_TCP_ADDR=172.17.0.8\r\nDB_PORT_6379_TCP_PORT=6379\r\n<\/pre>\n
and we want to use them in our PHP code using getenv<\/a> (e.g. getenv('DB_PORT_27017_TCP_ADDR')<\/code>) , maybe then your DB connection config might look something like this:<\/p>\n
\r\n 'db'=>['connectionString' =>'mysql:host='.getenv('DB_PORT_3306_TCP_ADDR').';port='.getenv('DB_PORT_3306_TCP_PORT').';dbname=mydb'],\r\n<\/pre>\n
This works fine on the CLI, but when running via php-fpm, the environment variables will be ignored. It turns out that you have to explicitly set the ENV vars in the php-fpm.conf<\/em>. Here’s an example, but the last 2 lines are the ones we need:<\/p>\n
\r\n[global]\r\npid = \/var\/run\/php5-fpm.pid\r\nerror_log = \/var\/log\/php5-fpm.log\r\n\r\n[www]\r\nuser = www-data\r\ngroup = www-data\r\nlisten = \/var\/run\/php5-fpm.sock\r\npm = dynamic\r\npm.max_children = 5\r\npm.start_servers = 2\r\npm.min_spare_servers = 1\r\npm.max_spare_servers = 3\r\nchdir = \/\r\nenv[DB_PORT_3306_TCP_ADDR] = 172.17.0.2\r\nenv[DB_PORT_3306_TCP_PORT] = 3306\r\n<\/pre>\n
Ok, so now the question is how do you put these environment variables in the fpm config file automatically?<\/p>\n
It took me 3 attempts before I was happyish. All attempts rely on using supervisord<\/a> in my webapp container.<\/p>\n
The first attempt used supervisord to run a little php cli script before the php5-fpm service. The php script adds the needed env vars to the config.<\/p>\n
This is the end portion of the Dockerfile that builds the PHP webapp container showing we add a custom supervisord config and use supervisor as our ENTRYPOINT:<\/p>\n
\r\n...\r\nRUN apt-get install --assume-yes supervisor\r\n\r\n# Put the supervisord configs file into place\r\nADD .\/containerconfig\/supervisord.conf \/etc\/supervisor\/supervisord.conf\r\nRUN mkdir -p \/var\/log\/supervisor\r\n\r\n# The whole container runs as if it was just the supervisord executable\r\nENTRYPOINT [\"\/usr\/bin\/supervisord\"]\r\n\r\n# We use CMD to supply a default argument to the entrypoint command (if none is specified)\r\n# NOTE1: CMD in a Dockerfile is completely replaced by what we provide on the commandline.\r\n# NOTE2: An image imported via docker pull (or docker import) won't know what command to run. Any image will lose all of its associated metadata on export\r\nCMD [\"--nodaemon\"]\r\n<\/pre>\n
This is the supervisord config that we add to the container:<\/p>\n
\r\n; This will run the webserver software - nginx and php5-fpm\r\n; This will also run a SSH daemon for debugging\r\n; It will also update php-fpm with any environment variables created by Docker\r\n\r\n[supervisord]\r\nlogfile=\/var\/log\/supervisor\/supervisord.log ; (main log file;default $CWD\/supervisord.log)\r\nchildlogdir=\/var\/log\/supervisor             ; ('AUTO' child log dir, default $TEMP)\r\n\r\n[program:sshd]\r\ncommand=\/usr\/sbin\/sshd -D\r\npriority=900\r\nautorestart=true\r\n\r\n[program:nginx]\r\ncommand=\/usr\/sbin\/nginx -g \"daemon off;\"\r\npriority=990\r\nusername=www-data\r\nautorestart=true\r\n\r\n; Run this script once before starting php5-fpm - it sets environment variables\r\n[program:pre-php5-fpm]\r\ncommand=\/opt\/setuplinks.php\r\npriority=998\r\nautostart=true\r\nstartretries=0\r\nexitcodes=0\r\nnodaemon=true\r\nstdout_logfile=\/var\/log\/supervisor\/%(program_name)s.log\r\nstderr_logfile=\/var\/log\/supervisor\/%(program_name)s.log\r\n\r\n[program:php5-fpm]\r\ncommand=\/usr\/sbin\/php5-fpm --nodaemonize\r\npriority=999\r\nusername=www-data\r\nautorestart=true\r\n<\/pre>\n
Notice we have a pre-php5-fpm<\/em> program that runs a script before the php5-fpm service runs (it has a higher priority value).<\/p>\n
Here is the \/opt\/setuplinks.php<\/code> script:<\/p>\n
\r\n#!\/usr\/bin\/php\r\n$envVal ) {\r\n\r\n    if (stristr($envName, '_PORT_') !== false) {\r\n        $fileText = file_get_contents($confFile);\r\n\r\n        # Either Add or Reset the variable\r\n        if (strstr($fileText, $envName) !== false) {\r\n            `sed -i \"s\/^env\\[$envName.*\/env[$envName] = $envVal\/g\" $confFile`;\r\n            echo \"MODIFIED $envName\\n\";\r\n        } else {\r\n            `echo \"env[$envName] = $envVal\" >>$confFile`;\r\n            echo \"ADDED    $envName\\n\";\r\n        }\r\n    }\r\n}\r\n\r\n\/\/ Log something - Helps use know this script has run because out will be in the supervisord log\r\necho \"DONE\\n\";\r\n<\/pre>\n
Unfortunately, the above script doesn’t work because it does not finish before supervisord starts fpm. I need to be sure the environment variables are added to the fpm conf before<\/em> fpm is started. In my 2nd attempt I solved this issue using a wrapper script around fpm:<\/p>\n
\r\n; This script sets environment variables before starting php5-fpm with --nodaemonize\r\n[program:php5-fpm]\r\ncommand=\/opt\/startFPMWithDockerEnvs.sh\r\npriority=999\r\nstdout_logfile=\/var\/log\/supervisor\/%(program_name)s.log\r\nstderr_logfile=\/var\/log\/supervisor\/%(program_name)s.log\r\nautorestart=true\r\n<\/pre>\n
I prefer not to use PHP for the wrapper script so this is the bash version that does the same thing (thanks to Robert Gr\u00fcndler<\/a>):<\/p>\n
\r\n#!\/bin\/bash\r\n\r\n# Function to update the fpm configuration to make the service environment variables available\r\nfunction setEnvironmentVariable() {\r\n\r\n    if [ -z \"$2\" ]; then\r\n        echo \"Environment variable '$1' not set.\"\r\n        return\r\n    fi\r\n\r\n    # Check whether variable already exists\r\n    if grep -q $1 \/etc\/php5\/fpm\/pool.d\/www.conf; then\r\n        # Reset variable\r\n        sed -i \"s\/^env\\[$1.*\/env[$1] = $2\/g\" \/etc\/php5\/fpm\/pool.d\/www.conf\r\n    else\r\n        # Add variable\r\n        echo \"env[$1] = $2\" >> \/etc\/php5\/fpm\/pool.d\/www.conf\r\n    fi\r\n}\r\n\r\n# Grep for variables that look like docker set them (_PORT_)\r\nfor _curVar in `env | grep _PORT_ | awk -F = '{print $1}'`;do\r\n    # awk has split them by the equals sign\r\n    # Pass the name and value to our function\r\n    setEnvironmentVariable ${_curVar} ${!_curVar}\r\ndone\r\n\r\n# Log something to the supervisord log so we know this script as run\r\necho \"DONE\"\r\n\r\n# Now start php-fpm\r\n\/usr\/sbin\/php5-fpm --nodaemonize\r\n<\/pre>\n
This worked but I did not like that the php5-fpm service is now not managed directly by supervisord. The 3rd and final attempt defines the php5-fpm program with autostart=false<\/code>. Then the prep script calls supervisorctl start php5-fpm<\/code> when it has done it’s stuff to get supervisord to start php5-fpm. This way I know that the prep script will finish before<\/em> the other one starts.<\/p>\n
This is the relevant portion of the supervisord conf:<\/p>\n
\r\n; Run this script once before starting php5-fpm - it adds environment variables to the fpm config\r\n[program:pre-php5-fpm]\r\ncommand=\/opt\/startFPMWithDockerEnvs.php\r\npriority=998\r\nautostart=true\r\nstartretries=0\r\nexitcodes=0\r\nnodaemon=true\r\nstdout_logfile=\/var\/log\/supervisor\/%(program_name)s.log\r\nstderr_logfile=\/var\/log\/supervisor\/%(program_name)s.log\r\n\r\n; Note: autostart=false. Once started keep in foreground via --nodaemonize\r\n[program:php5-fpm]\r\ncommand=\/usr\/sbin\/php5-fpm --nodaemonize\r\npriority=999\r\nstdout_logfile=\/var\/log\/supervisor\/%(program_name)s.log\r\nstderr_logfile=\/var\/log\/supervisor\/%(program_name)s.log\r\nautostart=false        ; Don't start this automatically. Leave it to the 'pre-php5-fpm' program to start it.\r\nautorestart=true\r\nusername=www-data\r\n\r\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\r\n; This next stuff is needed to be able to use supervisord from the command line.\r\n\r\n[unix_http_server]\r\nfile=%(here)s\/supervisor.sock\r\n\r\n[supervisorctl]\r\nserverurl=unix:\/\/%(here)s\/supervisor.sock\r\n\r\n[rpcinterface:supervisor]\r\nsupervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface\r\n<\/pre>\n
This is the new ending to the prep script:<\/p>\n
\r\n`supervisorctl -c \/etc\/supervisor\/supervisord_nonlive.conf start php5-fpm`;\r\necho \"DONE\\n\";\r\n<\/pre>\n
Now we can see from the logs that everything is starting in order:<\/p>\n
\r\n2014-03-13 15:57:55,830 INFO supervisord started with pid 1\r\n2014-03-13 15:57:56,834 INFO spawned: 'sshd' with pid 8\r\n2014-03-13 15:57:56,835 INFO spawned: 'nginx' with pid 9\r\n2014-03-13 15:57:56,836 INFO spawned: 'pre-php5-fpm' with pid 10\r\n2014-03-13 15:57:56,997 INFO spawned: 'php5-fpm' with pid 37\r\n2014-03-13 15:57:58,033 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)\r\n2014-03-13 15:57:58,033 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)\r\n2014-03-13 15:57:58,034 INFO success: pre-php5-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)\r\n2014-03-13 15:57:58,034 INFO success: php5-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)\r\n2014-03-13 15:57:58,058 INFO exited: pre-php5-fpm (exit status 0; expected)\r\n<\/pre>\n
Job Done! (Thanks goes to Brian Lalor<\/em> for help on the docker mailing list)<\/p>\n
Warning: Debugging is made extra hard because usually you would SSH into a container to debug it. However the problem is that --link<\/code> environment variables don’t appear in the SSH session. The environment variables exist, its just they aren’t visible to the SSH session. For more info see this github issue: https:\/\/github.com\/dotcloud\/docker\/issues\/2569<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
Update: Since 7\/May\/2014 docker v0.11.1 was released with a feature called “Link hostnames“. It is best to use them instead of environment variables and skip reading this blog post. If you use Docker with php FastCGI Process Manager (php-fpm) then you may find that container…<\/p>\n","protected":false},"author":40,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[23],"tags":[24,25],"class_list":["post-165","post","type-post","status-publish","format-standard","hentry","category-devops-2","tag-docker","tag-php"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p2CA4w-2F","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/posts\/165","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/users\/40"}],"replies":[{"embeddable":true,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/comments?post=165"}],"version-history":[{"count":29,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/posts\/165\/revisions"}],"predecessor-version":[{"id":195,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/posts\/165\/revisions\/195"}],"wp:attachment":[{"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/media?parent=165"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/categories?post=165"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/tags?post=165"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}