Environment variables in a dockerized Symfony


I have developed a Symfony Web-Application, and it runs locally in a dockerized environment with docker-compose. This app is going to be deployed to production as a docker container. In production the handling of environment variables and how they are passed to the container during development is different.

12 Factor App

A few points from the 12factor methodology:

  • III. Config: Store config in the environment since env vars are easy to change between deploys without changing any code
  • X. Dev/prod parity: Keep development, staging, and production as similar as possible

I was searching for options how to handle the differences how environment variables are passed and I found there are at least

7 ways to pass environment variables to a container

  1. ENV in dockerfile
  2. Dockerfile args passed at build time to ENV
  3. ENV passing in docker run as option
  4. Env_file in docker run as option
  5. Environment variables in docker-compose.yml
  6. Env_file in docker compose for each service
  7. .env in docker compose substitutes variables in docker-compose.yml

And there is even more. If variables are passed to a container there is an order of precedence as follows:

Order of Precedence
  1. Passed from the command line docker compose run --env <KEY[=[VAL]]>.
  2. Passed from/set in compose.yaml service’s configuration, from the environment key.
  3. Passed from/set in compose.yaml service’s configuration, from the env_file key.
  4. Passed from/set in Container Image in the ENV directive.

from https://docs.docker.com/compose/envvars-precedence/

How to deal with environment variables in a dockerized Symfony

The goal

All services regardless of which technology they use, should have one streamlined way of how the environment variables should be passed to the application.

The big picture

  • We use multiple services which all need to work together
  • Services run in docker container
  • We deploy and run services in different compositions for each environment
  • Each service has their own sensitive data
  • Each service might be a different technology or has a different tech stack

Steps towards the goal

  • The infrastructure config should be kept in env files but not in the same directory as the application
  • Each service gets its own env file to be completely independent of each other, and it gets explicitly set
  • During development each service gets the env variables passed via env file (env_file in docker-compose)
  • Every project that has a docker-compose.yml moves the application into an app directory to separate the application from its infrastructure configuration
  • We remove the DotEnv component from symfony and define each environment variable that we expect as parameter so the app tells us instantly when a key-value pair is missing
  • In development credentials can be added to the VCS
  • In all other envs the credentials can be either stored and linked on the server or be read from a vault

The implementation

In Symfony the DotEnv component is default installed and enabled in the frontcontroller, so when a new app is created there is always a .env file at the project root created with it. Read more in the documentation.

It is not the same .env that docker-compose.yml expects.

Symfony DotEnv and Docker Compose use the same file name .env
Docker compose is also using a file named .env to replace the variables in the docker-compose.yml if it is located in the same directory. If you don’t know that and put the web apps .env file in the same place then you accidentally might overwrite variables when you think you just updated a variable for the Symfony application.

We have two different stacks here that both want to use the .env file and both might, but not at the same time, obviously.

Since we want to use config variables explicitly and not by accident the Symfony DotEnv component is going to be removed and all config is moved inside environment variable files that are passed into the container.

The directory tree

To ease the separation of infrastructure and code the application code moves into the ./app directory to be completely separate from the code/config that defines the infrastructure. You see there is no .env file left from Symfony. All variables have now moved to the env files inside the devops/env directory.

├── app
│   ├── assets
│   ├── bin
│   ├── ci
│   ├── config
│   ├── migrations
│   ├── node_modules
│   ├── public
│   │   └── index.php
│   ├── src
│   ├── templates
│   ├── tests
│   ├── var
│   ├── vendor
│   ├── composer.json
│   ├── composer.lock
│   ├── Makefile
│   ├── package.json
│   ├── symfony.lock
│   ├── webpack.config.js
│   └── yarn.lock
├── devops
│   ├── database
│   ├── docker
│   │   └── frankenphp
│   │        └── Dockerfile
│   └── env
│       ├── app.env
│       └── database.env
├── docker-compose.prod.yml
├── docker-compose.yml
├── Makefile
└── README.md

The docker-compose.yml

Each service gets its own env_file where we can configure the sensitive data for each service.

version: '3.9'  
    image: ghcr.io/c-base/cbag3:dev-latest  
      dockerfile: ./devops/docker/frankenphp/Dockerfile  
      target: dev  
    env_file: ./devops/env/app.env  
      - 80:80  
      - 443:443  
      - './app:/app'  
    image: postgres:alpine  
    container_name: database  
    env_file: ./devops/env/database.env  
      - 15432:5432  
      - ./devops/database:/var/lib/postgresql
the .env file can be used with docker compose to configure variables inside the docker-compose.yml

Disable DotEnv in frontcontroller and console

The DotEnv component is disabled since all environment variables have already passed to the container.

# app/public/index.php
use Cbase\App\Kernel;  
$_SERVER['APP_RUNTIME_OPTIONS']['disable_dotenv'] = true; 
require_once dirname(__DIR__).'/vendor/autoload_runtime.php';  
return function (array $context) {  
    return new Kernel($context['APP_ENV'], (bool) $context['APP_DEBUG']);  
# app/bin/console
#!/usr/bin/env php  
use Cbase\App\Kernel;  
use Symfony\Bundle\FrameworkBundle\Console\Application;  
$_SERVER['APP_RUNTIME_OPTIONS']['disable_dotenv'] = true;  
require_once dirname(__DIR__) . '/vendor/autoload_runtime.php';  
return function (array $context) {  
    $kernel = new Kernel($context['APP_ENV'], (bool) $context['APP_DEBUG']);  
    return new Application($kernel);  
Symfony Runtime
run app only inside container
By disabling DotEnv we will no longer be able to run the application outside the container (our local machine) unless we set all environment variables there as well.
keep DotEnv for tests
For now, we leave the DotEnv for the tests since those environment variables won’t change regardless of where they are executed, and they will be executed in the dev container. We could change that by running the tests in their own container, but for now keep the .env.test.

Don’t forget to add the parameters in services.yml

# app/config/services.yaml
  images.upload.directory: '%env(resolve:IMAGES_UPLOAD_DIRECTORY)%'

    autowire: true
    autoconfigure: true
      string $imagesUploadDirectory: '%images.upload.directory%'

Since every environment has its own env_file there is the danger of forgetting to add an environment variable to the other environments.

In order to fail early we load the environment variable at the start of the application, by binding it. If we do not bind parameters to variables but just bin them to a service we might miss that we forgot to set an environment variable in the env file since the service might not be loaded in every request.

Run docker container in production with env-file

cat devops/env/app.env
# This is a comment

docker run --env-file devops/env/app.env app env | grep -E 'IMAGES'

Read more about it in the docker documentation.

Migration Path

There is a migration path for projects that use already many config yaml files and want to migrate to environment variables.

# config/my-app.yaml
  images.upload.directory: '%kernel.project_dir%/var/uploads'
# config/services.yaml
  env(IMAGES_UPLOAD_DIRECTORY): '%images.upload.directory%'

      string $imagesUploadDirectory: '%env(resolve:IMAGES_UPLOAD_DIRECTORY)%'
  1. the configuration processor looks up if there is an environment variable IMAGES_UPLOAD_DIRECTORY
  2. if that is the case, it will be taken,
  3. otherwise if it is not found '%images.upload.directory%' will be set to the environment variable.
  4. the '%env(resolve:IMAGES_UPLOAD_DIRECTORY)%' is bound to a variable $imagesUploadDirectory

Read more about configuration processors in the Symfony documentation about “Environment Variable Processors”.

This would result in the following migration path:

  1. Make it possible to set variables via environment variables
  2. Make sure all environments set the corresponding variables
  3. Remove many quirky unnecessary config files
  4. win


We removed the DotEnv from Symfony and will miss out on all the functionality that came with it, but chose using the env_file as it can be used for running a container, and it can be configured in the docker-compose.yml. The environment configs can be dumped from secret vaults regardless of the tech-stack that the cloud has to offer or kept in a shared directory that won’t change between deployments. There will be one explicit way of how each service will get configuration regardless of their environment or tech stack. Also, we learned that there is a simple way in Symfony to migrate to environment variables.

Happy continuously deploying everyone

More sources

Enter your instance's address

More posts like this

Build and push docker images to ghcr.io with GitHub Actions

2022-12-28 | #cd #docker #github #github-actions #github-packages

When you host your project code on GitHub and want to release it as a docker image for deployment or just publish it, the way to go are GitHub actions. Actions are basically hooks that can start CI/DC workflows on repository events. GitHub actions can be used to build and push images to GitHub’s Container Registry which are reachable under https://ghcr.io which is part of the package registry. The package registry is not only for docker images, it can also host quite a few other kinds of packages.

Continue reading 

Goodbye Jekyll, hello Hugo

2022-12-27 | #blog #cd #git #github #github-actions #hugo #jekyll

I started this blog in March 2013 when I was working for ImagineEasy when I had a few ideas to write down on how I’d work with Doctrine Repositories. I still like the idea, but I’d probably do it a bit different today. The blog and also how I’d work with doctrine. At the time Jekyll was the way to handle a static file blog. Since then, again, a few things have changed.

Continue reading 