AI VOID
DATE: 2026.05.22 00:00 CLEARANCE: PUBLIC

Handling Configuration and Secrets Securely

Docker Docker Compose Secrets Management Configuration Security Python

// table of contents

▼

Managing application configuration and sensitive data is a critical aspect of building production-ready applications. Hardcoding API keys, database credentials, or other environment-specific settings directly into your code or Dockerfiles is a significant security risk and a maintenance nightmare. In this chapter, we’ll learn how to separate configuration from code and handle sensitive information (secrets) securely within our Docker Compose stack.

By the end of this milestone, your multi-service application will properly load non-sensitive configuration from .env files and securely consume sensitive secrets using Docker’s built-in secrets management. This significantly improves the security posture and maintainability of your deployment.

Project Overview

This chapter focuses on a critical aspect of application hardening: secure configuration management. We’re enhancing our existing multi-service web application (Flask + PostgreSQL) to handle its settings and sensitive data in a production-minded way.

  • Finished Artifact: A Docker Compose deployed web application and database, where all environment-specific settings are externalized. Non-sensitive settings are loaded via .env files, and sensitive credentials are provided through Docker Secrets.
  • Target User: Any developer or operations engineer working with Dockerized applications who needs to deploy them securely and maintainably across different environments.
  • Success Criteria: The application successfully starts, connects to the database, and serves requests. All environment variables are correctly populated, and sensitive data is demonstrably consumed from Docker Secrets files, not exposed directly in environment variables or application code.

Tech Stack

For this chapter, we’ll continue working with our established stack, focusing on how its components interact with configuration and secrets:

  • Docker Engine: The underlying runtime for containers. (Version: Latest stable as of 2026-05-22, specific version unknown but assumed current).
  • Docker Compose: Used to define and run our multi-container application. (Version: Adheres to the Compose Specification, recommended approach without an explicit version field, as of 2026-05-22).
  • Python 3.11: The runtime for our Flask web application.
  • Flask: The web framework for our application service.
  • PostgreSQL 16-alpine: The database service. (Version: 16-alpine, latest stable as of 2026-05-22).

Milestones and Build Plan

This chapter is structured to incrementally build our secure configuration setup:

  1. Externalize Non-Sensitive Configuration: Move general application settings into a .env file and configure Docker Compose to inject these as environment variables.
  2. Introduce Docker Secrets: Create secure files for sensitive data (database password, application secret key) and define them as Docker Secrets in docker-compose.yml.
  3. Update Application Code: Modify the Flask application to read non-sensitive settings from environment variables and sensitive secrets from the special files mounted by Docker Secrets.

Architecture: Configuration and Secrets Flow

The core principle is the 12-Factor App methodology, which advocates for strictly separating configuration from code. Configuration should be injected into the application environment at runtime, rather than being part of the build artifact. This allows the same application image to be deployed across different environments (development, staging, production) with different configurations.

We’ll address two distinct types of configuration:

  1. Non-Sensitive Configuration: Settings like port numbers, API endpoints, or feature flags that don’t pose a security risk if exposed. These are ideal for .env files, which are simple and widely understood for local development.
  2. Sensitive Configuration (Secrets): Database passwords, API keys, encryption keys, or other credentials that must be protected. For these, we will use Docker Secrets, which provide a more robust and secure mechanism for passing sensitive data to containers.

Our web application service will be updated to read non-sensitive configuration from environment variables and sensitive secrets (like its own secret key and the database password) from designated files within the container’s filesystem. The db service will also consume its root password as a secret file.

Here’s a high-level view of how configuration and secrets will flow into our services:

flowchart LR DevOps[Developer Ops Engineer] --> DockerCompose[Docker Compose YML] DockerCompose --> EnvFile[Env File] DockerCompose --> SecretFiles[Secret Files on Host] subgraph Docker_Engine_Host["Docker Engine Host"] WebService[Web App Service] DbService[Database Service] end EnvFile --> WebService SecretFiles --> WebService SecretFiles --> DbService
  • EnvFile (non-sensitive): Provides general configuration to services via environment variables.
  • SecretFiles (sensitive): Provides sensitive data to services by mounting them as files within the container’s /run/secrets directory.
  • DockerCompose: Orchestrates how these configuration sources are injected into WebService and DbService.

Step-by-Step Implementation

Let’s assume we have a simple Flask web application (from previous chapters) that needs a database connection string, a secret key, and the database password to connect.

1. Externalizing Non-Sensitive Configuration with .env

First, we’ll create an .env file at the root of our project, alongside docker-compose.yml. This file will hold non-sensitive environment variables. Notice that the DATABASE_URL here no longer contains the password, as that will be handled as a secret.

Action: Create or update ./.env in your project root:

./.env

APP_PORT=5000
FLASK_ENV=development
DATABASE_HOST=db
DATABASE_PORT=5432
DATABASE_NAME=mydatabase
DATABASE_USER=user
  • APP_PORT: Defines the port our Flask app will listen on.
  • FLASK_ENV: Sets the Flask environment mode.
  • DATABASE_HOST, DATABASE_PORT, DATABASE_NAME, DATABASE_USER: These provide the non-sensitive connection details for our PostgreSQL database.

Next, modify your docker-compose.yml to instruct Docker Compose to load these variables into the web service and utilize them for the db service.

Action: Update your docker-compose.yml:

./docker-compose.yml

# docker-compose.yml
# This file adheres to the Compose Specification.
# For more information, see: https://docs.docker.com/compose/compose-file/
services:
  web:
    build: .
    ports:
      - "${APP_PORT}:${APP_PORT}" # Use APP_PORT from .env
    env_file: # Load environment variables from ./.env
      - ./.env
    # We will add secrets here shortly
    # ... other configurations for web service ...
  db:
    image: postgres:16-alpine # Latest stable as of 2026-05-22
    environment:
      POSTGRES_DB: ${DATABASE_NAME} # From .env
      POSTGRES_USER: ${DATABASE_USER} # From .env
      # POSTGRES_PASSWORD is now provided via a secret
    volumes:
      - db_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DATABASE_USER} -d ${DATABASE_NAME}"]
      interval: 10s
      timeout: 5s
      retries: 5
    # We will add secrets here shortly
    # ... other configurations for db service ...

volumes:
  db_data:
  • ports: - "${APP_PORT}:${APP_PORT}": This line now uses variable interpolation. Docker Compose will substitute ${APP_PORT} with the value found in /.env (which is 5000). This makes the port easily configurable without modifying the docker-compose.yml file itself.
  • env_file: - ./.env: This directive is crucial. It tells Docker Compose to read all key-value pairs from the specified .env file and expose them as environment variables to the web service’s containers. This is a common and convenient way to manage non-sensitive configuration for local development.
  • environment variables in the db service: ${DATABASE_NAME} and ${DATABASE_USER} are also populated from the .env file. This demonstrates how a single .env file can centralize configuration for multiple services, improving consistency.

2. Managing Sensitive Secrets with Docker Secrets

Now, let’s secure the database password for both the db service and the web service, as well as the web application’s own secret key. Docker Secrets offer a more secure way to handle sensitive data than environment variables, especially in multi-service deployments.

First, create files containing your sensitive secrets. It’s best practice to keep these files out of version control (e.g., add them to .gitignore). For this example, we’ll create simple text files.

Action: Create new files in your project root:

./db_password.txt (for the PostgreSQL database itself)

my_secure_db_password_123

./web_db_password.txt (for the web application to connect to the database)

my_secure_db_password_123

./app_secret_key.txt (for the Flask web application’s secret key)

this_is_a_very_secret_key_for_flask_app
  • db_password.txt: This file contains the password that the PostgreSQL database will use for its postgres superuser.
  • web_db_password.txt: This file contains the password that our web application will use to connect to the PostgreSQL database. While it’s the same value as db_password.txt in this example, in a real application, you might use different credentials for different services for a “least privilege” approach.
  • app_secret_key.txt: This is a secret key specific to our Flask application, used for session management and security features.

🧠 Important: In a real production scenario, these files would contain strong, randomly generated passwords/keys and would be managed by a dedicated secret management system or manually placed on the host, never committed to version control. Add these files to your .gitignore immediately to prevent accidental exposure.

Next, we’ll modify docker-compose.yml to define these as secrets at the top level and then make them available to the appropriate services.

Action: Update your docker-compose.yml with the secrets blocks:

./docker-compose.yml

# docker-compose.yml
# This file adheres to the Compose Specification.
services:
  web:
    build: .
    ports:
      - "${APP_PORT}:${APP_PORT}"
    env_file:
      - ./.env
    secrets: # Declare that the web service consumes secrets
      - app_secret_key
      - web_db_password # Web app's password for the DB
    # ...
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: ${DATABASE_NAME}
      POSTGRES_USER: ${DATABASE_USER}
      # POSTGRES_PASSWORD is removed. We use POSTGRES_PASSWORD_FILE instead.
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password # Tell Postgres to read from the secret file
    volumes:
      - db_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DATABASE_USER} -d ${DATABASE_NAME}"]
      interval: 10s
      timeout: 5s
      retries: 5
    secrets: # Declare that the db service consumes a secret
      - db_password # This refers to the secret defined in the top-level 'secrets' section
    # ...

volumes:
  db_data:

secrets: # Top-level declaration of secrets
  db_password: # The name of the secret for the DB's root password
    file: ./db_password.txt # Path to the file containing the secret's value
  web_db_password: # The name of the secret for the web app's DB connection
    file: ./web_db_password.txt
  app_secret_key: # Another secret for the web app
    file: ./app_secret_key.txt
  • Top-level secrets block: This new block defines all secrets available to our Compose application.
    • db_password:: This is a named secret. file: ./db_password.txt tells Docker Compose to read the content of db_password.txt from the host and securely make it available as a secret.
    • web_db_password:: Similarly, this secret is sourced from web_db_password.txt.
    • app_secret_key:: This secret is sourced from app_secret_key.txt.
  • Service-level secrets block:
    • For the db service: secrets: - db_password makes the db_password secret available to the db container. Docker Compose mounts this secret as a read-only file at /run/secrets/db_password inside the db container.
    • POSTGRES_PASSWORD_FILE: /run/secrets/db_password: This PostgreSQL-specific environment variable instructs the postgres image to read the database password from the specified file path, rather than from a POSTGRES_PASSWORD environment variable. This is a more secure way to provide passwords to PostgreSQL containers.
    • For the web service: secrets: - app_secret_key and secrets: - web_db_password make these secrets available. They will be mounted as files at /run/secrets/app_secret_key and /run/secrets/web_db_password respectively inside the web container.

3. Updating the Application Code

Now, let’s update our web service’s Dockerfile and app.py to consume these secrets and environment variables.

Action: Ensure your Dockerfile for the web service is up-to-date.

./Dockerfile

# Use a minimal base image for the web application
FROM python:3.11-slim-bookworm AS builder

# Set environment variables for the builder stage
ENV PYTHONUNBUFFERED 1

# Install build dependencies
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Final stage for the production image
FROM python:3.11-slim-bookworm

# Set environment variables for the runtime stage
ENV PYTHONUNBUFFERED 1
ENV FLASK_APP=app.py

WORKDIR /app
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY app.py .

EXPOSE 5000

# Run the application
CMD ["flask", "run", "--host=0.0.0.0", "--port", "5000"]
  • This Dockerfile uses a multi-stage build, first installing dependencies in a builder stage and then copying only the necessary runtime components to a smaller final image. This keeps our image size down and reduces the attack surface.

Action: Ensure your requirements.txt includes the PostgreSQL adapter.

./requirements.txt

Flask
psycopg2-binary # For PostgreSQL connection

Action: Update your app.py to read from environment variables and secret files.

./app.py

import os
from flask import Flask, jsonify
import psycopg2

app = Flask(__name__)

# --- Load non-sensitive configuration from environment variables ---
# These are populated by Docker Compose from the ./.env file
APP_PORT = os.getenv('APP_PORT', '5000') # Default to 5000 if not set
FLASK_ENV = os.getenv('FLASK_ENV', 'production')

# Database connection details (user, host, port, name) from .env
DB_HOST = os.getenv('DATABASE_HOST')
DB_PORT = os.getenv('DATABASE_PORT')
DB_NAME = os.getenv('DATABASE_NAME')
DB_USER = os.getenv('DATABASE_USER')

# --- Load sensitive secrets from Docker Secret files ---

# Flask application secret key
APP_SECRET_KEY_PATH = "/run/secrets/app_secret_key"
APP_SECRET_KEY = None
if os.path.exists(APP_SECRET_KEY_PATH):
    with open(APP_SECRET_KEY_PATH, 'r') as secret_file:
        APP_SECRET_KEY = secret_file.read().strip()
else:
    print(f"Warning: App secret key file not found at {APP_SECRET_KEY_PATH}")

if APP_SECRET_KEY:
    app.secret_key = APP_SECRET_KEY
else:
    # ⚠️ What can go wrong: This fallback should ONLY be for development/testing.
    # In production, a missing secret should halt startup or trigger an alert.
    app.secret_key = 'super-insecure-fallback-key' # Fallback for development, NOT for production

# Database password for the web application
WEB_DB_PASSWORD_PATH = "/run/secrets/web_db_password"
WEB_DB_PASSWORD = None
if os.path.exists(WEB_DB_PASSWORD_PATH):
    with open(WEB_DB_PASSWORD_PATH, 'r') as secret_file:
        WEB_DB_PASSWORD = secret_file.read().strip()
else:
    print(f"Warning: Web DB password file not found at {WEB_DB_PASSWORD_PATH}")

# Construct the full DATABASE_URL using secrets and environment variables
DATABASE_URL = None
if DB_USER and WEB_DB_PASSWORD and DB_HOST and DB_PORT and DB_NAME:
    DATABASE_URL = f"postgresql://{DB_USER}:{WEB_DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}"
else:
    print("Warning: Missing database connection details (user, password, host, port, or name).")
    # In a real app, you might raise an error here to prevent startup without critical config.

print(f"Flask Environment: {FLASK_ENV}")
print(f"Database Host: {DB_HOST}, Port: {DB_PORT}, Name: {DB_NAME}, User: {DB_USER}")
# ⚡ Quick Note: Do NOT log secrets in production!
# print(f"App Secret Key: {APP_SECRET_KEY}")
# print(f"Web DB Password: {WEB_DB_PASSWORD}")


# Database connection function
def get_db_connection():
    if not DATABASE_URL:
        print("Error: DATABASE_URL not constructed.")
        return None
    try:
        conn = psycopg2.connect(DATABASE_URL)
        return conn
    except Exception as e:
        print(f"Database connection error: {e}")
        return None

@app.route('/')
def hello():
    return jsonify(message="Hello from the Dockerized Flask App!", environment=FLASK_ENV)

@app.route('/db-test')
def db_test():
    conn = get_db_connection()
    if conn:
        cursor = conn.cursor()
        cursor.execute('SELECT 1;')
        result = cursor.fetchone()
        conn.close()
        return jsonify(message="Database connection successful!", result=result)
    else:
        return jsonify(message="Database connection failed!", error="Could not connect to database."), 500

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=int(APP_PORT))
  • os.getenv('APP_PORT', '5000'): Reads APP_PORT from the container’s environment variables. Docker Compose populates this from our .env file. The second argument ('5000') provides a default if the variable isn’t set, useful for local testing or when the .env file is optional.
  • DB_HOST, DB_PORT, DB_NAME, DB_USER: These are all read from environment variables, which are populated from the .env file.
  • APP_SECRET_KEY_PATH and WEB_DB_PASSWORD_PATH: Define the expected paths for the secret files. Docker Secrets are mounted as files inside the container at /run/secrets/<secret_name>.
  • The if os.path.exists(...) blocks: This is the standard and recommended way for an application to read Docker secrets. The application opens and reads the file content, which is the secret value.
  • DATABASE_URL = f"postgresql://{DB_USER}:{WEB_DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}": The full database connection string is now dynamically constructed within the application. It combines non-sensitive environment variables (user, host, port, name) with the sensitive password read from a secret file.
  • Production Awareness: The print statements are helpful for debugging, but you should avoid logging sensitive data (like APP_SECRET_KEY or WEB_DB_PASSWORD) in production environments. The warning about missing database details is a good practice for robust applications.

Testing & Verification

Now, let’s build and run our services to verify that configuration and secrets are being handled correctly.

  1. Build and Run the Services: Navigate to your project root in the terminal.

    docker compose build
docker compose up -d

    docker compose build ensures our Docker image for the web service is updated with the latest app.py changes. docker compose up -d starts our services in detached mode.

  2. Verify Environment Variables (Non-Sensitive Config): We can inspect the environment variables of the web container.

    docker compose exec web env | grep APP_PORT
docker compose exec web env | grep FLASK_ENV
docker compose exec web env | grep DATABASE_HOST
docker compose exec web env | grep DATABASE_USER

    You should see output similar to:

    APP_PORT=5000
FLASK_ENV=development
DATABASE_HOST=db
DATABASE_USER=user

    This confirms that the non-sensitive variables from .env are correctly loaded into the web service’s environment.

  3. Verify Secrets (Sensitive Config):

    • For the web service: Check if the app_secret_key and web_db_password files exist within the container and verify their content.
      docker compose exec web ls -l /run/secrets/
docker compose exec web cat /run/secrets/app_secret_key
docker compose exec web cat /run/secrets/web_db_password
      You should see both secret files listed (e.g., app_secret_key, web_db_password) and their content printed (e.g., this_is_a_very_secret_key_for_flask_app).
    • For the db service: Confirm that POSTGRES_PASSWORD is not in the environment variables, but POSTGRES_PASSWORD_FILE is set.
      docker compose exec db env | grep POSTGRES_PASSWORD
docker compose exec db env | grep POSTGRES_PASSWORD_FILE
      You should see no output for POSTGRES_PASSWORD (or only _FILE variants), but POSTGRES_PASSWORD_FILE=/run/secrets/db_password should be present. Also, verify the db_password secret file itself:
      docker compose exec db ls -l /run/secrets/
docker compose exec db cat /run/secrets/db_password
      You should see db_password listed and its content printed.

    ⚡ Real-world insight: By using POSTGRES_PASSWORD_FILE and Docker Secrets, the actual password string never appears in the container’s process environment, which can be easily inspected (e.g., via docker inspect). Similarly, for the web app, reading secrets from /run/secrets/ files is more secure than exposing them as environment variables. Docker mounts these secret files with restricted permissions (mode 0444), making them read-only for the container’s user.

  4. Test Application Endpoints: Open your browser or use curl to access the application:

    • http://localhost:5000/ Expected output: {"environment":"development","message":"Hello from the Dockerized Flask App!"}
    • http://localhost:5000/db-test Expected output: {"message":"Database connection successful!","result":[1]} This confirms that the application successfully connected to the database using the credentials provided via secrets and environment variables.

  5. Clean up:

    docker compose down

    This command stops and removes the containers, networks, and volumes created by docker compose up.

Production Considerations

Securing configuration and secrets is paramount in production. Here are key considerations:

  • .env vs. Docker Secrets in Production:

    • .env files: Ideal for local development and non-sensitive configuration. Do not use .env files for sensitive data in production environments, especially if they are committed to version control. They lack the security mechanisms of dedicated secret managers.
    • Docker Secrets: A significant step up for sensitive data in production for Docker Compose and Swarm deployments. They are mounted as files with restricted permissions inside the container, not exposed as easily inspectable environment variables. Docker handles their secure storage and transmission within the Docker daemon.
    • External Secret Managers: For highly sensitive, critical applications, consider integrating with dedicated secret management services like HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or Google Secret Manager. These services offer advanced features like secret rotation, auditing, and fine-grained access control, which are beyond the scope of basic Docker Secrets. Docker Secrets are a good intermediary step for smaller deployments or when a full-fledged secret manager is overkill.

  • Secret Rotation: Regularly rotate your secrets (e.g., database passwords, API keys). Docker Secrets don’t automate rotation, so you’d need an external process or script to update the secret files on the host and then redeploy your services (e.g., docker compose up -d).

  • Permissions on Host Files: Ensure your secret files (like db_password.txt) on the host machine have strict permissions, preventing unauthorized access. They should typically be readable only by the user running Docker. For example, chmod 600 db_password.txt makes the file readable and writable only by the owner.

  • Environment Variables & Security: While .env files inject environment variables, and some images allow consuming secrets as environment variables (e.g., POSTGRES_PASSWORD), be aware that environment variables can be inspected more easily (e.g., docker inspect <container_id> or docker exec <container_id> env). Always prefer file-based secret consumption (like /run/secrets/) when the application or image supports it, as it offers a higher level of isolation.

  • Security Auditing with docker-bench-security: For a comprehensive security posture, regularly audit your Docker host and container configurations. Tools like docker-bench-security (from the official Docker team, available at https://github.com/docker/docker-bench-security) can scan your setup against common best practices to identify potential vulnerabilities in your Docker daemon, images, and running containers. This tool helps ensure your Docker environment itself adheres to security best practices.

Common Issues & Solutions

  1. Secret File Not Found in Container:

    • Issue: The application reports that /run/secrets/<secret_name> does not exist, or the cat command shows an empty file or “No such file or directory”.
    • Solution: Double-check your docker-compose.yml.
      • Ensure the top-level secrets: block is correctly defined and that the file: path (e.g., ./db_password.txt) is correct relative to docker-compose.yml.
      • Verify that the service’s secrets: block (e.g., secrets: - db_password) correctly references the named secret defined at the top level.
      • Make sure the actual secret file (e.g., db_password.txt) exists on your host machine in the specified path.

  2. Application Not Reading Environment Variables:

    • Issue: os.getenv() returns None or a default value, even though the variable is defined in .env.
    • Solution:
      • Confirm env_file: - ./.env is present and correctly indented under the service in docker-compose.yml.
      • Ensure the .env file is in the correct location (usually the project root alongside docker-compose.yml).
      • Check for typos in the variable name in both .env and your application code (e.g., APP_PORT vs. app_port). Environment variable names are case-sensitive.
      • Remember to restart your services (docker compose up -d) after changing docker-compose.yml or .env files for changes to take effect. If you’ve previously built the image, docker compose up --build will ensure the latest image is used.

  3. Permissions on Secret Files:

    • Issue: You get “permission denied” when the container tries to read /run/secrets/<secret_name>, or Docker Compose fails to load the secret.
    • Solution: While Docker mounts secrets with 0444 permissions inside the container, ensuring the host file (e.g., db_password.txt) has appropriate permissions can prevent issues. Typically, chmod 600 db_password.txt is a good practice, making it readable only by the file owner. If Docker’s user mapping is complex, or if the Docker daemon itself doesn’t have access to the secret file on the host, this can cause problems.

Summary & Next Step

You’ve successfully implemented secure configuration and secret management for your Docker Compose application. You now understand the distinction between non-sensitive configuration and sensitive secrets, and how to leverage .env files for local development and Docker Secrets for production-grade security. This greatly enhances the security and maintainability of your application, paving the way for more robust deployments. Remember to also utilize tools like docker-bench-security to audit your overall Docker environment for security best practices.

In the next chapter, we’ll dive into implementing health checks for our services. Health checks are crucial for ensuring that our containers are not just running, but are actually healthy and ready to serve requests, which is vital for application reliability and automated recovery.

This page is AI-assisted and reviewed. It references official documentation and recognized resources where relevant.

References