Python API Development in Docker

Intention

In this article I will describe (in great detail) how I have configured a local dev environment to start up an application using Docker Compose and Docker containers. The result is a development environment that starts up quickly, is portable, and very closely resembles the final production environment.

Overview

Here are the steps you'll learn about in this post:

1. Create a specific Dockerfile that resembles the production dockerfile as closely as possible.
2. Create a docker-compose.yml file that builds the Dev Dockerfile. The docker-compose.yml contains default values for environment variables and other configurations.
3. Create a set_env_variables.sh shell script that can set environment variables with defaults and picks up values from a .env-dev file.
4. Create a reset_env_variables.sh shell script that can unset all environment variables and then sources the set_env_variables.sh shell script.
5. Create a start.sh shell script that makes configuring and starting the app a single simple call.
6. Create a stop.sh shell script that makes cleanly stopping the app a single simple call.
7. Leverage the "Dev Containers" VS Code extension by Microsoft for developing inside the container itself.
8. A full working example may be found at: https://github.com/generalui/python-flask-ariadne-api-starter.

Introduction

I often work on projects that must be handed off to other developers once created. Having been on the receiving end of this, I know that it can be a huge time-suck to try to get a project started locally with all the correct dependencies. A new developer on a project needs to get their dev environment set up quickly so they can get to the real work of building features and fixing bugs. A well designed dev environment can be the difference between new developers delivering value on day one versus getting stuck in setup for weeks.

Additionally, when I am working on local dev, I want my dev environment to match the production environment as closely as possible. How many times has code worked in local only to fail in production because the environment is slightly different?

Dockerfile

The app has two Dockerfiles. One for deployment and one for development. I wanted the development file to resemble the production file as much as possible. The reason for two files is that in development, I have a number of dependencies (for testing, linting, profiling, etc) that I didn't need or want in production.

Production Dockerfile

The production Dockerfile can be seen here.

Development Dockerfile

The development Dockerfile is similar to the production Dockerfile but with some additional apps and requirements installed just for dev.

Key similarities:

• Python version
• Running on Alpine
• Using the same code base
• Installing the same build tools for installing dependencies
• Building with the same requirements.txt file
• Removing the build tools after installing dependencies

Docker Compose

Using a docker compose file to start and stop the docker container allows much more configurability for my environment and app.

The docker compose file is counting on a number of environment variables being available. Defaults have been defined in the event the variables haven't been set.

Some of the defaults in the environment option are set for this app. Values for POSTGRES_DB and POSTGRES_DB_TEST, the database names for dev and test respectively, are set for this app.

Volume Configuration

The volumes option is very important in the development environment. We create a number of volumes here. All of them are delegated for better performance. This is geared towards developing INSIDE the container. I create these 4 volumes:

• I map the local project directory to the working directory inside the container. Thus, changes made in the local are reflected inside the container and vice versa.
• I map the local ~/.gitconfig file (in the user's home folder) to the root user's home folder in the container. This supports git commands in the container as though you are operating on your local command line.
• I map the local ~/.ssh folder (in the user's home folder) to the root user's home folder in the container. This makes the local ssh keys available inside the container for git fetches, pulls and pushes.
• I map the entire root user's home folder in the container to a volume. This volume persists any other files that are created inside the root user's home folder inside the container.

Start

There is a magic start.sh script in the root of the project. Starting the app in dev is as simple as running:

bash ./start.sh

This will help set environment variables such as DOT_ENV_FILE by identifying if a .env-dev file exists or not. The script accepts a few flags:

• -b or --build - Pass this flag if you explicitly want to rebuild the image.
• -r or --reset_env - Pass this flag to reset environment variables back to defaults.
• -n or --no_auto_start - Pass this flag to start the container without automatically starting the server.

Conclusion

This Docker-based development setup provides a robust, reproducible environment that closely mirrors production. By following these patterns, you can ensure that your development environment is consistent across team members and reduces the "works on my machine" problem.

For the complete working example with all configuration files and scripts, visit the Python Flask Ariadne API Starter repository.