1533 lines
53 KiB
Plaintext
1533 lines
53 KiB
Plaintext
[1]
|
||
[2]Hire Martians
|
||
[3]
|
||
Services
|
||
[4]
|
||
Clients
|
||
[5]
|
||
Products
|
||
[6]
|
||
Open Source
|
||
[7]
|
||
Blog
|
||
[8]
|
||
Events
|
||
[9]
|
||
Podcast
|
||
[10]
|
||
[11]
|
||
[12]
|
||
[13]
|
||
[14]
|
||
[15]Hire Martians
|
||
[16]
|
||
Services
|
||
[17]
|
||
Clients
|
||
[18]
|
||
Products
|
||
[19]
|
||
Open Source
|
||
[20]
|
||
Blog
|
||
[21]
|
||
Events
|
||
[22]
|
||
Podcast
|
||
[23]
|
||
[24]
|
||
[25]
|
||
[26]
|
||
[27]
|
||
7-
|
||
Oct
|
||
8
|
||
Meet us at Rocky Mountain Ruby in Boulder, Colorado!
|
||
[28]Hire Martians
|
||
[29]
|
||
[30]
|
||
[31]
|
||
[32]
|
||
|
||
Ruby on Whales: Dockerizing Ruby and Rails development
|
||
|
||
March 15, 2022
|
||
[svg]
|
||
Cover for Ruby on Whales: Dockerizing Ruby and Rails developmentCover for Ruby
|
||
on Whales: Dockerizing Ruby and Rails development
|
||
|
||
Topics
|
||
|
||
• [33]Backend
|
||
• [34]Full Cycle Software Development
|
||
• [35]Performance Optimization
|
||
• [36]Ruby on Rails
|
||
• [37]Ruby
|
||
• [38]Docker
|
||
• [39]PostgreSQL
|
||
• [40]Node.js
|
||
|
||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||
|
||
• [svg]
|
||
Vladimir Dementyev
|
||
|
||
Vladimir Dementyev
|
||
|
||
Principal Backend Engineer
|
||
|
||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||
|
||
Translations
|
||
|
||
• Japanese[41]クジラに乗ったRuby: Evil Martians流Docker+Ruby/Rails開発環境構
|
||
築
|
||
• Chinese[42]骑鲸之路——Docker模式下的Rails开发环境构筑
|
||
|
||
This post introduces the Docker configuration I use for developing my Ruby on
|
||
Rails projects. This configuration came out of—and then further evolved—during
|
||
[DEL:production:DEL] development at Evil Martians. Read on to learn all the
|
||
details, and feel free to use it, share it, and enjoy!
|
||
|
||
Notice: This article is regularly updated with the best and latest
|
||
recommendations; for details, take a look at the [43]Changelog.
|
||
|
||
So, where to start? This has been a pretty long journey: back in the day, I
|
||
used to develop using Vagrant, but its VMs were a bit too heavy for my 4GB RAM
|
||
laptop. In 2017, I decided to make the switch to containers, and this was how I
|
||
first began using Docker. But don’t get the impression that this was an instant
|
||
fix! I was in search of a configuration that was perfect for myself, my team,
|
||
and well, everyone else. And something which was just good enough would not cut
|
||
it. It took quite some time to develop a standard approach (as more formerly
|
||
enshrined with the first release of this article in 2019). Since that first
|
||
iteration of this post revealed my secret to the world, many Rails teams and
|
||
devs have adopted my technique, and actually, they’ve helped to contribute and
|
||
improve it!
|
||
|
||
With that out of the way, let me just go ahead and present the config itself.
|
||
Along the way, I’ll explain almost every line (because we’ve all had enough of
|
||
those cryptic tutorials that just assume you know stuff).
|
||
|
||
This post was originally adapted from my talk at RailsConf 2019: [44]
|
||
“Terraforming legacy Rails applications”.
|
||
|
||
The source code can be found in the [45]evilmartians/ruby-on-whales
|
||
repository on GitHub.
|
||
|
||
Before we get on with it, let’s note that we’ll be using up-to-date software
|
||
versions for this example: Docker Desktop 20.10+ and Docker Compose v2,
|
||
Ruby 3.1.0, PostgreSQL 14, etc.
|
||
|
||
The bulk of the post consists mostly of annotated code and configuration
|
||
examples, structured as follows:
|
||
|
||
• [46]The basics: Dockerfile and docker-compose.yml
|
||
• [47]Introducing Dip
|
||
• [48](Micro-)services vs Docker for development
|
||
• [49]From development to production
|
||
• [50]Introducing the Ruby on Whales interactive generator
|
||
|
||
Basic Docker configuration
|
||
|
||
[51]Dockerfile
|
||
|
||
The Dockerfile defines our Ruby application’s environment. This environment is
|
||
where we’ll run servers, access the console (rails c), perform tests, do Rake
|
||
tasks, and otherwise interact with our code in any way as developers:
|
||
|
||
# syntax=docker/dockerfile:1
|
||
|
||
ARG RUBY_VERSION
|
||
ARG DISTRO_NAME=bullseye
|
||
|
||
FROM ruby:$RUBY_VERSION-slim-$DISTRO_NAME
|
||
|
||
ARG DISTRO_NAME
|
||
|
||
# Common dependencies
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
--mount=type=tmpfs,target=/var/log \
|
||
rm -f /etc/apt/apt.conf.d/docker-clean; \
|
||
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache; \
|
||
apt-get update -qq \
|
||
&& DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
|
||
build-essential \
|
||
gnupg2 \
|
||
curl \
|
||
less \
|
||
git
|
||
|
||
# Install PostgreSQL dependencies
|
||
ARG PG_MAJOR
|
||
RUN curl -sSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | \
|
||
gpg --dearmor -o /usr/share/keyrings/postgres-archive-keyring.gpg \
|
||
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/postgres-archive-keyring.gpg] https://apt.postgresql.org/pub/repos/apt/" \
|
||
$DISTRO_NAME-pgdg main $PG_MAJOR | tee /etc/apt/sources.list.d/postgres.list > /dev/null
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
--mount=type=tmpfs,target=/var/log \
|
||
apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get -yq dist-upgrade && \
|
||
DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
|
||
libpq-dev \
|
||
postgresql-client-$PG_MAJOR
|
||
|
||
# Install NodeJS and Yarn
|
||
ARG NODE_MAJOR
|
||
ARG YARN_VERSION=latest
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
--mount=type=tmpfs,target=/var/log \
|
||
apt-get update && \
|
||
apt-get install -y curl software-properties-common && \
|
||
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add - && \
|
||
echo "deb https://deb.nodesource.com/node_${NODE_MAJOR}.x $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/nodesource.list && \
|
||
apt-get update && \
|
||
DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends nodejs
|
||
RUN npm install -g yarn@$YARN_VERSION
|
||
|
||
# Application dependencies
|
||
# We use an external Aptfile for this, stay tuned
|
||
COPY Aptfile /tmp/Aptfile
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
--mount=type=tmpfs,target=/var/log \
|
||
apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get -yq dist-upgrade && \
|
||
DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
|
||
$(grep -Ev '^\s*#' /tmp/Aptfile | xargs)
|
||
|
||
# Configure bundler
|
||
ENV LANG=C.UTF-8 \
|
||
BUNDLE_JOBS=4 \
|
||
BUNDLE_RETRY=3
|
||
|
||
# Store Bundler settings in the project's root
|
||
ENV BUNDLE_APP_CONFIG=.bundle
|
||
|
||
# Uncomment this line if you want to run binstubs without prefixing with `bin/` or `bundle exec`
|
||
# ENV PATH /app/bin:$PATH
|
||
|
||
# Upgrade RubyGems and install the latest Bundler version
|
||
RUN gem update --system && \
|
||
gem install bundler
|
||
|
||
# Create a directory for the app code
|
||
RUN mkdir -p /app
|
||
WORKDIR /app
|
||
|
||
# Document that we're going to expose port 3000
|
||
EXPOSE 3000
|
||
# Use Bash as the default command
|
||
CMD ["/bin/bash"]
|
||
|
||
This configuration only contains the essentials, and so it can be used as a
|
||
starting point. Let me illustrate what we’re are doing here a bit further. The
|
||
first three lines might look a bit strange:
|
||
|
||
ARG RUBY_VERSION
|
||
ARG DISTRO_NAME=bullseye
|
||
FROM ruby:$RUBY_VERSION-slim-$DISTRO_NAME
|
||
|
||
Why not just use FROM ruby:3.1.0, or whatever is the stable Ruby version du
|
||
jour? Well, we’re going this route because we want to make our environment
|
||
configurable from the outside using Dockerfile as a sort of a template:
|
||
|
||
• The exact versions of the runtime dependencies are specified in the
|
||
docker-compose.yml (see below 👇).
|
||
• The list of apt-installable dependencies is stored in a separate file
|
||
(also, see below 👇👇).
|
||
|
||
Additionally, we parameterize the Debian release (bullseye by default) to make
|
||
sure we’re adding the correct sources for our other dependencies (such as
|
||
PostgreSQL).
|
||
|
||
Alright, now, note that we declare the argument once again after the FROM
|
||
statement:
|
||
|
||
FROM ruby:$RUBY_VERSION-slim-$DISTRO_NAME
|
||
ARG DISTRO_NAME
|
||
|
||
That’s the tricky part of how Dockerfiles work: the args are reset after the
|
||
FROM statement. For more details, check out [52]this issue.
|
||
|
||
Moving on, the rest of the file contains the actual build steps. First, we’ll
|
||
need to manually install some common system dependencies (Git, cURL, etc.), as
|
||
we’re using the slim base Docker image to reduce the size:
|
||
|
||
# Common dependencies
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
--mount=type=tmpfs,target=/var/log \
|
||
apt-get update -qq \
|
||
&& DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
|
||
build-essential \
|
||
gnupg2 \
|
||
curl \
|
||
less \
|
||
git
|
||
|
||
We’ll explain all the details of installing system dependencies below,
|
||
alongside the application-specific dependencies.
|
||
|
||
Installing PostgreSQL and NodeJS via apt requires adding their deb package
|
||
repos to the sources list.
|
||
|
||
Here’s PostgreSQL (based on the [53]official documentation):
|
||
|
||
ARG PG_MAJOR
|
||
RUN curl -sSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | \
|
||
gpg --dearmor -o /usr/share/keyrings/postgres-archive-keyring.gpg \
|
||
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/postgres-archive-keyring.gpg] https://apt.postgresql.org/pub/repos/apt/" \
|
||
$DISTRO_NAME-pgdg main $PG_MAJOR | tee /etc/apt/sources.list.d/postgres.list > /dev/null
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
--mount=type=tmpfs,target=/var/log \
|
||
apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get -yq dist-upgrade && \
|
||
DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
|
||
libpq-dev \
|
||
postgresql-client-$PG_MAJOR
|
||
|
||
Since we aren’t expecting anyone to use this Dockerfile without [54]
|
||
Docker Compose, we don’t provide a default value for the PG_MAJOR argument (the
|
||
same applies to NODE_MAJOR below, and YARN_VERSION further below).
|
||
|
||
Also, notice that in the code above that the DISTRO_NAME argument which we
|
||
defined at the very beginning of the file comes back into play.
|
||
|
||
And, we repeat our apt-get ... apt-get clean spell again: we want to make sure
|
||
all the major pieces of our environment are built in an isolated way (this will
|
||
help us to better utilize Docker cache layers when performing upgrades).
|
||
|
||
For NodeJS (from the [55]NodeSource repo):
|
||
|
||
ARG NODE_MAJOR
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
--mount=type=tmpfs,target=/var/log \
|
||
apt-get update && \
|
||
apt-get install -y curl software-properties-common && \
|
||
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add - && \
|
||
echo "deb https://deb.nodesource.com/node_${NODE_MAJOR}.x $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/nodesource.list && \
|
||
apt-get update && \
|
||
DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends nodejs
|
||
|
||
Then, we install Yarn via NPM:
|
||
|
||
ARG YARN_VERSION=latest
|
||
RUN npm install -g yarn@$YARN_VERSION
|
||
|
||
So, why are we adding NodeJS and Yarn in the first place? Although Rails 7
|
||
allows you to [56]go Node-less via [57]import maps or precompiled binaries
|
||
(like [58]tailwindcss-rails), these additions increase the chances of
|
||
supporting legacy pipelines or adding modern Webpacker alternatives.
|
||
|
||
Now it’s time to install the application-specific dependencies:
|
||
|
||
COPY Aptfile /tmp/Aptfile
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
--mount=type=tmpfs,target=/var/log \
|
||
apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get -yq dist-upgrade && \
|
||
DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
|
||
$(grep -Ev '^\s*#' /tmp/Aptfile | xargs)
|
||
|
||
Let’s talk about that Aptfile trick a bit:
|
||
|
||
COPY Aptfile /tmp/Aptfile
|
||
RUN apt-get install \
|
||
$(grep -Ev '^\s*#' /tmp/Aptfile | xargs) \
|
||
|
||
I borrowed this idea from [59]heroku-buildpack-apt, which allows for installing
|
||
additional packages on Heroku. If you’re using this buildpack, you can even
|
||
re-use the same Aptfile for both the local and the production environment.
|
||
|
||
Our [60]default Aptfile contains only a single package (we’ll use Vim to edit
|
||
the Rails Credentials):
|
||
|
||
vim
|
||
|
||
In one of the previous projects I worked on, we generated PDFs using LaTeX and
|
||
[61]TexLive. In a case like that, our Aptfile might look like this:
|
||
|
||
vim
|
||
# TeX packages
|
||
texlive
|
||
texlive-latex-recommended
|
||
texlive-fonts-recommended
|
||
texlive-lang-cyrillic
|
||
|
||
By doing this we can keep task-specific dependencies in a separate file, thus
|
||
making our Dockerfile more universal.
|
||
|
||
With regards to DEBIAN_FRONTEND=noninteractive, I kindly ask you to take a look
|
||
at this [62]answer on Ask Ubuntu.
|
||
|
||
The --no-install-recommends option helps save some space (and makes our image
|
||
smaller) by disabling the installation of recommended packages. You can see
|
||
more [63]about saving disk space here.
|
||
|
||
That first (fairly cryptic) part of every RUN statement that installs packages
|
||
also serves the same purpose: it moves out the local repository of retrieved
|
||
package files into a cache that will be preserved between builds. We need this
|
||
magic to be in every RUN statement that installs packages to make sure this
|
||
particular [64]Docker layer doesn’t contain any garbage. It also greatly speeds
|
||
up image build!
|
||
|
||
[65]RUN --mount is relatively new feature of Docker. Traditionally, every
|
||
package installation step would be appended with a dark spell full of rm and
|
||
truncate commands to clean up temporary files.
|
||
|
||
The final part of the Dockerfile is mostly devoted to Bundler:
|
||
|
||
# Configure bundler
|
||
ENV LANG=C.UTF-8 \
|
||
BUNDLE_JOBS=4 \
|
||
BUNDLE_RETRY=3 \
|
||
|
||
# Store Bundler settings in the project's root
|
||
ENV BUNDLE_APP_CONFIG=.bundle
|
||
|
||
# Uncomment this line if you want to run binstubs without prefixing with `bin/` or `bundle exec`
|
||
# ENV PATH /app/bin:$PATH
|
||
|
||
# Upgrade RubyGems and install the latest Bundler version
|
||
RUN gem update --system && \
|
||
gem install bundler
|
||
|
||
Using LANG=C.UTF-8 sets the default locale to UTF-8. This is an emotional
|
||
setting, as otherwise, Ruby would use US-ASCII for strings—and that’d mean
|
||
waving goodbye to those sweet, sweet emojis! 👋
|
||
|
||
Setting BUNDLE_APP_CONFIG is required if you’ll use the <root>/.bundle folder
|
||
to store project-specicic Bundler settings (like credentials for private gems).
|
||
The default Ruby image [66]defines this variable so Bundler doesn’t fall back
|
||
to the local config.
|
||
|
||
Optionally, you can add your <root>/bin folder to the PATH in order to run
|
||
commands without bundle exec. We don’t do this by default, because it could
|
||
break in a multi-project environment (for instance, when you have local gems or
|
||
engines in your Rails app).
|
||
|
||
Previously, we also had to specify the Bundler version (taking advantage of
|
||
[67]some hacks to make sure it’s picked up by the system). Luckily, since
|
||
Bundler 2.3.0, we no longer need to manually install the version defined in the
|
||
Gemfile.lock (BUNDLED_WITH). Instead, to avoid conflicts, Bundler [68]does this
|
||
for us.
|
||
|
||
[69]compose.yml
|
||
|
||
[70]Docker Compose is a tool we can use to orchestrate our containerized
|
||
environment. It allows us to link containers to each other, and to define
|
||
persistent volumes and services.
|
||
|
||
Below is the compose file for developing a typical Rails application with
|
||
PostgreSQL as the database, and with Sidekiq as the background job processor:
|
||
|
||
x-app: &app
|
||
build:
|
||
context: .
|
||
args:
|
||
RUBY_VERSION: '3.2.2'
|
||
PG_MAJOR: '15'
|
||
NODE_MAJOR: '18'
|
||
image: example-dev:1.0.0
|
||
environment: &env
|
||
NODE_ENV: ${NODE_ENV:-development}
|
||
RAILS_ENV: ${RAILS_ENV:-development}
|
||
tmpfs:
|
||
- /tmp
|
||
- /app/tmp/pids
|
||
|
||
x-backend: &backend
|
||
<<: *app
|
||
stdin_open: true
|
||
tty: true
|
||
volumes:
|
||
- ..:/app:cached
|
||
- bundle:/usr/local/bundle
|
||
- rails_cache:/app/tmp/cache
|
||
- node_modules:/app/node_modules
|
||
- packs:/app/public/packs
|
||
- packs-test:/app/public/packs-test
|
||
- history:/usr/local/hist
|
||
- ./.psqlrc:/root/.psqlrc:ro
|
||
- ./.bashrc:/root/.bashrc:ro
|
||
environment: &backend_environment
|
||
<<: *env
|
||
REDIS_URL: redis://redis:6379/
|
||
DATABASE_URL: postgres://postgres:postgres@postgres:5432
|
||
WEBPACKER_DEV_SERVER_HOST: webpacker
|
||
MALLOC_ARENA_MAX: 2
|
||
WEB_CONCURRENCY: ${WEB_CONCURRENCY:-1}
|
||
BOOTSNAP_CACHE_DIR: /usr/local/bundle/_bootsnap
|
||
XDG_DATA_HOME: /app/tmp/caches
|
||
YARN_CACHE_FOLDER: /app/node_modules/.yarn-cache
|
||
HISTFILE: /usr/local/hist/.bash_history
|
||
PSQL_HISTFILE: /usr/local/hist/.psql_history
|
||
IRB_HISTFILE: /usr/local/hist/.irb_history
|
||
EDITOR: vi
|
||
depends_on: &backend_depends_on
|
||
postgres:
|
||
condition: service_healthy
|
||
redis:
|
||
condition: service_healthy
|
||
|
||
services:
|
||
rails:
|
||
<<: *backend
|
||
command: bundle exec rails
|
||
|
||
web:
|
||
<<: *backend
|
||
command: bundle exec rails server -b 0.0.0.0
|
||
ports:
|
||
- '3000:3000'
|
||
depends_on:
|
||
webpacker:
|
||
condition: service_started
|
||
sidekiq:
|
||
condition: service_started
|
||
|
||
sidekiq:
|
||
<<: *backend
|
||
command: bundle exec sidekiq -C config/sidekiq.yml
|
||
|
||
postgres:
|
||
image: postgres:14
|
||
volumes:
|
||
- .psqlrc:/root/.psqlrc:ro
|
||
- postgres:/var/lib/postgresql/data
|
||
- history:/user/local/hist
|
||
environment:
|
||
PSQL_HISTFILE: /user/local/hist/.psql_history
|
||
POSTGRES_PASSWORD: postgres
|
||
ports:
|
||
- 5432
|
||
healthcheck:
|
||
test: pg_isready -U postgres -h 127.0.0.1
|
||
interval: 5s
|
||
|
||
redis:
|
||
image: redis:6.2-alpine
|
||
volumes:
|
||
- redis:/data
|
||
ports:
|
||
- 6379
|
||
healthcheck:
|
||
test: redis-cli ping
|
||
interval: 1s
|
||
timeout: 3s
|
||
retries: 30
|
||
|
||
webpacker:
|
||
<<: *app
|
||
command: bundle exec ./bin/webpack-dev-server
|
||
ports:
|
||
- '3035:3035'
|
||
volumes:
|
||
- ..:/app:cached
|
||
- bundle:/usr/local/bundle
|
||
- node_modules:/app/node_modules
|
||
- packs:/app/public/packs
|
||
- packs-test:/app/public/packs-test
|
||
environment:
|
||
<<: *env
|
||
WEBPACKER_DEV_SERVER_HOST: 0.0.0.0
|
||
YARN_CACHE_FOLDER: /app/node_modules/.yarn-cache
|
||
|
||
volumes:
|
||
bundle:
|
||
node_modules:
|
||
history:
|
||
rails_cache:
|
||
postgres:
|
||
redis:
|
||
packs:
|
||
packs-test:
|
||
|
||
We define six services and two extension fields (x-app and x-backend). [71]
|
||
Extension fields allow us to define common parts of the configuration. We can
|
||
attach YAML anchors to them, and later, embed anywhere in the file.
|
||
|
||
NOTE: In the end, we don’t use Docker Compose or execute the docker compose up
|
||
command in order to run our application. Instead, we use Dip (see [72]below),
|
||
and thus, the compose.yml file only acts as a services registry. Another
|
||
important thing to note is that we put the compose.yml file into the .dockerdev
|
||
/ folder. This is why we mount the source code as ..:/app and not .:/app.
|
||
Please, keep this in mind if you’re considering using this configuration
|
||
without Dip (which is not recommended).
|
||
|
||
On that note, let’s go ahead and take a thorough look at each service.
|
||
|
||
x-app
|
||
|
||
The main purpose of this extension is to provide all the required information
|
||
to build our application container (as defined in the Dockerfile above):
|
||
|
||
x-app: &app
|
||
build:
|
||
context: .
|
||
args:
|
||
RUBY_VERSION: '3.2.2'
|
||
PG_MAJOR: '15'
|
||
NODE_MAJOR: '18'
|
||
|
||
What is the context? The context directory defines the [73]build context for
|
||
Docker. This is something like a working directory for the build process—for
|
||
example, when we execute the COPY command. As this directory is packaged and
|
||
sent to the Docker daemon every time an image is built, it’s better to keep it
|
||
as small as possible. We’re good here, since our context is just the .dockerdev
|
||
folder.
|
||
|
||
And, as we mentioned earlier, we’ll specify the exact version of our
|
||
dependencies using the args as declared in the Dockerfile.
|
||
|
||
It’s also a good idea to pay attention to the way we tag images:
|
||
|
||
image: example-dev:1.0.0
|
||
|
||
One of the benefits of using Docker for development is the ability to
|
||
automatically synchronize configuration changes across the team. This means the
|
||
only time you need to upgrade the local image version is when you make changes
|
||
to it (or to the arguments or files it relies on). Using example-dev:latest is
|
||
like shooting yourself in the foot.
|
||
|
||
Keeping an image version also helps work with two different environments
|
||
without any additional hassle. For example, when working on a long-standing
|
||
“chore/upgrade-to-ruby-3” branch, you can easily switch to master and use the
|
||
older image with the older version of Ruby: no need to rebuild anything.
|
||
|
||
Rule of thumb: Increase the version number in the image tag every time you
|
||
change Dockerfile or its arguments (upgrading dependencies, etc.)
|
||
|
||
Next, we add some common environment variables (those shared by multiple
|
||
services, e.g., Rails and Webpacker):
|
||
|
||
environment: &env
|
||
NODE_ENV: ${NODE_ENV:-development}
|
||
RAILS_ENV: ${RAILS_ENV:-development}
|
||
|
||
There are several things going on here, but I’d like to focus on just one: the
|
||
X=${X:-smth} syntax. This could be translated as “For X variable within the
|
||
container, if present, use the host machine’s X env variable, otherwise, use
|
||
another value”. Thus, we make it possible to run a service in a different
|
||
environment specified along with a command, e.g., RAILS_ENV=test docker-compose
|
||
up rails.
|
||
|
||
Note that we’re using a dictionary value (NODE_ENV: xxx) and not a list value (
|
||
- NODE_ENV=xxx) for the environment field. This allows us to re-use the common
|
||
settings (see below).
|
||
|
||
We also tell Docker to [74]use tmpfs for the /tmp folder within a container—and
|
||
also for the tmp/pids folder of our application. This way, we ensure that no
|
||
server.pid survives a container exit (say goodbye to any “A server is already
|
||
running” errors):
|
||
|
||
tmpfs:
|
||
- /tmp
|
||
- /app/tmp/pids
|
||
|
||
x-backend
|
||
|
||
Alright, so now, we’ve finally reached the most interesting part of this post.
|
||
|
||
This service defines the shared behavior of all Ruby services.
|
||
|
||
Let’s talk about the volumes first:
|
||
|
||
x-backend: &backend
|
||
<<: *app
|
||
stdin_open: true
|
||
tty: true
|
||
volumes:
|
||
- ..:/app:cached
|
||
- rails_cache:/app/tmp/cache
|
||
- bundle:/usr/local/bundle
|
||
- history:/usr/local/hist
|
||
- node_modules:/app/node_modules
|
||
- packs:/app/public/packs
|
||
- packs-test:/app/public/packs-test
|
||
- ./.psqlrc:/root/.psqlrc:ro
|
||
- ./.bashrc:/root/.bashrc:ro
|
||
- ./.pryrc:/root/.pryrc:ro
|
||
environment: &backend_environment
|
||
<<: *env
|
||
REDIS_URL: redis://redis:6379/
|
||
DATABASE_URL: postgres://postgres:postgres@postgres:5432
|
||
WEBPACKER_DEV_SERVER_HOST: webpacker
|
||
MALLOC_ARENA_MAX: 2
|
||
WEB_CONCURRENCY: ${WEB_CONCURRENCY:-1}
|
||
BOOTSNAP_CACHE_DIR: /usr/local/bundle/_bootsnap
|
||
XDG_DATA_HOME: /app/tmp/caches
|
||
YARN_CACHE_FOLDER: /app/node_modules/.yarn-cache
|
||
HISTFILE: /usr/local/hist/.bash_history
|
||
PSQL_HISTFILE: /usr/local/hist/.psql_history
|
||
IRB_HISTFILE: /usr/local/hist/.irb_history
|
||
EDITOR: vi
|
||
depends_on: &backend_depends_on
|
||
postgres:
|
||
condition: service_healthy
|
||
redis:
|
||
condition: service_healthy
|
||
volumes:
|
||
- ..:/app:cached
|
||
- bundle:/usr/local/bundle
|
||
- rails_cache:/app/tmp/cache
|
||
- node_modules:/app/node_modules
|
||
- packs:/app/public/packs
|
||
- packs-test:/app/public/packs-test
|
||
- history:/usr/local/hist
|
||
- ./.psqlrc:/root/.psqlrc:ro
|
||
- ./.bashrc:/root/.bashrc:ro
|
||
|
||
The Docker team is striving to make Docker work faster on MacOS. The latest
|
||
releases (since [75]4.6.0) come with VirtioFS accelerated directory sharing and
|
||
virtualization.framework support. Go check the “Experimental Features” tab in
|
||
the Docker Desktop Preferences. You might find the resulting performance
|
||
improvement to be pretty amazing: ([76]regular actions become ~2x faster)!
|
||
|
||
The first item in the volumes list mounts the project directory to the /app
|
||
folder within a container using the cached strategy. This cached modifier was
|
||
the key to efficient Docker development on macOS.
|
||
|
||
Wait, was?
|
||
|
||
Yeah. Was. That’s because since the release of gRPC FUSE synchronization, it’s
|
||
[77]no longer needed. Still, I decided to keep it for a while, for two reasons:
|
||
first, some of your teammates may still be using older Docker desktop versions,
|
||
and second, I ran some benchmarks and found that using older osxfs file sharing
|
||
could have better performance (but only when using :cached). So, even on modern
|
||
versions of Docker, it could make sense to uncheck the “Use gRPC FUSE for file
|
||
sharing” option inside the preferences menu.
|
||
|
||
The next line tells our container to use a volume named bundle to store the
|
||
contents of /usr/local/bundle (this is where gems are stored [78]by default).
|
||
By doing this, we persist our gem data across runs: all the volumes defined in
|
||
compose.yml will stay put until we run compose down --volumes.
|
||
|
||
The following lines have also been dutifully placed in order to nullify the
|
||
“Docker is slow on Mac” curse. We put all the generated files into Docker
|
||
volumes to avoid any heavy disk operations on the host machine:
|
||
|
||
- rails_cache:/app/tmp/cache
|
||
- node_modules:/app/node_modules
|
||
- packs:/app/public/packs
|
||
- packs-test:/app/public/packs-test
|
||
|
||
To give Docker a suitably fast speed on macOS, follow these two rules: use
|
||
:cached to mount source files (if not using gRPC FUSE), and use volumes for
|
||
generated content (assets, bundle, etc.).
|
||
|
||
NOTE: If you’re using Sprockets (or Propshaft), don’t forget to add a dedicated
|
||
volume to store the assets (assets:/app/public/assets). For tailwindcss-rails,
|
||
add something like assets_builds:/app/assets/builds.
|
||
|
||
We’ll then mount different command line tools configuration files and a volume
|
||
to persist their history:
|
||
|
||
- history:/usr/local/hist
|
||
- ./.psqlrc:/root/.psqlrc:ro
|
||
- ./.bashrc:/root/.bashrc:ro
|
||
|
||
Oh, and why is psql in the Ruby container? That’s because it’s used internally
|
||
when you run rails dbconsole.
|
||
|
||
Pressing onward, our [79].psqlrc file contains the following trick which makes
|
||
it possible to specify the path to the history file via the env variable—thus
|
||
allowing us to specify the path to the history file via the PSQL_HISTFILE env
|
||
variable, or otherwise, fall back to the default $HOME/.psql_history:
|
||
|
||
\set HISTFILE `[[ -z $PSQL_HISTFILE ]] && echo $HOME/.psql_history || echo $PSQL_HISTFILE`
|
||
|
||
The .bashrc file allows us to add terminal customizations within a container:
|
||
|
||
alias be="bundle exec"
|
||
|
||
Alright, let’s talk about the environment variables:
|
||
|
||
environment: &backend_environment
|
||
<<: *env
|
||
# ----
|
||
# Service discovery
|
||
# ----
|
||
REDIS_URL: redis://redis:6379/
|
||
DATABASE_URL: postgres://postgres:postgres@postgres:5432
|
||
WEBPACKER_DEV_SERVER_HOST: webpacker
|
||
# ----
|
||
# Application configuration
|
||
# ----
|
||
MALLOC_ARENA_MAX: 2
|
||
WEB_CONCURRENCY: ${WEB_CONCURRENCY:-1}
|
||
# -----
|
||
# Caches
|
||
# -----
|
||
BOOTSNAP_CACHE_DIR: /usr/local/bundle/_bootsnap
|
||
# This env variable is used by some tools (e.g., RuboCop) to store caches
|
||
XDG_DATA_HOME: /app/tmp/cache
|
||
# Puts the Yarn cache into a mounted volume for speed
|
||
YARN_CACHE_FOLDER: /app/node_modules/.yarn-cache
|
||
# ----
|
||
# Dev tools
|
||
# ----
|
||
HISTFILE: /usr/local/hist/.bash_history
|
||
PSQL_HISTFILE: /usr/local/hist/.psql_history
|
||
IRB_HISTFILE: /usr/local/hist/.irb_history
|
||
EDITOR: vi
|
||
|
||
First of all, we “inherit” variables from the common environment variables (<<:
|
||
*env).
|
||
|
||
The first group of variables (DATABASE_URL, REDIS_URL, and
|
||
WEBPACKER_DEV_SERVER_HOST) connect our Ruby application to other services.
|
||
|
||
The DATABASE_URL and WEBPACKER_DEV_SERVER_HOST variables are supported by Rails
|
||
(ActiveRecord and Webpacker respectively) out of the box. Some libraries
|
||
(Sidekiq) also support REDIS_URL, but not all of them: for instance, Action
|
||
Cable must be explicitly configured.
|
||
|
||
The second group contains some application-wide settings. For example, we
|
||
define MALLOC_ARENA_MAX and WEB_CONCURRENCY to help us keep Ruby memory
|
||
handling in check.
|
||
|
||
Read more about Ruby memory spells and techniques:
|
||
|
||
[80]Cables vs. malloc_trim, or yet another Ruby memory usage benchmark
|
||
|
||
Cables vs. malloc_trim, or yet another Ruby memory usage benchmark
|
||
|
||
March 19, 2019
|
||
Read also
|
||
|
||
Also, we have the variables responsible for storing caches in Docker volumes (
|
||
BOOTSNAP_CACHE_DIR, XDG_DATA_HOME, YARN_CACHE_FOLDER).
|
||
|
||
We use [81]bootsnap to speed up application load time. We store its cache in
|
||
the same volume as the Bundler data. This is because this cache mostly contains
|
||
the gem data, and we want to make sure the cache is reset every time we drop
|
||
the Bundler volume (for instance, during a Ruby version upgrade).
|
||
|
||
The final group of variables aim to improve the developer experience. HISTFILE:
|
||
/usr/local/hist/.bash_history is the most significant here: it tells Bash to
|
||
store its history in the specified location, thus making it persistent. The
|
||
same goes for PSQL_HISTFILE and IRB_HISTFILE.
|
||
|
||
NOTE: You need to configure IRB to store history in the specified location. To
|
||
do that, drop these lines into your .irbrc file:
|
||
|
||
IRB.conf[:HISTORY_FILE] = ENV["IRB_HISTFILE"] if ENV["IRB_HISTFILE"]
|
||
|
||
Finally, EDITOR: vi is used, for example, by the rails credentials:edit command
|
||
to manage credentials files.
|
||
|
||
And with that, the only lines in this service we’ve yet to cover are:
|
||
|
||
stdin_open: true
|
||
tty: true
|
||
|
||
These lines make this service interactive, that is, they provide a TTY. We need
|
||
this, for example, to run the Rails console or Bash within a container.
|
||
|
||
This is the same as running a Docker container with the -it option.
|
||
|
||
rails
|
||
|
||
The rails server is our default backend service. The only thing it overrides is
|
||
the command to execute:
|
||
|
||
rails:
|
||
<<: *backend
|
||
command: bundle exec rails
|
||
|
||
This service is meant for executing all the commands needed in development (
|
||
rails db:migrate, rspec, etc.).
|
||
|
||
web
|
||
|
||
The web service is meant for launching a web server. It defines the exposed
|
||
ports and the required dependencies to run the app itself.
|
||
|
||
webpacker
|
||
|
||
The only thing I want to mention here is the WEBPACKER_DEV_SERVER_HOST: 0.0.0.0
|
||
setting: it makes the Webpack dev server accessible from the outside (it runs
|
||
on localhost by default).
|
||
|
||
Health checks
|
||
|
||
When running common Rails commands such as db:migrate, we want to ensure that
|
||
the DB is up and ready to accept connections. How can we tell Docker Compose to
|
||
wait until a dependent service is ready? We can use [82]health checks!
|
||
|
||
You’ve probably noticed that our depends_on definition isn’t just a list of
|
||
services:
|
||
|
||
backend:
|
||
# ...
|
||
depends_on:
|
||
postgres:
|
||
condition: service_healthy
|
||
redis:
|
||
condition: service_healthy
|
||
|
||
postgres:
|
||
# ...
|
||
healthcheck:
|
||
test: pg_isready -U postgres -h 127.0.0.1
|
||
interval: 5s
|
||
|
||
redis:
|
||
# ...
|
||
healthcheck:
|
||
test: redis-cli ping
|
||
interval: 1s
|
||
timeout: 3s
|
||
retries: 30
|
||
|
||
Introducing Dip
|
||
|
||
If you still think that Docker Compose way is too complicated, there’s a tool
|
||
called [83]Dip (developed by one of my colleages at Evil Martians) which aims
|
||
to make the developer experience even smoother.
|
||
|
||
[84]Reusable development containers with Docker Compose and Dip
|
||
|
||
Reusable development containers with Docker Compose and Dip
|
||
|
||
November 17, 2020
|
||
Read also
|
||
|
||
[85]Dip is a thin wrapper over docker compose, which provides a switch from
|
||
infrastructure-oriented flow to development-oriented one. The key benefits of
|
||
using Dip are as follows:
|
||
|
||
• The ability to define application-specific interactive commands and
|
||
sub-commands.
|
||
• The dip provision flow to quickly set up a development environment from
|
||
scratch.
|
||
• Support for multiple compose.yml files (including OS-specific
|
||
configurations).
|
||
|
||
With Dip in place, to start working on the app locally, you just need to
|
||
execute a few commands:
|
||
|
||
# Builds a Docker image if none, runs additional commands
|
||
$ dip provision
|
||
# Runs a Rails server with the defined dependencies
|
||
$ dip rails s
|
||
=> Booting Puma
|
||
=> Rails 7.0.2.2 application starting in development
|
||
=> Run `bin/rails server --help` for more startup options
|
||
[1] Puma starting in cluster mode...
|
||
...
|
||
[1] - Worker 0 (PID: 9) booted in 0.0s, phase: 0
|
||
|
||
Here is our typical [86]dip.yml file:
|
||
|
||
version: '7.1'
|
||
|
||
# Define default environment variables to pass
|
||
# to Docker Compose
|
||
environment:
|
||
RAILS_ENV: development
|
||
|
||
compose:
|
||
files:
|
||
- .dockerdev/compose.yml
|
||
project_name: example_demo
|
||
|
||
interaction:
|
||
# This command spins up a Rails container with the required dependencies (such as databases),
|
||
# and opens a terminal within it.
|
||
runner:
|
||
description: Open a Bash shell within a Rails container (with dependencies up)
|
||
service: rails
|
||
command: /bin/bash
|
||
|
||
# Run a Rails container without any dependent services (useful for non-Rails scripts)
|
||
bash:
|
||
description: Run an arbitrary script within a container (or open a shell without deps)
|
||
service: rails
|
||
command: /bin/bash
|
||
compose_run_options: [ no-deps ]
|
||
|
||
# A shortcut to run Bundler commands
|
||
bundle:
|
||
description: Run Bundler commands
|
||
service: rails
|
||
command: bundle
|
||
compose_run_options: [ no-deps ]
|
||
|
||
# A shortcut to run RSpec (which overrides the RAILS_ENV)
|
||
rspec:
|
||
description: Run RSpec commands
|
||
service: rails
|
||
environment:
|
||
RAILS_ENV: test
|
||
command: bundle exec rspec
|
||
|
||
rails:
|
||
description: Run Rails commands
|
||
service: rails
|
||
command: bundle exec rails
|
||
subcommands:
|
||
s:
|
||
description: Run Rails server at http://localhost:3000
|
||
service: web
|
||
compose:
|
||
run_options: [service-ports, use-aliases]
|
||
|
||
yarn:
|
||
description: Run Yarn commands
|
||
service: rails
|
||
command: yarn
|
||
compose_run_options: [ no-deps ]
|
||
|
||
psql:
|
||
description: Run Postgres psql console
|
||
service: postgres
|
||
default_args: anycasts_dev
|
||
command: psql -h postgres -U postgres
|
||
|
||
'redis-cli':
|
||
description: Run Redis console
|
||
service: redis
|
||
command: redis-cli -h redis
|
||
|
||
provision:
|
||
- dip compose down --volumes
|
||
- dip compose up -d postgres redis
|
||
- dip bash -c bin/setup
|
||
|
||
Let me explain some bits of this in further detail.
|
||
|
||
First, the compose section:
|
||
|
||
compose:
|
||
files:
|
||
- .dockerdev/compose.yml
|
||
project_name: example_demo
|
||
|
||
Here we should specify the path to our Compose configuration (.dockerdev/
|
||
compose.yml). Accordingly, we can run dip from the project root, and the
|
||
correct configuration will be picked up.
|
||
|
||
The project_name is important: if we don’t specify it, the folder containing
|
||
the compose.yml file would be used (“dockerdev”), which could lead to
|
||
collisions between different projects.
|
||
|
||
The rails command is also worth some additional attention:
|
||
|
||
rails:
|
||
description: Run Rails commands
|
||
service: rails
|
||
command: bundle exec rails
|
||
subcommands:
|
||
s:
|
||
description: Run Rails server at http://localhost:3000
|
||
service: web
|
||
compose:
|
||
run_options: [service-ports, use-aliases]
|
||
|
||
By default, the dip rails command would call bundle exec rails within a Rails
|
||
container. However, we use the subcommand feature of Dip here to treat dip
|
||
rails s differently:
|
||
|
||
• We use the web service, not rails (so, the deps are up).
|
||
• We expose the service ports (3000 in our case).
|
||
• We also enable network aliases, so other services can access this container
|
||
via the web hostname.
|
||
|
||
Under the hood, this will result in the following Docker Compose command:
|
||
|
||
docker compose run --rm --service-ports --use-aliases web
|
||
|
||
Note that it uses run, and not up. This difference makes our server
|
||
terminal-accessible. For example, this means that we can attach a debugger and
|
||
use it without any problems (with the up command the terminal is
|
||
non-interactive).
|
||
|
||
Interactive provisioning
|
||
|
||
To learn how to keep configuration under control, check out this “Terraforming
|
||
Rails” series:
|
||
|
||
[87]Anyway Config: Keep your Ruby configuration sane
|
||
|
||
Anyway Config: Keep your Ruby configuration sane
|
||
|
||
April 14, 2020
|
||
[svg]
|
||
Cover for Anyway Config: Keep your Ruby configuration sane
|
||
Read also
|
||
|
||
For most applications, building an image and setting up a database is not
|
||
enough to start developing: beyond this, some kind of secrets, or credentials,
|
||
or .env files are required. Here, we’ve managed to use Dip to help new
|
||
engineers quickly assemble all these wayfallen parts by providing an
|
||
interactive provision experience.
|
||
|
||
Let’s consider, for example, that we need to put a .env.development.local file
|
||
with some secret info and also configure RubyGems to download packages from a
|
||
private registry (say, Sidekiq Pro). For this, I’ll write the following
|
||
provision script:
|
||
|
||
# The command is extracted, so we can use it alone
|
||
configure_bundler:
|
||
command: |
|
||
(test -f .bundle/config && cat .bundle/config | \
|
||
grep BUNDLE_ENTERPRISE__CONTRIBSYS__COM > /dev/null) ||
|
||
\
|
||
(echo "Sidekiq ent credentials: "; read -r creds; dip bundle config --local enterprise.contribsys.com $creds)
|
||
|
||
provision:
|
||
- (test -f .env.development.local) || (echo "\n\n ⚠️ .env.development.local file is missing\n\n"; exit 1)
|
||
- dip compose down --volumes
|
||
- dip configure_bundler
|
||
- (test -f config/database.yml) || (cp .dockerdev/database.yml.example config/database.yml)
|
||
- dip compose up -d postgres redis
|
||
- dip bash -c bin/setup
|
||
|
||
Below you can see a demonstration of this command running in action:
|
||
|
||
An interactive Dip provisioning example
|
||
|
||
Services vs Docker for development
|
||
|
||
You can use a good ‘ol [88]Makefile to do the same, for sure. However, we’ve
|
||
found that using a dedicated tool (like Dip) to define everything in a
|
||
declarative manner is more efficient.
|
||
|
||
One more use case for standardizing the development setup is to make it
|
||
possible to run multiple independent services locally. Let me quickly
|
||
demonstrate how we do this with Dip. First, you need to dockerize each
|
||
application (following this post). After that, we need to connect the apps to
|
||
each other. How can we do this? With the help of Docker Compose [89]external
|
||
networks.
|
||
|
||
We add the following line to the dip.yml for each app:
|
||
|
||
# ...
|
||
provision:
|
||
# Make sure the named network exists
|
||
- docker network inspect my_project > /dev/null 2>&1 || \
|
||
docker network create my_project
|
||
# ...
|
||
|
||
Finally, we attach services to this network via aliases in the compose.yml
|
||
files:
|
||
|
||
# service A: compose.yml
|
||
service:
|
||
ruby:
|
||
# ...
|
||
networks:
|
||
default:
|
||
project:
|
||
aliases:
|
||
- project-a
|
||
|
||
networks:
|
||
project:
|
||
external:
|
||
name: my_project
|
||
|
||
# service B: compose.yml
|
||
service:
|
||
web:
|
||
# ...
|
||
environment:
|
||
# We can access the service A via its alias defined for the external network
|
||
SERVICE_URL: http://project-a:3000
|
||
|
||
networks:
|
||
project:
|
||
external:
|
||
name: my_project
|
||
|
||
From development to production
|
||
|
||
So, here’s one of the most popular questions we’ve been asked since launching
|
||
the first version of this article: how to go live with Docker? To answer this,
|
||
we’d need to write a entirely new article… and we will 😉.
|
||
|
||
For now, let me give a sneak preview of how can we extend the current
|
||
development setup to cover the production environment as well.
|
||
|
||
First of all, we’re not going to talk about a Docker Compose-based deployment,
|
||
so compose.yml is out. All we need is to update our Docker image to reflect the
|
||
difference between development and production:
|
||
|
||
1. For security reasons, we should execute the code on behalf of the regular,
|
||
non-root user.
|
||
2. We should keep all the required dependencies and artifacts within the image
|
||
itself; we cannot use volumes (the image should be self-contained).
|
||
3. We should keep and copy the source code into a container.
|
||
4. The resulting image should be as slim as possible.
|
||
|
||
To achieve this, we’ll refactor our existing Dockerfile to define multiple
|
||
stages (and to support [90]multi-stage builds). Below is the annotated example:
|
||
|
||
# syntax=docker/dockerfile:1
|
||
|
||
ARG RUBY_VERSION
|
||
ARG DISTRO_NAME=bullseye
|
||
|
||
# Here we add the the name of the stage ("base")
|
||
FROM ruby:$RUBY_VERSION-slim-$DISTRO_NAME AS base
|
||
|
||
ARG PG_MAJOR
|
||
ARG NODE_MAJOR
|
||
ARG YARN_VERSION
|
||
|
||
# Common dependencies
|
||
# ...
|
||
# The following lines are exactly the same as before
|
||
# ...
|
||
# ...
|
||
WORKDIR /app
|
||
|
||
EXPOSE 3000
|
||
CMD ["/bin/bash"]
|
||
|
||
# Then, we define the "development" stage from the base one
|
||
FROM base AS development
|
||
|
||
ENV RAILS_ENV=development
|
||
|
||
# The major difference from the base image is that we may have development-only system
|
||
# dependencies (like Vim or graphviz).
|
||
# We extract them into the Aptfile.dev file.
|
||
COPY Aptfile.dev /tmp/Aptfile.dev
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
--mount=type=tmpfs,target=/var/log \
|
||
apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get -yq dist-upgrade && \
|
||
DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
|
||
$(grep -Ev '^\s*#' /tmp/Aptfile.dev | xargs)
|
||
|
||
# The production-builder image is responsible for installing dependencies and compiling assets
|
||
FROM base as production-builder
|
||
|
||
# First, we create and configure a dedicated user to run our application
|
||
RUN groupadd --gid 1005 my_user \
|
||
&& useradd --uid 1005 --gid my_user --shell /bin/bash --create-home my_user
|
||
USER my_user
|
||
RUN mkdir /home/my_user/app
|
||
WORKDIR /home/my_user/app
|
||
|
||
# Then, we re-configure Bundler
|
||
ENV RAILS_ENV=production \
|
||
LANG=C.UTF-8 \
|
||
BUNDLE_JOBS=4 \
|
||
BUNDLE_RETRY=3 \
|
||
BUNDLE_APP_CONFIG=/home/my_user/bundle \
|
||
BUNDLE_PATH=/home/my_user/bundle \
|
||
GEM_HOME=/home/my_user/bundle
|
||
|
||
# Install Ruby gems
|
||
COPY --chown=my_user:my_user Gemfile Gemfile.lock ./
|
||
RUN mkdir $BUNDLE_PATH \
|
||
&& bundle config --local deployment 'true' \
|
||
&& bundle config --local path "${BUNDLE_PATH}" \
|
||
&& bundle config --local without 'development test' \
|
||
&& bundle config --local clean 'true' \
|
||
&& bundle config --local no-cache 'true' \
|
||
&& bundle install --jobs=${BUNDLE_JOBS} \
|
||
&& rm -rf $BUNDLE_PATH/ruby/${RUBY_VERSION}/cache/* \
|
||
&& rm -rf /home/my_user/.bundle/cache/*
|
||
|
||
# Install JS packages
|
||
COPY --chown=my_user:my_user package.json yarn.lock ./
|
||
RUN yarn install --check-files
|
||
|
||
# Copy code
|
||
COPY --chown=my_user:my_user . .
|
||
|
||
# Precompile assets
|
||
# NOTE: The command may require adding some environment variables (e.g., SECRET_KEY_BASE) if you're not using
|
||
# credentials.
|
||
RUN bundle exec rails assets:precompile
|
||
|
||
# Finally, our production image definition
|
||
# NOTE: It's not extending the base image, it's a new one
|
||
FROM ruby:$RUBY_VERSION-slim-$DISTRO_NAME AS production
|
||
|
||
# Production-only dependencies
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
--mount=type=tmpfs,target=/var/log \
|
||
apt-get update -qq \
|
||
&& apt-get dist-upgrade -y \
|
||
&& DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
|
||
curl \
|
||
gnupg2 \
|
||
less \
|
||
tzdata \
|
||
time \
|
||
locales \
|
||
&& update-locale LANG=C.UTF-8 LC_ALL=C.UTF-8
|
||
|
||
# Upgrade RubyGems and install the latest Bundler version
|
||
RUN gem update --system && \
|
||
gem install bundler
|
||
|
||
# Create and configure a dedicated user (use the same name as for the production-builder image)
|
||
RUN groupadd --gid 1005 my_user \
|
||
&& useradd --uid 1005 --gid my_user --shell /bin/bash --create-home my_user
|
||
RUN mkdir /home/my_user/app
|
||
WORKDIR /home/my_user/app
|
||
USER my_user
|
||
|
||
# Ruby/Rails env configuration
|
||
ENV RAILS_ENV=production \
|
||
BUNDLE_APP_CONFIG=/home/my_user/bundle \
|
||
BUNDLE_PATH=/home/my_user/bundle \
|
||
GEM_HOME=/home/my_user/bundle \
|
||
PATH="/home/my_user/app/bin:${PATH}" \
|
||
LANG=C.UTF-8 \
|
||
LC_ALL=C.UTF-8
|
||
|
||
EXPOSE 3000
|
||
|
||
# Copy code
|
||
COPY --chown=my_user:my_user . .
|
||
|
||
# Copy artifacts
|
||
# 1) Installed gems
|
||
COPY --from=production-builder $BUNDLE_PATH $BUNDLE_PATH
|
||
# 2) Compiled assets (by Webpacker in this case)
|
||
COPY --from=production-builder /home/my_user/app/public/packs /home/my_user/app/public/packs
|
||
# 3) We can even copy the Bootsnap cache to speed up our Rails server load!
|
||
COPY --chown=my_user:my_user --from=production-builder /home/my_user/app/tmp/cache/bootsnap* /home/my_user/app/tmp/cache/
|
||
|
||
CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0"]
|
||
|
||
Introducing the Ruby on Whales interactive generator
|
||
|
||
As a bonus, our Ruby on Whales [91]repository ships with a Rails template
|
||
(published on [92]Rails Bytes), which can help you quickly adopt Docker for
|
||
development by running a single command (and answering a few questions).
|
||
|
||
Without further ado, check out the demonstartion below:
|
||
|
||
An interactive Ruby on Whales installer
|
||
|
||
You can give it a try by running a single command:
|
||
|
||
rails app:template LOCATION='https://railsbytes.com/script/z5OsoB'
|
||
|
||
Radio wave representing wind sounds on Mars
|
||
|
||
Acknoledgements
|
||
|
||
I would like to thank:
|
||
|
||
• [93]Sergey Ponomarev for sharing performance tips and helping battle-test
|
||
the initial dockerization attempts.
|
||
• [94]Mikhail Merkushin for his work on Dip.
|
||
• [95]Dmitriy Nemykin for helping with the major (v2) upgrade.
|
||
• [96]Oliver Klee ([97]Brain Gourmets) for continuous PRs with the
|
||
configuration improvements and actualization.
|
||
|
||
Radio wave representing wind sounds on Mars
|
||
|
||
Changelog
|
||
|
||
2.0.3 (2023-09-21)
|
||
|
||
• Upgrade Node.js installation script.
|
||
|
||
2.0.2 (2022-11-30)
|
||
|
||
• Use RUN --mount for caching packages between builds instead of manual
|
||
cleanup.
|
||
|
||
2.0.1 (2022-03-22)
|
||
|
||
• Replace deprecated apt-key with gpg.
|
||
|
||
2.0.0 (2022-03-02)
|
||
|
||
• Major upgrade and new chapters.
|
||
|
||
1.1.4 (2021-10-12)
|
||
|
||
• Added tmp/pids to tmpfs (to deal with “A server is already running”
|
||
errors).
|
||
|
||
1.1.3 (2021-03-30)
|
||
|
||
• Updated Dockerfile to mitigate MiniMagic licensing issues. See [98]
|
||
terraforming-rails#35
|
||
• Use dictionary to organize environment variables. See [99]
|
||
terraforming-rails#6
|
||
|
||
1.1.2 (2021-02-26)
|
||
|
||
• Update dependencies versions. See [100]terraforming-rails#28
|
||
• Allow to use comments in Aptfile. See [101]terraforming-rails#31
|
||
• Fix path to Aptfile inside Dockerfile. See [102]terraforming-rails#33
|
||
|
||
1.1.1 (2020-09-15)
|
||
|
||
• Use .dockerdev directory as build context instead of project directory. See
|
||
[103]terraforming-rails#26 for details.
|
||
|
||
1.1.0 (2019-12-10)
|
||
|
||
• Change base Ruby image to slim.
|
||
• Specify Debian release for Ruby version explicitly and upgrade to buster.
|
||
• Use standard Bundler path (/usr/local/bundle) instead of /bundle.
|
||
• Use Docker Compose file format v2.4.
|
||
• Add health checking to postgres and redis services.
|
||
|
||
Join our email newsletter
|
||
|
||
Get all the new posts delivered directly to your inbox. Unsubscribe anytime.
|
||
|
||
[104][ ]Your email[105][ ]
|
||
Subscribe
|
||
Or [107]subscribe to a feed
|
||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||
[108]chronicles@evilmartians.com
|
||
|
||
United States
|
||
|
||
[109]+1 888 400 5485
|
||
|
||
Portugal
|
||
|
||
[110]+351 308 808 570
|
||
|
||
Japan
|
||
|
||
[111]+81 6 6225 1242
|
||
|
||
• [112]Contact us
|
||
• [113]Careers
|
||
• [114]日本語版
|
||
|
||
[115]Privacy policy
|
||
[116]Cookie & privacy preferences
|
||
[117]Notice at collection
|
||
|
||
Designed and developed by Evil Martians
|
||
|
||
|
||
References:
|
||
|
||
[1] https://evilmartians.com/
|
||
[2] https://cal.com/team/evilmartians/exploration
|
||
[3] https://evilmartians.com/services
|
||
[4] https://evilmartians.com/clients
|
||
[5] https://evilmartians.com/products
|
||
[6] https://evilmartians.com/opensource
|
||
[7] https://evilmartians.com/chronicles
|
||
[8] https://evilmartians.com/events
|
||
[9] https://evilmartians.com/devpropulsionlabs
|
||
[10] https://x.com/evilmartians
|
||
[11] https://www.linkedin.com/company/evil-martians
|
||
[12] https://github.com/evilmartians
|
||
[13] https://www.youtube.com/@evil.martians
|
||
[14] https://evilmartians.com/
|
||
[15] https://cal.com/team/evilmartians/exploration
|
||
[16] https://evilmartians.com/services
|
||
[17] https://evilmartians.com/clients
|
||
[18] https://evilmartians.com/products
|
||
[19] https://evilmartians.com/opensource
|
||
[20] https://evilmartians.com/chronicles
|
||
[21] https://evilmartians.com/events
|
||
[22] https://evilmartians.com/devpropulsionlabs
|
||
[23] https://x.com/evilmartians
|
||
[24] https://www.linkedin.com/company/evil-martians
|
||
[25] https://github.com/evilmartians
|
||
[26] https://www.youtube.com/@evil.martians
|
||
[27] https://evilmartians.com/events/evolution-of-real-time-and-anycable-rocky-mountain
|
||
[28] https://cal.com/team/evilmartians/exploration
|
||
[29] https://x.com/evilmartians
|
||
[30] https://www.linkedin.com/company/evil-martians
|
||
[31] https://github.com/evilmartians
|
||
[32] https://www.youtube.com/@evil.martians
|
||
[33] https://evilmartians.com/chronicles?categories=backend
|
||
[34] https://evilmartians.com/chronicles?services=software-development
|
||
[35] https://evilmartians.com/chronicles?services=audit-and-optimization
|
||
[36] https://evilmartians.com/chronicles?skills=rubyonrails
|
||
[37] https://evilmartians.com/chronicles?skills=ruby
|
||
[38] https://evilmartians.com/chronicles?skills=docker
|
||
[39] https://evilmartians.com/chronicles?skills=postgresql
|
||
[40] https://evilmartians.com/chronicles?skills=nodejs
|
||
[41] https://techracho.bpsinc.jp/hachi8833/2022_04_07/116843
|
||
[42] https://xfyuan.github.io/2020/07/dockeerizing-rails-development/
|
||
[43] https://evilmartians.com/chronicles/ruby-on-whales-docker-for-ruby-rails-development#changelog
|
||
[44] https://noti.st/palkan/vhsbxO/terraforming-legacy-rails-applications
|
||
[45] https://github.com/evilmartians/ruby-on-whales
|
||
[46] https://evilmartians.com/chronicles/ruby-on-whales-docker-for-ruby-rails-development#basic-docker-configuration
|
||
[47] https://evilmartians.com/chronicles/ruby-on-whales-docker-for-ruby-rails-development#introducing-dip
|
||
[48] https://evilmartians.com/chronicles/ruby-on-whales-docker-for-ruby-rails-development#services-vs-docker-for-development
|
||
[49] https://evilmartians.com/chronicles/ruby-on-whales-docker-for-ruby-rails-development#from-development-to-production
|
||
[50] https://evilmartians.com/chronicles/ruby-on-whales-docker-for-ruby-rails-development#introducing-the-ruby-on-whales-interactive-generator
|
||
[51] https://github.com/evilmartians/ruby-on-whales/blob/main/example/.dockerdev/Dockerfile
|
||
[52] https://github.com/moby/moby/issues/34129
|
||
[53] https://www.postgresql.org/download/linux/debian/
|
||
[54] https://docs.docker.com/compose/
|
||
[55] https://github.com/nodesource/distributions/blob/master/README.md#debinstall
|
||
[56] https://world.hey.com/dhh/modern-web-apps-without-javascript-bundling-or-transpiling-a20f2755
|
||
[57] https://github.com/WICG/import-maps
|
||
[58] https://github.com/rails/tailwindcss-rails
|
||
[59] https://github.com/heroku/heroku-buildpack-apt
|
||
[60] https://github.com/evilmartians/terraforming-rails/blob/master/examples/dockerdev/.dockerdev/Aptfile
|
||
[61] https://www.tug.org/texlive/
|
||
[62] https://askubuntu.com/a/972528
|
||
[63] http://xubuntugeek.blogspot.com/2012/06/save-disk-space-with-apt-get-option-no.html
|
||
[64] https://docs.docker.com/storage/storagedriver/#images-and-layers
|
||
[65] https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/reference.md#run---mount
|
||
[66] https://github.com/docker-library/ruby/issues/129#issue-229195231
|
||
[67] https://github.com/evilmartians/terraforming-rails/pull/24
|
||
[68] https://github.com/rubygems/rubygems/pull/4076
|
||
[69] https://github.com/evilmartians/ruby-on-whales/blob/main/example/.dockerdev/compose.yml
|
||
[70] https://docs.docker.com/compose/
|
||
[71] https://github.com/compose-spec/compose-spec/blob/master/spec.md#extension
|
||
[72] https://evilmartians.com/chronicles/ruby-on-whales-docker-for-ruby-rails-development#introducing-dip
|
||
[73] https://docs.docker.com/compose/compose-file/#context
|
||
[74] https://docs.docker.com/v17.09/engine/admin/volumes/tmpfs/#choosing-the-tmpfs-or-mount-flag
|
||
[75] https://www.docker.com/blog/speed-boost-achievement-unlocked-on-docker-desktop-4-6-for-mac/
|
||
[76] https://twitter.com/palkan_tula/status/1504499523216945167
|
||
[77] https://github.com/docker/for-mac/issues/5402
|
||
[78] https://github.com/infosiftr/ruby/blob/9b1f77c11d663930f4175c683b1c5f268d4d8191/Dockerfile.template#L47
|
||
[79] https://github.com/evilmartians/ruby-on-whales/blob/main/example/.dockerdev/.psqlrc
|
||
[80] https://evilmartians.com/chronicles/cables-vs-malloc_trim-or-yet-another-ruby-memory-usage-benchmark
|
||
[81] https://www.github.com/Shopify/bootsnap
|
||
[82] https://docs.docker.com/compose/compose-file/compose-file-v3/#healthcheck
|
||
[83] https://evilmartians.com/opensource/dip
|
||
[84] https://evilmartians.com/chronicles/reusable-development-containers-with-docker-compose-and-dip
|
||
[85] https://evilmartians.com/opensource/dip
|
||
[86] https://github.com/evilmartians/ruby-on-whales/blob/main/example/dip.yml
|
||
[87] https://evilmartians.com/chronicles/anyway-config-keep-your-ruby-configuration-sane
|
||
[88] https://makefile.site/
|
||
[89] https://docs.docker.com/compose/networking/#use-a-pre-existing-network
|
||
[90] https://docs.docker.com/develop/develop-images/multistage-build/
|
||
[91] https://github.com/evilmartians/ruby-on-whales
|
||
[92] https://railsbytes.com/public/templates/z5OsoB
|
||
[93] https://github.com/sponomarev
|
||
[94] https://github.com/bibendi
|
||
[95] https://github.com/fargelus/
|
||
[96] https://github.com/oliverklee
|
||
[97] https://www.braingourmets.com/
|
||
[98] https://github.com/evilmartians/terraforming-rails/pull/35
|
||
[99] https://github.com/evilmartians/terraforming-rails/pull/6
|
||
[100] https://github.com/evilmartians/terraforming-rails/pull/28
|
||
[101] https://github.com/evilmartians/terraforming-rails/pull/31
|
||
[102] https://github.com/evilmartians/terraforming-rails/issues/33
|
||
[103] https://github.com/evilmartians/terraforming-rails/issues/26
|
||
[107] https://evilmartians.com/chronicles.atom
|
||
[108] mailto:chronicles@evilmartians.com
|
||
[109] tel:+18884005485
|
||
[110] tel:+351308808570
|
||
[111] tel:+81662251242
|
||
[112] https://evilmartians.com/contact-us
|
||
[113] https://wellfound.com/company/evilmartians
|
||
[114] https://evilmartians.jp/
|
||
[115] https://evilmartians.com/privacy
|
||
[116] https://evilmartians.com/cookies
|
||
[117] https://evilmartians.com/privacy#notice_at_collection
|