From aa49cb1cfe640bfb75c5c411e509ab1e61708198 Mon Sep 17 00:00:00 2001 From: Joe Doss Date: Thu, 28 May 2020 12:11:51 -0500 Subject: [PATCH] Bugfix for #7663 and refactor of container setup (#7747) * Move from Alpine Linux to Fedora Linux. This changes the base container image away from Alpine Linux to Fedora Linux to address some musl libc issues with some of our gems. It also moved the main file to a Containerfile and symlinks the Dockerfile to it. This makes our container setup less Docker-centric as Linux users most likely will be using Podman as their container runtime. Lastly, it moves the WORKDIR from /usr/src/app to /opt/apps/devto. The Linux FHS states that /opt is the spot for Optional application software packages and /usr/src is for kernel source code. https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard /opt Optional application software packages /usr/src Source code, e.g., the kernel source code with its header files. Also, if SELinux becomes a thing in our future, moving this makes it easier to manage contexts out of /opt rather than /usr/src. * Adjust the Containerfile a bit to add some missing packages, make it more generic and move env vars to the docker-compose. * Move the Dockerfile to a symlink to the Containerfile. We need to be able to support more than just Docker for a container runtime. Moving everything to a Containerfile and symlinking the Dockerfile helps users run runtimes such as Podman. * Add in new entrypoint files and fix the app path in docker-entrypoint.sh * Refactor the docker-compose.yml file so it doesn't build the main application container three times in a row. We can use the same container for web, webpacker, and sidekiq. Also make the volume names very explicit on what their contents and add in SELinux context support with :Z * Fix ELASTICSEARCH_URL. * Remove the absolute path on ip binary and add in iproute to Containerfile. * Symlink the Dockerfile to Containerfile and add in a container-compose.yml file for usage with podman-compose. We are waiting for this issue to get resolved https://github.com/containers/libpod/issues/6153 and for podman-compose to mature a bit more. A user using podman-compose can use podman-compose -f container-compose.yml to launch the DEV container stack with Podman. * Update the .gitignore file to reflect the new container volumes. * Fix the entrypoint script on the Containerfile and clean up a bunch of things. * Rework the Containerfile to prep it to run as a non-root user. We have to wait for this issue to get fixed: https://github.com/containers/libpod/issues/6153 for Linux users and then we can uncomment the USER line so we are running Rails as a non-root user. :toot: * This reworks the compose files so each task that is needed to start the app has a correct wait command with dockerize. It also adds in containers for doing yarn and bundle things for development. Since we mount the code directory inside the container we lose our pre-containerized gems and node_modules. This means users can run: Linux podman-compose -f container-compose.yml up Mac docker-compose up and it should correctly build the app stack with containers! * Clean up old container related things. * Add in some container pre-reqs and my name to the "Core team" section! :toot: * Adjust the seed and sidekiq compose entries so they do not use the web entrypoint and set the REDIS_URL and REDIS_SESSIONS_URL env vars for seed. Add in some echos to the entrypoint.sh. Also set docker-compose to use :delegated on mount points to speed things up. https://docs.docker.com/storage/bind-mounts/#configure-mount-consistency-for-macos * Just call yarn install --dev on the yarn container. * Update documentation to support Docker and Podman as Container Engines and move the page from docker to containers to reflect the fact not all users run their containers via Docker. There are dozens of us... DOZENS! * Set --local on bundle config so it just impacts this Ruby app. * Renamed bin/docker-setup to bin/container-setup to reflect the fact that that we can use Podman as a container engine. I also refactored it so it does some basic pre-flight checks for docker and docker-compose or podman and podman-compose to make sure users have a container engine installed. * Set cache_all_platforms true for bundler. * Bump docker-compose dockerize -timeout to 45min for slower macOS hardware. * Move to the consolidated app_initializer:setup rake task for bootstrapping the app. * Update docs/installation/readme.md --- .gems/.gitignore | 4 + .gitignore | 6 +- Containerfile | 52 ++++ Dockerfile | 82 +----- README.md | 15 ++ bin/container-setup | 106 ++++++++ bin/docker-setup | 26 -- bin/setup | 3 +- config/initializers/docker.rb | 2 +- container-compose.yml | 151 +++++++++++ docker-compose.yml | 121 +++++++-- docker-entrypoint.sh | 84 ------- docker-run.sh | 429 -------------------------------- dockerhub-readme.md | 18 -- docs/installation/containers.md | 117 +++++++++ docs/installation/docker.md | 98 -------- docs/installation/readme.md | 12 +- scripts/bundle.sh | 10 + scripts/entrypoint.sh | 12 + 19 files changed, 586 insertions(+), 762 deletions(-) create mode 100644 .gems/.gitignore create mode 100644 Containerfile mode change 100644 => 120000 Dockerfile create mode 100755 bin/container-setup delete mode 100755 bin/docker-setup create mode 100644 container-compose.yml delete mode 100755 docker-entrypoint.sh delete mode 100755 docker-run.sh delete mode 100644 dockerhub-readme.md create mode 100644 docs/installation/containers.md delete mode 100644 docs/installation/docker.md create mode 100755 scripts/bundle.sh create mode 100755 scripts/entrypoint.sh diff --git a/.gems/.gitignore b/.gems/.gitignore new file mode 100644 index 000000000..5e7d2734c --- /dev/null +++ b/.gems/.gitignore @@ -0,0 +1,4 @@ +# Ignore everything in this directory +* +# Except this file +!.gitignore diff --git a/.gitignore b/.gitignore index 27cd23b01..c27da3df9 100644 --- a/.gitignore +++ b/.gitignore @@ -75,5 +75,7 @@ docs/.static/api/ # asdf .tool-versions -# ES docker volume -docker_data/ +# Container volumes +db_data +es_data +gem_cache diff --git a/Containerfile b/Containerfile new file mode 100644 index 000000000..326b8820e --- /dev/null +++ b/Containerfile @@ -0,0 +1,52 @@ +FROM quay.io/devto/ruby:2.7.1 + +USER root + +RUN curl -sL https://dl.yarnpkg.com/rpm/yarn.repo -o /etc/yum.repos.d/yarn.repo && \ + dnf install -y bash git ImageMagick iproute less libcurl libcurl-devel \ + libffi-devel libxml2-devel libxslt-devel nodejs pcre-devel \ + postgresql postgresql-devel ruby-devel tzdata yarn + +ENV APP_USER=devto +ENV APP_UID=1000 +ENV APP_GID=1000 +ENV APP_HOME=/opt/apps/devto/ +RUN mkdir -p ${APP_HOME} && chown "${APP_UID}":"${APP_GID}" "${APP_HOME}" +RUN groupadd -g "${APP_GID}" "${APP_USER}" && \ + adduser -u "${APP_UID}" -g "${APP_GID}" -d "${APP_HOME}" "${APP_USER}" + +ENV BUNDLER_VERSION=2.1.4 +RUN gem install bundler:"${BUNDLER_VERSION}" +ENV GEM_HOME=/opt/apps/bundle/ +ENV BUNDLE_SILENCE_ROOT_WARNING=1 BUNDLE_APP_CONFIG="${GEM_HOME}" +ENV PATH "${GEM_HOME}"/bin:$PATH +RUN mkdir -p "${GEM_HOME}" && chown "${APP_UID}":"${APP_GID}" "${GEM_HOME}" + +ENV DOCKERIZE_VERSION=v0.6.1 +RUN wget https://github.com/jwilder/dockerize/releases/download/"${DOCKERIZE_VERSION}"/dockerize-linux-amd64-"${DOCKERIZE_VERSION}".tar.gz \ + && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-"${DOCKERIZE_VERSION}".tar.gz \ + && rm dockerize-linux-amd64-"${DOCKERIZE_VERSION}".tar.gz + +WORKDIR "${APP_HOME}" + +# As soon as this: +# https://github.com/containers/libpod/issues/6153 +# issue is fixed for Linux users we can uncomment the line below so we are not +# running the app as the root user. :toot: +# USER "${APP_USER}" + +COPY ./.ruby-version "${APP_HOME}" +COPY ./Gemfile ./Gemfile.lock "${APP_HOME}" +RUN bundle check || bundle install --jobs 20 --retry 5 + +COPY ./package.json ./yarn.lock ./.yarnrc "${APP_HOME}" +COPY ./.yarn "${APP_HOME}"/.yarn +RUN yarn install + +RUN mkdir -p "${APP_HOME}"/public/{uploads,images,podcasts} + +COPY . "${APP_HOME}" + +ENTRYPOINT ["./scripts/entrypoint.sh"] + +CMD ["bundle", "exec", "rails","server","-b","0.0.0.0","-p","3000"] diff --git a/Dockerfile b/Dockerfile deleted file mode 100644 index 3958307ff..000000000 --- a/Dockerfile +++ /dev/null @@ -1,81 +0,0 @@ -FROM ruby:2.7.1-alpine3.10 - -#------------------------------------------------------------------------------ -# -# Install Project dependencies -# -#------------------------------------------------------------------------------ -RUN apk update -qq && apk add git nodejs postgresql-client ruby-dev build-base \ - less libxml2-dev libxslt-dev pcre-dev libffi-dev postgresql-dev tzdata imagemagick \ - libcurl curl-dev yarn - -#------------------------------------------------------------------------------ -# -# Install required bundler version -# -#------------------------------------------------------------------------------ -RUN gem install bundler:2.1.4 - -#------------------------------------------------------------------------------ -# -# Define working directory -# -#------------------------------------------------------------------------------ -WORKDIR /usr/src/app - -#------------------------------------------------------------------------------ -# -# Copy Gemfile and run bundle install -# -#------------------------------------------------------------------------------ -COPY ./.ruby-version . -COPY ./Gemfile ./Gemfile.lock ./ -RUN bundle install --jobs 20 --retry 5 - -#------------------------------------------------------------------------------ -# -# Copy Package.json and yarn.lock -# -#------------------------------------------------------------------------------ -COPY ./package.json ./yarn.lock ./.yarnrc ./ -COPY ./.yarn ./.yarn - -#------------------------------------------------------------------------------ -# -# Install packages -# -#------------------------------------------------------------------------------ -RUN yarn install - -# timeout extension required to ensure -# system work properly on first time load -ENV RACK_TIMEOUT_WAIT_TIMEOUT=10000 \ - RACK_TIMEOUT_SERVICE_TIMEOUT=10000 \ - STATEMENT_TIMEOUT=10000 - -# Run mode configuration between dev / demo -# for entrypoint script behaviour -ENV RUN_MODE="demo" - -# Database URL configuration - with user/pass -ENV DATABASE_URL="postgresql://devto:devto@db:5432/PracticalDeveloper_development" - -# DB setup / migrate script triggers on boot -ENV DB_SETUP="false" \ - DB_MIGRATE="false" - -# In order for redis to work, these should be set -ENV REDIS_URL="redis://redis:6379" \ - REDIS_SESSIONS_URL="redis://redis:6379" - -# -# Let's setup the public uploads folder volume -# -RUN mkdir -p /usr/src/app/public/uploads -VOLUME /usr/src/app/public/uploads - -# Entrypoint and command to start the server -COPY docker-entrypoint.sh /usr/bin/ -RUN chmod +x /usr/bin/docker-entrypoint.sh - -COPY . . diff --git a/Dockerfile b/Dockerfile new file mode 120000 index 000000000..5240dc01e --- /dev/null +++ b/Dockerfile @@ -0,0 +1 @@ +Containerfile \ No newline at end of file diff --git a/README.md b/README.md index 93b492b1f..af24c8a3f 100644 --- a/README.md +++ b/README.md @@ -89,6 +89,8 @@ A more complete overview of our stack is available in ### Prerequisites +#### Local + - [Ruby](https://www.ruby-lang.org/en/): we recommend using [rbenv](https://github.com/rbenv/rbenv) to install the Ruby version listed on the badge. @@ -100,6 +102,18 @@ A more complete overview of our stack is available in - [Redis](https://redis.io/) 4 or higher. - [Elasticsearch](https://www.elastic.co) 7 or higher. +#### Containers + +**Linux** + +- [Podman](https://github.com/containers/libpod) 1.9.2 or higher +- [Podman Compose](https://github.com/containers/podman-compose) 0.1.5 or higher + +**OS X** + +- [Docker Desktop for Mac](https://docs.docker.com/docker-for-mac/install/) + + ### Installation Documentation [View Full Installation Documentation](https://docs.dev.to/installation/). @@ -128,6 +142,7 @@ A more complete overview of our stack is available in - [@ridhwana](https://dev.to/ridhwana) - [@fdoxyz](https://dev.to/fdoxyz) - [@msarit](https://dev.to/msarit) +- [@jdoss](https://dev.to/jdoss) ## Vulnerability disclosure diff --git a/bin/container-setup b/bin/container-setup new file mode 100755 index 000000000..5c47fd1d9 --- /dev/null +++ b/bin/container-setup @@ -0,0 +1,106 @@ +#!/usr/bin/env bash + +set -e + +if [[ "$OSTYPE" == "darwin"* ]]; then + + if hash docker 2>/dev/null; then + CONTAINER_RUNTIME=docker + else + + echo "Error! We could not find docker on your Mac!" + echo "Please install Docker for Mac:" + echo "https://docs.docker.com/docker-for-mac/install/" + + exit 1 + + fi + + if hash docker-compose 2>/dev/null; then + CONTAINER_COMPOSE=docker-compose + else + + echo "Error! We could not find a docker-compose on your Mac!" + echo "Please install Docker for Mac:" + echo "https://docs.docker.com/docker-for-mac/install/" + + exit 1 + + fi + +echo "Building the DEV container image!" + +if [ "${CONTAINER_RUNTIME}" = "docker" ]; then + echo "This will take a while if you are building the container for the first time." + + docker-compose build + +elif [ "${CONTAINER_RUNTIME}" = "podman" ]; then + + podman-compose build + +fi + +elif [[ "$OSTYPE" == "linux-gnu"* ]]; then + + if hash podman 2>/dev/null; then + CONTAINER_RUNTIME=podman + elif hash docker 2>/dev/null; then + CONTAINER_RUNTIME=docker + else + + echo "Error! We could not find a container runtime on your Linux install!" + echo "Please install Podman v1.9.3 or greater:" + echo "https://podman.io/getting-started/installation.html" + + exit 1 + + fi + + if hash podman-compose 2>/dev/null; then + CONTAINER_COMPOSE=podman-compose + elif hash docker 2>/dev/null; then + CONTAINER_COMPOSE=docker-compose + else + + echo "Error! We could not find ${CONTAINER_COMPOSE} on your Linux install!" + echo "Please install Podman Compose:" + echo "https://github.com/containers/podman-compose#installation" + + exit 1 + + fi + +else + + echo "Oof! Sorry! This OS is unsupported! :(" + +fi + +echo "Starting the DEV container stack..." + +if [ "${CONTAINER_RUNTIME}" = "docker" ]; then + + if [[ ! -d .gems/ || ! "$(ls -A .gems)" ]]; then + echo "The .gems directory is empty or does not exist." + echo "This will cause bundle install to run which takes a long time with ${CONTAINER_RUNTIME}." + echo "Depending on your hardware specs, the initial setup can take" + echo "30 to 45 minutes. Please stand by..." + sleep 10 + fi + + docker-compose up + +elif [ "${CONTAINER_RUNTIME}" = "podman" ]; then + + if podman pod exists devto; then + echo "The devto pod is already setup. Please run:" + echo "" + echo "podman-compose -p devto -f container-compose.yml down" + echo "" + echo "to bring down the DEV container stack before running container-setup." + else + podman-compose -p devto -f container-compose.yml up + fi + +fi diff --git a/bin/docker-setup b/bin/docker-setup deleted file mode 100755 index 116d5cf92..000000000 --- a/bin/docker-setup +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env sh - -set -e - -echo "== Building Docker image (this may take a while) ==" -docker-compose build - -echo "== Ensuring Elasticsearch data directory is owned by the correct uid ==" -docker-compose run elasticsearch chown elasticsearch:elasticsearch /usr/share/elasticsearch/data - -echo "== Setting up database ==" -on_app_setup_failed() { - echo "Application setup failed with exit code $1." >&2 - echo "" >&2 - echo "Check the Docker logs using:" >&2 - echo " docker-compose logs" >&2 - echo "" >&2 - echo "Once you fixed the error, try running the script again." >&2 - echo "To also delete all data, run this before retrying:" >&2 - echo " docker-compose down -v && sudo rm -rf docker_data" >&2 - exit $1 -} -docker-compose run web sh -c 'yarn global add wait-on && wait-on -t 30000 http-get://elasticsearch:9200/ && rails app_initializer:setup' || on_app_setup_failed $? - -echo "== Starting app ==" -docker-compose up diff --git a/bin/setup b/bin/setup index 6c2fad1f6..6b8f47400 100755 --- a/bin/setup +++ b/bin/setup @@ -15,7 +15,8 @@ FileUtils.chdir APP_ROOT do puts "== Installing dependencies ==" system! "gem install bundler --conservative" - system! "bundle config set cache_all true" + system! "bundle config --local cache_all true" + system! "bundle config --local cache_all_platforms true" system("bundle check") || system!("bundle install") system! "gem install foreman" diff --git a/config/initializers/docker.rb b/config/initializers/docker.rb index 44f672d64..dbafa39bf 100644 --- a/config/initializers/docker.rb +++ b/config/initializers/docker.rb @@ -1,7 +1,7 @@ # Docker specific development configuration if Rails.env.development? && File.file?("/.dockerenv") # Using shell tools so we don't need to require Socket and IPAddr - host_ip = `/sbin/ip route|awk '/default/ { print $3 }'`.strip + host_ip = `ip route|awk '/default/ { print $3 }'`.strip logger = Logger.new(STDOUT) logger.info "Allowing #{host_ip} for BetterErrors and Web Console" diff --git a/container-compose.yml b/container-compose.yml new file mode 100644 index 000000000..96e0919da --- /dev/null +++ b/container-compose.yml @@ -0,0 +1,151 @@ +version: "3" +services: + web: + build: . + image: devto-web:latest + container_name: dev_web + ports: + - "3000:3000" + depends_on: + - bundle + - db + - elasticsearch + - redis + - yarn + environment: + RAILS_ENV: development + DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development + ELASTICSEARCH_URL: http://elasticsearch:9200 + REDIS_URL: redis://redis:6379 + REDIS_SESSIONS_URL: redis://redis:6379 + RACK_TIMEOUT_WAIT_TIMEOUT: 10000 + RACK_TIMEOUT_SERVICE_TIMEOUT: 10000 + STATEMENT_TIMEOUT: 10000 + volumes: + - .:/opt/apps/devto:z + - ./.gems:/opt/apps/bundle:z + entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "http://elasticsearch:9200", "-wait", "tcp://redis:6379", "-wait", "file:///opt/apps/bundle/bundle_finished", "-timeout", "300s", "./scripts/entrypoint.sh"] + command: ["bundle", "exec", "rails","server","-b","0.0.0.0","-p","3000"] + + bundle: + image: devto-web:latest + container_name: dev_bundle + environment: + RAILS_ENV: development + REDIS_SIDEKIQ_URL: redis://redis:6379 + DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development + ELASTICSEARCH_URL: http://elasticsearch:9200 + volumes: + - .:/opt/apps/devto:z + - ./.gems:/opt/apps/bundle:z + entrypoint: ["bash"] + command: ["./scripts/bundle.sh"] + + yarn: + image: devto-web:latest + container_name: dev_yarn + environment: + RAILS_ENV: development + REDIS_SIDEKIQ_URL: redis://redis:6379 + DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development + ELASTICSEARCH_URL: http://elasticsearch:9200 + volumes: + - .:/opt/apps/devto:z + entrypoint: ["bash", "-c"] + command: ["yarn install --dev"] + + webpacker: + image: devto-web:latest + container_name: dev_webpacker + depends_on: + - web + - yarn + environment: + RAILS_ENV: development + REDIS_SIDEKIQ_URL: redis://redis:6379 + DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development + ELASTICSEARCH_URL: http://elasticsearch:9200 + volumes: + - .:/opt/apps/devto:z + - ./.gems:/opt/apps/bundle:z + entrypoint: ["dockerize", "-wait", "file:///opt/apps/devto/node_modules/.bin/webpack-dev-server", "-timeout", "300s"] + command: ["./bin/webpack-dev-server"] + + seed: + image: devto-web:latest + container_name: dev_seed + depends_on: + - web + - redis + - db + - elasticsearch + environment: + RAILS_ENV: development + REDIS_SIDEKIQ_URL: redis://redis:6379 + REDIS_URL: redis://redis:6379 + REDIS_SESSIONS_URL: redis://redis:6379 + DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development + ELASTICSEARCH_URL: http://elasticsearch:9200 + volumes: + - .:/opt/apps/devto:z + - ./.gems:/opt/apps/bundle:z + entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "http://elasticsearch:9200", "-wait", "tcp://redis:6379", "-wait", "http://web:3000", "-timeout", "300s"] + command: ["bundle", "exec", "rake","db:seed"] + + sidekiq: + image: devto-web:latest + container_name: dev_sidekiq + depends_on: + - web + - redis + - db + - elasticsearch + environment: + RAILS_ENV: development + REDIS_SIDEKIQ_URL: redis://redis:6379 + DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development + ELASTICSEARCH_URL: http://elasticsearch:9200 + volumes: + - .:/opt/apps/devto:z + - ./.gems:/opt/apps/bundle:z + entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "http://elasticsearch:9200", "-wait", "tcp://redis:6379", "-wait", "http://web:3000", "-timeout", "300s"] + command: ["bundle", "exec", "sidekiq","-c","2"] + + db: + image: postgres:9.6.15-alpine + container_name: dev_postgresql + environment: + POSTGRES_USER: devto + POSTGRES_PASSWORD: devto + POSTGRES_DB: PracticalDeveloper_development + ports: + - "5432:5432" + volumes: + - db_data:/var/lib/postgresql/data:Z + + redis: + image: "redis" + container_name: dev_redis + ports: + - "6379:6379" + + elasticsearch: + image: docker.elastic.co/elasticsearch/elasticsearch:7.5.2 + container_name: dev_elasticsearch + environment: + - cluster.name=devto + - bootstrap.memory_lock=true + - "ES_JAVA_OPTS=-Xms512m -Xmx512m" + - "discovery.type=single-node" + - xpack.security.enabled=false + - xpack.monitoring.enabled=false + - xpack.graph.enabled=false + - xpack.watcher.enabled=false + ports: + - "9200:9200" + # volumes: + # - es_data:/usr/share/elasticsearch/data:Z + +volumes: + db_data: + # es_data: diff --git a/docker-compose.yml b/docker-compose.yml index 7d5d7de78..1bba69a11 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -2,29 +2,101 @@ version: "3" services: web: build: . + image: devto-web:latest + container_name: dev_web ports: - "3000:3000" depends_on: + - bundle - db - - redis - elasticsearch + - redis + - yarn environment: RAILS_ENV: development DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development ELASTICSEARCH_URL: http://elasticsearch:9200 REDIS_URL: redis://redis:6379 REDIS_SESSIONS_URL: redis://redis:6379 + RACK_TIMEOUT_WAIT_TIMEOUT: 10000 + RACK_TIMEOUT_SERVICE_TIMEOUT: 10000 + STATEMENT_TIMEOUT: 10000 volumes: - - .:/usr/src/app - - /usr/src/app/node_modules - command: bundle exec rails server -b 0.0.0.0 -p 3000 + - .:/opt/apps/devto:delegated + - ./.gems:/opt/apps/bundle:delegated + entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "http://elasticsearch:9200", "-wait", "tcp://redis:6379", "-wait", "file:///opt/apps/bundle/bundle_finished", "-timeout", "2700s", "./scripts/entrypoint.sh"] + command: ["bundle", "exec", "rails","server","-b","0.0.0.0","-p","3000"] + + bundle: + image: devto-web:latest + container_name: dev_bundle + environment: + RAILS_ENV: development + REDIS_SIDEKIQ_URL: redis://redis:6379 + DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development + ELASTICSEARCH_URL: http://elasticsearch:9200 + volumes: + - .:/opt/apps/devto:delegated + - ./.gems:/opt/apps/bundle:delegated + entrypoint: ["bash"] + command: ["./scripts/bundle.sh"] + + yarn: + image: devto-web:latest + container_name: dev_yarn + environment: + RAILS_ENV: development + REDIS_SIDEKIQ_URL: redis://redis:6379 + DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development + ELASTICSEARCH_URL: http://elasticsearch:9200 + volumes: + - .:/opt/apps/devto:delegated + entrypoint: ["bash", "-c"] + command: ["yarn install --dev"] + webpacker: - build: . - command: ./bin/webpack-dev-server - sidekiq: - build: . - command: bundle exec sidekiq -c 2 + image: devto-web:latest + container_name: dev_webpacker depends_on: + - web + - yarn + environment: + RAILS_ENV: development + REDIS_SIDEKIQ_URL: redis://redis:6379 + DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development + ELASTICSEARCH_URL: http://elasticsearch:9200 + volumes: + - .:/opt/apps/devto:delegated + - ./.gems:/opt/apps/bundle:delegated + entrypoint: ["dockerize", "-wait", "file:///opt/apps/devto/node_modules/.bin/webpack-dev-server", "-timeout", "2700s"] + command: ["./bin/webpack-dev-server"] + + seed: + image: devto-web:latest + container_name: dev_seed + depends_on: + - web + - redis + - db + - elasticsearch + environment: + RAILS_ENV: development + REDIS_SIDEKIQ_URL: redis://redis:6379 + REDIS_URL: redis://redis:6379 + REDIS_SESSIONS_URL: redis://redis:6379 + DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development + ELASTICSEARCH_URL: http://elasticsearch:9200 + volumes: + - .:/opt/apps/devto:delegated + - ./.gems:/opt/apps/bundle:delegated + entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "http://elasticsearch:9200", "-wait", "tcp://redis:6379", "-wait", "http://web:3000", "-timeout", "2700s"] + command: ["bundle", "exec", "rake","db:seed"] + + sidekiq: + image: devto-web:latest + container_name: dev_sidekiq + depends_on: + - web - redis - db - elasticsearch @@ -33,23 +105,35 @@ services: REDIS_SIDEKIQ_URL: redis://redis:6379 DATABASE_URL: postgresql://devto:devto@db:5432/PracticalDeveloper_development ELASTICSEARCH_URL: http://elasticsearch:9200 + volumes: + - .:/opt/apps/devto:delegated + - ./.gems:/opt/apps/bundle:delegated + entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "http://elasticsearch:9200", "-wait", "tcp://redis:6379", "-wait", "http://web:3000", "-timeout", "2700s"] + command: ["bundle", "exec", "sidekiq","-c","2"] + db: - image: postgres:${POSTGRES_VERSION:-9.6.15-alpine} + image: postgres:9.6.15-alpine + container_name: dev_postgresql environment: POSTGRES_USER: devto POSTGRES_PASSWORD: devto POSTGRES_DB: PracticalDeveloper_development ports: - "5432:5432" + volumes: + - db_data:/var/lib/postgresql/data:delegated + redis: image: "redis" + container_name: dev_redis ports: - "6379:6379" + elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.5.2 - container_name: elasticsearch + container_name: dev_elasticsearch environment: - - cluster.name=docker-cluster + - cluster.name=devto - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - "discovery.type=single-node" @@ -57,11 +141,16 @@ services: - xpack.monitoring.enabled=false - xpack.graph.enabled=false - xpack.watcher.enabled=false + volumes: + - es_data:/usr/share/elasticsearch/data:delegated + ports: + - "9200:9200" ulimits: memlock: soft: -1 hard: -1 - volumes: - - ./docker_data/elasticsearch/data:/usr/share/elasticsearch/data - ports: - - "9200:9200" + +volumes: + db_data: + es_data: + diff --git a/docker-entrypoint.sh b/docker-entrypoint.sh deleted file mode 100755 index ba4806833..000000000 --- a/docker-entrypoint.sh +++ /dev/null @@ -1,84 +0,0 @@ -#!/bin/sh - -# -# Lets setup the alias file -# @TODO - add as scripts instead within /bin? - this will help auto fill? -# -echo "" > ~/.bashrc -echo "alias devto-setup='cd /usr/src/app/ && gem install bundler && bundle install --jobs 20 --retry 5 && yarn install && bin/setup'" >> ~/.bashrc -echo "alias devto-migrate='cd /usr/src/app/ && bin/rails db:migrate'" >> ~/.bashrc -echo "alias devto-start='cd /usr/src/app/ && bundle exec rails server -b 0.0.0.0 -p 3000'" >> ~/.bashrc - -# -# Lets ensure we are in the correct workspace -# -cd /usr/src/app/ - -# -# Lets handle "DEV" RUN_MODE -# -if [[ "$RUN_MODE" = "DEV" ]] -then - echo ">---" - echo "> [dev.to/docker-entrypoint.sh] DEV mode" - echo "> " - echo "> Welcome to the dev.to, DEVELOPMENT container, for convenience your repository" - echo "> should be mounted onto '/usr/src/app/', and port 3000 should be forwarded to your host machine" - echo "> " - echo "> In addition the following alias commands has been preconfigured to get you up and running quickly" - echo "> " - echo "> devto-setup : Does the gem/yarn dependency installation, along with database setup" - echo "> devto-migrate : Calls the database migration script" - echo "> devto-start : Start the rails application server on port 3000" - echo "> " - echo "> Finally to exit this container bash terminal (and stop the container), use the command 'exit'" - echo ">---" - - # Lets startup bash for the user to interact with - /bin/bash - exit $?; -fi - -# -# Lets handle "DEMO" RUN_MODE -# -echo ">---" -echo "> [dev.to/docker-entrypoint.sh] DEMO mode" -echo "> " -echo "> Time to rock & roll" -echo ">---" - -# -# DB setup -# note this will fail (intentionally), if DB was previously setup -# -if [[ "$DB_SETUP" == "true" ]] -then - echo ">---" - echo "> [dev.to/docker-entrypoint.sh] Performing DB_SETUP : you can skip this step by setting DB_SETUP=false" - echo ">---" - bin/setup -fi - -# -# DB migration script -# -if [[ "$DB_MIGRATE" == "true" ]] -then - echo ">---" - echo "> [dev.to/docker-entrypoint.sh] Performing DB_MIGRATE : you can skip this step by setting DB_MIGRATE=false" - echo ">---" - bin/rails db:migrate -fi - -# -# Execute rails server on port 3000 -# -if [[ "$APP_SERVER" == "true" ]] -then - echo ">---" - echo "> [dev.to/docker-entrypoint.sh] Starting the rails servers - whheee!" - echo ">---" - rm -f tmp/pids/server.pid - bundle exec rails server -b 0.0.0.0 -p 3000 -fi diff --git a/docker-run.sh b/docker-run.sh deleted file mode 100755 index 98b571f65..000000000 --- a/docker-run.sh +++ /dev/null @@ -1,429 +0,0 @@ -#!/bin/bash - -########################################### -# -# Interactive mode script handling -# -########################################### - -if [ "$1" = "INTERACTIVE-DEMO" ] -then - # echo "#---" - # echo "# Starting up INTERACTIVE-DEMO mode" - # echo "#---" - - # Configure RUN_MODE as DEMO - RUN_MODE="DEMO" - - echo "|---" - echo "|" - echo "| Welcome to DEV.TO interactive docker demo setup guide." - echo "|" - echo "| For logins to work, we will need either GITHUB or TWITTER API keys" - echo "|" - echo "| See ( https://docs.dev.to/getting-started/config-env/ ) " - echo "| for instructions on how to get the various API keys " - echo "|" - echo "| Once you got your various API keys, please proceed to the next step" - echo "|" - echo "|---" - - echo "|---" - echo "| Setting up GITHUB keys" - echo "| (OPTIONAL, leave blank and press enter to skip)" - echo "|---" - echo -n "| Please indicate your GITHUB_KEY : " - read INPUT_KEY - if [ ! -z "$INPUT_KEY" ] - then - export GITHUB_KEY="$INPUT_KEY" - fi - - echo -n "| Please indicate your GITHUB_SECRET : " - read INPUT_KEY - if [ ! -z "$INPUT_KEY" ] - then - export GITHUB_SECRET="$INPUT_KEY" - fi - - echo -n "| Please indicate your TWITTER_KEY : " - read INPUT_KEY - if [ ! -z "$INPUT_KEY" ] - then - export TWITTER_KEY="$INPUT_KEY" - fi - - echo -n "| Please indicate your TWITTER_SECRET : " - read INPUT_KEY - if [ ! -z "$INPUT_KEY" ] - then - export TWITTER_SECRET="$INPUT_KEY" - fi - -fi - -########################################### -# -# Script header guide -# -########################################### - -echo "#---" -echo "#" -echo "# This script will perform the following steps ... " -echo "#" -echo "# 1) Stop and remove any docker container with the name 'dev-to-postgres' and 'dev-to'" -echo "# 2) Reset any storage directories if RUN_MODE starts with 'RESET-'" -echo "# 3) Build the dev.to docker image, with the name of 'dev-to:dev' or 'dev-to:demo'" -echo "# 4) Deploy the postgres container, mounting '_docker-storage/postgres' with the name 'dev-to-postgres'" -echo "# 5) Deploy the dev-to container, with the name of 'dev-to-app', and sets up its port to 3000" -echo "#" -echo "# To run this script properly, execute with the following (inside the dev.to repository folder)..." -echo "# './docker-run.sh [RUN_MODE] [Additional docker environment arguments]'" -echo "#" -echo "# Alternatively to run this script in 'interactive mode' simply run" -echo "# './docker-run.sh INTERACTIVE-DEMO'" -echo "#" -echo "#---" -echo "#---" -echo "#" -echo "# RUN_MODE can either be the following" -echo "#" -echo "# - 'DEV' : Start up the container into bash, with a quick start guide" -echo "# - 'DEMO' : Start up the container, and run dev.to" -echo "# - 'RESET-DEV' : Resets postgresql and upload data directory for a clean deployment, before running as DEV mode" -echo "# - 'RESET-DEMO' : Resets postgresql and upload data directory for a clean deployment, before running as DEMO mode" -echo "# - 'INTERACTIVE-DEMO' : Runs this script in 'interactive' mode to setup the 'DEMO'" -echo "#" -echo "# So for example to run a development container in bash it's simply" -echo "# './docker-run.sh DEV'" -echo "#" -echo "# Finally to run a working demo, you will need to provide either..." -echo "# './docker-run.sh .... -e GITHUB_KEY= -e GITHUB_SECRET=" -echo "#" -echo "# And / Or ..." -echo "# './docker-run.sh .... -e TWITTER_KEY= -e TWITTER_SECRET=" -echo "#" -echo "# Note that all of this can also be configured via ENVIRONMENT variables prior to running the script" -echo "#" -echo "#---" - -########################################### -# -# Core script logic -# -########################################### - -# -# Arguments / Environment handling -# - -# Terminate without argument -if [ -z "$1" ] -then - # Invalid RUN_MODE - echo "#---" - echo "# [FATAL ERROR] Missing RUN_MODE argument (see example above)" - echo "#---" - exit 1 -fi - -# Initialize : docker-run.sh RUN_MODE -# also parses it from argument array -if [ -z "$RUN_MODE" ] -then - RUN_MODE="DEMO" -fi -# Argument processing -if [ "$1" = "DEMO" ] || [ "$1" = "DEV" ] || [ "$1" = "RESET-DEV" ] || [ "$1" = "RESET-DEMO" ] -then - # The mode is passed into the command line script, process it - RUN_MODE="$1" - # Process argument array without RUN_MODE - ARG_ARRAY_STR="${@:2}" -else - if [ "$1" = "INTERACTIVE-DEMO" ] - then - RUN_MODE="DEMO" - # Process argument array without RUN_MODE - ARG_ARRAY_STR="${@:2}" - else - # Process argument array with $0 - ARG_ARRAY_STR="${@:1}" - fi -fi -if [ "$RUN_MODE" = "DEMO" ] || [ "$RUN_MODE" = "DEV" ] || [ "$RUN_MODE" = "RESET-DEV" ] || [ "$RUN_MODE" = "RESET-DEMO" ] -then - # OK we validated run mode - RUN_MODE=$RUN_MODE -else - # Invalid RUN_MODE - echo "#---" - echo "# [FATAL ERROR] Invalid RUN_MODE : $RUN_MODE (see example above)" - echo "#---" - exit 2 -fi - -# Initialize : the currrent repository directory (and navigate to it) -if [ -z "$REPO_DIR" ] -then - REPO_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" -fi -cd "$REPO_DIR"; - -# Initialize : the general storage directory (used to magic bash the upload / postgres directory) -if [ -z "$STORAGE_DIR" ] -then - STORAGE_DIR="$REPO_DIR/_docker-storage" -fi - -# Initialize : Postgresql data directory -if [ -z "$POSTGRES_DIR" ] -then - POSTGRES_DIR="$STORAGE_DIR/postgresql-data/" -fi - -# Initialize : dev-to public upload data -if [ -z "$UPLOAD_DIR" ] -then - UPLOAD_DIR="$STORAGE_DIR/public-upload/" -fi - -echo "#---" -echo "# Ok, to start with - lets assume the following settings (provided or auto default)..." -echo "#" -echo "# RUN_MODE = $RUN_MODE" -echo "# REPO_DIR = $REPO_DIR" -echo "# STORAGE_DIR = $STORAGE_DIR" -echo "# UPLOAD_DIR = $UPLOAD_DIR" -echo "# POSTGRES_DIR = $POSTGRES_DIR" -echo "#" -echo "# PS : These settings can also be overwritten by ENVIRONMENT variables, extremely useful in a CI setup =)" -echo "#---" - -# -# ENV variables to support forwarding, and the compulsory list from bash script to docker (if detected) -# -ENV_FORWARDING_LIST=( - # login via GITHUB - "GITHUB_KEY" - "GITHUB_SECRET" - # login via TWITTER - "TWITTER_KEY" - "TWITTER_SECRET" - # PUSHER integration - "PUSHER_APP_ID" - "PUSHER_KEY" - "PUSHER_SECRET" - "PUSHER_CLUSTER" - # @TODO : anything else to pass forward? S3 -) - -# -# dev.to docker command flags to pass forward -# -DEVTO_DOCKER_FLAGS="$ARG_ARRAY_STR" - -# -# Scan for ENV variables to forward -# -echo "#---" -echo "# Lets scan for dev.to environment variables that will automatically be passed" -echo "# forward into the container if present (very useful for CI testing)" -echo "#---" -for i in "${ENV_FORWARDING_LIST[@]}" -do - if [[ $DEVTO_DOCKER_FLAGS == *"$i"* ]] - then - echo "[detected in arguments] - $i" - continue - fi - - if [ ! -z "$(printenv $i)" ] - then - echo "[detected env variable] - $i" - DEVTO_DOCKER_FLAGS="$DEVTO_DOCKER_FLAGS -e $i=$(printenv $i)" - continue - fi - - echo "[skipped env variable] - $i" -done - -# -# Stop and remove existing containers -# -EXISTING_POSTGRES=$(docker ps -a | grep "dev-to-postgres" | awk '{print $1}') -EXISTING_DEVTO=$(docker ps -a | grep "dev-to-app" | awk '{print $1}') -if [[ ! -z "$EXISTING_POSTGRES" ]] || [[ ! -z "$EXISTING_DEVTO" ]] -then - echo "#---" - echo "# Removing dev-to-postgres / dev-to-app containers" - echo "# Found the following : $EXISTING_POSTGRES $EXISTING_DEVTO" - echo "#---" - - # I dunno how docker does a race condition against the previous echo step - # this works around that - sleep 0.001 - - # Stopping and removing containers - docker stop $EXISTING_POSTGRES $EXISTING_DEVTO; - docker rm $EXISTING_POSTGRES $EXISTING_DEVTO; -fi - -# -# Reset the postgresql / upload folder -# -if [ "$RUN_MODE" = "RESET-DEV" ] || [ "$RUN_MODE" = "RESET-DEMO" ] -then - echo "#---" - echo "# Detected RESET based run mode : $RUN_MODE" - echo "#" - echo "# Will delete the following directory X_X " - echo "# - $UPLOAD_DIR" - echo "# - $POSTGRES_DIR" - echo "#" - echo "# And change RUN_MODE to : ${RUN_MODE:6}" - echo "#---" - - # Remove files - rm -R "$POSTGRES_DIR" - rm -R "$UPLOAD_DIR" - - # Update RUN_MODE - RUN_MODE="${RUN_MODE:6}" -fi - -# -# Build the dev-to container -# Exits on failure -# -echo "#---" -echo "# Building dev-to $RUN_MODE docker container (yay!) ... " -echo "# If this is your first time running this build step, go grab a coffee - it may take a long while" -echo "# Alternatively, some paper swords with chairs : https://xkcd.com/303/" -echo "#---" - -if [ "$RUN_MODE" = "DEV" ] -then - # Build the DEV mode container - docker build --target alpine-ruby-node -t dev-to:dev . || exit $? -else - # Build the DEMO mode container - docker build -t dev-to:demo . || exit $? -fi - -# -# Deploy postgresql container -# -echo "#---" -echo "# Deploying postgresql container" -echo "#" -echo "# POSTGRES_DIR : $POSTGRES_DIR" -echo "#---" -mkdir -p "$POSTGRES_DIR" -docker run -d --name dev-to-postgres -e POSTGRES_PASSWORD=devto -e POSTGRES_USER=devto -e POSTGRES_DB=PracticalDeveloper_development -v "$POSTGRES_DIR:/var/lib/postgresql/data" postgres:10.7-alpine - -# -# Wait for postgresql server -# this waits up to ~6*10 seconds -# -echo "#---" -echo "# Waiting for postgres server (this commonly takes 10 ~ 60 seconds) ... " -echo -n "# ." -RETRIES=12 -until docker exec dev-to-postgres psql -U devto -d PracticalDeveloper_development -c "select 1" > /dev/null 2>&1 || [ $RETRIES -eq 0 ]; do - echo -n "." - sleep 5 - RETRIES=$((RETRIES - 1)) -done -echo "" -echo "# Wait completed, moving on ... " -echo "#---" - -# -# Deploy dev-to container -# -echo "#---" -echo "# Deploying dev-to container" -echo "#" -echo "# RUN_MODE : $RUN_MODE" -echo "# REPO_DIR : $REPO_DIR" -echo "# UPLOAD_DIR : $UPLOAD_DIR" -echo "#---" -mkdir -p "$UPLOAD_DIR" - -# -# in DEV mode - lets run in interactive mode -# -if [ "$RUN_MODE" = "DEV" ] -then - docker run -it -p 3000:3000 \ - --name dev-to-app \ - --link dev-to-postgres:db \ - -v "$REPO_DIR:/usr/src/app" \ - -e RUN_MODE="DEV" \ - -e DATABASE_URL=postgresql://devto:devto@db:5432/PracticalDeveloper_development \ - --entrypoint "/usr/src/app/docker-entrypoint.sh" \ - $DEVTO_DOCKER_FLAGS \ - dev-to:dev - - # End of dev mode - exit 0; -fi - -# -# in DEMO mode - lets run it in the background -# -docker run -d -p 3000:3000 \ ---name dev-to-app \ ---link dev-to-postgres:db \ --v "$UPLOAD_DIR:/usr/src/app/public/uploads/" \ --e RUN_MODE="DEMO" \ --e DATABASE_URL=postgresql://devto:devto@db:5432/PracticalDeveloper_development \ -$DEVTO_DOCKER_FLAGS \ -dev-to:demo - -# -# Wait for dev.to server -# this waits up to ~20*30 seconds -# -echo "#---" -echo "# Waiting for dev.to server... " -echo "#" -echo "# this commonly takes 2 ~ 10 minutes, basically, a very long time .... =[ " - -# Side note, looped to give 4 set of distinct lines -# especially if long wait times occur (to make it more manageable) -for i in 1 2 3 4 -do - RETRIES=30 - echo -n "# ." - until docker exec dev-to-app curl -I --max-time 5 -f http://localhost:3000/ > /dev/null 2>&1 || [ $RETRIES -eq 0 ]; do - echo -n "." - sleep 5 - RETRIES=$((RETRIES - 1)) - done - echo "" -done - -echo "# Wait completed, moving on ... " -echo "#---" - -# -# Dumping out docker info -# -echo "#---" -echo "# Displaying relevant docker information" -echo "#---" -DOCKER_INFO=$(docker ps) -echo "$DOCKER_INFO" | head -1 -echo "$DOCKER_INFO" | grep dev-to- - -# -# Finishing message -# -echo "#---" -echo "# Container deployed on address ( http://localhost:3000 )" -echo "# " -echo "# Time to dev.to(gether)" -echo "#---" diff --git a/dockerhub-readme.md b/dockerhub-readme.md deleted file mode 100644 index 537b680b6..000000000 --- a/dockerhub-readme.md +++ /dev/null @@ -1,18 +0,0 @@ -# How do I get this up and running quickly - -Note: You will need replace all the various `` - -```bash -# Run a postgres server configured for dev.to -docker run -d --name dev-to-postgres \ - -e POSTGRES_PASSWORD=devto \ - -e POSTGRES_USER=devto \ - -e POSTGRES_DB=PracticalDeveloper_development \ - -v ":/var/lib/postgresql/data" \ - postgres:10.7-alpine; - -# Wait about 30 seconds, to give the postgres container time to start -sleep 30 - -> PS : Someone from official dev.to team should create their own container namespace and update this segment after merger (if any) -``` diff --git a/docs/installation/containers.md b/docs/installation/containers.md new file mode 100644 index 000000000..f7e1f0787 --- /dev/null +++ b/docs/installation/containers.md @@ -0,0 +1,117 @@ +--- +title: Containers +--- + +# Installing DEV using Containers + +If you encounter any errors with our Container setup, please kindly +[report any issues](https://github.com/thepracticaldev/dev.to/issues/new/choose)! + +## Installing prerequisites + +_These prerequisites assume you're working on an operating system supported by +Docker or Podman._ + +### Choosing a Container Engine + +A container engine is software that runs and manages containers on a computer. +One of the most widely known Container Engines is Docker, but there are many +other Container Engines available, such as [Podman](https://podman.io/), [CRI-O](https://cri-o.io/), and [LXD](https://linuxcontainers.org/lxd/introduction/). + +DEV supports two Container Engines: Docker and Podman. + +### Docker + +DEV can be setup with Docker and Docker Compose on macOS or Linux systems. + +Docker is available for many different operating systems. You may use Docker as your Container Engine on both macOS and Linux workstations. As of right now Docker is the only Container Engine for macOS and we recommend you follow the [Docker Desktop on Mac](https://docs.docker.com/docker-for-mac/install/), +install instructions to get Docker and Docker Compose installed. + +Docker also works well on Linux distributions that have not moved to cgroup v2. +You can install it by following their [Installation per distro](https://docs.docker.com/engine/install/) +to get Docker and you can install Docker Compose by following these +[instructions](https://docs.docker.com/compose/install/). + +### Podman + +DEV can be setup with Podman and Podman Compose on Linux systems. + +[Podman](https://podman.io/) is an FOSS project that provides a Container Engine +that is daemonless which only runs on Linux systems. It can be run as the root +user or as a non-privileged user. It also provides a Docker-compatible +command line interface. Podman is available on many different Linux distributions +and it can be installed by following these [instructions](https://podman.io/getting-started/installation). + +[Podman Compose](https://github.com/containers/podman-compose) is a an early +project under development that is implementing docker-compose like experience +with Podman. You can install it by following these [instructions](https://github.com/containers/podman-compose#installation). + +## Setting up DEV + +1. Fork DEV's repository, e.g. +1. Clone your forked repository, eg. + `git clone https://github.com//dev.to.git` +1. Set up your environment variables/secrets + + - Take a look at `Envfile`. This file lists all the `ENV` variables we use + and provides a fake default for any missing keys. + - The [backend guide](/backend) will show you how to get free API keys for + additional services that may be required to run certain parts of the app. + - For any key that you wish to enter/replace: + + 1. Create `config/application.yml` by copying from the provided template + (i.e. with bash: + `cp config/sample_application.yml config/application.yml`). This is a + personal file that is ignored in git. + 1. Obtain the development variable and apply the key you wish to + enter/replace. i.e.: + + ```shell + GITHUB_KEY: "SOME_REAL_SECURE_KEY_HERE" + GITHUB_SECRET: "ANOTHER_REAL_SECURE_KEY_HERE" + ``` + + - You do not need "real" keys for basic development. Some features require + certain keys, so you may be able to add them as you go. + +## Running DEV with Docker via docker-compose + +1. Run `bin/container-setup` +2. That's it! Navigate to + +The script executes the following steps: + +1. `docker-compose build` +2. `docker-compose up` + +## Running DEV with Podman via podman-compose + +1. Run `bin/container-setup` +2. That's it! Navigate to + +The script executes the following steps: + +1. `podman-compose build` +2. `podman-compose up` + +## Known Problems & Solutions + +### Docker on Mac + +- Should you experience problems with the Elasticsearch container, try to + increase the memory and/or swap allocation for Docker. On macOS this can be + done via the GUI: + + ![docker gui](https://user-images.githubusercontent.com/47985/74210448-b63b7c80-4c83-11ea-959b-02249b2a6952.png) + +- In case `rails server` doesn't start with the following message: + + ``` + Data update scripts need to be run before you can start the application. Please run rails data_updates:run (RuntimeError) + ``` + + run the following command: + + ```shell + docker-compose run web rails data_updates:run + ``` diff --git a/docs/installation/docker.md b/docs/installation/docker.md deleted file mode 100644 index a17e5ec28..000000000 --- a/docs/installation/docker.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -title: Docker ---- - -# Installing DEV with Docker - -If you encounter any errors with our Docker setup, please kindly -[report any issues](https://github.com/thepracticaldev/dev.to/issues/new/choose)! - -## Installing prerequisites - -_These prerequisites assume you're working on an operating system supported by -Docker._ - -### Docker and Docker Compose - -Docker is available for many different operating systems. We recommend you -follow the [Docker CE install guide](https://docs.docker.com/install/), which -illustrates multiple installation options for each OS. - -You're also going to need Docker Compose, to start multiple containers. We -recommend you follow the -[Docker Compose install guide](https://docs.docker.com/compose/install/) as -well. - -## Installing DEV - -1. Fork DEV's repository, e.g. -1. Clone your forked repository, eg. - `git clone https://github.com//dev.to.git` -1. Set up your environment variables/secrets - - - Take a look at `Envfile`. This file lists all the `ENV` variables we use - and provides a fake default for any missing keys. - - The [backend guide](/backend) will show you how to get free API keys for - additional services that may be required to run certain parts of the app. - - For any key that you wish to enter/replace: - - 1. Create `config/application.yml` by copying from the provided template - (i.e. with bash: - `cp config/sample_application.yml config/application.yml`). This is a - personal file that is ignored in git. - 1. Obtain the development variable and apply the key you wish to - enter/replace. i.e.: - - ```shell - GITHUB_KEY: "SOME_REAL_SECURE_KEY_HERE" - GITHUB_SECRET: "ANOTHER_REAL_SECURE_KEY_HERE" - ``` - - - You do not need "real" keys for basic development. Some features require - certain keys, so you may be able to add them as you go. - -## Running the Docker app via docker-compose (recommended) - -_Docker compose will by default use postgres:9.6 as the database version, should -you want to update that set the `POSTGRES_VERSION` variable in your environment -and start the container again_ - -1. Run `bin/docker-setup` -2. That's it! Navigate to - -The script executes the following steps: - -1. `docker-compose build` -2. `docker-compose run web rails app_initializer:setup` -3. `docker-compose up` - -## Running the Docker app (advanced) - -DEV provides a `docker-run.sh` script which can be used to run the Docker app -with custom options. - -Please execute the script itself to view all additional options: - -```shell -./docker-run.sh -``` - -## Known Problems & Solutions - -- Should you experience problems with the Elasticsearch container, try to - increase the memory and/or swap allocation for Docker. On macOS this can be - done via the GUI: - - ![docker gui](https://user-images.githubusercontent.com/47985/74210448-b63b7c80-4c83-11ea-959b-02249b2a6952.png) - -- In case `rails server` doesn't start with the following message: - - ``` - Data update scripts need to be run before you can start the application. Please run rails data_updates:run (RuntimeError) - ``` - - run the following command: - - ```shell - docker-compose run web rails data_updates:run - ``` diff --git a/docs/installation/readme.md b/docs/installation/readme.md index 6a1b7d6e4..03ef53c94 100644 --- a/docs/installation/readme.md +++ b/docs/installation/readme.md @@ -3,7 +3,7 @@ items: - mac.md - windows.md - linux.md - - docker.md + - containers.md - gitpod.md - postgresql.md - others.md @@ -20,15 +20,15 @@ You can install DEV to your local machine and we have instructions for [Mac](/installation/mac), [Windows](/installation/windows) and [Linux](/installation/linux). -## Running Docker +## Running with containers Installing to your local machine can be troublesome for many reasons such as a conflicting database and runtime versions. -Another way you can get a development environment up and running is with Docker. -Docker will install everything you need in an isolated container, and you need -not concern about the details. We have instructions for installing with -[Docker](/installation/docker). +Another way you can get a development environment up and running is with containers. +Using containers will setup everything you need in an isolated environment, and you need +not concern about the details of setting everything up locally. We have +instructions for getting setup with [containers](/installation/containers) quickly. ## GitPod _- beginner friendly!_ diff --git a/scripts/bundle.sh b/scripts/bundle.sh new file mode 100755 index 000000000..8f69f24dd --- /dev/null +++ b/scripts/bundle.sh @@ -0,0 +1,10 @@ +#!/usr/bin/env bash + +set -e + +if [ -f /opt/apps/bundle/bundle_finished ]; then + rm -f /opt/apps/bundle/bundle_finished +fi + +bundle install --jobs 20 --retry 5 +echo $(date --utc +%FT%T%Z) > /opt/apps/bundle/bundle_finished diff --git a/scripts/entrypoint.sh b/scripts/entrypoint.sh new file mode 100755 index 000000000..915d7cc8e --- /dev/null +++ b/scripts/entrypoint.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash + +set -e + +if [ -f tmp/pids/server.pid ]; then + rm -f tmp/pids/server.pid +fi + +echo "Running rake app_initializer:setup..." +bundle exec rake app_initializer:setup + +exec "$@"