This "rebases" our application images off of the new Ruby base image
added in #19632, and fixes numerous problems and quirks with how the
images were built along the way. Notably:
- Issues where layers attempted to delete files in prior layers have
been resolved (this caused build failures on some Docker filesystem
drivers, notably overlay2).
- Bundler is no longer allowed to deviate from or modify the lockfile
(`BUNDLE_FROZEN` is now `true`).
- `git(1)` is no longer required to live inside the container and `.git`
is no longer required to be copied into the Docker build context, as
these were only used to calculate `FOREM_BUILD_SHA`, which is now
passed in as a Build Argument to the container build context.
- The entire source tree is no longer `chmod` in one giant swing, which
ran so long on my system (as just one example) that I gave up after
15-20 minutes and issued it a `SIGTERM`. Instead, `COPY --chown` is
used more heavily and ensures the `APP_USER` will have access to the
requisite files.
This new container image appears to build successfully for
`linux/arm64`, which refs (but does not complete) #19626. Currently,
such builds aren't automated , and must be built on a developer
workstation. For example:
```sh
docker buildx build --platform linux/amd64,linux/arm64 -f Containerfile . -t ghcr.io/forem/forem:klardotsh-test --push --build-arg VCS_REF=$(git rev-parse --short HEAD)
```
In the meantime, the existing `linux/amd64`-only BuildKite scripts have
been updated to allow this PR to merge as a separate unit, and CI
refactors to enable the multiarch builds of `linux/arm64,linux/amd64`
can come later when more time is available.
This is one of several blockers on the path to getting #19603 merged.
The next step in that chronology will be rebasing that work on top of
this work, which *should* be, on the containerization side, as
straightforward as bumping `Containerfile.base` to reference the new
upstream image, rebuilding the base container, and then bumping the
reference in `Containerfile`.
* Only print BUILDKITE environment variables when they were present
This was causing errors running the build_containers.sh script
locally, since only buildkite was passing these (and errors stop the
script, so nothing was built).
* Simplify conditional guards to check only for BUILDKITE variable
https://buildkite.com/docs/pipelines/environment-variables#bk-env-vars-buildkite
describes the simplest "this is buildkite" check, BUILDKITE is always
true when running in buildkite.
* Add release scripts
* Use `delete` over `gsub` with an empty string
Co-authored-by: Michael Kohl <citizen428@dev.to>
* wip
* wip
* Include stable release-channel branches in builds
* Add changelog stub
* Include the union of changes to the changelog
* Add things
* Add stuff
* Remove test changelog content
* Fix String#delete call
Extraneous `.` made Ruby parse it as a range
* Fix suggested release command
* Use backticks to denote a command to run
* Add debugging output to script
* Build more appropriate containers for stable
This includes tags
* Check if branch *starts with* release channel name
* Cache tag builds from production tag
* Skip trying to build containers for untagged pushes
* Clear out stubbed changelog
* Update PR template to incorporate CHANGELOG.md
* Run Rubocop on release scripts
This excludes some linter rules that only make sense in code that is
running as part of the Rails app. For example, we can't rely on
`ActiveSupport::TimeWithZone` because these scripts don't load Rails,
and we define top-level methods because they're scripts rather than
complex applications.
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Fix bundle issues with the Multistage container
and adjusts a few other things:
- Disable ruby docs on bundler gem install
- Fix the bundle config
- chmod +x on scripts/build_containers.sh
* Add app/assets back into the builder image.
* Looks like running:
yarn install --frozen-lockfile \
&& RAILS_ENV=test NODE_ENV=test bundle exec rails webpacker:compile'
is super busted.
https://buildkite.com/forem/build-containers/builds/48#97451356-6552-40a1-8333-ac6fc0fccc3a
so let's just comment that out for now.
* Fix bug on creating the testing container for PRs and create a function to
run docker image prune -f that will get run when the script exits cleanly or
uncleanly. This prevent junk containers from building up over time.
* Add in some labels and keys. Add in building containers on master branch.
* Let's also build on main branch for when we eventually move away from master.
* Let's use a previous builder image as a cache as it might speed things up.
* WIP multi-stage container build.
* Move to a multistage build process in the Containerfile.
* Enable testing container
* Finalize the multistage build script and Containerfile. Ooof!
I got blocked for a bit with this bugger:
Step 27/32 : COPY --from=builder --chown=${APP_USER}:${APP_USER} ${APP_HOME} ${APP_HOME}
failed to copy files: failed to copy directory: Error processing tar file(exit status 1): Container ID 100999 cannot be mapped to a host ID
but https://github.com/phusion/passenger-docker/issues/235#issuecomment-636318827
was a good shove in the right direction!
* Fix bugs on container build script.
* Adjust compose files to use multistage builds.
* Add in Buildkite pipeline for building containers.
* Update CODEOWNERS to claim ownership over things that Systems Eng cares about
