Commit graph

10 commits

Author SHA1 Message Date
Joe Doss
b485ef2eb9
Jdoss/more buildkite container fixes (#11503)
* Fix bug on creating the testing container for PRs and create a function to
run docker image prune -f that will get run when the script exits cleanly or
uncleanly. This prevent junk containers from building up over time.

* Add in some labels and keys. Add in building containers on master branch.

* Let's also build on main branch for when we eventually move away from master.

* Let's use a previous builder image as a cache as it might speed things up.
2020-11-19 13:34:20 -06:00
Joe Doss
1d4685219d
Multistage container builds (#11468)
* WIP multi-stage container build.

* Move to a multistage build process in the Containerfile.

* Enable testing container

* Finalize the multistage build script and Containerfile. Ooof!

I got blocked for a bit with this bugger:

Step 27/32 : COPY --from=builder --chown=${APP_USER}:${APP_USER} ${APP_HOME} ${APP_HOME}
failed to copy files: failed to copy directory: Error processing tar file(exit status 1): Container ID 100999 cannot be mapped to a host ID

but https://github.com/phusion/passenger-docker/issues/235#issuecomment-636318827

was a good shove in the right direction!

* Fix bugs on container build script.

* Adjust compose files to use multistage builds.

* Add in Buildkite pipeline for building containers.

* Update CODEOWNERS to claim ownership over things that Systems Eng cares about
2020-11-19 10:03:35 -06:00
Molly Struve
0f6030262f
Bug Fix:Make entrypoint-dev.sh Runnable (#10359) 2020-09-16 19:08:43 -05:00
Joe Doss
f34dbf4693
Better production container precompiling and bootstrapping (#10280)
* Create entrypoint-dev.sh to be used with devel workflows with docker/podman compose
and adjust the compose files to use it.

* Adjust the entrypoint script so we have better control over when we run
rake assets:precompile and rake app_initializer:setup.

* Copy vendor/cache so we can bundle install from cached gems instead of
fetching them from the Internet.

* Run rake assets:precompile in the container build.

* Add in clobber and clean tasks to entrypoint.
2020-09-16 16:59:19 -05:00
Molly Struve
b67a750319
Optimization:Use Locally Bundled Gems in Containers (#10344) 2020-09-16 10:00:21 +07:00
Joe Doss
6fa954c9d0
Add FOREM_BUILD_SHA and FOREM_BUILD_DATE to the container. (#9999)
In the past it was always useful to have meta data within regards to the SHA and
build date/time. This adds in two new env vars that are written to disk as flat
files and then exported to environment variables when the container starts up.

This motivation comes from this comment:

https://github.com/forem/forem/pull/9987#discussion_r476123829
2020-08-25 18:19:04 -05:00
Joe Doss
4e1fc4755b
Export RELEASE_FOOTPRINT on container start. (#9987)
RELEASE_FOOTPRINT will be set to the output of date -u +'%Y-%m-%dT%H:%M:%SZ'
(2020-08-25T02:02:21Z for example) which will fix https://github.com/forem/forem/pull/9904

This will be reset to a new date and time string every time the container starts.
2020-08-25 11:06:53 -04:00
Joe Doss
aa49cb1cfe
Bugfix for #7663 and refactor of container setup (#7747)
* Move from Alpine Linux to Fedora Linux.

This changes the base container image away from Alpine Linux to
Fedora Linux to address some musl libc issues with some of our gems.

It also moved the main file to a Containerfile and symlinks the Dockerfile
to it. This makes our container setup less Docker-centric as Linux users
most likely will be using Podman as their container runtime.

Lastly, it moves the WORKDIR from /usr/src/app to /opt/apps/devto.
The Linux FHS states that /opt is the spot for Optional application
software packages and /usr/src is for kernel source code.

https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

/opt Optional application software packages
/usr/src Source code, e.g., the kernel source code with its header files.

Also, if SELinux becomes a thing in our future, moving this makes it
easier to manage contexts out of /opt rather than /usr/src.

* Adjust the Containerfile a bit to add some missing packages, make it
more generic and move env vars to the docker-compose.

* Move the Dockerfile to a symlink to the Containerfile.

We need to be able to support more than just Docker for a container
runtime. Moving everything to a Containerfile and symlinking the
Dockerfile helps users run runtimes such as Podman.

* Add in new entrypoint files and fix the app path in docker-entrypoint.sh

* Refactor the docker-compose.yml file so it doesn't build the main
application container three times in a row. We can use the same
container for web, webpacker, and sidekiq.

Also make the volume names very explicit on what their contents and add
in SELinux context support with :Z

* Fix ELASTICSEARCH_URL.

* Remove the absolute path on ip binary and add in iproute to Containerfile.

* Symlink the Dockerfile to Containerfile and add in a container-compose.yml file
for usage with podman-compose. We are waiting for this issue to get resolved

https://github.com/containers/libpod/issues/6153

and for podman-compose to mature a bit more. A user using podman-compose can use

podman-compose -f container-compose.yml

to launch the DEV container stack with Podman.

* Update the .gitignore file to reflect the new container volumes.

* Fix the entrypoint script on the Containerfile and clean up a bunch of things.

* Rework the Containerfile to prep it to run as a non-root user. We have to wait
for this issue to get fixed:

https://github.com/containers/libpod/issues/6153

for Linux users and then we can uncomment the USER line so we are running Rails
as a non-root user. :toot:

* This reworks the compose files so each task that is needed to start the app has
a correct wait command with dockerize. It also adds in containers for doing
yarn and bundle things for development. Since we mount the code directory inside
the container we lose our pre-containerized gems and node_modules.

This means users can run:

Linux
podman-compose -f container-compose.yml up

Mac
docker-compose up

and it should correctly build the app stack with containers!

* Clean up old container related things.

* Add in some container pre-reqs and my name to the "Core team" section! :toot:

* Adjust the seed and sidekiq compose entries so they do not use the web entrypoint
and set the REDIS_URL and REDIS_SESSIONS_URL env vars for seed.

Add in some echos to the entrypoint.sh.

Also set docker-compose to use :delegated on mount points to speed things up.

https://docs.docker.com/storage/bind-mounts/#configure-mount-consistency-for-macos

* Just call yarn install --dev on the yarn container.

* Update documentation to support Docker and Podman as Container Engines and
move the page from docker to containers to reflect the fact not all users run
their containers via Docker.

There are dozens of us... DOZENS!

* Set --local on bundle config so it just impacts this Ruby app.

* Renamed bin/docker-setup to bin/container-setup to reflect the fact that that
we can use Podman as a container engine. I also refactored it so it does some
basic pre-flight checks for docker and docker-compose or podman and
podman-compose to make sure users have a container engine installed.

* Set cache_all_platforms true for bundler.

* Bump docker-compose dockerize -timeout to 45min for slower macOS hardware.

* Move to the consolidated app_initializer:setup rake task for bootstrapping the
app.

* Update docs/installation/readme.md
2020-05-28 12:11:51 -05:00
Ridhwana
813389d36a
Keeping your forked branch up to date (#6531)
* chore: updates to sync your forked ranch

* chore: use rease instead

* feat: update the script to not bomb out if you have added the remote before

* chore: add the comment back
2020-03-09 09:37:46 -04:00
Robin
d7174923a2 Git scripts to sync or reset repository fork (#5055)
* Add script to clean / reset the fork and start from upstream

* Add script to sync fork with latest changes from upstream

* Remove swap file accidentially checked in

* Add exit on break to shell scripts and documentation of shell scripts to workflow-suggested workflow

Co-authored-by: Mac Siri <krairit.siri@gmail.com>
2019-12-30 13:58:16 -05:00