Commit graph

562 commits

Author SHA1 Message Date
Molly Struve
eda0a78d47
Add Health Check Endpoint for Cache (#8222) 2020-06-01 15:19:56 -05:00
Joe Doss
aa49cb1cfe
Bugfix for #7663 and refactor of container setup (#7747)
* Move from Alpine Linux to Fedora Linux.

This changes the base container image away from Alpine Linux to
Fedora Linux to address some musl libc issues with some of our gems.

It also moved the main file to a Containerfile and symlinks the Dockerfile
to it. This makes our container setup less Docker-centric as Linux users
most likely will be using Podman as their container runtime.

Lastly, it moves the WORKDIR from /usr/src/app to /opt/apps/devto.
The Linux FHS states that /opt is the spot for Optional application
software packages and /usr/src is for kernel source code.

https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

/opt Optional application software packages
/usr/src Source code, e.g., the kernel source code with its header files.

Also, if SELinux becomes a thing in our future, moving this makes it
easier to manage contexts out of /opt rather than /usr/src.

* Adjust the Containerfile a bit to add some missing packages, make it
more generic and move env vars to the docker-compose.

* Move the Dockerfile to a symlink to the Containerfile.

We need to be able to support more than just Docker for a container
runtime. Moving everything to a Containerfile and symlinking the
Dockerfile helps users run runtimes such as Podman.

* Add in new entrypoint files and fix the app path in docker-entrypoint.sh

* Refactor the docker-compose.yml file so it doesn't build the main
application container three times in a row. We can use the same
container for web, webpacker, and sidekiq.

Also make the volume names very explicit on what their contents and add
in SELinux context support with :Z

* Fix ELASTICSEARCH_URL.

* Remove the absolute path on ip binary and add in iproute to Containerfile.

* Symlink the Dockerfile to Containerfile and add in a container-compose.yml file
for usage with podman-compose. We are waiting for this issue to get resolved

https://github.com/containers/libpod/issues/6153

and for podman-compose to mature a bit more. A user using podman-compose can use

podman-compose -f container-compose.yml

to launch the DEV container stack with Podman.

* Update the .gitignore file to reflect the new container volumes.

* Fix the entrypoint script on the Containerfile and clean up a bunch of things.

* Rework the Containerfile to prep it to run as a non-root user. We have to wait
for this issue to get fixed:

https://github.com/containers/libpod/issues/6153

for Linux users and then we can uncomment the USER line so we are running Rails
as a non-root user. :toot:

* This reworks the compose files so each task that is needed to start the app has
a correct wait command with dockerize. It also adds in containers for doing
yarn and bundle things for development. Since we mount the code directory inside
the container we lose our pre-containerized gems and node_modules.

This means users can run:

Linux
podman-compose -f container-compose.yml up

Mac
docker-compose up

and it should correctly build the app stack with containers!

* Clean up old container related things.

* Add in some container pre-reqs and my name to the "Core team" section! :toot:

* Adjust the seed and sidekiq compose entries so they do not use the web entrypoint
and set the REDIS_URL and REDIS_SESSIONS_URL env vars for seed.

Add in some echos to the entrypoint.sh.

Also set docker-compose to use :delegated on mount points to speed things up.

https://docs.docker.com/storage/bind-mounts/#configure-mount-consistency-for-macos

* Just call yarn install --dev on the yarn container.

* Update documentation to support Docker and Podman as Container Engines and
move the page from docker to containers to reflect the fact not all users run
their containers via Docker.

There are dozens of us... DOZENS!

* Set --local on bundle config so it just impacts this Ruby app.

* Renamed bin/docker-setup to bin/container-setup to reflect the fact that that
we can use Podman as a container engine. I also refactored it so it does some
basic pre-flight checks for docker and docker-compose or podman and
podman-compose to make sure users have a container engine installed.

* Set cache_all_platforms true for bundler.

* Bump docker-compose dockerize -timeout to 45min for slower macOS hardware.

* Move to the consolidated app_initializer:setup rake task for bootstrapping the
app.

* Update docs/installation/readme.md
2020-05-28 12:11:51 -05:00
Molly Struve
14c2372ae7
[deploy] Add API Key to API Write Throttle (#8110) 2020-05-28 09:45:01 -05:00
Molly Struve
8a4c2c1cb4
[deploy] Add App, Database, and Search Health Check Endpoints (#8071) 2020-05-28 08:33:31 -05:00
Michael Kohl
7f75f99560
[deploy] Rename classified listings (#7910)
* Change models and related files

* Update controllers and specs

* More renaming

* Seek and destroy, I mean search and replace

* Round up the stragglers

* Ground control to Major Travis...

* More fixes

* PR feedback

* Various fixes

* Rename view

* Fix list query builder

* Unify request specs

* Fix some API spec errors

* Fix remaining API specs

* Make spec conform to API

* Fix leftover problems

* Fix JS tests

* Fix column name in select

* Fix API specs

* Fix search specs

* Paging Mr. Travis
2020-05-27 13:35:09 +00:00
Josh Puetz
1a1b5b544e
Add Feature Flags support to DEV (#7996)
* Create tables for Flipper gem

* Enable Flipper gem

* Remove concurrent ruby from Gemfile

* Add Flipper UI

* Codeland landing page

* Remove codeland controller and sample page

* Remove oops

* Rename Flipper-UI endpoint

* Wrap Flipper in a FeatureFlag module

* [deploy] Lock node to 12.16 (#8057)

Co-authored-by: Mac Siri <krairit.siri@gmail.com>
2020-05-26 15:26:33 -05:00
rhymes
0976132425
[deploy] RSS feed: add article tags as categories (#8046) 2020-05-26 15:51:33 -04:00
Ben Halpern
cd18d2f436
[deploy] Add alternate comment-oriented feed algos (#8042) 2020-05-26 13:23:03 -04:00
Andy Zhao
c42fd3461e
[deploy] Rename positive_reactions_count to public_reactions_count in logic (#7926)
* Rename positive_reactions_count to public_reactions_count

* Add positive reactions count back in so we can remove it

* Use public_category method for reactions

* Add positive_reactions_count in case any old caches rely on it

* Add positive_rxn_count to account for API endpoints

* Remove unused method

* One more spot...

* Add method back in because of caches

* Update specs to match new functionality

* Fix typo

* Remove unused methods
2020-05-26 12:36:28 -04:00
rhymes
1112cbf316
[deploy] Update webpacker configuration (#8004) 2020-05-25 12:03:18 -04:00
Ben Halpern
c23c4fbb2d
[deploy] Adjust a/b weights for feed (#8037) 2020-05-22 17:17:36 -04:00
Mac Siri
1abd7cd46b
[deploy] Add async_info to ReservedWords (#8022) 2020-05-21 17:02:26 -04:00
Alex
49d4ec8d1c
[deploy] Revert "Update safe params for Array params (#7914)" (#8018)
This reverts commit 4eba14f150.
2020-05-21 12:39:55 -04:00
Andy Zhao
e686d63d78
[deploy] Add public reactions count fields to Elasticsearch (#7956)
* Add public_reactions_count field to elasticsearch

* Add new data update scripts and update old ones

* Use BulkIndexWorker to batch indexing

* Use proper search class for handling feed content

* Remove redundant script oops

* Rename data update script and remove old specs

* Combine specs into one file
2020-05-20 13:06:48 -04:00
Molly Struve
1c461f8891
[deploy] Remove Unused Pro Memberships (#7894) 2020-05-20 11:54:30 -05:00
rhymes
300ba1b33f
[deploy] Decouple Twitter authentication from fetching tweets (#7920)
* Add TwitterClient::Client

* Use TwitterClient::Client for Tweet

* Test retweets

* Test assignment to user

* Fix TweetTag spec

* Remove TwitterBot

* Fix some specs

* Fix RSS Reader specs
2020-05-19 18:05:41 -04:00
Alex
4eba14f150
[deploy] Update safe params for Array params (#7914) 2020-05-19 13:01:50 -04:00
Molly Struve
6a4a99f162
[deploy] Track Deploys In Honeycomb (#7935) 2020-05-19 09:10:41 -05:00
Molly Struve
dbb5c752ab
[deploy] Sample Noisy Honeycomb Spans (#7891) 2020-05-18 10:36:01 -05:00
Molly Struve
7293df3191
[deploy] Return if there is no ip_address from Method (#7874) 2020-05-14 18:13:24 -05:00
Molly Struve
7dd249ac21
[deploy] Add Back Tracking API-keys in Honeycomb (#7868) 2020-05-14 15:10:15 -05:00
Molly Struve
8ae486a362
[deploy] Track and Limit API writes by IP Address (#7847) 2020-05-14 14:13:56 -05:00
Mac Siri
321b24d2a0
[deploy] Create write_articles oauth scope (#7834) 2020-05-14 09:55:53 -04:00
Molly Struve
3a49d3f0d1
[deploy] Revert "filter/sample noisy spans (#7833)" (#7837)
This reverts commit 716bb099fa.
2020-05-13 17:45:49 -05:00
Liz Fong-Jones
716bb099fa
[deploy] filter/sample noisy spans (#7833) 2020-05-13 17:07:09 -05:00
Anna Buianova
82799eb752
Remove code related to /live_articles (#7822) 2020-05-13 09:26:13 -04:00
Molly Struve
fb04925c29
[deploy] Decrease General Site Request Limit (#7771) 2020-05-12 09:41:19 -05:00
rhymes
81fff248dd
Refactoring GitHub Repos functionality - step 1 (#7764)
* Rename find_or_create to upsert and improve validation and testing

* Add User.authenticated_through?

* Refactor settings/integrations

* Refactor profile github repositories rendering

* Refactor repos fetching

* Only fetch a single repo and improve error messages

* Remove unused code

* Cleanups

* Fix specs

* Remove trailing whitespace

* Fix spec

* Trigger Travis
2020-05-12 13:48:19 +02:00
Molly Struve
1570580279
[deploy] Return the First 300 characters for Redis Commands (#7778) 2020-05-11 15:01:23 -05:00
Molly Struve
bbff0f757e
[deploy] dont configure Honeycomb if its disabled (#6618) 2020-05-11 11:59:18 -05:00
Michael Kohl
1d578aa3ea
[deploy] Add org credit management to org mod page (#7761)
* Add org credit management to org mod page

* Fix typo
2020-05-11 16:49:05 +02:00
Sarthak Sharma
90d60bf73f
[deploy] 🚀 Feature: Ability to search discoverable channels and send request to join (#7385)
* Feature 🚀 : Ability to delete messages in chat channels

- Sending message ID to frontend
- Deleting Message
- Use pusher to delete message realtime

* Minor Bug 🐞: Show message action only for current user

- User can delete or edit their own messages

* Test cases added

* Bug 🐞: Update message id for receiver

Message id was not sent to receiver by pusher

* Refactoring🛠: Message controller refactoring

* Test Cases📝 : Specs for Delete message added

* Feature 🚀 : Ability to edit messages

* Test Cases📝 : Specs for Edit message added

* added new column discoverable for public channel and changes in elasticsearch

* fix corrections

* changes in serializer file, added channel_discoverable

* added checkbox with discoverable policy

* changes in code for discoverable channels

* accept and reject member invitation by mod

* add joining request to closed groups

* fixed merge error

* changes in joining member to closed groups

* join to closed group

* changes in message action of joining

* changes in chat upopened json

* 🚀Feature : Ability to search Global Channels

* touch updated_at in after commit update

* 🚀Feature : Ability to send request to discoverable channel

* 🚀Feature : Ability to send request to discoverable channel

* created new route for joining closed channel

* 🚀 Success handle on joining request sent

* removed redundant code

* join_channel improvement, removed authorization for join channel

* list of joining requests

* added joining_request status channels on search list

* changes in mailer and query builder optimized

* 🛠 Adding filter for new Query

* added rspec for add membership method in controller

* rspec for sending join channel request

* invite join request channel list rspec

* test case for query builder discoverable

* viewable and discoverable channel list

* refactored logic for search channels

* 🚀 Check if Request already sent

* 🛠 Optimizing code and making channelButton Component

* 🛠 Optimizing codefor SVG problem

* 🛠 Optimized action.js

* changes in search controller query

* hot fix

* removed unwanted code

* 🛠  Fix the Channel Name problem

* 🛠  Optimizing code further for CodeClimate

* added new column discoverable for public channel and changes in elasticsearch

* fix corrections

* changes in serializer file, added channel_discoverable

* added checkbox with discoverable policy

* changes in code for discoverable channels

* accept and reject member invitation by mod

* add joining request to closed groups

* fixed merge error

* changes in joining member to closed groups

* join to closed group

* changes in message action of joining

* changes in chat upopened json

* touch updated_at in after commit update

* 🚀Feature : Ability to search Global Channels

* 🚀Feature : Ability to send request to discoverable channel

* 🚀Feature : Ability to send request to discoverable channel

* created new route for joining closed channel

* 🚀 Success handle on joining request sent

* removed redundant code

* join_channel improvement, removed authorization for join channel

* list of joining requests

* added joining_request status channels on search list

* changes in mailer and query builder optimized

* added rspec for add membership method in controller

* rspec for sending join channel request

* invite join request channel list rspec

* 🛠 Adding filter for new Query

* test case for query builder discoverable

* viewable and discoverable channel list

* refactored logic for search channels

* 🚀 Check if Request already sent

* 🛠 Optimizing code and making channelButton Component

* 🛠 Optimizing codefor SVG problem

* 🛠 Optimized action.js

* changes in search controller query

* hot fix

* 🛠  Fix the Channel Name problem

* 🛠  Fixing merge problem

* 🛠  Optimizing code further for CodeClimate

* updated UI for membership edit

* fixed test casses and fixed UI for chat_channel_edit

* 🛠napshots added

* test cases fixed

* 🛠 Test cases added for new component

* channel settings accesible only by mod

* 🛠 More Test cases added

* 🛠 Svg code optimized

* 🛠 Svg code optimized in videocontent

* optimized code for search query

* fix test case for query builder

* changes in joining closed channel logic

* refactored join channel

* redirect to edit channel after joining request

* changes in test case for redirect

* optimized code

* 🛠 Eslint bugs fixed

* test case fixed

* optimization

* olving channel repetition problem

* optimization of code for query builder

* changes in query builder

* test cases fixed

* 🛠 Handling reduntant data on frontend

* 🛠  Don't show data if no filter query

* 🛠 Optimized code for fixing bugs and code coverage

* 🛠  Optimizing code further for CodeClimate

Co-authored-by: Parasgr-code <paras.gaur@skynox.tech>
2020-05-11 09:29:15 -04:00
Molly Struve
ea1d38d3b8
[deploy] Bulk Fetch Follow Button Data for Users (#7676) 2020-05-08 13:58:58 -05:00
Molly Struve
81c9b27a7f
Track IPs and API Tokens for Throttled Requets, Send Retry After Header (#7735) 2020-05-08 10:16:49 -04:00
Molly Struve
f190f4c6c0
[deploy] Track Client IPs to Help Block Bad Actors (#7730) 2020-05-07 12:45:30 -05:00
Jacob Herrington
a4f71c22b6
[deploy] Add automated email ownership emails (#7556)
* Add automated email ownership emails

This is task Michael does manually a lot. There are a lot of cases where
we need to validate that an individual has access to the email address
associated with their DEV account. This automates a singificant portion
of that, and hopefully we can use it in a lot of places.
2020-05-06 12:07:54 -05:00
Alex
b74b035e22
[deploy] Add new VCL snippets for Fastly (#7654)
* Add new VCL snippets

* Fix specs

* Partial match Fuzz Faster

* Remove PHP
2020-05-04 10:18:19 -04:00
Michael Kohl
fcd0639d2a
[deploy] Remove column and minor cleanup (#7642)
* Remove column and minor cleanup

* More cleanup

* Fix spec

* Remove category from select

* Wake up Travis

* No space for you
2020-05-03 11:52:03 -04:00
Alex
7149070bca
[deploy] Create FastlyConfig & refactor params safe list (#7630)
* Fastly refactor

- Create FastlyConfig
- Create FastlyConfig::Base
- Create FastlyConfig::Snippets
- Move safe_params to its own VCL file
- Rescue unauthorized errors in development
- Add FastlyConfig errors

* Move active version to method

* Update naming option --> config

* Refactor InvalidConfigsFormat msg to Error class

* Refactor active version logic

* Fix log_to_datadog call

* Move get_active_version logic back into method

* Add some specs ¯\_(ツ)_/¯

* Update Fastly rake task

* Remove old Fastly way

* Update docs for Fastly

* Move snippets to config/ & remove old config

* Change update_config to upsert_config

* Change error type to SubclassResponsibility

* Refactor get_updated_files with filter_map

* Refactor update

* Cleanup update_config --> upsert_config

* Fix updater specs
2020-05-01 11:23:48 -04:00
Molly Struve
2ba6c98e81
Limit 1 message every second for single user (#7631)
* limit 1 message every second for  single user

* increase message limit to 2 in a second
2020-04-30 19:06:19 -04:00
rhymes
7ebd95f193
Upgrade field_test to 0.3.2 (#7592)
* Make sure tests are testing the right thing

* Upgrade gem field_test to 0.3.2

* Fix things

* Add test

* Fix typo
2020-04-30 18:43:30 -04:00
rhymes
c7c498834f
[deploy] Authentication refactoring: generalize account settings (#7597)
* Rename remove_association to remove_identity and future proof its tests

* Use the signin_path everywhere

* Generalize account settings

* Use delegation
2020-04-30 17:29:53 +02:00
Mac Siri
b7a29c7144
Make DATABASE_URL_TEST optional (#7604)
* Make DATABASE_URL_TEST optional

* Add documentation

* Update wording
2020-04-30 11:19:16 +02:00
Molly Struve
4c779be2ed
[deploy] Remove ALL Algolia Code (#7577) 2020-04-29 10:11:57 -05:00
rhymes
8407bf9b4e
[deploy] Authentication refactoring: cleanups (#7576)
* Use Devise registry to list available providers

* Cleanups following Devise+Omniauth docs

* More cleanups

* Use helpers

* More cleanups

* Oops, forgot to remove these specs
2020-04-29 10:33:30 -04:00
Ridhwana
8140e46ceb
Add some placeholder pages (#7441)
* feat: add placeholders for some routes

* about_listings_url

* feat: tag-moderation

* chore: about_listings_url test

* chore: use post a job url

* chore: add pages methods

* chore: just reordered stuff

* oops forgot to save - oh man

* chore: change to single quotes

* chore: hopefully retrigger cc

* chore: update tests

* feat: PR suggestions:)

* chore: spec
2020-04-28 15:55:59 +02:00
rhymes
2d26318cf9
[deploy] Replace (black|white)list with more inclusive language (#7459)
* Replace isBlacklisted

* Update fastly.rake task

* More renames

* Rename Faslty config file

* More changes

* Replace isForbiddenFromPreloading with isNotPreloadable

* Skip problematic test

* Re-enable
2020-04-27 18:00:12 +02:00
Tam Chau
ffba42cf43
Add expires_in option for ActiveSupport::Cache::RedisStore#increment method (#7515)
* rails patch to fix expires_in for increment method

* fixup! rails patch to fix expires_in for increment method

correct test case

* fix redundant file name & class name

* add comment to explain monkey patch for RedisCacheStore#increment
2020-04-27 16:40:49 +02:00
rhymes
f5a6ff32cc
Finalize Slack messengers refactoring (#7484)
* Remove dead code

* Rename SlackBotPingWorker to Slack::Messengers::Worker

* Remove useless Slack mock

* Fix specs
2020-04-24 21:15:10 +02:00
rhymes
db9186043e
Removed yarn integrity check (#7478) 2020-04-23 19:23:12 -04:00