Commit graph

222 commits

Author SHA1 Message Date
Anna Buianova
ba1c13bbb6
Asynchronous organization delete (#13388)
* Asynchronous organization destroy

* Notification after an organization was deleted

* Send data to Datadog and Honeybadger when org deletion failed

* Delete unused code

* Fixed Organization#destroyable? and organization delete specs

* Reorganized organization deletion specs

* Removed redundant specs

* Improved org deleted email text template

* Don't peform the org delete when a user or an org were not found

* Renamed user_id in the org delete worker

* Audit logging when deleten an organization

* Removed specs for tracking pixel and UTM params

* Changed slug for audit log on org delete

* Fixed schema.rb
2021-04-19 18:30:35 +03:00
Vaidehi Joshi
97678456d6
[15 Min Fix]: Extract a post's followers out of service and onto Article model (#13229)
* Move authors followers out of Notifications::NotifiableAction::Send and into Article model

* Rename authors_followers to followers

* Update followers method in service, oops

* Update app/services/notifications/notifiable_action/send.rb

Co-authored-by: Michael Kohl <citizen428@dev.to>

Co-authored-by: Michael Kohl <citizen428@dev.to>
2021-04-07 06:46:32 -07:00
Daniel Uber
16d17445a5
reenable article rescoring when moderators flag users (#12983)
* Update worker calculation and disable early return

* Enable service tests

This updates the test to look at the individual article scores
changing, rather than the sum of the articles scores changing

The scores lambda was added as a way to observe this eagerly

* Correctly handle raising the score when vomit is cleared

The initial implementation loaded the articles reactions and the
user's reactions in order to rescore. This restores that activity.

It occurs to me that article.reactions.sum(:points) +
user.reactions.sum(:points) feels like something articles should know
how to do, in which case we can move this into article (#rescore!)?
and call that instead of knowing how to handle moderation specially here.

* Move logic to Article#update_score

Having realized "sum of reaction scores" was an Article concern, I
found that we have a worker to score articles that does _exactly_ what
we're doing here.

Let's use that and allow the vomit reaction presence or absence on a
user to automatically affect the article calculations.

This is at least n + 1 and maybe worse.
2021-03-16 14:04:13 -05:00
Fernando Valverde
cf3bde4259
Add mobile push notifications to Forem (#12419)
* First commit with iOS PN working

* RPush cleanup worker + unique jobs config

* Remove rpush tables from schema.rb

* PR feedback

* Feature flag and test for route

* Tests and feature flag PushNotification ::Send

* Update app/controllers/devices_controller.rb

Co-authored-by: Michael Kohl <citizen428@dev.to>

* Update spec/routing/devices_routes_spec.rb

Co-authored-by: Michael Kohl <citizen428@dev.to>

* Update spec/services/push_notifications/send_spec.rb

Co-authored-by: Michael Kohl <citizen428@dev.to>

* PR feedback

* Set Rpush driver and url

* More PR feedback

* Apply suggestions from code review

Co-authored-by: Jamie Gaskins <jgaskins@gmail.com>

* PR feedback from Rhymes

* Don’t double render

* Sure

Co-authored-by: Josh Puetz <hi@joshpuetz.com>
Co-authored-by: Josh Puetz <josh@dev.to>
Co-authored-by: Michael Kohl <citizen428@dev.to>
Co-authored-by: Jamie Gaskins <jgaskins@gmail.com>
2021-03-12 14:08:18 -06:00
Molly Struve
004b62ac98
Optimization:Enforce Uniqueness for Follows::UpdatePointsWorker and RatingVotes::AssignRatingWorker (#12852) 2021-03-02 10:50:44 -06:00
Alex
d109b6824e
Refactor cache purging (#12811)
* Refactor caching

* Bring back old bust class

* Refactor with PR feedback

* Remove useless return values
2021-03-02 10:09:05 -05:00
Molly Struve
ac91299952
Bug Fix:Update updated_at Timestamp Everytime We Run Repo Sync Worker (#12853) 2021-03-01 13:45:10 -06:00
Michael Kohl
a432ab7b3f
Spike: Add ThrottledCall utility class (#12779)
* Add ThrottledCall utility class

* Rename class level interface method

* Update worker to use ThrottledCall

* Add ThrottledCall spec to worker
2021-03-01 08:51:04 +07:00
Michael Kohl
2555b59d87
Extract page view updates into workers (#12686)
* Extract page view updates into workers

* Restructure code

* Update specs

* Fix and reactive 2 specs

* Update according to PR feedback

* Use create! over create
2021-02-17 09:24:05 +07:00
Michael Kohl
98ec301a3c
Add guard clause to worker (#12632) 2021-02-11 10:43:48 +07:00
Ben Halpern
e1b1fe3ed7
Declare winner in feed_top_articles_query test (#12308)
* Declare winner in feed_top_articles_query test

* Some WIP modifications to lfe

* Fix up tests and generalize

* Fix style

* Adjust tests to handle different field test scenarios

* Remove socre_randomness tests

* Fix test to fit stub

* Update app/workers/users/record_field_test_event_worker.rb
2021-02-04 13:02:00 -05:00
Michael Kohl
e258e36c92
Remove language settings (#12502)
* Remove language settings

* Remove more language related code

* Remove leftover spec

* Remove language from ES mapping
2021-02-04 08:35:00 +07:00
Michael Kohl
60813703e8
Fix tag alias problems (#12480) 2021-02-03 11:58:59 -05:00
Ridhwana
530e927060
Add ability to re-run a data script from the the data update script page (#12424)
* feat: add an error column to the data update script

* feat: save the error to the error field

* feat: save the error when the script fails

* feat: show the script error on the data update script page

* chore: pass the error to mark_as_failed instead of having its own function

* refactor: use presence

* test: ensure that we test an error case of a data update script

* chore: rename errorneous to failing

* test: update some specs, working on the others

* chore: update tests now that there are two files

* chore: change error from a string to a text to allow for more char

* feat: order the data update scripts by the latest script that ran

* feat: when the script has succeeded reset error to nil

* feat: create a model function that will allow the script to be force run

* chore: oops remove the functions form the worker to the model so it can be re-used

* feat: create an endpoint that will call the model method force run when we hit the api by clicking the button

* feat: ensure that the we add an ajax call that calls the controller endpoint

* chore: remove newline

* refactor: change to a more Restful route

* refactor: use Stimulus

* fix: move the code from the model back into the worker so that we can reuse it in the controller

* feat:call the worker in the force_run action and create a show route

* feat: very first draft of using a polling mechanism on the show method after we kickoff the sidekiq job (still need to error handle)

* chore: some syntax changes and unused variables

* fix: call the method correctly with the paramters

* feat: do some error handling

* feat: error handing on the frontend

* chore: use e instead of err

* refactor: just pass the id instead of the whole script

* chore: remove the button

* fix: allow the id to be passed

* feat: handle errors better

* feat: limit the filename column width

* refactor: use a common function to set the banner error

* test: add a test fro rerun button

* test: v1 of the data update script request

* test: write some specs for the js controller

* Update app/controllers/admin/data_update_scripts_controller.rb

Co-authored-by: Michael Kohl <me@citizen428.net>

* tests: update the data worker spec

* chore: clean up the js controller and its tests

* chore: remove whitespaces

* chore: swap the functions based on the controller

* chore: updates to the UI

* chore: remove the standard error catch

* chore: update the alert and error messages

* chore: remove test for error handling for sidekiq run

Co-authored-by: Michael Kohl <me@citizen428.net>
2021-02-02 17:24:39 +02:00
rhymes
304b8f0e13
Add Message to the postgres.db_table_size dataset on Datadog (#12516) 2021-02-01 12:41:12 -05:00
Mac Siri
5d86877e02
Limit user.touch usage in RepoSyncWorker (#12458) 2021-01-28 12:35:58 -05:00
Kirk Haines
ed74f2f245
Abstract DatadogStatsClient to ForemStatsClient (#12304)
* This change abstracts the DatadogStatsClient into a ForemStatsClient.
The purpose of this abstraction is to set the foundation for a subsequent PR that will allow one to use New Relic for recording Forem stats, instead of Datadog, if there is a New Relic configuration found.
This specific change creates an abstraction layer that can be built upon, without changing any actual default behavior. All specs still pass.

* Use delegate instead of explicit methods.

* Delegate instead of explicit methods.

* Fix the error.

* Refactor according to the suggestions in the comments.

* Ooops.  Stats work better when all of the code is committed.

* Removing the alias of count to increment since that was done in error.
2021-01-27 11:25:44 -05:00
Ridhwana
2dad0c42f8
Save Data Update Script errors to database and show them on http://localhost:3000/admin/data_update_scripts (#12348)
* feat: add an error column to the data update script

* feat: save the error to the error field

* feat: save the error when the script fails

* feat: show the script error on the data update script page

* chore: pass the error to mark_as_failed instead of having its own function

* refactor: use presence

* test: ensure that we test an error case of a data update script

* chore: rename errorneous to failing

* test: update some specs, working on the others

* chore: update tests now that there are two files

* chore: change error from a string to a text to allow for more char
2021-01-21 14:48:34 +01:00
Alex
dfeccf6c11
Move MailchimpBot to Mailchimp::Bot service (#12205)
* Move MailchimpBot to Mailchimp::Bot service

* Update specs

* Put back old Mailchimp Bot
2021-01-12 10:28:48 -05:00
rhymes
e7f9735354
Remove RssReader (#12169)
* Add script to remove feature flag

* Remove :feeds_import feature flag and RssReader and related classes

* Change Feeds::ImportArticlesWorker signature to support Sidekiq Cron serialization

* Replace RssReader::Assembler with Feeds::AssembleArticleMarkdown

* Removing Assembler
2021-01-08 11:45:15 -05:00
rhymes
4aaeb3e955
Feeds::Import: only import feeds that haven't been recently updated (#12126)
* Add earlier_than to Feeds::Import

* Ignore feeds that have been imported in the last 4 hours

* Make earlier_than mandatory in Feeds::ImportArticlesWorker
2021-01-06 16:58:42 +01:00
Ben Halpern
0ec2a40fe0
Test variants for which posts are fetched on initial "hot articles" home feed query. (#11982)
* Declare winner in update points worker

* Remove tests

* Test variants for which posts are fetched on home feed

* Add field test

* Rewrite field tests

* Change user to @user

* Fix tests

* Fix user thing

* Check for existence of user before setting experiment

* Fix experiment -> @experiment

* Change spec

* Fix tests

* Fix tests

* Fix tests

* Fix rubocop issues

* Remove unneeded offset value

* Fix up specs

* Final adjustments

* Add comments

* Update app/services/articles/feeds/large_forem_experimental.rb

Co-authored-by: Molly Struve <mollylbs@gmail.com>

* Update app/services/articles/feeds/large_forem_experimental.rb

Co-authored-by: Molly Struve <mollylbs@gmail.com>

* Update app/services/articles/feeds/large_forem_experimental.rb

Co-authored-by: Fernando Valverde <fdov88@gmail.com>

* Update app/services/articles/feeds/large_forem_experimental.rb

Co-authored-by: Fernando Valverde <fdov88@gmail.com>

* Better initial user query

Co-authored-by: Molly Struve <mollylbs@gmail.com>
Co-authored-by: Fernando Valverde <fdov88@gmail.com>
2021-01-05 12:29:46 -05:00
Molly Struve
f4315489b2
Refactor:Use dev_to? to Check for DEV (#12083) 2020-12-30 14:34:00 -06:00
Molly Struve
b94c95f164
Bug Fix:Update dev.to check to use DEV Community (#12082) 2020-12-30 12:41:51 -06:00
Alex
17b2ff63f2
Cleanup bust_user (#12066) 2020-12-30 10:07:42 -05:00
Alex
f70af66258
Update bust to EdgeCache::Bust (#12052)
* Update bust to EdgeCache::Bust

* Update specs

* Fix more specs

* Fix more specs :)
2020-12-29 09:53:15 +01:00
Alex
2a345d5214
Update bust_listings to service (#12050) 2020-12-29 09:52:34 +01:00
Alex
c7ae198179
Update bust_podcast to EdgeCache::BustPodcast (#12047) 2020-12-29 09:49:59 +01:00
Alex
6195d6cf36
Update bust_podcast_episode to EdgeCache::BustPodcastEpisode (#12049)
* Update bust_podcast_episode to service

* Update argument checks
2020-12-29 09:49:13 +01:00
Alex
e0d0fd6c96
Update bust_organization to EdgeCache::BustOrganization (#12048)
* Update bust_organization to service

* Add spec for nil slug

* Update argument check
2020-12-29 09:48:36 +01:00
Michael Kohl
63cf95d811
Split BadgeRewarder into multiple services (#11902) 2020-12-23 09:55:06 -05:00
Alex
d0c70e72d7
Update bust_tag to EdgeCache::BustTag (#11953) 2020-12-21 14:43:42 -05:00
Alex
e9886d106c
Update bust_events to EdgeCache::BustEvents (#11955) 2020-12-21 14:43:28 -05:00
Ben Halpern
5cd947336b
Declare winner in update points worker (#11974)
* Declare winner in update points worker

* Remove tests

* Fix finalized_points style issue
2020-12-21 12:59:54 -05:00
Alex
f31470d8e5
Update .bust_page to EdgeCache::BustPage (#11952) 2020-12-18 11:21:54 -05:00
Anna Buianova
1079c4e402
Removed slack notification for admins when user is deleted (#11929) 2020-12-17 20:31:22 +03:00
Alex
d03d03bc39
Update bust_article to BustArticle (#11931)
* Update bust_article to BustArticle

* Fix spec
2020-12-17 12:20:44 -05:00
Anna Buianova
5430448305
Confirmations that users GDPR data was deleted (#11039)
* Display gdpr delete requests for deleted users in admin

* Added a test for destroy a gdpr request

* Reorganized gdpr requests menu

* Update schema version

* Added a link to the flash notice after user delete by admin

* Fix namespace for tests

* Aligh schema.rb with master

* Added a missing newline

* Removed unused partial

* Fix typo

Co-authored-by: Michael Kohl <me@citizen428.net>

* Replaced string with symbols for AR

Co-authored-by: Michael Kohl <me@citizen428.net>

* Added an AuditLog record on gdpr delete confirmation

* Make email and user_id required in Users::GdprDeleteRequest

* Checked out package.json from master

* Fix Gdpr => GDPR

* Added missing space

* Remove unneeded freeze

Co-authored-by: Michael Kohl <me@citizen428.net>
2020-12-14 12:49:45 +03:00
Michael Kohl
59e2f5a025
Remove app/labor/color_from_image.rb (#11825)
* Remove app/labor/color_from_image.rb

* Remove miro gem

* Delete TODO
2020-12-10 10:08:06 +07:00
Ben Halpern
31b83a4511
Remove concept of "ongoing" field tests and clarify a/b test instructions/expectations. (#11734)
* Remove concept of 'ongoing' field tests

* Fix specs

* Feeds spec

* Fix spec

* Fix style issues

* More details on field tests
2020-12-07 13:52:54 -05:00
Ben Halpern
12efe40e63
Add a/b test for implicit follow points (#11675)
* Add a/b test for implicit follow points

* Add field test specs

* Add a/b testing docs

* Fix tests

* Fix tests

* Update docs/technical-overview/ab_testing.md

Co-authored-by: Molly Struve <mollylbs@gmail.com>

* Add comment

* Update docs/technical-overview/ab_testing.md

Co-authored-by: Jacob Herrington <jacobherringtondeveloper@gmail.com>

* Update config/field_test.yml

* Update docs/technical-overview/ab_testing.md

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>

* Update docs/technical-overview/ab_testing.md

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>

Co-authored-by: Molly Struve <mollylbs@gmail.com>
Co-authored-by: Jacob Herrington <jacobherringtondeveloper@gmail.com>
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
2020-12-03 08:39:04 -05:00
Alex
7ee8523ba3
Refactor EdgeCache (#11684)
* Refactor EdgeCache

* Update specs

* Add bust comment spec

* Fix more specs

* Use const_get

* Make methods private

* Add .discussion? decorator to Article

* Change variable name to article for clarity
2020-12-02 14:20:22 -05:00
Molly Struve
fd97007364
Bug Fix:Remove nil cached_tag_lists when Collecting Tags (#11648) 2020-11-30 07:18:19 -06:00
Michael Kohl
06fd95f93f
Move LogWorkerQueueStats from labor to services (#11642) 2020-11-30 09:31:15 +07:00
Ben Halpern
d759ac4627
Add implicit points to follows to improve feed relevancy (in all algorithms) (#11513)
* Fix menu dropdown disable issue

* Initial base work

* Add real algorithm

* Add working object and test

* Fix schema

* Fix schema

* Fix schema

* Finalize

* Add update script and explict_points

* Proper implicit/explicit points

* Fix tests

* Add docs

* Fix tests

* Fix data update script test

* Fix data update script test

* Fix typo

* Improve efficiency and add more logic

* Changed to find_each

* Change to bulk insert

* Update tests and refactor logic

* Various improvements

* Fix typo

* Fix tests

* Remove extra line

* Fix tests

* Fix test typo

* Cache site-wide tag names
2020-11-25 16:09:51 -05:00
rhymes
1741e52e83
Add missing features to new feed importer (#11501)
* Add explanatory comment to how RssReaderFetchUserWorker works

* Add Feeds::ImportWorker worker

* Enable Feeds::Import for single user feed fetching behind a feature flag

* Add Feeds::ValidateUrl service

* Add feature flag to user's validate feed URL feature

* Remove todo notices

* Add feature flag to fetch_all_rss task

* Add feature flag to RssReaderWorker

* Bring back short-circuit for Articles::RssReaderWorker
2020-11-24 18:11:13 +01:00
Molly Struve
6aa41bfba2
Optimization:Only Check Past Day for DataUpdateScript failures (#11448) 2020-11-17 06:39:58 -06:00
Molly Struve
60b7572f5c
Feature:Send Alerts for Failed DataUpdateScripts to Datadog (#11381) 2020-11-12 10:59:54 -06:00
rhymes
47624ddfcd
Add users param to Feeds::Import (#11234)
* Add users param to Feeds::Import

* Fix init and specs

* Replace explicit nil check

Co-authored-by: Michael Kohl <me@citizen428.net>

Co-authored-by: Michael Kohl <me@citizen428.net>
2020-11-03 07:02:17 -06:00
rhymes
51a4a35448
Add Feeds::Import service class (#10998)
* Add the parallel gem

* Add prototype version of Feeds::Import with parallel URL fetching and parsing

* Tune Feeds::Import and add rake task for local tests

* Add rough measurer tool

* Add specs

* Apply suggestions by @citizen428

* Replace silence with silent

* No need for eager loading

* Add temporary Articles::DevFeedsImportWorker

* Remove temporary rake task

* Add basic error handling, copied from RssReader

* Fix error handling

* Remove temporary measuring rake task

* Remove logging added for development purposes

* Add info and error logging on error level
2020-10-30 15:01:44 +01:00