Commit graph

206 commits

Author SHA1 Message Date
Kirk Haines
ed74f2f245
Abstract DatadogStatsClient to ForemStatsClient (#12304)
* This change abstracts the DatadogStatsClient into a ForemStatsClient.
The purpose of this abstraction is to set the foundation for a subsequent PR that will allow one to use New Relic for recording Forem stats, instead of Datadog, if there is a New Relic configuration found.
This specific change creates an abstraction layer that can be built upon, without changing any actual default behavior. All specs still pass.

* Use delegate instead of explicit methods.

* Delegate instead of explicit methods.

* Fix the error.

* Refactor according to the suggestions in the comments.

* Ooops.  Stats work better when all of the code is committed.

* Removing the alias of count to increment since that was done in error.
2021-01-27 11:25:44 -05:00
Ridhwana
2dad0c42f8
Save Data Update Script errors to database and show them on http://localhost:3000/admin/data_update_scripts (#12348)
* feat: add an error column to the data update script

* feat: save the error to the error field

* feat: save the error when the script fails

* feat: show the script error on the data update script page

* chore: pass the error to mark_as_failed instead of having its own function

* refactor: use presence

* test: ensure that we test an error case of a data update script

* chore: rename errorneous to failing

* test: update some specs, working on the others

* chore: update tests now that there are two files

* chore: change error from a string to a text to allow for more char
2021-01-21 14:48:34 +01:00
Alex
dfeccf6c11
Move MailchimpBot to Mailchimp::Bot service (#12205)
* Move MailchimpBot to Mailchimp::Bot service

* Update specs

* Put back old Mailchimp Bot
2021-01-12 10:28:48 -05:00
rhymes
e7f9735354
Remove RssReader (#12169)
* Add script to remove feature flag

* Remove :feeds_import feature flag and RssReader and related classes

* Change Feeds::ImportArticlesWorker signature to support Sidekiq Cron serialization

* Replace RssReader::Assembler with Feeds::AssembleArticleMarkdown

* Removing Assembler
2021-01-08 11:45:15 -05:00
rhymes
4aaeb3e955
Feeds::Import: only import feeds that haven't been recently updated (#12126)
* Add earlier_than to Feeds::Import

* Ignore feeds that have been imported in the last 4 hours

* Make earlier_than mandatory in Feeds::ImportArticlesWorker
2021-01-06 16:58:42 +01:00
Ben Halpern
0ec2a40fe0
Test variants for which posts are fetched on initial "hot articles" home feed query. (#11982)
* Declare winner in update points worker

* Remove tests

* Test variants for which posts are fetched on home feed

* Add field test

* Rewrite field tests

* Change user to @user

* Fix tests

* Fix user thing

* Check for existence of user before setting experiment

* Fix experiment -> @experiment

* Change spec

* Fix tests

* Fix tests

* Fix tests

* Fix rubocop issues

* Remove unneeded offset value

* Fix up specs

* Final adjustments

* Add comments

* Update app/services/articles/feeds/large_forem_experimental.rb

Co-authored-by: Molly Struve <mollylbs@gmail.com>

* Update app/services/articles/feeds/large_forem_experimental.rb

Co-authored-by: Molly Struve <mollylbs@gmail.com>

* Update app/services/articles/feeds/large_forem_experimental.rb

Co-authored-by: Fernando Valverde <fdov88@gmail.com>

* Update app/services/articles/feeds/large_forem_experimental.rb

Co-authored-by: Fernando Valverde <fdov88@gmail.com>

* Better initial user query

Co-authored-by: Molly Struve <mollylbs@gmail.com>
Co-authored-by: Fernando Valverde <fdov88@gmail.com>
2021-01-05 12:29:46 -05:00
Molly Struve
f4315489b2
Refactor:Use dev_to? to Check for DEV (#12083) 2020-12-30 14:34:00 -06:00
Molly Struve
b94c95f164
Bug Fix:Update dev.to check to use DEV Community (#12082) 2020-12-30 12:41:51 -06:00
Alex
17b2ff63f2
Cleanup bust_user (#12066) 2020-12-30 10:07:42 -05:00
Alex
f70af66258
Update bust to EdgeCache::Bust (#12052)
* Update bust to EdgeCache::Bust

* Update specs

* Fix more specs

* Fix more specs :)
2020-12-29 09:53:15 +01:00
Alex
2a345d5214
Update bust_listings to service (#12050) 2020-12-29 09:52:34 +01:00
Alex
c7ae198179
Update bust_podcast to EdgeCache::BustPodcast (#12047) 2020-12-29 09:49:59 +01:00
Alex
6195d6cf36
Update bust_podcast_episode to EdgeCache::BustPodcastEpisode (#12049)
* Update bust_podcast_episode to service

* Update argument checks
2020-12-29 09:49:13 +01:00
Alex
e0d0fd6c96
Update bust_organization to EdgeCache::BustOrganization (#12048)
* Update bust_organization to service

* Add spec for nil slug

* Update argument check
2020-12-29 09:48:36 +01:00
Michael Kohl
63cf95d811
Split BadgeRewarder into multiple services (#11902) 2020-12-23 09:55:06 -05:00
Alex
d0c70e72d7
Update bust_tag to EdgeCache::BustTag (#11953) 2020-12-21 14:43:42 -05:00
Alex
e9886d106c
Update bust_events to EdgeCache::BustEvents (#11955) 2020-12-21 14:43:28 -05:00
Ben Halpern
5cd947336b
Declare winner in update points worker (#11974)
* Declare winner in update points worker

* Remove tests

* Fix finalized_points style issue
2020-12-21 12:59:54 -05:00
Alex
f31470d8e5
Update .bust_page to EdgeCache::BustPage (#11952) 2020-12-18 11:21:54 -05:00
Anna Buianova
1079c4e402
Removed slack notification for admins when user is deleted (#11929) 2020-12-17 20:31:22 +03:00
Alex
d03d03bc39
Update bust_article to BustArticle (#11931)
* Update bust_article to BustArticle

* Fix spec
2020-12-17 12:20:44 -05:00
Anna Buianova
5430448305
Confirmations that users GDPR data was deleted (#11039)
* Display gdpr delete requests for deleted users in admin

* Added a test for destroy a gdpr request

* Reorganized gdpr requests menu

* Update schema version

* Added a link to the flash notice after user delete by admin

* Fix namespace for tests

* Aligh schema.rb with master

* Added a missing newline

* Removed unused partial

* Fix typo

Co-authored-by: Michael Kohl <me@citizen428.net>

* Replaced string with symbols for AR

Co-authored-by: Michael Kohl <me@citizen428.net>

* Added an AuditLog record on gdpr delete confirmation

* Make email and user_id required in Users::GdprDeleteRequest

* Checked out package.json from master

* Fix Gdpr => GDPR

* Added missing space

* Remove unneeded freeze

Co-authored-by: Michael Kohl <me@citizen428.net>
2020-12-14 12:49:45 +03:00
Michael Kohl
59e2f5a025
Remove app/labor/color_from_image.rb (#11825)
* Remove app/labor/color_from_image.rb

* Remove miro gem

* Delete TODO
2020-12-10 10:08:06 +07:00
Ben Halpern
31b83a4511
Remove concept of "ongoing" field tests and clarify a/b test instructions/expectations. (#11734)
* Remove concept of 'ongoing' field tests

* Fix specs

* Feeds spec

* Fix spec

* Fix style issues

* More details on field tests
2020-12-07 13:52:54 -05:00
Ben Halpern
12efe40e63
Add a/b test for implicit follow points (#11675)
* Add a/b test for implicit follow points

* Add field test specs

* Add a/b testing docs

* Fix tests

* Fix tests

* Update docs/technical-overview/ab_testing.md

Co-authored-by: Molly Struve <mollylbs@gmail.com>

* Add comment

* Update docs/technical-overview/ab_testing.md

Co-authored-by: Jacob Herrington <jacobherringtondeveloper@gmail.com>

* Update config/field_test.yml

* Update docs/technical-overview/ab_testing.md

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>

* Update docs/technical-overview/ab_testing.md

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>

Co-authored-by: Molly Struve <mollylbs@gmail.com>
Co-authored-by: Jacob Herrington <jacobherringtondeveloper@gmail.com>
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
2020-12-03 08:39:04 -05:00
Alex
7ee8523ba3
Refactor EdgeCache (#11684)
* Refactor EdgeCache

* Update specs

* Add bust comment spec

* Fix more specs

* Use const_get

* Make methods private

* Add .discussion? decorator to Article

* Change variable name to article for clarity
2020-12-02 14:20:22 -05:00
Molly Struve
fd97007364
Bug Fix:Remove nil cached_tag_lists when Collecting Tags (#11648) 2020-11-30 07:18:19 -06:00
Michael Kohl
06fd95f93f
Move LogWorkerQueueStats from labor to services (#11642) 2020-11-30 09:31:15 +07:00
Ben Halpern
d759ac4627
Add implicit points to follows to improve feed relevancy (in all algorithms) (#11513)
* Fix menu dropdown disable issue

* Initial base work

* Add real algorithm

* Add working object and test

* Fix schema

* Fix schema

* Fix schema

* Finalize

* Add update script and explict_points

* Proper implicit/explicit points

* Fix tests

* Add docs

* Fix tests

* Fix data update script test

* Fix data update script test

* Fix typo

* Improve efficiency and add more logic

* Changed to find_each

* Change to bulk insert

* Update tests and refactor logic

* Various improvements

* Fix typo

* Fix tests

* Remove extra line

* Fix tests

* Fix test typo

* Cache site-wide tag names
2020-11-25 16:09:51 -05:00
rhymes
1741e52e83
Add missing features to new feed importer (#11501)
* Add explanatory comment to how RssReaderFetchUserWorker works

* Add Feeds::ImportWorker worker

* Enable Feeds::Import for single user feed fetching behind a feature flag

* Add Feeds::ValidateUrl service

* Add feature flag to user's validate feed URL feature

* Remove todo notices

* Add feature flag to fetch_all_rss task

* Add feature flag to RssReaderWorker

* Bring back short-circuit for Articles::RssReaderWorker
2020-11-24 18:11:13 +01:00
Molly Struve
6aa41bfba2
Optimization:Only Check Past Day for DataUpdateScript failures (#11448) 2020-11-17 06:39:58 -06:00
Molly Struve
60b7572f5c
Feature:Send Alerts for Failed DataUpdateScripts to Datadog (#11381) 2020-11-12 10:59:54 -06:00
rhymes
47624ddfcd
Add users param to Feeds::Import (#11234)
* Add users param to Feeds::Import

* Fix init and specs

* Replace explicit nil check

Co-authored-by: Michael Kohl <me@citizen428.net>

Co-authored-by: Michael Kohl <me@citizen428.net>
2020-11-03 07:02:17 -06:00
rhymes
51a4a35448
Add Feeds::Import service class (#10998)
* Add the parallel gem

* Add prototype version of Feeds::Import with parallel URL fetching and parsing

* Tune Feeds::Import and add rake task for local tests

* Add rough measurer tool

* Add specs

* Apply suggestions by @citizen428

* Replace silence with silent

* No need for eager loading

* Add temporary Articles::DevFeedsImportWorker

* Remove temporary rake task

* Add basic error handling, copied from RssReader

* Fix error handling

* Remove temporary measuring rake task

* Remove logging added for development purposes

* Add info and error logging on error level
2020-10-30 15:01:44 +01:00
Andy Zhao
b80dae1436
Update data exporter to handle admin send (#10274)
* Update data exporter to handle admin send

* Match button with everything else

* Use proper redirect path

* Stub SiteConfig definition instead of setting it

Co-authored-by: Mac Siri <krairit.siri@gmail.com>

* Removed if statement by accident oops

* Remove non-functional boolean param and pass email directly

* Use refinement to conv to boolean instead of JSON.parse

* Rename to StringToBoolean

* Use 'using' in proper scope (not in method)

* Rename to_bool to to_boolean

* Refactor if statement, thanks rhymes!

* Fix small bugs in tests

* Remove tracking for export_email b/c no @user

Co-authored-by: Mac Siri <krairit.siri@gmail.com>
2020-10-26 18:00:56 -04:00
rhymes
b6762fcc7b
[deploy] Add temporary RSS reader worker to gather monitoring data (#11037) 2020-10-23 12:02:08 -05:00
Michael Kohl
a2b55f889c
Remove Twitch integration (#10732)
* Remove Twitch integration

* Ignore columns
2020-10-08 12:44:10 +00:00
Anna Buianova
f6a7a42acc
Slack notification with reminder to delete user gdpr data (#10532) 2020-10-02 10:12:12 -04:00
rhymes
fe75dcde1c
Routine rubocop fixes (#10478) 2020-09-29 10:14:26 -04:00
Molly Struve
13fb5c4278
[deploy] Optimization:Store Podcast Ep Data in Redis, Rather than Sending as Sidekiq Argument (#10331) 2020-09-16 08:26:38 -05:00
Molly Struve
bfcba3f560
[deploy] Optimization:Use delete_by when Destroying HtmlVariant Trials and Successes (#10332) 2020-09-14 15:40:30 -05:00
rhymes
f6e24bc85b
Stop pinging sitemap tools locally (#10314) 2020-09-14 16:10:52 +02:00
Molly Struve
34593b7ec3
[deploy] Optimization:Select Article Columns for Digest Email, Limit 6, Use User Attributes (#10147) 2020-09-02 11:31:43 -05:00
Diogo Osório
a5e7b1cf60
Rely on error types to determine when a repo should be removed on the RepoSyncWorker (#10131)
* Registers the Github AcocuntSuspended & RepositoryUnavailable errors

Adds the above-mentioned errors to the list of known errors raised by
Octokit as to make possible for the rest of the application to
specifically trap those.

* Makes the RepoSyncWorker error handling rely on types

The current implementation attempted to parse the error message to
determine if the error reported by Github's API can/should be handled,
this commit changes the implementation so that the exception type is
used instead.
2020-09-01 15:01:23 -04:00
Molly Struve
667f3c9edf
[deploy] Refactor:Email Digest Sending Logic (#10107) 2020-08-31 14:28:55 -05:00
Molly Struve
0b8df875da
[deploy] Bug Fix:Destroy GithubRepos with Suspended or Blocked Access (#10103) 2020-08-31 11:32:15 -05:00
Molly Struve
4ab5bf5255
[deploy] Feature:Add Ability to Run RssReader via Sidekiq Cron, Prevent for DEV (#10066) 2020-08-28 14:57:22 -05:00
Molly Struve
7891db594f
[deploy] Bug Fix:Add send_email_digest Rake Task for DEV (#10065) 2020-08-28 14:12:10 -05:00
Molly Struve
92b9af0261
Bug Fix:Destroy Repos that We No Longer Have Credentials For (#10056) 2020-08-28 14:06:38 -05:00
Molly Struve
8a44eb9421
[deploy] Refactor:Cleanup Naming and Optimize Email Digest Code (#10054) 2020-08-28 13:36:11 -05:00