Turns out this caused a *different* problem. It would work for the first
delete batch in `BulkSqlDelete`, but subsequent `DELETE` queries would
then get their `statement_timeout`s reset to the global setting. The
issue was that `BulkSqlDelete` uses methods that don't respect nested
transactions, and we were already resetting the value of the setting to
its previous value anyway, so there was really no reason to run it
inside a transaction.
The only failure mode I know of here is that the `ensure` block breaks
because `connection` is severed. In that case, the value is reset anyway
when the connection pool replaces the connection so the failure does not
propagate.
* Allow Emails::RemoveOldEmailsWorker to run longer
We set the SQL statement timeout based on the STATEMENT_TIMEOUT
environment variable (defaulting to 2.5 seconds in production), but not
all SQL queries are created equal. Since some may take longer out of
necessity, this PR introduces a `Model.with_statement_timeout` method
that allows you to change the SQL statement timeout only for a given
block.
* EmailMessage doesn't inherit ApplicationRecord
It inherits from `Ahoy::Message`, which inherits directly from
`ActiveRecord::Base`.
[3] pry(main)> EmailMessage.ancestors
=> [EmailMessage(id: integer, clicked_at: datetime, content: text, feedback_message_id: integer, mailer: string, sent_at: datetime, subject: text, to: text, token: string, user_id: integer, user_type: string, utm_campaign: string, utm_content: string, utm_medium: string, utm_source: string, utm_term: string),
EmailMessage::GeneratedAssociationMethods,
EmailMessage::GeneratedAttributeMethods,
Ahoy::Message(id: integer, clicked_at: datetime, content: text, feedback_message_id: integer, mailer: string, sent_at: datetime, subject: text, to: text, token: string, user_id: integer, user_type: string, utm_campaign: string, utm_content: string, utm_medium: string, utm_source: string, utm_term: string),
Kaminari::ConfigurationMethods,
Kaminari::ActiveRecordModelExtension,
Ahoy::Message::GeneratedAssociationMethods,
Ahoy::Message::GeneratedAttributeMethods,
Bullet::SaveWithBulletSupport,
ActiveRecord::Base,
...
* Use milliseconds to match STATEMENT_TIMEOUT units
* show top tags when search is empty
* add cypress tests
* nudge travis
* tweak to e2e test
* show a 'top tags' heading
* only suggest tags that are supported
* Add supported scope to Tag
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Separate followed and "anti-followed" tags
* Minor code change
* Indicate "anti-followed" tags in sidebar
* Fix UpdatePointsWorker spec
* Cries in Arel
* Use antifollow scope more selectively
* Add spec for not_cached_tagged_with_any scope
* Add explanatory comment
* Remove unused route
* Update comment
* Remove unrelated change, figure out why > 1 later
* Need more ☕
* Avoid false positives in spec
* Change handling of ingored tags in sidebar
* Add explanatory comment
* Boost priority of queue stats worker
We were missing stats for hours at a time this week because this worker
was blocked behind other jobs. We *especially* need stats when Sidekiq
is saturated, so this commit puts these stats at the highest priority.
The tradeoff should be minimal. These jobs take 6-30ms once every 10
minutes.
* Change queue in worker spec, as well
Touching the GithubRepo invokes GithubRepo#clear_caches, so calling
`update` and then `touch` calls `clear_caches` *twice*. This pulls it
back to calling it once.
* schema file undelete description
* feat: v1 of the script
* Flesh out remaining enums under their categories
* complete UsersSettings data update script
* complete DUS for relevant attributes in users and profiles tables
* complete DUS for users_notification_settings
* alphabetize user_settings sql file
* safeguard against null values for "null: false" settings
* Set up actual UsersSettings DUS and specs files
* fix broken DUS script
* complete specs for UsersSetting DUS
* Address QA of specs
* complete specs for users_notification_settings DUS
* fix the typos (thanks Julianna!)
* begin implementation
* still building
* add missing attribute "email_membership_newsletter"
* complete sync code (except race condition for user profile)
* complete implementation, remains tests
* Address PR review and fix Travis fails
* remove superfluous Profile.new
* fix travis fails
* feat: update the users_notification_setting attributes from the user model
* feat: use the config fonts enums to display the fonts
* feat: loop through the keys
* fix profile = nil blowing up; add specs for notification_setting model
* remove unneeded spec
* remove feed validation until after sync code removed; fixes feed_import spec failures
* remove spec associated with feed_url validation in user_setting model
* fix failing spec 😅
* add TODO
* feat: set the user settings in the user controller and use it in the customization form
* feat: move some update logic to the users settings controller thats being used from customization
* feat: show the updated values form the users_settingd and not the user instance
* Generalize redirect back to current tab
* still trying to reflect changed theme upon refresh
* customizations take effect on refresh
* remove 'with_feed' scope from user model
Co-authored-by: Jamie Gaskins <jamie@forem.com>
* start with takeover for fields previously in profiles table
* Takeover code for `publishing_from_rss` section in Settings (#13914)
* implement takeover code part 1
* implement takeover code
* fix feed fetch
* need rhymes help
* complete implementation; specs pending
* fix STUPID omission that caused so many headaches 😫
* implement profile fields pointing to users_settings 🎉
* run migrations
* implement inbox type & guidelines takeover code; specs pending (#13911)
* Point changes in notification settings to `users_notification_settings` table (#13910)
* implement takeover code; remains specs
* address PR feedback; remove related sync code
* address PR review feedback
* need help with routing and specs
* address pr review
* addressing pr review
* Treat implementation edge cases and omissions 😅
* fix uncommented comment
* fixing implementation cases
* address more PR review feedback
* fixing notifications use-cases
* refactor settings controller
* more pr review changes
* solving bugs
* fix broken onboarding
* handle eperience_level calls
* more fixes
* remove unneeded mappings
* add To-dos for quety updates
* remove done TODO
* purge done TODOs
* update notification_settings-related queries
* start fixing specs
* fixing specs
* fix notification and lrg_forem specs
* fixing broken specs
* still fixing
* fix line dif and remove reloads from user.rb
* run specs
* silence bullet and other fixes
* remove setting migration scripts and specs, fix more settings for specs
* handle missing user for article builder and fix notification specs
* fix some final controller specs and re-add incorrectly removed specs
* remove deprecated data update scripts and related workers, put travis back
* refactor admin tags mods controller, write/move specs for users notifications settings controller
* schema cleanup and other small refactors for consistency
* set field we can invalidate in spec via active record instead of at the db level
* remove I think an uneccessary hook call from subscribe_to_mailchimp_newsletter
* use bnefore_create to setup settings, please dont blow up the test suite
* mailchimp bot fix
* remove decorator in favor of single model method
Co-authored-by: Arit Amana <msarit@gmail.com>
Co-authored-by: Ridhwana <ridhwana.khan16@gmail.com>
Co-authored-by: Arit Amana <32520970+msarit@users.noreply.github.com>
Co-authored-by: Jamie Gaskins <jamie@forem.com>
* Add inline animated image detection
* Do not detect animation during preview
* Promote FastImage as a dependency
* Add Articles::DetectAnimatedImages service
* Restore previous parsing
* Add and use Articles::DetectAnimatedImagesWorker
* Remove obsolete poc
* Properly detect images in the after commit callback
* Simplify logic
* Use a second guard clause
* Fix parsing of relative paths and add tests
* Have Articles::DetectAnimatedImages correctly detect relative images uploaded locally
* Change Articles::DetectAnimatedImagesWorker priority to medium
* Only one & required
* Put the user id filter inside the CTEs
One thing I forgot about Postgres CTEs is that they materialize the
entire CTE and the top-level query filters _that_.
* Use the correct column name
* Run user setting DUS per-user via Sidekiq
Doing it as a single query trips the query timeout beyond a certain
quantity of users.
* Run DataUpdateScripts::MigrateRelevantFieldsFromUsersToUsersNotificationSettings asynchronously
* Rename SiteConfig
* More renaming
* Update spec
* Update mandatory settings mapping
* More renaming
* e2e test fixes
* You have a rename, and you have a rename
* Spec fix
* More changes
* Temporarily disable specs
* After-merge update
* Undo rename for migration
* undo rename of DUS
* Fix DUS
* Fix merge problem
* Remove redundant DUS
* Fix specs
* Remove unused code
* Change wrong class name
* More cleanup
* Re-add missing values to constant
* Fix constant
* Fix spec
* Remove obsolete fields
* Add accidentally removed field
* Update spec
* Move methods from Settings::General to ForemInstance
* Remove unneeded model
* Change mentions of 'site config'
* Initial work for @-mention notifications from posts
* Revert article.published changes to article updater, add clarifying comments
* Extract article preview into reusable partial for notification views
* Clean up Article Updater
* Address + remove some FIXMEs
* Add a whole buncha specs for @-mention functionality in posts YAY
* Refactor create all spec to use shared examples, add clarifying comments
* Add guard clause to create all service
* Update new mention and notifiable action specs
* Some additional cleanup
* Add specs + shared examples to SendEmailNotificationWorker spec
* Use aggregate_failures where applicable
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Cleanup and address code review comments
* Add MentionDecorator + relevant specs
* Address comments/issues flagged by @rhymes
* Optimize plucking user_ids when checking for article followers
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Handle GitHub "Repository access blocked" error
We've been seeing a lot of these errors on DEV, and it's a pretty
reasonable expectation that if we're being blocked from seeing them
using the user's OAuth access token that this isn't temporary.
* Re-raise so we don't swallow this error
* Constant name != company name???!?!?!
* Asynchronous organization destroy
* Notification after an organization was deleted
* Send data to Datadog and Honeybadger when org deletion failed
* Delete unused code
* Fixed Organization#destroyable? and organization delete specs
* Reorganized organization deletion specs
* Removed redundant specs
* Improved org deleted email text template
* Don't peform the org delete when a user or an org were not found
* Renamed user_id in the org delete worker
* Audit logging when deleten an organization
* Removed specs for tracking pixel and UTM params
* Changed slug for audit log on org delete
* Fixed schema.rb
* Move authors followers out of Notifications::NotifiableAction::Send and into Article model
* Rename authors_followers to followers
* Update followers method in service, oops
* Update app/services/notifications/notifiable_action/send.rb
Co-authored-by: Michael Kohl <citizen428@dev.to>
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Update worker calculation and disable early return
* Enable service tests
This updates the test to look at the individual article scores
changing, rather than the sum of the articles scores changing
The scores lambda was added as a way to observe this eagerly
* Correctly handle raising the score when vomit is cleared
The initial implementation loaded the articles reactions and the
user's reactions in order to rescore. This restores that activity.
It occurs to me that article.reactions.sum(:points) +
user.reactions.sum(:points) feels like something articles should know
how to do, in which case we can move this into article (#rescore!)?
and call that instead of knowing how to handle moderation specially here.
* Move logic to Article#update_score
Having realized "sum of reaction scores" was an Article concern, I
found that we have a worker to score articles that does _exactly_ what
we're doing here.
Let's use that and allow the vomit reaction presence or absence on a
user to automatically affect the article calculations.
This is at least n + 1 and maybe worse.
* Extract page view updates into workers
* Restructure code
* Update specs
* Fix and reactive 2 specs
* Update according to PR feedback
* Use create! over create
* Declare winner in feed_top_articles_query test
* Some WIP modifications to lfe
* Fix up tests and generalize
* Fix style
* Adjust tests to handle different field test scenarios
* Remove socre_randomness tests
* Fix test to fit stub
* Update app/workers/users/record_field_test_event_worker.rb
* feat: add an error column to the data update script
* feat: save the error to the error field
* feat: save the error when the script fails
* feat: show the script error on the data update script page
* chore: pass the error to mark_as_failed instead of having its own function
* refactor: use presence
* test: ensure that we test an error case of a data update script
* chore: rename errorneous to failing
* test: update some specs, working on the others
* chore: update tests now that there are two files
* chore: change error from a string to a text to allow for more char
* feat: order the data update scripts by the latest script that ran
* feat: when the script has succeeded reset error to nil
* feat: create a model function that will allow the script to be force run
* chore: oops remove the functions form the worker to the model so it can be re-used
* feat: create an endpoint that will call the model method force run when we hit the api by clicking the button
* feat: ensure that the we add an ajax call that calls the controller endpoint
* chore: remove newline
* refactor: change to a more Restful route
* refactor: use Stimulus
* fix: move the code from the model back into the worker so that we can reuse it in the controller
* feat:call the worker in the force_run action and create a show route
* feat: very first draft of using a polling mechanism on the show method after we kickoff the sidekiq job (still need to error handle)
* chore: some syntax changes and unused variables
* fix: call the method correctly with the paramters
* feat: do some error handling
* feat: error handing on the frontend
* chore: use e instead of err
* refactor: just pass the id instead of the whole script
* chore: remove the button
* fix: allow the id to be passed
* feat: handle errors better
* feat: limit the filename column width
* refactor: use a common function to set the banner error
* test: add a test fro rerun button
* test: v1 of the data update script request
* test: write some specs for the js controller
* Update app/controllers/admin/data_update_scripts_controller.rb
Co-authored-by: Michael Kohl <me@citizen428.net>
* tests: update the data worker spec
* chore: clean up the js controller and its tests
* chore: remove whitespaces
* chore: swap the functions based on the controller
* chore: updates to the UI
* chore: remove the standard error catch
* chore: update the alert and error messages
* chore: remove test for error handling for sidekiq run
Co-authored-by: Michael Kohl <me@citizen428.net>
