The download app broadcast (in the Generator) returns early if the
user was created less than 7 days ago. It seems counterproductive to
filter out _only_ users created in the last 7 days.
Add one day to the query range, so users created less than 8 days ago,
but more than 7 days ago, receive the final message.
* Name the created after timestamp and remove temporary variables
The logic used to calculate the live at time, the week ago time, then
compare them to find the more recent value.
Move all of this logic to a method (there's a guard to exit if
the setting is nil, so this should be safe).
* Mark method private
This is not part of the public api (for workers, this should be
`#perform` alone). Mark it private
* Proposing a new feed experiment
This commit involves tweaking a few subtle aspects of the challenger:
1) Remove the sort by published date on the relevance score. The
articles will now be listed in the order of perceived relevance.
2) Disable a few levers that did little. There really aren't any
`articles.featured = true` articles in the database.
3) Gently increase the weight of each comment in a linear manner.
4) Give a little more weight to posts that don't have followed tags.
In addition, it involves renaming the conversions to better convey their
implementation. This renaming helps create more descriptive labels for
the test results at `/admin/abtests`.
This commit also adds logic to repurpose the AbExperiment feed_strategy
logic; I envision adjusting the weighted feed strategy levers with some
frequency.
Per discussions, I'm also disabling the "not logged in" feed testing.
We'll use the LargeForemExperimental for this.
* Flipping attribute name to positive
Mentally "not disabled" is harder to parse than "enabled". This change
helps with setting an optimistic attribute.
* Bump for travis
* Bump for travis
* Removing file committed by mistake
* Extracting method
* Load article before creating a page view for an invalid one
This should resolve a validation error in PageView.create! when the
article does not exist (perhaps it was deleted, or the user suspended,
or it was invalid data sent from the client).
This had been happening dozens of times per day for the last 10
months.
* Use an intention revealing symbol instead of calculated id
Since we only require that find_by not find anything, pass a symbol
that's not going to be the id for any article under any conditions.
As an added benefit, this provides a clear indication of the purpose
of the symbol, without needing to mentally confirm that
```sql
SELECT * FROM articles
WHERE id IN (
SELECT (1 + MAX(id)) FROM articles
) LIMIT 1
```
actually never gives any articles back.
* Use the tag_list method, rather than the cached_tag_list attribute
This is a follow on to #15456 and #15637
Saving the article in #15637 was a work-around to force the article to
have a valid cached tag list. However, we can rely on the tag_list
method to find the associated tags.
Update the worker to use tag_list (rather than the decorator's
cached_tag_list.array) and flatten the tag list rather than flat
mapping String#split over the non-nil cached_tag_lists.
* Remove unused join on tags
The query logging showed each article's tags were loaded twice in a
row (select by id, immediately followed by select by id). Removing the
join removed this behavior. I suspect mapping `#tag_list` over
articles is ignoring the loaded tags.
Performance (in my local environment) improved about 30% when this join was removed.
* Guarding against missing experiments
This is a short-circuit to ensure we don't throw exceptions in the
worker. In #15240 we introduce an AbExperiment module that we could use
to insulate against FieldTest implementations.
* Update app/workers/users/record_field_test_event_worker.rb
Co-authored-by: Jamie Gaskins <jamie@forem.com>
* Favoring guard condition over coercion
Co-authored-by: Jamie Gaskins <jamie@forem.com>
* Remove Connect
* Remove more Connect specs
* Remove a lot more Connect code
* 🚮
* It all has to go
* Explicitly add httpclient
* Update application layout
* Remove messages association from User
* Start fixing specs
* reintroduce util function and refactor references
* Remove Connect Cypress test
* Fix more specs
* Remove Connect from listings
* Ignore contact_via_connect column on listings
* Remove contact_via_connect usages
* Ignore mod_chat_channel_id on tags
* Drop Connect tables
* Remove email_connect_messages from user notification settings
* Re-add httpclient 2.8.3
This was mistakenly removed as a merge conflict
* Don't need to exclude removed chat channel file
* Remove unneeded style for chat channels
* Remove unneeded channel list prop type
* Remove chat channels index/connect-link from getPageEntries
* Re-add comment from httpclient in Gemfile
* Remove connect references from mailers
Tag Moderators no longer have a chat channel
No longer will users be notified about new messages (there won't be
any)
No longer will users be notified about channel invites (you can't
invite anyone anymore)
* Don't configure Pusher and remove PUSHER_* from .env_sample
since it's removed from gemfile, the Pusher constant will not resolve, if this is
configured in the environment variables we'll fail to boot.
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
Co-authored-by: Dan Uber <dan@forem.com>
* Add PGHero for more insights into the DB
Andrew Kane's Ruby gems will continue being added until morale improves!
* Add linux x86_64 protobuf
* Fix discrepancy between Gemfile and Gemfile.lock
No idea how this happened, but it happened when merging `main` back into
this branch.
Co-authored-by: Dan Uber <dan@forem.com>
* Initial work to expand detection functionality
* Finish up functionality and tests
* Fix class name
* Fix tests
* Update spec/services/articles/enrich_image_attributes_spec.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Update spec/services/articles/enrich_image_attributes_spec.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
Turns out this caused a *different* problem. It would work for the first
delete batch in `BulkSqlDelete`, but subsequent `DELETE` queries would
then get their `statement_timeout`s reset to the global setting. The
issue was that `BulkSqlDelete` uses methods that don't respect nested
transactions, and we were already resetting the value of the setting to
its previous value anyway, so there was really no reason to run it
inside a transaction.
The only failure mode I know of here is that the `ensure` block breaks
because `connection` is severed. In that case, the value is reset anyway
when the connection pool replaces the connection so the failure does not
propagate.
* Allow Emails::RemoveOldEmailsWorker to run longer
We set the SQL statement timeout based on the STATEMENT_TIMEOUT
environment variable (defaulting to 2.5 seconds in production), but not
all SQL queries are created equal. Since some may take longer out of
necessity, this PR introduces a `Model.with_statement_timeout` method
that allows you to change the SQL statement timeout only for a given
block.
* EmailMessage doesn't inherit ApplicationRecord
It inherits from `Ahoy::Message`, which inherits directly from
`ActiveRecord::Base`.
[3] pry(main)> EmailMessage.ancestors
=> [EmailMessage(id: integer, clicked_at: datetime, content: text, feedback_message_id: integer, mailer: string, sent_at: datetime, subject: text, to: text, token: string, user_id: integer, user_type: string, utm_campaign: string, utm_content: string, utm_medium: string, utm_source: string, utm_term: string),
EmailMessage::GeneratedAssociationMethods,
EmailMessage::GeneratedAttributeMethods,
Ahoy::Message(id: integer, clicked_at: datetime, content: text, feedback_message_id: integer, mailer: string, sent_at: datetime, subject: text, to: text, token: string, user_id: integer, user_type: string, utm_campaign: string, utm_content: string, utm_medium: string, utm_source: string, utm_term: string),
Kaminari::ConfigurationMethods,
Kaminari::ActiveRecordModelExtension,
Ahoy::Message::GeneratedAssociationMethods,
Ahoy::Message::GeneratedAttributeMethods,
Bullet::SaveWithBulletSupport,
ActiveRecord::Base,
...
* Use milliseconds to match STATEMENT_TIMEOUT units
* show top tags when search is empty
* add cypress tests
* nudge travis
* tweak to e2e test
* show a 'top tags' heading
* only suggest tags that are supported
* Add supported scope to Tag
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Separate followed and "anti-followed" tags
* Minor code change
* Indicate "anti-followed" tags in sidebar
* Fix UpdatePointsWorker spec
* Cries in Arel
* Use antifollow scope more selectively
* Add spec for not_cached_tagged_with_any scope
* Add explanatory comment
* Remove unused route
* Update comment
* Remove unrelated change, figure out why > 1 later
* Need more ☕
* Avoid false positives in spec
* Change handling of ingored tags in sidebar
* Add explanatory comment
* Boost priority of queue stats worker
We were missing stats for hours at a time this week because this worker
was blocked behind other jobs. We *especially* need stats when Sidekiq
is saturated, so this commit puts these stats at the highest priority.
The tradeoff should be minimal. These jobs take 6-30ms once every 10
minutes.
* Change queue in worker spec, as well
Touching the GithubRepo invokes GithubRepo#clear_caches, so calling
`update` and then `touch` calls `clear_caches` *twice*. This pulls it
back to calling it once.
* schema file undelete description
* feat: v1 of the script
* Flesh out remaining enums under their categories
* complete UsersSettings data update script
* complete DUS for relevant attributes in users and profiles tables
* complete DUS for users_notification_settings
* alphabetize user_settings sql file
* safeguard against null values for "null: false" settings
* Set up actual UsersSettings DUS and specs files
* fix broken DUS script
* complete specs for UsersSetting DUS
* Address QA of specs
* complete specs for users_notification_settings DUS
* fix the typos (thanks Julianna!)
* begin implementation
* still building
* add missing attribute "email_membership_newsletter"
* complete sync code (except race condition for user profile)
* complete implementation, remains tests
* Address PR review and fix Travis fails
* remove superfluous Profile.new
* fix travis fails
* feat: update the users_notification_setting attributes from the user model
* feat: use the config fonts enums to display the fonts
* feat: loop through the keys
* fix profile = nil blowing up; add specs for notification_setting model
* remove unneeded spec
* remove feed validation until after sync code removed; fixes feed_import spec failures
* remove spec associated with feed_url validation in user_setting model
* fix failing spec 😅
* add TODO
* feat: set the user settings in the user controller and use it in the customization form
* feat: move some update logic to the users settings controller thats being used from customization
* feat: show the updated values form the users_settingd and not the user instance
* Generalize redirect back to current tab
* still trying to reflect changed theme upon refresh
* customizations take effect on refresh
* remove 'with_feed' scope from user model
Co-authored-by: Jamie Gaskins <jamie@forem.com>
* start with takeover for fields previously in profiles table
* Takeover code for `publishing_from_rss` section in Settings (#13914)
* implement takeover code part 1
* implement takeover code
* fix feed fetch
* need rhymes help
* complete implementation; specs pending
* fix STUPID omission that caused so many headaches 😫
* implement profile fields pointing to users_settings 🎉
* run migrations
* implement inbox type & guidelines takeover code; specs pending (#13911)
* Point changes in notification settings to `users_notification_settings` table (#13910)
* implement takeover code; remains specs
* address PR feedback; remove related sync code
* address PR review feedback
* need help with routing and specs
* address pr review
* addressing pr review
* Treat implementation edge cases and omissions 😅
* fix uncommented comment
* fixing implementation cases
* address more PR review feedback
* fixing notifications use-cases
* refactor settings controller
* more pr review changes
* solving bugs
* fix broken onboarding
* handle eperience_level calls
* more fixes
* remove unneeded mappings
* add To-dos for quety updates
* remove done TODO
* purge done TODOs
* update notification_settings-related queries
* start fixing specs
* fixing specs
* fix notification and lrg_forem specs
* fixing broken specs
* still fixing
* fix line dif and remove reloads from user.rb
* run specs
* silence bullet and other fixes
* remove setting migration scripts and specs, fix more settings for specs
* handle missing user for article builder and fix notification specs
* fix some final controller specs and re-add incorrectly removed specs
* remove deprecated data update scripts and related workers, put travis back
* refactor admin tags mods controller, write/move specs for users notifications settings controller
* schema cleanup and other small refactors for consistency
* set field we can invalidate in spec via active record instead of at the db level
* remove I think an uneccessary hook call from subscribe_to_mailchimp_newsletter
* use bnefore_create to setup settings, please dont blow up the test suite
* mailchimp bot fix
* remove decorator in favor of single model method
Co-authored-by: Arit Amana <msarit@gmail.com>
Co-authored-by: Ridhwana <ridhwana.khan16@gmail.com>
Co-authored-by: Arit Amana <32520970+msarit@users.noreply.github.com>
Co-authored-by: Jamie Gaskins <jamie@forem.com>
* Add inline animated image detection
* Do not detect animation during preview
* Promote FastImage as a dependency
* Add Articles::DetectAnimatedImages service
* Restore previous parsing
* Add and use Articles::DetectAnimatedImagesWorker
* Remove obsolete poc
* Properly detect images in the after commit callback
* Simplify logic
* Use a second guard clause
* Fix parsing of relative paths and add tests
* Have Articles::DetectAnimatedImages correctly detect relative images uploaded locally
* Change Articles::DetectAnimatedImagesWorker priority to medium
* Only one & required
* Put the user id filter inside the CTEs
One thing I forgot about Postgres CTEs is that they materialize the
entire CTE and the top-level query filters _that_.
* Use the correct column name
* Run user setting DUS per-user via Sidekiq
Doing it as a single query trips the query timeout beyond a certain
quantity of users.
* Run DataUpdateScripts::MigrateRelevantFieldsFromUsersToUsersNotificationSettings asynchronously
* Rename SiteConfig
* More renaming
* Update spec
* Update mandatory settings mapping
* More renaming
* e2e test fixes
* You have a rename, and you have a rename
* Spec fix
* More changes
* Temporarily disable specs
* After-merge update
* Undo rename for migration
* undo rename of DUS
* Fix DUS
* Fix merge problem
* Remove redundant DUS
* Fix specs
* Remove unused code
* Change wrong class name
* More cleanup
* Re-add missing values to constant
* Fix constant
* Fix spec
* Remove obsolete fields
* Add accidentally removed field
* Update spec
* Move methods from Settings::General to ForemInstance
* Remove unneeded model
* Change mentions of 'site config'
* Initial work for @-mention notifications from posts
* Revert article.published changes to article updater, add clarifying comments
* Extract article preview into reusable partial for notification views
* Clean up Article Updater
* Address + remove some FIXMEs
* Add a whole buncha specs for @-mention functionality in posts YAY
* Refactor create all spec to use shared examples, add clarifying comments
* Add guard clause to create all service
* Update new mention and notifiable action specs
* Some additional cleanup
* Add specs + shared examples to SendEmailNotificationWorker spec
* Use aggregate_failures where applicable
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Cleanup and address code review comments
* Add MentionDecorator + relevant specs
* Address comments/issues flagged by @rhymes
* Optimize plucking user_ids when checking for article followers
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Handle GitHub "Repository access blocked" error
We've been seeing a lot of these errors on DEV, and it's a pretty
reasonable expectation that if we're being blocked from seeing them
using the user's OAuth access token that this isn't temporary.
* Re-raise so we don't swallow this error
* Constant name != company name???!?!?!
