* Refactoring Spam Handler
There's considerable repeated logic between checking spam for an article
and spam for a comment and user.
This attempts to send things through channels that are similar and close
in organization.
* Fixing broken spec
* Fixing spec around recent user
* Update app/models/reaction.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Update app/models/reaction.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Consolidating new user query logic
Prior to this commit there were two separate queries around new user
logic. With this commit, we're changing the logic to repurpose a site
wide setting.
* Generalizing a previously specific message
* Fixing method name
As part of a recommended refactor, I extracted a method, then renamed
it. I failed to account for that renaming.
This commit fixes that.
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Favoring dependent: :delete_all over :destroy
The `dependent: :destroy` callback is slower than `dependent: :delete`
and `dependent: :delete_all`. We need only favor the `dependent:
:destroy` when there's callbacks that happen.
In the case of :destroy, ActiveRecord instantiates each object and then
runs destroy. Whereas in the case of :delete, ActiveRecord issues a SQL
command to delete the related files.
It is often "safer" to use :destroy, as it guarantees that you'll
instantiate the record and run it's callbacks. But sometimes you have
to go with the speed of SQL.
This relates to #14140. Note there is still more to consider, but given
that we're looking at moving from :destroy on all of an article's page
views to :delete, we might buy enough time in the callback.
* Removing redundancy of article destroy
Prior to this commit, `before_destroy_actions` called the `bust_cache`
method which in turn called `touch_actor_latest_article_updated_at` but
`bust_cache` did not pass the destroying parameter. Then
`before_destroy_actions` immediately called
`touch_actor_latest_article_updated_at` with `destroying: true`.
With this change, we remove one of those duplicate calls.
* Fixing specs regarding relationships
* Fixing specs regarding relationships
* Refactoring and extending article spam behavior
Prior to this refactor, we were checking an Article's title for
spaminess. This change now checks the Article's title and
body_markdown.
In addition, the encapsulates the regular expression implementations in
favor of asking the Settings::RateLimit class to determine if the passed
in text is "spammy".
Furthermore, instead of having lots of inline logic within the article,
this refactor introduces a `Spam::ArticleHandler` class. This helps
tidy up the already busy Article specs by delegating the business logic
of Spam handling to it's own class.
There are further refactors for Comments and Users that would follow
this pattern, but this change is more important to get out the door.
Closes#15396
* Adding comments to clarify behavior of spam handling
* Adding missing expectation to spec
* Caching sum privileged reaction scores
This relates to work on improving the feed. Namely that in the current
proposal in PR #15240 I'm not accounting for how privileged users have
given feedback on an article.
With this change, I will now have a cached value on the articles table
that can be a proxy for how the privileged users have reacted.
In addition there's a small refactor that moves a constant to the
correct scoping object.
Dependent on #15283.
* Fixing method name to `exists?`
Prior to this commit, I was using `exist?` which doesn't work for
ActiveRecord::Relation objects.
* Initial work to expand detection functionality
* Finish up functionality and tests
* Fix class name
* Fix tests
* Update spec/services/articles/enrich_image_attributes_spec.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Update spec/services/articles/enrich_image_attributes_spec.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Separate followed and "anti-followed" tags
* Minor code change
* Indicate "anti-followed" tags in sidebar
* Fix UpdatePointsWorker spec
* Cries in Arel
* Use antifollow scope more selectively
* Add spec for not_cached_tagged_with_any scope
* Add explanatory comment
* Remove unused route
* Update comment
* Remove unrelated change, figure out why > 1 later
* Need more ☕
* Avoid false positives in spec
* Change handling of ingored tags in sidebar
* Add explanatory comment
* fix: artiles from the same feed can be fetched from multiple accounts
* undo unintentional changes to schema.rb
* modify: article_spec and import_spec tests
* fix: failing test
* modify: import_spec:213 for user.setting
* fix: failing test import_spec:213
* add: unique index on articles canonical_url where published
* revert: schema.rb
* modify: unique url error message to include admin email
* modify: custom error message
* modify: .update to .update! (bang)
* Create DiscussionLocks
* Fix specs
* Update nullify_blank_notes_and_reason
* Update before_validation call
* Updated DiscussionLockPolicy for clarity
* Move permitted_attributes to a constant
* Update route
* Apply suggestions from code review for frontend
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
* Add title tags
* Wrap unlock confirm in main element
* Wrap flash messages up in div
* Actually fix title tags
* Hide comment reply button when discussion is locked
* Add E2E tests
* Try to fix E2E tests
* Cypress...you work locally but not in CI...why!?
* PR feedback
* Update E2E tests
* More E2E updates 😭
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
* Add inline animated image detection
* Do not detect animation during preview
* Promote FastImage as a dependency
* Add Articles::DetectAnimatedImages service
* Restore previous parsing
* Add and use Articles::DetectAnimatedImagesWorker
* Remove obsolete poc
* Properly detect images in the after commit callback
* Simplify logic
* Use a second guard clause
* Fix parsing of relative paths and add tests
* Have Articles::DetectAnimatedImages correctly detect relative images uploaded locally
* Change Articles::DetectAnimatedImagesWorker priority to medium
* Only one & required
* Rename SiteConfig
* More renaming
* Update spec
* Update mandatory settings mapping
* More renaming
* e2e test fixes
* You have a rename, and you have a rename
* Spec fix
* More changes
* Temporarily disable specs
* After-merge update
* Undo rename for migration
* undo rename of DUS
* Fix DUS
* Fix merge problem
* Remove redundant DUS
* Fix specs
* Remove unused code
* Change wrong class name
* More cleanup
* Re-add missing values to constant
* Fix constant
* Fix spec
* Remove obsolete fields
* Add accidentally removed field
* Update spec
* Move methods from Settings::General to ForemInstance
* Remove unneeded model
* Change mentions of 'site config'
* Initial work for @-mention notifications from posts
* Revert article.published changes to article updater, add clarifying comments
* Extract article preview into reusable partial for notification views
* Clean up Article Updater
* Address + remove some FIXMEs
* Add a whole buncha specs for @-mention functionality in posts YAY
* Refactor create all spec to use shared examples, add clarifying comments
* Add guard clause to create all service
* Update new mention and notifiable action specs
* Some additional cleanup
* Add specs + shared examples to SendEmailNotificationWorker spec
* Use aggregate_failures where applicable
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Cleanup and address code review comments
* Add MentionDecorator + relevant specs
* Address comments/issues flagged by @rhymes
* Optimize plucking user_ids when checking for article followers
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Update format specifiers in readable_publish_date
* Trigger Travis manually
* Update format specifiers in readable_publish_date
* Update format specifiers in readable_publish_date
Co-authored-by: rhymes <rhymes@hey.com>
* Use trigger and tsvector column to speed up reading list search
* Add organization destroy spec and todo note
* Fix failing data update script due to not null constraint
* Remove the leading anchor in the trigger regexp
* Fix reading list specs
* Address feedback
* Move authors followers out of Notifications::NotifiableAction::Send and into Article model
* Rename authors_followers to followers
* Update followers method in service, oops
* Update app/services/notifications/notifiable_action/send.rb
Co-authored-by: Michael Kohl <citizen428@dev.to>
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Rename banned and comment_banned roles
* Add data update script to rename roles containing 'ban'
* Add named error for Suspended users
* Update unidiomatic method names
* Rename misc banned to suspended
* Apply suggestions from code review
Co-authored-by: Michael Kohl <me@citizen428.net>
* Add unit tests for suspended methods
This commit also adds TODO comments for removing banned and
comment_banned from the codebase after data update scripts have
successfully run on all of our Forems.
Co-authored-by: Michael Kohl <me@citizen428.net>
* Limit mentions in articles
* Bump MAX_USER_MENTION_LIVE_AT to April 7th, 2021 UTC time
* Use stubbed constants in article and comment specs
* Add TODOs around extracting MAX_USER_MENTIONS into constant
* User#followed_articles->Article.cached_tagged_with
This should improve performance on StoriesController#index
* Make cached_* work like their tagged_with cousins
* Swap to `cached_tagged_with_any`
This was using `tagged_with(tags, any: true)` before so this commit
swaps to using a scope that aligns with that method.
* Allow for searching by Tag models
* do the thing
* abracadabra run the build
* Add comment explaining Postgres regex anchors
* Optimize Article.published scope
The index on `articles.published` is not being invoked due to lack of
diversity in the values in that column (it only has 2 possible values
because it's a boolean).
The index on `articles.published_at` is extremely diverse and will be
invoked any time it can reduce the scope of a query by an order of
magnitude or more.
On DEV, this does not change the row count for the scope:
irb(main):001:0> Article.where(published: true).count == Article.where(published: true).where(Arel.sql("published_at < now()")).count
=> true
The query plan for the `Articles::Feeds::LargeForemExperimental` service
invoked in the `Stories::FeedController#show` endpoint goes from this:
Planning Time: 0.271 ms
Execution Time: 329.258 ms
to this:
Planning Time: 0.330 ms
Execution Time: 0.468 ms
This is a reduction in query time of 99.7%
* Set published_at in articles factory
An article that is published should always have a `published_at`
timestamp
* Use Time.current for Zonebie
* Add clarifying comment to Article.published scope
* Generate timestamp in Ruby instead of SQL
I think using transactions for specs was interfering with comparing
timestamps generated in SQL, so this commit generates the timestamp in
Ruby.
* Move published_at outside of the transient block
This was causing articles to be marked as `published_at` right now even
if `published_at` was specified in the `FactoryBot.create` call.
* published_at: nil is no longer the factory default
* published_at is no longer nil by default
* published_at is no longer nil by default
* We didn't actually want to clear this published_at
This was intentionally left out to show that published_at does not get
cleared when we flip published true->false.
Fixes#13008
Add a test case for blackbox stability
This failed as expected on master and passed on this branch, when a
reaction (which would change the article's calculated score during
update) is added, the hotness should be stable, and not modified
during the second call to `#update_score`.
When https://github.com/forem/forem/pull/11268 we didn't test it thoroughly and
thus didn't detect that it wasn't working correctly and that it
resulted in an empty string in the `<description>` field.
* Replace #active_help scope with #active_help class method
* WIP: Add tests around #active_help class method
* Refactor #active_help and update article_spec.rb tests to pass
* Replaces .active_help class method with .active_help scope
- Reverts change from scope to class method, back to scope
- Updates to use newer AR query syntax using endless ranges
- Refactors scope to be more concise
* Allow any admin to publish welcome thread
* Update app/models/article.rb
Co-authored-by: rhymes <rhymes@hey.com>
* Fix specs by making user for welcome thread an admin
* revert spec changes and update code to include either admin, super_admin, or site mascot
* add specs for scope
* Modify tests and include mascot user
* Fix test
Co-authored-by: rhymes <rhymes@hey.com>
Co-authored-by: mstruve <mollylbs@gmail.com>
* Adds a lower() query to the DB in Stories#show
* Restructure spec
* bypass article callbacks in order to test edge case
* Fixes spec
* Change the approach to unsure downcase path in Article model
* Move .downcase call to set_caches before_save callback
* Adds data update script for fixing offending Article paths
* Moves model tests to correct block
* Apply suggestions from code review
Co-authored-by: Vaidehi Joshi <vaidehi.sj@gmail.com>
Co-authored-by: Vaidehi Joshi <vaidehi.sj@gmail.com>
* Implement changes
* display "Suspended" if user role.name is banned
* Correct equality symbol
Co-authored-by: Michael Kohl <me@citizen428.net>
Co-authored-by: Michael Kohl <me@citizen428.net>
* Replaces second_user_id and third_user_id with co_author_ids on the Articles table
* Adjusts migration that adds co_author_ids to also add array: true
* Replaces second_user_id and third_user_id with co_author_ids in all applicable files
- Use co_author_ids in _individual_article view
- Use co_author_ids in articles and stories controllers
- Use co_author_ids in article and podcast_episode models
- Use co_author_ids in article_dashboard
- Use co_author_ids in comments_helper
- Use co_author_ids in articles_spec
* Replaces second_user_id and third_user_id with co_author_ids in article.rb model
* Remove unused #assign_co_authors method from the Stories::Controller
* Add #co_authors and #co_author_name_and_path to article_decorator.rb
* Adds a descriptive form field for co_author_ids for Admins in /admin/articles
* Adds a conditional in article/show.html.erb to render co_author names on an Article in a human-readable way
* Replaces get_co_author_name_and_path with proper method name, co_author_name_and_path in show.html.erb
* Adds latest schema to (hopefully) resolve Travis conflicts
* Adds the safe operator to show.html.erb
* Adjusts articles_spec.rb to use an array in the first test
* Replaces unless &.empty? with if ./present? in articles/show.html.erb
* Replaces second_user_id with co_author_ids in stories_show_spec.rb test
* Replaces elsif with else in articles/show.html.erb
* Cleans up show.html.erb by removing conditional in favor of decorator method
* Refactors splitting of params in Articles::Controller and optimizes query in #co_authors
* Reverts removal of second_user_id and third_user_id and migration file
* Adds a data_update script to update co_author_ids with existing second and third user_ids
* Adds validations to co_authors and flash_messages to indicate whether an update was successful or not
- Adds to methods to article.rb to validate the IDs of co_authors and authors
- Adds flash messages to the Admin::Articles::Controller and a redirect to the show page
- Removes the JS highlighting upon submit when updating an Article in Admin
- Refactors #update action in Admin::Articles::Controller
* Adds tests to article_spec.rb around co_author_id validations in article.rb
* Adjusts #validate_co_authors_must_not_be_the_same to use .include? instead
* Uses Field::Select.with_options in article_dashboard.rb to properly display co_author_ids
* Reverts removal of assigning co_author_ids in the Stories::Controller
* Adjusts error message and adjusts return logic in article.rb (thanks, Fernando!)
* Fixes failing article_spec.rb test that checks for co_author_ids uniqueness
* Adds a default array to co_author_ids and checks if they are .blank?
* Refactor data_update script to use a single SQL statement (thank you, Michael)
* Preserve array order of co_author_ids in article_decorator.rb
* Add db file for default: []
* Add validation to fix bug related to text inputs and invalid users when adding co_authors
* Adds tests to ensure that co_author_ids are both an integer and an integer > 0
* Updates admin/articles_spec.rb to default [] instead of nil
* Adjusts validations in article.rb to be DRY-er and more consistent
* Consolidates validations further
* Refactors validations in article.rb to use procs
* Refactors data_update script to remove nil values from array
