* Favoring delete_all on user relationships
Prior to this commit, several of the user's "has_many" were marked as
`depenedent: :destroy`.
In the case of :destroy, ActiveRecord instantiates each object and then
runs destroy. Whereas in the case of :delete, ActiveRecord issues a SQL
command to delete the related files.
It is often "safer" to use :destroy, as it guarantees that you'll
instantiate the record and run it's callbacks. But sometimes you have
to go with the speed of SQL.
And while not directly related to #15424 it is representative of our
callback ecosystem creating some unexpected computational loads.
Related to #15442 and #15424
* Noting models that user no longer cascade destroys
* Updating valid domain registration
Prior to this commit, our regular expression did not account for the `-`
character as valid within the domain. The `-` character cannot be the
first nor last character of the domain (e.g. `-hello.com` nor
`hello-.com` are invalid but `hell-o.com` is valid).
In addition I tidied up the default value of the
`blocked_registration_email_domains` to match it's sibling `allowed_registration_email_domains`.
This relates to the spammer seo-hunt.com
* Adding validator domain validator spec
* Moving request spec to unit spec
This change does two things:
1) Allows for 2 character domains
2) Moves the spec for 2 character domains from an expensive spec to a
cheaper to run spec (e.g., request-cycle to unit-test validator)
It preserves PR #12268
* Remove Connect
* Remove more Connect specs
* Remove a lot more Connect code
* 🚮
* It all has to go
* Explicitly add httpclient
* Update application layout
* Remove messages association from User
* Start fixing specs
* reintroduce util function and refactor references
* Remove Connect Cypress test
* Fix more specs
* Remove Connect from listings
* Ignore contact_via_connect column on listings
* Remove contact_via_connect usages
* Ignore mod_chat_channel_id on tags
* Drop Connect tables
* Remove email_connect_messages from user notification settings
* Re-add httpclient 2.8.3
This was mistakenly removed as a merge conflict
* Don't need to exclude removed chat channel file
* Remove unneeded style for chat channels
* Remove unneeded channel list prop type
* Remove chat channels index/connect-link from getPageEntries
* Re-add comment from httpclient in Gemfile
* Remove connect references from mailers
Tag Moderators no longer have a chat channel
No longer will users be notified about new messages (there won't be
any)
No longer will users be notified about channel invites (you can't
invite anyone anymore)
* Don't configure Pusher and remove PUSHER_* from .env_sample
since it's removed from gemfile, the Pusher constant will not resolve, if this is
configured in the environment variables we'll fail to boot.
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
Co-authored-by: Dan Uber <dan@forem.com>
* Refactoring and extending article spam behavior
Prior to this refactor, we were checking an Article's title for
spaminess. This change now checks the Article's title and
body_markdown.
In addition, the encapsulates the regular expression implementations in
favor of asking the Settings::RateLimit class to determine if the passed
in text is "spammy".
Furthermore, instead of having lots of inline logic within the article,
this refactor introduces a `Spam::ArticleHandler` class. This helps
tidy up the already busy Article specs by delegating the business logic
of Spam handling to it's own class.
There are further refactors for Comments and Users that would follow
this pattern, but this change is more important to get out the door.
Closes#15396
* Adding comments to clarify behavior of spam handling
* Adding missing expectation to spec
Prior to this commit, we had filtering options for email registration:
1. Specific allowed domains.
2. Implicitly allow all domains if no allowed domains specified.
With this commit, we add another option: The adminstrator may block one
or more domains from registering.
Closesforem/rfcs#281
This commit removes a handful of magic numbers and instead relies on a
constant.
This has a small impact in that the Basic feed will now return 50
articles instead of 25. However, normalizing the feed pagination window
size helps reduce some oddities in reporting.
In addition, this might be something we consider giving administrators
the ability to set (with default options, because we shouldn't allow
page sizes of 10_000 as that's a massive memory hog).
Related to #14709
Prior to this commit, this instantiation of `Redcarpet::Markdown` was
different from the other instantiations. The other ones all included
the second paramenter (e.g., `Constants::Redcarpet::CONFIG`).
My question is, should we (or can we) be using the same configuration in
all cases?
* Caching sum privileged reaction scores
This relates to work on improving the feed. Namely that in the current
proposal in PR #15240 I'm not accounting for how privileged users have
given feedback on an article.
With this change, I will now have a cached value on the articles table
that can be a proxy for how the privileged users have reacted.
In addition there's a small refactor that moves a constant to the
correct scoping object.
Dependent on #15283.
* Fixing method name to `exists?`
Prior to this commit, I was using `exist?` which doesn't work for
ActiveRecord::Relation objects.
* Add attribute cleaner
* Start using AttributeCleaner
* Add additional check
* Also work with non AR classes
* Remove unnecessray :aggregate_failures
* Rename to StringAttributeCleaner
* Only permit valid class names as sponsorable type
https://github.com/forem/forem/issues/14386 identified an error could
arise if input was provide to the sponsorable_type field in the admin
creation form, but it was not a valid constant (since we include the
related model in the index when loading @sponsorships, this permits
creating a sponsorship that can't easily be managed or deleted).
Add a validation to ensure when the sponsorable type is present, it's
a class (really, we probably want to also ensure it's available as an
associated model type, since we'll be looking it up with find(:sponsorable_id), but this
is an initial attempt at adding some guard rails to this form).
A more realistic solution would be to add a `Sponsorship::SPONSORABLE_TYPES` constant
to the class, and validate the supplied sponsorable type is in
`SPONSORABLE_TYPES.map(&:name)`. That requires more domain knowledge
about what kind of things can be sponsored than I have, it would
certainly include at least ActsAsTaggableOn::Tag but may include other
classes (or why would it be polymorphic).
* Explicitly list the allowed types for sponsorship
I see we can sponsor tags, and suspect that's all we can sponsor
meaningfully (the view only shows details when it is a tag).
* Use inclusion validation with options to replicate custom method
And remove the code, we don't need it any more.
* Update app/models/sponsorship.rb
Co-authored-by: Michael Kohl <citizen428@dev.to>
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Initial work to expand detection functionality
* Finish up functionality and tests
* Fix class name
* Fix tests
* Update spec/services/articles/enrich_image_attributes_spec.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Update spec/services/articles/enrich_image_attributes_spec.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Fix and clean up podcast pages
* Adjust tests
* Fix a test
* Fix missing i18n
* Fix tests
* Fix tests
* Fiddle with test
* Sure up css and tests
* Add featured as allowed param
* Fix a couple tests
* xit out test
* Update app/views/podcast_episodes/index.html.erb
* Update app/views/podcast_episodes/index.html.erb
Co-authored-by: ludwiczakpawel <ludwiczakpawel@gmail.com>
Co-authored-by: ludwiczakpawel <ludwiczakpawel@gmail.com>
Turns out this caused a *different* problem. It would work for the first
delete batch in `BulkSqlDelete`, but subsequent `DELETE` queries would
then get their `statement_timeout`s reset to the global setting. The
issue was that `BulkSqlDelete` uses methods that don't respect nested
transactions, and we were already resetting the value of the setting to
its previous value anyway, so there was really no reason to run it
inside a transaction.
The only failure mode I know of here is that the `ensure` block breaks
because `connection` is severed. In that case, the value is reset anyway
when the connection pool replaces the connection so the failure does not
propagate.
* Allow Emails::RemoveOldEmailsWorker to run longer
We set the SQL statement timeout based on the STATEMENT_TIMEOUT
environment variable (defaulting to 2.5 seconds in production), but not
all SQL queries are created equal. Since some may take longer out of
necessity, this PR introduces a `Model.with_statement_timeout` method
that allows you to change the SQL statement timeout only for a given
block.
* EmailMessage doesn't inherit ApplicationRecord
It inherits from `Ahoy::Message`, which inherits directly from
`ActiveRecord::Base`.
[3] pry(main)> EmailMessage.ancestors
=> [EmailMessage(id: integer, clicked_at: datetime, content: text, feedback_message_id: integer, mailer: string, sent_at: datetime, subject: text, to: text, token: string, user_id: integer, user_type: string, utm_campaign: string, utm_content: string, utm_medium: string, utm_source: string, utm_term: string),
EmailMessage::GeneratedAssociationMethods,
EmailMessage::GeneratedAttributeMethods,
Ahoy::Message(id: integer, clicked_at: datetime, content: text, feedback_message_id: integer, mailer: string, sent_at: datetime, subject: text, to: text, token: string, user_id: integer, user_type: string, utm_campaign: string, utm_content: string, utm_medium: string, utm_source: string, utm_term: string),
Kaminari::ConfigurationMethods,
Kaminari::ActiveRecordModelExtension,
Ahoy::Message::GeneratedAssociationMethods,
Ahoy::Message::GeneratedAttributeMethods,
Bullet::SaveWithBulletSupport,
ActiveRecord::Base,
...
* Use milliseconds to match STATEMENT_TIMEOUT units
* Check if email_newsletter is changed
* Check if Settings::General.mailchimp_api_key is defined
* Add spec for Users::NotificationSetting
* Remove spec for boolean column
http://www.chrisrolle.com/en/blog/boolean-attribute-validation
* Check if email is set when changing newsletter subscription setting
* Make it easy to read
* Remove specs for a case never happening
* Eager-load comment user data
Profiles and user settings are loaded for on each comment author. On
posts with many comments, this invokes a *lot* of DB queries. This
commit replaces 2 queries per comment with 2 queries for all comments.
* Fix where I misunderstood limit's purpose
This implementation is too clever. It looks like it should just be
getting all but the last, but it uses the fact that the default is 0 to
default to getting all of them. This isn't readily understandable to
someone reading the code.
I don't have a suggestion to fix it yet so I'm just going to leave it
as-is for now.
* Remove check_box type from profile fields
* Fix profile field spec
* Delete emphasized_comments_experiment.rb
This file was not supposed to be part of this PR
* Initial forem omniauth strategy setup work
* Finish basic raw proof of concept
* Some playing around
* use OAuth payloads + PASSPORT_OAUTH_URL for local dev
* Use FeatureFlag for Forem Passport Auth
* Working on tests
* Fix tests 🤞🏼 & some cleanup
* Use correct namespace within lib directory (match class namespace)
* Test to ensure Forem Passport auth is restricted by FeatureFlag
* Add broadcast + work on tests
* Update spec/lib/data_update_scripts/insert_forem_connect_broadcast_message_spec.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Hash format
* Schema cleanup + inline comments
* Use temprorary Heroku domain
* More cleanup
* Missed one
* Back to passport.forem.com
* Require correct path in lib
* Apply suggestions from code review
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Use with_indifferent_access for symbol hash access in Forem strategy
Co-authored-by: benhalpern <bendhalpern@gmail.com>
Co-authored-by: Michael Kohl <citizen428@forem.com>
* show top tags when search is empty
* add cypress tests
* nudge travis
* tweak to e2e test
* show a 'top tags' heading
* only suggest tags that are supported
* Add supported scope to Tag
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Separate followed and "anti-followed" tags
* Minor code change
* Indicate "anti-followed" tags in sidebar
* Fix UpdatePointsWorker spec
* Cries in Arel
* Use antifollow scope more selectively
* Add spec for not_cached_tagged_with_any scope
* Add explanatory comment
* Remove unused route
* Update comment
* Remove unrelated change, figure out why > 1 later
* Need more ☕
* Avoid false positives in spec
* Change handling of ingored tags in sidebar
* Add explanatory comment
* Move logic to determine noindex from view to model
This brings attention to the code and allows for faster unit tests.
I expect we'll be modifying this soon to deemphasize the code tag as a
marker of quality.
* Replace "5" constant with home feed minimum score
* Remove check for "code" tags in low quality article check
This made sense for DEV and other devrel communities, but is no longer
necessary for Forems.
* Add UserExperience setting for minimum index score
- add setting, I used `index_minimum_score` to match the
`home_feed_minimum_score`, `minimum_index_score` feels natural but
breaks the pattern.
- use this to determine whether to include article in the sitemap
- use this to determine whether to add noindex meta to article show page
* Remove unneeded tests
Since there's no longer any check for <code> tags in the body, omit
those tests (which passed anyway).
* Add new setting to admin page
* Admin-configurable display locale
* Add i18n-js and namespacing
* Basic tests and clean up
* A few test adjustments
* Update vendor cache
* Fix a few tests
* Fix a few tests
* Update app/views/articles/_actions.html.erb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Update app/views/articles/_comments_actions.html.erb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Update app/views/articles/_single_story.html.erb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Update app/views/articles/_single_story.html.erb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Update app/views/comments/_comment_header.html.erb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Update app/views/layouts/_sidebar_tags.html.erb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Update app/views/listings/index.html.erb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Update spec/system/homepage/user_visits_homepage_articles_spec.rb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Update spec/system/user/view_user_index_spec.rb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Alphabetical locale page
* Add activerecord custom validation error translations
* Add i18n to webpacker
* Fix a few tests
* Adjust error messages
* Add i18n-tasks
* Adjust JS to get working with jest
* Adjust the way translations are pulled in
* Adjust jest tests
* Remove time localization
* Remove superfluous public js
* Add basic tests for i18n application controller
* Remove unnecessary content
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Constants for controller allowed params
* Apply suggestions from code review
Co-authored-by: Michael Kohl <me@citizen428.net>
Co-authored-by: Michael Kohl <me@citizen428.net>
* If there is no body of the linked issue or PR, don't render markdown
Return a readable and actionable error to the user when they're
linking a PR that doesn't have a body (and we can't render).
This is not needed if there's a fallback path to render _something_ in
the absence of a body (or if it's safe to just skip the `body` rendered
inside the github issue liquid template partial).
* When there is no issue body to render, just show PR title and link
Set the body to an empty string and let github return an empty html
body.
It would be possible to substitute this with a conditional to only
post if a body was present, or assign processed_html directly if not.
* Don't use github api when we have an empty body
Save a network trip and just replace the processed html body with an
empty string when the issue body was nil.
* Relax tag validation restriction to permit alphanumeric strings
Non-word symbols like musical notes are still blocked, but most
languages should work.
* Handle redirect after tag edit
The URI.parse(tag.name) was not updated in #13720
Move reused logic into tag_path method in URL module.
* FactoryBot generated tags are non-ascii
This should help expose any cases where we're mistakenly counting on
ascii-only content in tags by causing all generated tags without a
specified name to be of the form `tag你好#{n}` instead of `tag#{n}`.
* Revert "FactoryBot generated tags are non-ascii"
This reverts commit ed1035cd1e91619b2c1599e521b6ddf962c97f80.
It was useful to see how many places the path knowledge is in the
tests (a lot of requests have GET /t/tag.name and need to have GET
URL.tag_path instead).
https://app.travis-ci.com/github/forem/forem/builds/237542600 has the
results.
Reverting because those test-side changes seem like a secondary
concern to the change in this PR (I'll file a follow on issue)
* Add feature flag for connect
* Start using Connect feature flag
* Add feature flag check to plaintext mailers
* Minor fixes/formatting issues
* Fix typo
* Enable Connect feature flag for tests
* Fix feature flag in tests
* Update rails_helper
* Fix typo
* (╯°□°）╯︵ ┻━┻
* One more try
* connect announcement
* Use connect feature flag in admin and admin routes
* Add aria-live attribute to announcement
* Undo schema.rb changes
Co-authored-by: Paweł Ludwiczak <ludwiczakpawel@gmail.com>
