* Refactoring to consolidate logic
Prior to this commit, two controllers had nearly identical chunks of
logic. This refactor extracts the logic to a common and more canonical
location.
* Addressing rubocop's aggressive auto-fix
* Adding spat operator for pluck
* Renaming method for greater clarity
* Refactoring to leverage an Article scope
I've been looking at the queries that involve filtering articles based
on scores. There's several places that seem to be close to consistent,
but have some nuanced difference.
This refactor consolidates one of those "almost the same" cases.
* Bump for travis
* Remove slash characters from user supplied user search input
Prevents an error when the search term includes '\'
PG::SyntaxError: ERROR: syntax error in tsquery
https://app.honeybadger.io/projects/66984/faults/79391397
I had originally thought to add this cleanup to Search::Username but
decided to move it as close to the generated (invalid) query as
possible to prevent alternate paths finding their way here.
* Add spec
Since there's no existing tests for the scope - I put the test code on
the caller (Search::Username) rather than the model (User), this seems reasonable.
* When the term is empty (or only slashes) just return null relation
* Handle nil input (search for nothing) correctly
One of the request specs sends a username search with no query, so we
can't call nil.delete or nil.empty?, use blank? of empty?
* Favoring single SQL and computation over enum
Prior to this commit, we ran a SQL statement to generate a list of
articles then applied some minor randomization.
With this commit, I'm removing the randomization in favor of expected
values, and collapsing the result set into a single inline SQL and some
post query simple arithmatic.
Let's check my math. For each article we sum:
* `article.score`
* `article.comments_count` * 14
* a random number between 0 and 5 (`rand(6)`) with expected value of 2.5
* the tag's (taggings_count + 1) / 2
The new SQL query sums the `article.score` and the
`article.comments_count` * 14. I then reduce the remainder of the
equation. From "for each article sum `(rand(5) + (taggings_count + 1) /
2)`" we have the following:
`article_count * (2.5 + (taggings_count + 1) / 2)`
Which is equivalent to:
`article_count * ((taggings_count + 1 + (2.5 * 2)) / 2)`
Which is equivalent to:
`article_count * ((taggings_count + 6) / 2)`
This change reduces computation time by favoring expected values.
* Fixing broken behavior and adding comments
* Refactoring questions asked of user
In this pull request, I'm extracting and normalizing role-based
questions asked of the user.
Prior to this commit, our codebase has asked two very similar questions
of our user model:
- `user.has_role?(:admin)`
- `user.admin?`
In asking `has_role?(:admin)` we are relying on implementation details
of the rolify gem. In addition, the `has_role?` question asked
throughout controllers or views means that it's harder to create
hieararchies of permissions.
In favoring `user.admin?` as our question, we can use that indirection
as an opportunity to discuss and decide "Should someone with the
`:super_admin` role be `user.admin? == true`?"
The details of this commit is to do three primary things:
1. Ask the `has_role?` questions in "one place" in the code (e.g. the
`Authorizer` module)
2. Extract the role based questions that are on the `User` model and
provde backwards compatable delegation.
3. Structure the code so that it's harder to accidentally call
`user.has_role?` (e.g., make `User#has_role?` and `User#has_any_role?`
private).
This is related to #15624 and the updates are informed by discussion in
PR #15691. This commit supplants #15691.
* Refactoring the liquid tag policy tests
* Fixing typo
* Bump for travis
* Extracting container for allowed tags & attrs
Prior to this commit, we had several different locations in which we
specified ALLOWED_TAGS and ALLOWED_ATTRIBUTES for HTML rendering and
sanitization.
Curious to see how these either intersected or didn't, I opted to
create a container module that allows for us to more readily normalize
these allowed tags and attributes. It's possible that we won't do any
normalization, but this work helps make that easier.
Ideally, I'd love us to contextualize "why did we choose the
tags/attributes we chose?" But for now, I think consolidating these
tags and attributes will help make adding a `details` and `summary` tag
easier.
This relates to forem/rfcs#296
See [Google Sheet][1] for analysis of what tags/attributes are used, the
intersection and union.
[1]:https://docs.google.com/spreadsheets/d/1yj-a1qus1o0o4cj-_gOMP5yteeg-_f3s5z7kvK0Y7RM/edit#gid=0
* Fixing misnamed constant
* Fixing misnamed constant
* Extracting additional HtmlRendering use cases
* Adding comparative documentation for HTML tags
* Fixing broken parameter signature
* Moving constants into MarkdownProcessor
* Consolidating role query logic
Prior to this commit, we had two logically identical chunks of code.
With this commit, we consolidate that logic into a single method. In
addition, we remove one place where the application knows about the
roles implementation.
Loosely related to #15624.
* Bump for travis
* Remove reactions to a user when deleting the user
Fixes#15245
---
I have no idea why `create(:vomit_reaction, reactable: user)` gave a
validation error about category being invalid, but `build().save`
worked fine.
* Move relation details to reaction scope `for_user`
Add a user method to access this by passing self to reaction scope
* Create a moderator when flagging user in factory
This might be a missing requirement in the reactions factory - but
this suppresses an error about invalid category I was getting when
setting a privileged reaction on the user via
create(:vomit_reaction, reactable: user)
Everywhere else it looks like we do this with `user: moderator` (where
there's a trusted user in the surrounding context).
* Move trusted user requirement into vomit_reaction factory
* Add a data update script to remove reactions to deleted users
This cleans up the remaining invalid references and should make the
change in #15249 obsolete (i.e. we don't need to limit the view to
exclude items that no longer exist).
* WIP: add a creatore settings form
* WIP: updat the controller to use the Creator Settings FOREM
* feat: use the creator settings form for the new action
* feat: add some default values for the new action
* a note about form data
* update the initiaize function to set some default values
* feat: update the form to use the model data
* feat: permit adn use the attributes within creator_settings_form
* update the flash error
* refactor: require and permit parameters
* chore: use booleans, set defaults and validate the form
* spec: update all the creator_settings tests
* chore: remove comment
* refactor: use self
* feat: aggregate failures'
* chore: remove the logo uploader in the controller
* refactor: update error handling
* feat: update the wasy the controller handles success and error
* chore: remove the resource errors
* feat: show flash message on new line
* fix: use a redirect so that we can get back to /new
* refactor: pass these values through as they seem to be caching whne setting them as default
* chore: change default values
* spec: update tests
* Fix CreatorSettingsForm specs
* fix: use a boolean for public
* spec: add another test for the success var
* fix: radio button labels to correspond + cyress specs
* spec: update based on new changes
* spec: update the params and the expected output
* spec: update the comments and status
* feat: no need for the initialize as we use Active Record Attributes
* feat: update the tac and coc to be persisted when ticked
* fix: amend spec
* blank space
* Message
Co-authored-by: Michael Kohl <me@citizen428.net>
* User decorator (and spec) should use `trusted?`
Fixes a few issues seen in an rspec run that show as:
DEPRECATION WARNING: User#trusted is deprecated, favor
User#trusted? (called from config_body_class at
/opt/apps/forem/app/decorators/user_decorator.rb:58)
And here:
/opt/apps/forem/spec/decorators/user_decorator_spec.rb:112
/opt/apps/forem/spec/decorators/user_decorator_spec.rb:121
/opt/apps/forem/spec/decorators/user_decorator_spec.rb:130
/opt/apps/forem/spec/decorators/user_decorator_spec.rb:139
* prefer User trusted? to trusted
DEPRECATION WARNING: User#trusted is deprecated, favor
User#trusted? (called from permissions at
/opt/apps/forem/app/models/rating_vote.rb:25)
* Prefer trusted? to trusted in user spec
* Use warned? rather than warned in admin article partial
* use trusted? rather than trusted in moderator requests spec
* Prefer trusted? to trusted in moderations controller
* Prefer trusted? to trusted in moderations view
* User auditable? should call trusted? and not trusted
Deprecations go rolling right along.
* Invert guard clause logic to be positive
The original "return unless multiple negated conditions hold" guard
was cumbersome.
Invert to return if any of the exceptions apply, namely:
- this is a comment or readinglist rating (rather than explicit),
allowed for all
- this rating is from a moderator/trusted user (allowed)
- this rating is offered by the article's author (allowed)
I had intended to also remove the safe navigation operators (since it
wasn't clear why there would be a null user or null article, as
rating_vote joins users to articles with a score), but the builtin
validation tests (is expected to validate ...) build objects with
missing attributes, and raise errors when the spec is run.
* Extracting cross-model uniqueness validator
Prior to this commit, we had somewhat duplicated logic across four
models. In adding this validator we're consolidating the logic and
tidying up each of the other models; letting the validator know which
models are part of the "uniqueness gang."
* Adding conditional test for validator
* Bump for travis
* Apply suggestions from code review
Co-authored-by: Michael Kohl <citizen428@forem.com>
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Refactoring to use StringAttributeCleaner
In #15281, we introduced the StringAttributeCleaner. Let's use it!
* Adding StringAttributeCleaner for User properties
* Parameterizing callback for attribute cleaning
* Reading and applying the example for attribute cleaner
`User#trusted?` is more Ruby idiomatic than `User#trusted`. This helps
align the various "User#question?" methods.
I've added a deprecation warning, but don't believe that this is active
in the code-base.
* wip - Got logo upload working
* Now have the logo rendering in the header and in the admin image config section.
* Small layout tweak for admin -> config -> images -> logo.
* feat: create a logo uploader with some tests
* feat: use the logoUploader instead of the ArticleImageUploader
* feat: return early because svg's do not contain exif or gps data
* chore: we can move the raise outside the transaction as the rest of the transaction won't execute if we raise an error
* feat: add a size range
* WIP: resize an image to a random number for now
* hid the logo behind a feature flag and kept logo_svg as is in the site header.
* Added the jpe file type to the logo uploader.
* Skipped the resizing of an image if it's an SVG in the logo uploader.
* Added content types to the content type logo uploader allow list.
* Synced logo validation with frontend and backend.
* Removed unnecessary ALLOWED_PARAMS elements.
* feat: update the logo upoader and tests
* chore: remove comments
* chore: remove comments
* feat: update the resizing for the images + add the correct content type
* spec: test the versions
* fix: update the Constant
* feat: add the versions of the logo
* feat: populate the settings correctly and consistently
* feat: add an random string to the file name to avoid caching issues
* feat: amend the logo layout
* chore: remove comments
* spec: update
* feat: image type whitelist
* feat: update the logo css and also just use resized_logo and remove mobile resize
* feat: add a max-height
* only add site-logo if the feature flag is off
* Renamed IMAGE_TYPE_WHITELIST to IMAGE_TYPE_ALLOWLIST
* Update app/controllers/admin/creator_settings_controller.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Update app/uploaders/logo_uploader.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Update app/uploaders/logo_uploader.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Update app/uploaders/logo_uploader.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Update spec/uploaders/logo_uploader_spec.rb
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* Update spec/uploaders/logo_uploader_spec.rb
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* chore: revert admin change
* refactor: use a static value for directory
* feat: freeze constants
* feat: remove the logo requirement
* chore: spacing
* remove logo requirement
Co-authored-by: Ridhwana <ridhwana.khan16@gmail.com>
Co-authored-by: Michael Kohl <citizen428@forem.com>
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
Given that we have the roles of `:tech_admin`, `:admin`, and
`:super_admin`, I don't want the surprise of assuming that `user.admin?`
means that they have the role of `:admin`.
* Refactoring to use scope
Prior to this commit, four methods had similar purpose but different
implementations. This commit normalizes that and extracts a few
`ActiveRecord::Base` scopes.
Related to #15624
* Fixing typo and tests
In #15412 I introduced faulty logic, and didn't reuse existing scopes.
For comparison, the original code for comment threshold checks was here:
5d5dbaca95/app/models/comment.rb (L303)
```ruby
Reaction.comment_vomits.where(reactable_id: user.comments.pluck(:id)).size > 2
```
This change restores that and tightens up the scope usage.
* Refactoring Spam Handler
There's considerable repeated logic between checking spam for an article
and spam for a comment and user.
This attempts to send things through channels that are similar and close
in organization.
* Fixing broken spec
* Fixing spec around recent user
* Update app/models/reaction.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Update app/models/reaction.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Consolidating new user query logic
Prior to this commit there were two separate queries around new user
logic. With this commit, we're changing the logic to repurpose a site
wide setting.
* Generalizing a previously specific message
* Fixing method name
As part of a recommended refactor, I extracted a method, then renamed
it. I failed to account for that renaming.
This commit fixes that.
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Fix Rpush app when creating Android notification
* Include Android devices in Service spec
* Apply suggestions from code review
Co-authored-by: Michael Kohl <citizen428@forem.com>
Co-authored-by: Michael Kohl <citizen428@forem.com>
Prior to this commit, we trusted all user profiles. There are folks
using DEV.to profiles to help with their spam efforts.
With this commit, we're trying to thwart their efforts.
Closesforem/rfcs#334
* Refactoring poll/vote/skip logic
Favor asking the poll if the user has already voted instead of relying
on the poll vote or skip to know about the implementation
details (e.g. should a poll vote know that there's a related model call
poll skipped? likely not)
This refactor also tidies up the specs, for which there's quite a bit of
chatter.
Found via the `flay` gem.
* Adding method documentation
Also need to bump gitpod's build as it halted for the prior commit.
* AB Test for Feed Strategy
Adding the plumbing for performing AB Tests on the feed strategy.
You can add the environment variable `AB_TEST_SERVICE_FEED_STRATEGY` to
specify the field test to run.
For example, from the forem application home directory you can run the
following to force a strategy for all users:
```console
AB_TEST_SERVICE_FEED_STRATEGY=original bin/start
```
Forcing the strategy can significantly help with quality assurance testing.
* Adding configurable weighted feed strategy
I wrote about this exploration on [DEV.to][1]. This particular
implementation provides significant configuration options for A/B
testing.
At it's core this `Articles::Feeds::WeightedQueryStrategy` provides a
means to query articles that are relevant to the user. The
configuration helps us test and refine those values.
Further more this implementation introduces the idea of some common
interfaces for feed strategies. I'm not happy with the naming
convention but we can work to change that.
This even hints at a mechanism for administrators to configure.
Related to forem/rfcs#314, as I work towards understanding the feed
algorithm. And #15180
[1]:https://dev.to/jeremyf/practicing-postgresql-and-postulating-improvements-5m
* Renaming AbTestService to AbExperiment
I'm loath to end any production class's filename with "test". I don't
want minitest or rspec coming along and attempting to "test" that file.
In addition, I find that Experiment better describes the concept.
Also, per conversations with Michael K, we favor nouns objects to go
in app/models and verb objects to go in app/services. That's a great
separation.
* Factoring AbExperiment into a Controller concern
I wish this could be a stand-alone module but the implementation of
field test has cases that require controller context.
* Sanitizing configurable weighted strategy cases
* Minor refactoring of coding structure
* Renaming parameter for clarity
* Renaming variable to avoid confusion
Prior to the commit, `env` is a controller variable. The injected `env`
variable didn't collide, but did create just a bit of confusion.
With this change, I'm trying to disentangle from that variable.
* Factoring AbExperiment away from mixin concern
Per discussion with Michael K, we both agree that we do not like
controller concern mixins. They can quickly complicate the legibility
of any given codebase.
This refactor provides a common and simplified interface for a Plain Old
Ruby Object (PORO) that accounts for the implementation considerations
of the underlying "field_test" controller method.
* Adding Datadog.tracer.trace around feed actions
* Adding featured_article_factor
Prior to this commit, there was no consideration for how moderators
engage with an article. This is a first nudge towards that effort.
* Removing spec
Add a spec that says yes or no we're feature testing something seems
fragile. Namely, why are we wanting to assert that as a truism.
* Helping ensure we're "tracing" the same things
From the code comment:
> Hey, why the to_a you say? Because the
> LargeForemExperimental has already done this. But the
> weighted strategy has not. I also don't want to alter the
> weighted query implementation as it returns a lovely
> ActiveRecord::Relation. So this is a concession.
* Addressing parameters for tracing function
* For weight feed query only count positive tags
A user assigns explicit points to the tags that they do or do not want
to see. This change treats the negative explicit_points as an indicator
that they don't want to see the tag.
* Adding weighted factor for privileged user reactions
* Updating comments to better clarify
* Adding additional constraint for published_at
* Updating documentation and adding logger
* Adding documentation concerning method call
* Upping logging level for experiments
See conversations in https://github.com/forem/forem/pull/15240
* Adding group by clauses to appease postgres configs
* Injecting relevant scoring method's joins
Prior to this commit, we included all of the joins regardless of the
scoring methods chosen. With this commit, we only inject joins that are
used with the commit.
What this would allow us to do is to then have some pre-cooked scoring
methods but not turn them on. Or to turn off the more expensive scoring
methods.
* Adding feature flag for feed strategy experiment
* Favoring dependent: :delete_all over :destroy
The `dependent: :destroy` callback is slower than `dependent: :delete`
and `dependent: :delete_all`. We need only favor the `dependent:
:destroy` when there's callbacks that happen.
In the case of :destroy, ActiveRecord instantiates each object and then
runs destroy. Whereas in the case of :delete, ActiveRecord issues a SQL
command to delete the related files.
It is often "safer" to use :destroy, as it guarantees that you'll
instantiate the record and run it's callbacks. But sometimes you have
to go with the speed of SQL.
This relates to #14140. Note there is still more to consider, but given
that we're looking at moving from :destroy on all of an article's page
views to :delete, we might buy enough time in the callback.
* Removing redundancy of article destroy
Prior to this commit, `before_destroy_actions` called the `bust_cache`
method which in turn called `touch_actor_latest_article_updated_at` but
`bust_cache` did not pass the destroying parameter. Then
`before_destroy_actions` immediately called
`touch_actor_latest_article_updated_at` with `destroying: true`.
With this change, we remove one of those duplicate calls.
* Fixing specs regarding relationships
* Fixing specs regarding relationships
* Ensuring reading list excludes unpublished articles
Prior to this commit, if a given user adds an article to their reading
list then the author unpublishes the article, the article remains in the
reading list. When the given user would then "click" on the now
unpublished article, they would get a 404 Not Found notice.
With this commit, we now exclude unpublished articles from the search
results for the reading list.
This should also address the issue of attempting to archive the reading
list item (because it won't be visible in the listing).
One of the behaviors that is expected is if, in the above scenario, the
author again publishes the article, that article will again "appear" on
the reading list for the given user.
Closes#14796
* Updating documentation and adding spec
* Addressing pull request feedback
Yes, I should've used a scope! And also, no need to test present. Just
let it's "truthy"-ness speak for itself!
* Update spec/services/search/reading_list_spec.rb
Co-authored-by: Jamie Gaskins <jamie@forem.com>
Co-authored-by: Jamie Gaskins <jamie@forem.com>
* window.ForemMobile namespaced functions
* Fix broken Buildkite
* Wait for data-loaded before using Runtime in pack
* Bring back sprockets base
* Better error handling
* Smal fixes to ConsumerApp to support Android platform
* Fix typo
* utilities/waitForDataLoaded.js
* Update app/javascript/utilities/waitForDataLoaded.js
Co-authored-by: Nick Taylor <nick@forem.com>
* Review feedback
* Fix failing spec
* Cleanup promise
* Remove old utility file
* Add Cypress check for namespaced availability
* More specs
* Refactor to rely on ForemMobile for native bridge messaging
* Fix spec/requests/api/v0/instances_spec.rb
* Fix devices spec
* Remove changes to /api/instance
* Refactor to dynamic import
* Fix typo
* Fixed custom event detail payload for tests.
* mock ForemMobile function instead of webkit call directly
* failing jest test debug
* Another attempt
* Fixed broken test that was missing an onMainImageUrlChange prop on the component.
* Fixed mobile bridging E2E tests.
* Move Cypress tests to mobileFlows
* Add JSDocs + cypress spec for injectJSMessage when logged out
* Cleanup + Disable native image upload until AppStore approval
Co-authored-by: Nick Taylor <nick@forem.com>
Co-authored-by: Nick Taylor <nick@dev.to>
* Favoring delete_all on user relationships
Prior to this commit, several of the user's "has_many" were marked as
`depenedent: :destroy`.
In the case of :destroy, ActiveRecord instantiates each object and then
runs destroy. Whereas in the case of :delete, ActiveRecord issues a SQL
command to delete the related files.
It is often "safer" to use :destroy, as it guarantees that you'll
instantiate the record and run it's callbacks. But sometimes you have
to go with the speed of SQL.
And while not directly related to #15424 it is representative of our
callback ecosystem creating some unexpected computational loads.
Related to #15442 and #15424
* Noting models that user no longer cascade destroys
* Updating valid domain registration
Prior to this commit, our regular expression did not account for the `-`
character as valid within the domain. The `-` character cannot be the
first nor last character of the domain (e.g. `-hello.com` nor
`hello-.com` are invalid but `hell-o.com` is valid).
In addition I tidied up the default value of the
`blocked_registration_email_domains` to match it's sibling `allowed_registration_email_domains`.
This relates to the spammer seo-hunt.com
* Adding validator domain validator spec
* Moving request spec to unit spec
This change does two things:
1) Allows for 2 character domains
2) Moves the spec for 2 character domains from an expensive spec to a
cheaper to run spec (e.g., request-cycle to unit-test validator)
It preserves PR #12268
* Remove Connect
* Remove more Connect specs
* Remove a lot more Connect code
* 🚮
* It all has to go
* Explicitly add httpclient
* Update application layout
* Remove messages association from User
* Start fixing specs
* reintroduce util function and refactor references
* Remove Connect Cypress test
* Fix more specs
* Remove Connect from listings
* Ignore contact_via_connect column on listings
* Remove contact_via_connect usages
* Ignore mod_chat_channel_id on tags
* Drop Connect tables
* Remove email_connect_messages from user notification settings
* Re-add httpclient 2.8.3
This was mistakenly removed as a merge conflict
* Don't need to exclude removed chat channel file
* Remove unneeded style for chat channels
* Remove unneeded channel list prop type
* Remove chat channels index/connect-link from getPageEntries
* Re-add comment from httpclient in Gemfile
* Remove connect references from mailers
Tag Moderators no longer have a chat channel
No longer will users be notified about new messages (there won't be
any)
No longer will users be notified about channel invites (you can't
invite anyone anymore)
* Don't configure Pusher and remove PUSHER_* from .env_sample
since it's removed from gemfile, the Pusher constant will not resolve, if this is
configured in the environment variables we'll fail to boot.
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
Co-authored-by: Dan Uber <dan@forem.com>
* Refactoring and extending article spam behavior
Prior to this refactor, we were checking an Article's title for
spaminess. This change now checks the Article's title and
body_markdown.
In addition, the encapsulates the regular expression implementations in
favor of asking the Settings::RateLimit class to determine if the passed
in text is "spammy".
Furthermore, instead of having lots of inline logic within the article,
this refactor introduces a `Spam::ArticleHandler` class. This helps
tidy up the already busy Article specs by delegating the business logic
of Spam handling to it's own class.
There are further refactors for Comments and Users that would follow
this pattern, but this change is more important to get out the door.
Closes#15396
* Adding comments to clarify behavior of spam handling
* Adding missing expectation to spec
Prior to this commit, we had filtering options for email registration:
1. Specific allowed domains.
2. Implicitly allow all domains if no allowed domains specified.
With this commit, we add another option: The adminstrator may block one
or more domains from registering.
Closesforem/rfcs#281
This commit removes a handful of magic numbers and instead relies on a
constant.
This has a small impact in that the Basic feed will now return 50
articles instead of 25. However, normalizing the feed pagination window
size helps reduce some oddities in reporting.
In addition, this might be something we consider giving administrators
the ability to set (with default options, because we shouldn't allow
page sizes of 10_000 as that's a massive memory hog).
Related to #14709
Prior to this commit, this instantiation of `Redcarpet::Markdown` was
different from the other instantiations. The other ones all included
the second paramenter (e.g., `Constants::Redcarpet::CONFIG`).
My question is, should we (or can we) be using the same configuration in
all cases?
