* Select max from a subquery of two things
max(int, int) not a function, says postgres, once it stopped yelling
about parenthesis balancing.
* prefer greatest to select max from subquery
* Ensuring that we don't divide by zero
Adding one to a "always-ish greater than or equal to 0" value. This
should resolve a flakey test and a seemingly erratic production error.
Resolves https://app.honeybadger.io/projects/66984/faults/83596547
* Favoring min over adding 1
Due to syncrhonization antics, let's not assume time is consistent
across servers.
Timey Whimey Wibbley Wobbley
* Update app/services/articles/feeds/weighted_query_strategy.rb
Co-authored-by: Daniel Uber <djuber@gmail.com>
Co-authored-by: Daniel Uber <djuber@gmail.com>
* Ensuring we don't track views of author or unpublished
Prior to this commit, I was surprised to learn that we:
1) Tracked an author's view of their article.
2) Tracked views of an unpublished article.
This came up from a Forem creator asking if they could reset the view
counter. Or trigger the reset on publication.
I think a general business logic policy of don't track views for the
author and don't track views for unpublished articles is a reasonable
default.
Were we to pursue the clear views on publication, we'd need to consider
something that went from unpublished -> published -> unpublished ->
published. Without a more explicit state machine, triggering a
busineiss logic behavior seems a bit unexpected.
In other words, I wrote an article. There are 20 views when I realize
that I need to unpublished it. I make the changes in the unpublished
state, and re-publish. I'd assume that those 20 views would still be
"recorded" and counted towards my article's view counts.
* Adjusting condition structure
Prior to this commit, the `if` clause was rather far to the right. This
helps make the if clause more pronounced.
We are loading the badge for each included tag in the
Search::TagSerializer and seeing warnings from bullet
```
Bullet::Notification::UnoptimizedQueryError:
GET /search/tags?name=ta
USE eager loading detected
Tag => [:badge]
Add to your query: .includes([:badge])
Call stack
/home/travis/build/forem/forem/app/serializers/search/tag_serializer.rb:5:in `block in <class:TagSerializer>'
/home/travis/build/forem/forem/app/services/search/tag.rb:11:in `serialize'
/home/travis/build/forem/forem/app/services/search/tag.rb:7:in `search_documents'
/home/travis/build/forem/forem/app/controllers/search_controller.rb:54:in `tags'
/home/travis/build/forem/forem/app/lib/middlewares/set_time_zone.rb:10:in `call'
```
Follow the advice, now when multiple tags are in the result set,
having multiple badges, badges are loaded only once (at query time)
and not one by one (at serialization time).
In my quest to find `where(attribute: value)` in controllers and even
services, I stumbled upon this pattern.
As of this commit (and prior) an Article's associated user is it's author.
* Don't update social information for suspended/banished accounts
* Prevent suspended users/accounts from updating their profile information
* Add tests and fix some logic
* Refactoring to add helper method
Prior to this commit, we made view level calls to service modules. This
refactor provides convenience methods on the model.
Furthermore, it addresses a few Rubocop violations that "come along for
the ride."
* Ensuring cached entity squaks like User
* Fixing broken spec
* Fixing typo
* Don't send auth broadcasts for providers that are in beta
Currently we have :apple as a restricted provider (you can enable it,
but it's treated as beta here, rather than generally available).
While we were correctly checking if you had all GA providers enabled
in authenticated_with_all_providers?, we were incorrectly pulling
all enabled provider names in find_auth_broadcast (the message to send
the user), and picking apple_connect.
Since it doesn't make sense to omit apple id login from consideration
when checking if all available auth methods are used, then recommend
that it be used consistently, capture this "GA" state as a method, and
use it both in the test "does this user have all available identity
providers enabled?" and the selection "which identity provider can I
suggest they setup?" consistently.
Since we're about to enable google as an auth source (in #15986) I'll
check with Josh if he expects this to be GA on release or in limited
beta.
* Clean up authenticated_with_all_providers?
We have a method identities that returns the enabled identities for
the user (a relation), and a method ga_providers that returns a list
of enabled and not beta provider symbols.
Change the set difference to use Array#all? (which will exit early on
the first failure). Efficiency note: while I think this reads
better,it's possible this issues a number of small (cheap) queries for
identity by user id and provider id, but there's a unique index on
(provider, user_id) that should be effective.
* Only check providers that have active broadcast messages
An admin can stop sending "connect using apple" follow ups by
disabling that broadcast.
I randomized the enabled/active broadcasts for connection options so
they're not always pulling the same (facebook? apple?) option every
time.
* Clean up lost thought in comment
ActiveSupport adds DateAndTime::Calculations#after? (and before?) -
which clarifies intent (users newer than the relative time are
skipped) of the early returns.
* Modify welcome notification generator to handle errors per-message
The original flow aborted all processing when any
ActiveRecord::RecordNotFound exception was raised. This causes a
situation where a missing broadcast message (by title) causes that
notification message to fail, and each day the same failure to
occur (since it wasn't successfully sent the day before), blocking all
further messages.
We want these to proceed on the schedule implicit in the checks, and
failing early prevents that.
Modify the error handling to catch RecordNotFound on each message, and
continue throught the checks until a notification is enqueued or all
checks have been attempted.
* Run welcome broadcast check 4 hours after the user was created
This started to fail when I modified the generator (why? because the
user was 7 days old and _other_ notifications would be sent).
Ensure the user is 4 hours old before checking that the disabled
welcome broadcast message is checked, and not sent.
* Initial work
* Implement strategy injection into WeightedQueryStrategy
* Modify field_test config and adding variants
* Change orginal to constant and make some other adjustments
* Fix hardcoded test values
Prior to this commit, we had two places that need to know the nuances
ofquerying for tag flares and what we should include in our queries for
serialization.
With this commit, we're factoring towards a common source of knowledge
and providing a much needed test for the expected output of this
serialization.
Loosely related to #15916, #15983, #15994, and #16032.
* add needed fields to tag search
* specs
* add badge to tags suggest response (#15840)
* Add badge to tags suggest response
* Update tags_spec.rb
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
* limit tag search badge to only badge_image
* only unpack badge image if badge exists
Co-authored-by: Dwight Scott <dwight@forem.com>
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
* Send custom user agent string when fetching feeds
dev.to blocks access to the feeds for user agent "Ruby"
Use "Forem Feeds Import" as alternate user agent when fetching feeds.
fixes https://github.com/forem/forem/issues/15939
* Update app/services/feeds/import.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Proposing a new feed experiment
This commit involves tweaking a few subtle aspects of the challenger:
1) Remove the sort by published date on the relevance score. The
articles will now be listed in the order of perceived relevance.
2) Disable a few levers that did little. There really aren't any
`articles.featured = true` articles in the database.
3) Gently increase the weight of each comment in a linear manner.
4) Give a little more weight to posts that don't have followed tags.
In addition, it involves renaming the conversions to better convey their
implementation. This renaming helps create more descriptive labels for
the test results at `/admin/abtests`.
This commit also adds logic to repurpose the AbExperiment feed_strategy
logic; I envision adjusting the weighted feed strategy levers with some
frequency.
Per discussions, I'm also disabling the "not logged in" feed testing.
We'll use the LargeForemExperimental for this.
* Flipping attribute name to positive
Mentally "not disabled" is harder to parse than "enabled". This change
helps with setting an optimistic attribute.
* Bump for travis
* Bump for travis
* Removing file committed by mistake
* Extracting method
* Refactoring and documenting class
I came to this commit by looking for raw `where` method calls to the
Article object. Originally, I was looking at `where.not(user_id: :id)`
but found this service class.
In this class, I saw some significant duplication and went with a bit of
refactoring and some documentation.
This refactoring removed some unnecessary guards and calculations (as
documented inline).
* Naming a constant and adding more documentation
* Use the most recent timestamp from sent digest messages
The original logic here had queried for up to 10 (unordered) messages
from the database for this user, perhaps leveraging the knowledge that
we purge old messages after 90 days to ensure 10 was enough.
Since the last message in the limited and unordered list could have
had any sent_at time in the past 90 days, it's possible users could
pass the "should send email" check multiple times in a day (and get
multiple digest emails, I'm not certain when the rake task runs but it
could be as frequently as once per deployment?)
Since we're only using this list of messages to find the most recently
sent message, select the maximum sent time.
* convert periodic_email_digest setting to days
Time.current - last_email_sent_at gives an integer count of elapsed
seconds
Settings::General.periodic_email_digest is an integer count of days
between digests.
Convert to days (so we're only sending digests when enough time has
elapsed) instead of defaulting to once every 2 seconds (or at least
several times per day).
* Clean up comparison
Rather than subtracting the current time from the last time, and
checking that the difference in time is greater than the periodic
setting, use days.ago to get the timestamp far enough in the past, and
ensure last email was sent before that ( `sent_at < n.days.ago` ).
This seems like it reads clearer than the original implementation.
* Prefer Time.before? to numeric comparison
Change the method name from last_email_sent_at to last_email_sent, so
that the comparison reads like English (the other callers use it as a
number)
And since the code reveals its intention clearly, there's no need for
an inline comment about the next line any more.
* Refactoring to leverage an Article scope
I've been looking at the queries that involve filtering articles based
on scores. There's several places that seem to be close to consistent,
but have some nuanced difference.
This refactor consolidates one of those "almost the same" cases.
* Bump for travis
* Add missing spaces
* Use filter_map over map + reject/compact
* Simplify FactoryBot calls
* Use to_h with block instead of map + to_h
* Use guard clause
* Refactoring questions asked of user
In this pull request, I'm extracting and normalizing role-based
questions asked of the user.
Prior to this commit, our codebase has asked two very similar questions
of our user model:
- `user.has_role?(:admin)`
- `user.admin?`
In asking `has_role?(:admin)` we are relying on implementation details
of the rolify gem. In addition, the `has_role?` question asked
throughout controllers or views means that it's harder to create
hieararchies of permissions.
In favoring `user.admin?` as our question, we can use that indirection
as an opportunity to discuss and decide "Should someone with the
`:super_admin` role be `user.admin? == true`?"
The details of this commit is to do three primary things:
1. Ask the `has_role?` questions in "one place" in the code (e.g. the
`Authorizer` module)
2. Extract the role based questions that are on the `User` model and
provde backwards compatable delegation.
3. Structure the code so that it's harder to accidentally call
`user.has_role?` (e.g., make `User#has_role?` and `User#has_any_role?`
private).
This is related to #15624 and the updates are informed by discussion in
PR #15691. This commit supplants #15691.
* Refactoring the liquid tag policy tests
* Fixing typo
* Bump for travis
* Remove modify_hr_tags method definition
this breaks things that call it.
* Remove direct uses of modify_hr_tags fixer method
Remove from class METHODS lists, remove test cases about this behavior, and remove from
methods lists in test cases.
* Remove hr tag modification spec
Still don't understand what problem this was fixing, but I've removed
the test case.
* Extracting container for allowed tags & attrs
Prior to this commit, we had several different locations in which we
specified ALLOWED_TAGS and ALLOWED_ATTRIBUTES for HTML rendering and
sanitization.
Curious to see how these either intersected or didn't, I opted to
create a container module that allows for us to more readily normalize
these allowed tags and attributes. It's possible that we won't do any
normalization, but this work helps make that easier.
Ideally, I'd love us to contextualize "why did we choose the
tags/attributes we chose?" But for now, I think consolidating these
tags and attributes will help make adding a `details` and `summary` tag
easier.
This relates to forem/rfcs#296
See [Google Sheet][1] for analysis of what tags/attributes are used, the
intersection and union.
[1]:https://docs.google.com/spreadsheets/d/1yj-a1qus1o0o4cj-_gOMP5yteeg-_f3s5z7kvK0Y7RM/edit#gid=0
* Fixing misnamed constant
* Fixing misnamed constant
* Extracting additional HtmlRendering use cases
* Adding comparative documentation for HTML tags
* Fixing broken parameter signature
* Moving constants into MarkdownProcessor
* Consolidating role query logic
Prior to this commit, we had two logically identical chunks of code.
With this commit, we consolidate that logic into a single method. In
addition, we remove one place where the application knows about the
roles implementation.
Loosely related to #15624.
* Bump for travis
* Remove reactions to a user when deleting the user
Fixes#15245
---
I have no idea why `create(:vomit_reaction, reactable: user)` gave a
validation error about category being invalid, but `build().save`
worked fine.
* Move relation details to reaction scope `for_user`
Add a user method to access this by passing self to reaction scope
* Create a moderator when flagging user in factory
This might be a missing requirement in the reactions factory - but
this suppresses an error about invalid category I was getting when
setting a privileged reaction on the user via
create(:vomit_reaction, reactable: user)
Everywhere else it looks like we do this with `user: moderator` (where
there's a trusted user in the surrounding context).
* Move trusted user requirement into vomit_reaction factory
* Add a data update script to remove reactions to deleted users
This cleans up the remaining invalid references and should make the
change in #15249 obsolete (i.e. we don't need to limit the view to
exclude items that no longer exist).
* Use the feed source url when the feed url (from links) is empty
This probably should just be using @feed_source_url always to make a
base path for absolute paths without a hostname, rather than expecting
this from the feed.
An example why we should not be using @feed.url would be
individual posts at domain/slug but the "index" page at
domain/blog (with a generic screen on /), and images hosted from
/assets/ without /blog/ anywhere in the asset path.
Partial fix for #15488
* Use item.url as base_url for relative links
It's possible the rss feed and the contained articles are served from
separate hosts/subdomains/paths. Don't assume the entries are related
to the feed or the feed_source_url, use the item's url for relative
links in that item.
* Update failing spec to add url to item double
* Add two guards when purchasing credits
This prevents sending an empty card to stripe, and avoids a
foreseeable error when purchasing 0 credits. Error returned from
stripe is "This value must be greater than or equal to 1".
* Move redirect back into action method
Only use validation methods to check input and set appropriate messages.
* Accept organization id if set
The payment service will use the organization id (and charge the
organization) when that's the purchaser. Don't fail when no stripe
token or card selected if an organization admin is purchasing credits.
* Move validation into the payment processing service
This cleans up the number of details the controller needs to worry
about
Now only if the process payment service had any tests...
* Prevent only organization_id from permitting purchase
and move this check to a method, with an intention revealing name
* Add i18n translation for error message
There weren't any service side error messages - this will be displayed
in a view (credits/purchase) but comes from the payment service -
other messages (from stripe) are passed through as-is there.
If this is the wrong _place_ for the translation I can retool.
* Add unit test for payment processing
Only checking errors are raised as expected and the internal error
message is translated.
* Leave translation alone
Revert test translation
Prior to this fix, in the production rails console we ran the following:
```ruby
Articles::Feeds::WeightedQueryStrategy
.new(user: nil, page: 1, tags: nil)
.featured_story_and_default_home_feed(user_signed_in: false)
```
The result was: `[nil, []]`. Which in the
[StoriesController#assign_feed_stories][1] is where we have the
following:
```ruby
@featured_story, @stories = feed
.featured_story_and_default_home_feed(
user_signed_in: user_signed_in?)
```
In Blazer, I ran the following query at 2021-12-02 Thu 20:41 EST:
```sql
SELECT * FROM articles
WHERE featured = true
AND published = true
AND published_at > (NOW() - interval '7 days')
AND published_at < NOW()
AND main_image IS NOT NULL
```
So the logic of filtering with `only_featured` resulted in the
`@featured_story` being `nil`. Which then cascaded into the logic for
determining the remainder of the results.
For further discussion see the inline comments and specs that I wrote.
In addition to the above, I'm also reusing
`Articles::Feeds::FindFeaturedStory`
Prior to this commit, the `Articles::Feeds::WeightedQueryStrategy` first
grabbed the featured story, then grabbed the remaining articles. This
is different than the implementation of the
`Articles::Feeds::LargeForemExperimental` in which we first grab all the
articles then claim one as the featured.
In the WeightedQueryStrategy, the articles would not include the
featured. However, in the LargeForemExperimental it would. This
results in upstream logic in [app/javascript/articles/Feed.jsx][2]
going through and removing one of the articles.
[1]:c1a3ba99eb/app/controllers/stories_controller.rb (L237-L271)
[2]:c1a3ba99eb/app/javascript/articles/Feed.jsx (L57-L63)
`User#trusted?` is more Ruby idiomatic than `User#trusted`. This helps
align the various "User#question?" methods.
I've added a deprecation warning, but don't believe that this is active
in the code-base.
