Adding additional order by levers (#17847)

In the general sense we have an existing `order by` that follows the
following format: `order(x) = f(x) * rand() ^ ( 1 / g(x))`

For the present [20220603-variant-b][1], `g(x)` is the article’s score.
And `f(x)` is 1.

This pull request introduces a few named variations for the above
`order(x)` formula.

The three named formats each use a different `g(x)` function; namely
`greatest(0.1, ln(1 + greatest(0, public_reactions_count)))`.  The more
public_reactions the article has the closer to zero the resulting
positive number will be.  And raising a random number, with range
`(0..1)`, to the inverse of that number will tend to mean that artciles
with many reactions will tend to group towards the higher end of the
random range, whereas articles with few reactions will have a more even
distribution across the range.

Further the `ln` function helps dampen the weight we give to reactions.

The variation within these three order levers is on the `f(x)` portion:

1. Using `published_at`
2. Using `last_commented_at`
3. Using a mix of `published_at` and `last_commented_at`

In all cases, those dates are converted to an integer.  A date 2 weeks
ago will result in a number less than a date 1 week ago which will be
less than a today.

The idea for the mix of `published_at` and `last_commented_at` is to see
a mix of things where some “older” articles (from the underlying result
set) will get a chance to show up earlier in the feed.

My rational for this order structure is because the `RANDOM() ^ (1 /
score)` is showing possible success in a current experiment.  And in
past trials long past, it rose to the top as a likely and viable
contender.

These new `order_by` lever patterns introduce three things:

1. A variable based on a date in time
2. A dampening of what can be a large number; (we have scores in the
   publication range that are between -200 and 750 so I think it is
   important to apply a dampening.  For example `(0.1 ^ (1/100)) =
   0.977` and `(0.01 ^ (1/100)) = 0.955`; in other words these large
   scores aggressively pushed their corresponding posts to the top of
   the relevance; even as we weren't explicitly querying for those
   scores.
3. With the score being quasi-magical, it doesn’t make sense to sort on
   that value but to instead sort on a property that represents what
   we’re setting as our experiment goals (e.g. publishing something or
   commenting on something and/or reacting to something)

Note: it would be feasible to extract these into a singular order lever
that we would configure.  But that's a future concern.

Also note, none of these are yet part of any experiment, but are instead
being "queued up" to add to our available corpus.

Closes forem/forem#17834

Related to:

- forem/forem#17827
- forem/forem#17826
- forem/forem#17833
- forem/forem#16128 :: for past order queries to seek as inspiration

[1]:https://github.com/forem/forem/blob/main/config/feed-variants/20220603-variant-b.json
This commit is contained in:
Jeremy Friesen 2022-06-10 16:37:21 -04:00 committed by GitHub
parent c62d899f05
commit 736079e28c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23

View file

@ -90,6 +90,29 @@ module Articles
order_by_fragment: "article_relevancies.randomized_value " \
"^ (1.0 / greatest(articles.score, 0.1)) DESC")
order_by_lever(:published_at_with_randomization_favoring_public_reactions,
label: "Favor recent articles with more reactions, "\
"but apply randomness to mitigate stagnation.",
order_by_fragment: "(cast(extract(epoch FROM published_at) as integer)) * "\
"(article_relevancies.randomized_value ^ (1.0 /" \
" greatest(0.1, ln(1 + greatest(0, public_reactions_count))))) DESC")
order_by_lever(:last_comment_at_with_randomization_favoring_public_reactions,
label: "Favor articles with recent comments and more reactions, " \
" but apply randomness to mitigate stagnation.",
order_by_fragment: "(cast(extract(epoch FROM last_comment_at) as integer)) * "\
"(article_relevancies.randomized_value ^ (1.0 / " \
"greatest(0.1, ln(1 + greatest(0, public_reactions_count))))) DESC")
order_by_lever(:random_pick_of_which_date_to_use_with_randomization_favoring_public_reactions,
label: "Favor articles with recent comments or published at and more reactions, " \
"but apply randomness to mitigate stagnation.",
order_by_fragment: "(cast(extract(epoch FROM "\
"(CASE WHEN RANDOM() > 0.5 THEN published_at ELSE last_comment_at END)) " \
"as integer)) * "\
"(article_relevancies.randomized_value ^ (1.0 / "\
"greatest(0.1, ln(1 + greatest(0, public_reactions_count))))) DESC")
relevancy_lever(:comments_count_by_those_followed,
label: "Weight to give for the number of comments on the article from other users" \
"that the given user follows.",