maverick/docbrown
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Adding ability to "seed" feed's SQL randomization (#17827)

Browse source 
The commit includes three changes that work to allow us to "seed" the
randomization.  When using the same seed, and randomizer should/must
return the same sequence of numbers.

First, I added the "seed" parameter to the variant query.  If you want
the same sequence of random numbers from query to query, pass the same
seed.  Otherwise, we'll use a Ruby generated random seed.

Second, I added a virtual column to the virtual `article_relevancies`
table.  If you provide a seed, and call the query twice, the first row
in each of `article_relevancies` will have the same `randomized_value`,
likewise the second row, etc.

Third, I amended the existing `order_by_lever` that had a `RANDOM()`
postgresql function call.  I replaced that with the `randomized_value`
which is computed based on the provided seed.

Fundamentally the idea is to allow for randomness but introduce possible
repeatability; a hard thing considering that some of the inner selection
criteria is not fixed (e.g. number of reactions can change from moment
to moment).

I wrote a [complimentary blog post][1] to further explain what's
happening.

Related to forem/forem#17826

[1]:https://takeonrules.com/2022/06/03/adding-reproducible-randomization-to-sql-queries/
This commit is contained in:
Jeremy Friesen 2022-06-06 11:58:16 -04:00 • committed by GitHub
parent cdde8f3a82
commit 2e19ea9255
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 17 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
app/models/articles/feeds.rb
View file

@ -87,7 +87,8 @@ module Articles
order_by_lever(:final_order_by_random_weighted_to_score,
label: "Order by conflating a random number and the score (see forem/forem#16128)",
order_by_fragment: "RANDOM() ^ (1.0 / greatest(articles.score, 0.1)) DESC")
order_by_fragment: "article_relevancies.randomized_value " \
"^ (1.0 / greatest(articles.score, 0.1)) DESC")
relevancy_lever(:comments_count_by_those_followed,
label: "Weight to give for the number of comments on the article from other users" \

19
app/services/articles/feeds/variant_query.rb
View file

@ -42,12 +42,14 @@ module Articles
# @param number_of_articles [Integer, #to_i]
# @param page [Integer, #to_i]
# @param tag [NilClass] not used
def initialize(config:, user: nil, number_of_articles: 50, page: 1, tag: nil)
# @param seed [Number] used in the `setseed` Postgresql function to set the randomization seed.
def initialize(config:, user: nil, number_of_articles: 50, page: 1, tag: nil, seed: nil)
@user = user
@number_of_articles = number_of_articles
@page = page
@tag = tag
@config = config
@seed = Float(seed || rand)
oldest_published_at = Articles::Feeds.oldest_published_at_to_consider_for(
user: @user,
days_since_published: config.max_days_since_published,
@ -56,7 +58,7 @@ module Articles
configure!
end
attr_reader :config, :query_parameters
attr_reader :config, :query_parameters, :seed
# Query for articles relevant to the user's interest.
#
@ -108,8 +110,17 @@ module Articles
# articles and then sort on the attributes on either the article or the result set
# (e.g. sort on the relevancy score).
join_fragment = Arel.sql(
"INNER JOIN (#{Article.sanitize_sql(unsanitized_sql_sub_query)}) " \
"AS article_relevancies ON articles.id = article_relevancies.id",
"INNER JOIN (" \
"\n--- The setseed needs to be called independently; later we reference seeder \n" \
"WITH seeder AS (SELECT setseed(#{Float(@seed)})) "\
"\n--- We are using the inner logic to build relevancy score" \
"\n--- The outer part, with seeder, is to create a stable randomized number \n" \
"SELECT inner_article_relevancies.id, "\
"inner_article_relevancies.relevancy_score, " \
"RANDOM() AS randomized_value " \
"FROM seeder, " \
"(#{Article.sanitize_sql(unsanitized_sql_sub_query)}) AS inner_article_relevancies" \
") AS article_relevancies ON articles.id = article_relevancies.id",
)
# This sub-query allows us to take the hard work of the hand-coded unsanitized sql and