27b82996ce
* Adding new relevancy lever for privileged reactions
This PR includes four changes:
1. Renaming the variant to remove Jennie's name
2. Starting a new experiment
3. Adding a new relevancy lever
4. Adding a new variant that uses the relevancy lever
1 and 2 is somewhat straight-forward.
For 3, we're looking at having a total of 5 "slots" for privileged user
reactions. Hence the four boundary values.
For 4, we're configuring the granular levers with the following:
- `[-∞..-5)` should have a 0.05
- `[-5..0)` should have a 0.5
- `[0..5)` should have a 0.9
- `[5..10)` should have a 0.98
- `[10..∞)` should have a 1.0
The 20220509-variant.json is a copy of 20220422-variant.json but
replaces the `privileged_user_reaction` lever with the
`privileged_user_reaction_granular` lever.
Below is the diff to highlight the similarities and differences between
the two variants.
```shell
❯ diff config/feed-variants/20220509-variant.json config/feed-variants/20220422-variant.json
90c90
< "privileged_user_reaction_granular": {
---
> "privileged_user_reaction": {
92,96c92,93
< [-2, 0.05],
< [-1, 0.5],
< [0, 0.9],
< [1, 0.98],
< [2, 1]
---
> [-1, 0.2],
> [1, 1]
98,102c95,97
< "fallback": 0.9,
< "very_negative_reaction_threshold": -5,
< "negative_reaction_threshold": 0,
< "positive_reaction_threshold": 5,
< "very_positive_reaction_threshold": 10
---
> "fallback": 0.95,
> "negative_reaction_threshold": -10,
> "positive_reaction_threshold": 10
```
Closes forem/forem#17584
* Update app/models/articles/feeds.rb
Co-authored-by: Josh Puetz <josh@dev.to>
* Adjusting testing logic to reflect latest experiment
* Adjusting negative threshold
Co-authored-by: Josh Puetz <josh@dev.to>
|
||
|---|---|---|
| .. | ||
| 20220415-incumbent.json | ||
| 20220422-variant.json | ||
| 20220509-variant.json | ||
| original.json | ||
| README.md | ||
Feed Variants
This folder contains the feed variants that we have, are, or will be testing. By
convention the file's basename without extension is the name of the variant
(e.g. "./config/feed-variants/20220415-incumbent.json" encodes the variant named
20220415-incumbent).
There exists one variant, original.json, which is our fallback variant. This
fallback is performant and adequate for generating the feed for DEV.to. It is
present in case we need to quickly turn off an experiment. The name original
is the same as the AbExperiment::ORIGINAL_VARIANT constant.
Do not remove original.json without consideration for how to rollback a feed
experiment.
Variants are loaded into production as needed. That "need" is determined by the
configuration of the ./config/field_test.yml. Once we load a variant into
production, we cache that variant (see
Articles::Feeds::VariantAssembler.variants_cache).
As part of our test suite we assemble and verify each defined variant.
How to Make a Feed Variant
A feed variant is written in JSON format. A feed variant configures one or more
Articles::Feeds::RelevancyLevers. The variant configuration defines:
- The cases and fallback for each of the selected relevancy levers
- Optionally the sort order of the query results
- Optionally a few high level parameters.
The relevancy levers and their SQL query fragments are defined in
Articles::Feeds::LEVER_CATALOG. Any levers not configured for a variant are
omitted for that variant (but available for other variants).
The available sort order is also defined in Articles::Feeds::LEVER_CATALOG.
The high level parameters are defined in Aritcles::Feeds::VariantQuery::Config
and as of <2022-05-06 Fri> are:
- max_days_since_published: only consider articles that were published no more than the max_days_since_published.