27b82996ce
* Adding new relevancy lever for privileged reactions
This PR includes four changes:
1. Renaming the variant to remove Jennie's name
2. Starting a new experiment
3. Adding a new relevancy lever
4. Adding a new variant that uses the relevancy lever
1 and 2 is somewhat straight-forward.
For 3, we're looking at having a total of 5 "slots" for privileged user
reactions. Hence the four boundary values.
For 4, we're configuring the granular levers with the following:
- `[-∞..-5)` should have a 0.05
- `[-5..0)` should have a 0.5
- `[0..5)` should have a 0.9
- `[5..10)` should have a 0.98
- `[10..∞)` should have a 1.0
The 20220509-variant.json is a copy of 20220422-variant.json but
replaces the `privileged_user_reaction` lever with the
`privileged_user_reaction_granular` lever.
Below is the diff to highlight the similarities and differences between
the two variants.
```shell
❯ diff config/feed-variants/20220509-variant.json config/feed-variants/20220422-variant.json
90c90
< "privileged_user_reaction_granular": {
---
> "privileged_user_reaction": {
92,96c92,93
< [-2, 0.05],
< [-1, 0.5],
< [0, 0.9],
< [1, 0.98],
< [2, 1]
---
> [-1, 0.2],
> [1, 1]
98,102c95,97
< "fallback": 0.9,
< "very_negative_reaction_threshold": -5,
< "negative_reaction_threshold": 0,
< "positive_reaction_threshold": 5,
< "very_positive_reaction_threshold": 10
---
> "fallback": 0.95,
> "negative_reaction_threshold": -10,
> "positive_reaction_threshold": 10
```
Closes forem/forem#17584
* Update app/models/articles/feeds.rb
Co-authored-by: Josh Puetz <josh@dev.to>
* Adjusting testing logic to reflect latest experiment
* Adjusting negative threshold
Co-authored-by: Josh Puetz <josh@dev.to>
33 lines
1.2 KiB
YAML
33 lines
1.2 KiB
YAML
# See AbExperiment namespace for details of goals and variants
|
|
experiments:
|
|
# NOTE: Our feed strategy testing experiment must begin with "feed_strategy"
|
|
feed_strategy_starting_20220509:
|
|
# NOTE: Required as we want only want to consider for conversion events that
|
|
# occurred on or after the given start_date.
|
|
start_date: 2022-05-09
|
|
variants:
|
|
- 20220422-variant
|
|
- 20220509-variant
|
|
weights:
|
|
- 80
|
|
- 20
|
|
goals:
|
|
- user_creates_comment
|
|
- user_creates_comment_on_at_least_four_different_days_within_a_week
|
|
- user_views_pages_on_at_least_four_different_days_within_a_week
|
|
- user_views_pages_on_at_least_four_different_hours_within_a_day
|
|
- user_views_pages_on_at_least_nine_different_days_within_two_weeks
|
|
- user_views_pages_on_at_least_twelve_different_hours_within_five_days
|
|
- user_publishes_post
|
|
- user_publishes_post_at_least_two_times_within_week
|
|
- user_publishes_post_at_least_two_times_within_two_weeks
|
|
exclude:
|
|
bots: true
|
|
|
|
# Dashboard performance
|
|
cache: true
|
|
precision: 2
|
|
|
|
# For logged-in users we'll explicitly pass participant.
|
|
# If we experiment on non-logged in, we don't want to collect cookies anyway.
|
|
cookies: false
|