docbrown/config/feed-variants
Ben Halpern 6d669c8a5c
Add new feed experiment for Jan 26 (#20563)
* Add a/b tests for Jan 26

* Fix feed lever

* Update config/field_test.yml

* Update config/field_test.yml

* Update app/models/articles/feeds.rb
2024-01-26 16:53:26 -05:00
..
20220415-incumbent.json Adding VariantQuery reseed randomization option (#17942) 2022-06-16 15:04:08 -04:00
20220422-variant.json Adding VariantQuery reseed randomization option (#17942) 2022-06-16 15:04:08 -04:00
20220509-variant.json Adding VariantQuery reseed randomization option (#17942) 2022-06-16 15:04:08 -04:00
20220518-variant.json Adding VariantQuery reseed randomization option (#17942) 2022-06-16 15:04:08 -04:00
20220525-variant.json Adding VariantQuery reseed randomization option (#17942) 2022-06-16 15:04:08 -04:00
20220603-variant-a.json Adding VariantQuery reseed randomization option (#17942) 2022-06-16 15:04:08 -04:00
20220603-variant-b.json Adding VariantQuery reseed randomization option (#17942) 2022-06-16 15:04:08 -04:00
20220617-variant-a.json Adding 2022-06-17 feed variant (#17950) 2022-06-16 15:32:38 -04:00
20220617-variant-b.json Adding 2022-06-17 feed variant (#17950) 2022-06-16 15:32:38 -04:00
20220617-variant-c.json Adding 2022-06-17 feed variant (#17950) 2022-06-16 15:32:38 -04:00
20220712-variant.json Starting 2022-07-12 Experiment (#18083) 2022-07-12 14:51:47 -04:00
20220719-variant-a.json Adding 2022-07-19 variant (#18156) 2022-07-19 14:59:53 -04:00
20220719-variant-b.json Adding 2022-07-19 variant (#18156) 2022-07-19 14:59:53 -04:00
20220726-variant-a.json Feed experiment for 2022-07-26 (#18227) 2022-07-27 08:19:32 -05:00
20220726-variant-b.json Feed experiment for 2022-07-26 (#18227) 2022-07-27 08:19:32 -05:00
20220801-variant-a.json Feed experiment for 2022-08-01 (#18269) 2022-08-04 11:06:58 -04:00
20220801-variant-b.json Feed experiment for 2022-08-01 (#18269) 2022-08-04 11:06:58 -04:00
20220823-variant.json Feed experiment for 08-23-2022 (#18371) 2022-08-26 10:30:01 -05:00
20220830-variant-a.json Feed experiment for week of Aug 29, 2022 (#18395) 2022-08-31 09:28:27 -05:00
20220830-variant-b.json Feed experiment for week of Aug 29, 2022 (#18395) 2022-08-31 09:28:27 -05:00
20220830-variant-c.json Feed experiment for week of Aug 29, 2022 (#18395) 2022-08-31 09:28:27 -05:00
20220907-variant-a.json Feed experiment for 09-07-2022 (#18437) 2022-09-09 08:17:00 -05:00
20220907-variant-b.json Feed experiment for 09-07-2022 (#18437) 2022-09-09 08:17:00 -05:00
20220907-variant-c.json Feed experiment for 09-07-2022 (#18437) 2022-09-09 08:17:00 -05:00
20220915-variant-a.json Feed experiment for 09-15-2022 (#18463) 2022-09-16 11:27:42 -05:00
20220915-variant-b.json Feed experiment for 09-15-2022 (#18463) 2022-09-16 11:27:42 -05:00
20220915-variant-c.json Feed experiment for 09-15-2022 (#18463) 2022-09-16 11:27:42 -05:00
20220920-variant-a.json Feed experiment for 09-20-2022 (#18475) 2022-09-21 12:19:38 -04:00
20220920-variant-b.json Feed experiment for 09-20-2022 (#18475) 2022-09-21 12:19:38 -04:00
20220920-variant-c.json Feed experiment for 09-20-2022 (#18475) 2022-09-21 12:19:38 -04:00
20220926-variant-a.json Feed experiment for 09-26-2022 (#18501) 2022-09-27 08:48:15 -06:00
20220926-variant-b.json Feed experiment for 09-26-2022 (#18501) 2022-09-27 08:48:15 -06:00
20220926-variant-c.json Feed experiment for 09-26-2022 (#18501) 2022-09-27 08:48:15 -06:00
20221004-variant-a.json Feed experiment for 10-04-2022 (#18533) 2022-10-05 06:41:51 -04:00
20221004-variant-b.json Feed experiment for 10-04-2022 (#18533) 2022-10-05 06:41:51 -04:00
20221011-variant-a.json Feed experiment for 10-11-2022 (#18566) 2022-10-11 08:56:49 -06:00
20221011-variant-b.json Feed experiment for 10-11-2022 (#18566) 2022-10-11 08:56:49 -06:00
20221018-variant-a.json Feed experiment for 10-18-2022 (#18600) 2022-10-19 10:43:52 -04:00
20221018-variant-b.json Feed experiment for 10-18-2022 (#18600) 2022-10-19 10:43:52 -04:00
20221018-variant-c.json Feed experiment for 10-18-2022 (#18600) 2022-10-19 10:43:52 -04:00
20221025-variant-a.json Feed experiment for 10-25-2022 (#18628) 2022-10-25 16:16:28 -04:00
20221025-variant-b.json Feed experiment for 10-25-2022 (#18628) 2022-10-25 16:16:28 -04:00
20221025-variant-c.json Feed experiment for 10-25-2022 (#18628) 2022-10-25 16:16:28 -04:00
20221031-variant-a.json Feed experiment for 10-31-2022 (#18670) 2022-11-02 09:00:59 -04:00
20221031-variant-b.json Feed experiment for 10-31-2022 (#18670) 2022-11-02 09:00:59 -04:00
20221031-variant-c.json Feed experiment for 10-31-2022 (#18670) 2022-11-02 09:00:59 -04:00
20221107-variant-a.json Feed experiment for 11-07-2022 (#18702) 2022-11-08 13:06:58 -05:00
20221107-variant-b.json Feed experiment for 11-07-2022 (#18702) 2022-11-08 13:06:58 -05:00
20221107-variant-c.json Feed experiment for 11-07-2022 (#18702) 2022-11-08 13:06:58 -05:00
20221107-variant-d.json Feed experiment for 11-07-2022 (#18702) 2022-11-08 13:06:58 -05:00
20221122-variant-a.json Feed experiment for 11-22-2022 (#18769) 2022-11-23 14:18:26 -05:00
20221122-variant-b.json Feed experiment for 11-22-2022 (#18769) 2022-11-23 14:18:26 -05:00
20221122-variant-c.json Feed experiment for 11-22-2022 (#18769) 2022-11-23 14:18:26 -05:00
20221215-variant-a.json Feed experiment for 12-15-2022 (#18866) 2022-12-16 10:15:01 -05:00
20221215-variant-b.json Feed experiment for 12-15-2022 (#18866) 2022-12-16 10:15:01 -05:00
20221215-variant-c.json Feed experiment for 12-15-2022 (#18866) 2022-12-16 10:15:01 -05:00
20230110-variant-a.json Feed experiment for 01-11-2023 (#18949) 2023-01-11 17:03:29 -05:00
20230110-variant-b.json Feed experiment for 01-11-2023 (#18949) 2023-01-11 17:03:29 -05:00
20230110-variant-c.json Feed experiment for 01-11-2023 (#18949) 2023-01-11 17:03:29 -05:00
20230424-variant-a.json Add feed experiment for 04/24 (#19388) 2023-04-25 07:49:06 -04:00
20230424-variant-b.json Add feed experiment for 04/24 (#19388) 2023-04-25 07:49:06 -04:00
20230424-variant-c.json Add feed experiment for 04/24 (#19388) 2023-04-25 07:49:06 -04:00
20230430-variant-a.json Add feed experiment for 4/30 (#19406) 2023-05-02 07:05:12 -04:00
20230430-variant-b.json Add feed experiment for 4/30 (#19406) 2023-05-02 07:05:12 -04:00
20230430-variant-c.json Add feed experiment for 4/30 (#19406) 2023-05-02 07:05:12 -04:00
20230511-variant-a.json Add feed experiment for 5/11 (#19458) 2023-05-12 12:17:25 -04:00
20230511-variant-b.json Add feed experiment for 5/11 (#19458) 2023-05-12 12:17:25 -04:00
20230511-variant-c.json Add feed experiment for 5/11 (#19458) 2023-05-12 12:17:25 -04:00
20230814-variant-a.json Add feed experiment for Aug 14 (#19937) 2023-08-15 10:01:08 -04:00
20230814-variant-b.json Add feed experiment for Aug 14 (#19937) 2023-08-15 10:01:08 -04:00
20230814-variant-c.json Add feed experiment for Aug 14 (#19937) 2023-08-15 10:01:08 -04:00
20230918-variant-a.json New feed experiments for 9/18 (#20130) 2023-09-18 14:40:09 -07:00
20230918-variant-b.json New feed experiments for 9/18 (#20130) 2023-09-18 14:40:09 -07:00
20230918-variant-c.json New feed experiments for 9/18 (#20130) 2023-09-18 14:40:09 -07:00
20230927-variant-a.json Add new feed experiments (#20171) 2023-09-27 08:21:21 -04:00
20230927-variant-b.json Add new feed experiments (#20171) 2023-09-27 08:21:21 -04:00
20231001-variant-a.json Add new feed experiment for 10-1 (#20194) 2023-10-01 13:52:33 +07:00
20231001-variant-b.json Add new feed experiment for 10-1 (#20194) 2023-10-01 13:52:33 +07:00
20231017-variant-a.json Add feed test for oct 17 (#20252) 2023-10-17 11:33:49 -04:00
20231017-variant-b.json Add feed test for oct 17 (#20252) 2023-10-17 11:33:49 -04:00
20231017-variant-c.json Add feed test for oct 17 (#20252) 2023-10-17 11:33:49 -04:00
20231023-variant-a.json New AB test for October 23 (#20274) 2023-10-23 10:30:12 -04:00
20231023-variant-b.json New AB test for October 23 (#20274) 2023-10-23 10:30:12 -04:00
20231108-variant-a.json New Feed experiment for Nov 8 (#20328) 2023-11-08 13:33:26 -05:00
20231108-variant-b.json New Feed experiment for Nov 8 (#20328) 2023-11-08 13:33:26 -05:00
20231127-variant-a.json Add recommended articles to feed experiment (Nov 27 experiment) (#20399) 2023-11-29 18:22:18 +00:00
20231127-variant-b.json Add recommended articles to feed experiment (Nov 27 experiment) (#20399) 2023-11-29 18:22:18 +00:00
20231127-variant-c.json Add recommended articles to feed experiment (Nov 27 experiment) (#20399) 2023-11-29 18:22:18 +00:00
20231205-variant-a.json Feed experiment for Dec 5 (#20427) 2023-12-06 16:05:50 -05:00
20231205-variant-b.json Feed experiment for Dec 5 (#20427) 2023-12-06 16:05:50 -05:00
20231205-variant-c.json Feed experiment for Dec 5 (#20427) 2023-12-06 16:05:50 -05:00
20240104-variant-a.json Add articles clickbait_score as factor in final feed ordering (#20493) 2024-01-05 09:37:53 -05:00
20240104-variant-b.json Add articles clickbait_score as factor in final feed ordering (#20493) 2024-01-05 09:37:53 -05:00
20240104-variant-c.json Add articles clickbait_score as factor in final feed ordering (#20493) 2024-01-05 09:37:53 -05:00
20240126-variant-a.json Add new feed experiment for Jan 26 (#20563) 2024-01-26 16:53:26 -05:00
20240126-variant-b.json Add new feed experiment for Jan 26 (#20563) 2024-01-26 16:53:26 -05:00
20240126-variant-c.json Add new feed experiment for Jan 26 (#20563) 2024-01-26 16:53:26 -05:00
original.json Adding VariantQuery reseed randomization option (#17942) 2022-06-16 15:04:08 -04:00
README.md Adding VariantQuery reseed randomization option (#17942) 2022-06-16 15:04:08 -04:00

Feed Variants

This folder contains the feed variants that we have, are, or will be testing. By convention the file's basename without extension is the name of the variant (e.g. "./config/feed-variants/20220415-incumbent.json" encodes the variant named 20220415-incumbent).

There exists one variant, original.json, which is our fallback variant. This fallback is performant and adequate for generating the feed for DEV.to. It is present in case we need to quickly turn off an experiment. The name original is the same as the AbExperiment::ORIGINAL_VARIANT constant.

Do not remove original.json without consideration for how to rollback a feed experiment.

Variants are loaded into production as needed. That "need" is determined by the configuration of the ./config/field_test.yml. Once we load a variant into production, we cache that variant (see Articles::Feeds::VariantAssembler.variants_cache).

As part of our test suite we assemble and verify each defined variant.

How to Make a Feed Variant

A feed variant is written in JSON format. A feed variant configures one or more Articles::Feeds::RelevancyLevers. The variant configuration defines:

  • The cases and fallback for each of the selected relevancy levers
  • Optionally the sort order of the query results
  • Optionally a few high level parameters.

The relevancy levers and their SQL query fragments are defined in Articles::Feeds::LEVER_CATALOG. Any levers not configured for a variant are omitted for that variant (but available for other variants).

The available sort order is also defined in Articles::Feeds::LEVER_CATALOG.

The high level parameters are defined in Aritcles::Feeds::VariantQuery::Config and as of <2022-05-06 Fri> are:

  • max_days_since_published: only consider articles that were published no more than the max_days_since_published.

  • reseed_randomizer_on_each_request: when true, each time you call the query you will get different randomized numbers; when false, the resulting randomized numbers will be the same within a window of time.