Commit graph

8 commits

Author SHA1 Message Date
Fernando Valverde
227fc66705
Timecop freeze ab_experiment spec (#18655) 2022-11-01 11:35:37 -04:00
Ben Halpern
cd5c86c760
Feed experiment for 09-26-2022 (#18501)
* Feed experiment for 09-26-2022

* Fix variant

* Add proper tests
2022-09-27 08:48:15 -06:00
Anna Buianova
ff76cdd3c5
Scheduling articles (#17939)
* Article query spec for scheduled articles

* Added scheduled article badge on the user dashboard

* Added published_at field to editor options

* Accept and validate published_at from editor

* Refactor published_at validation

* Allow 1-minute difference in published_at

* Notice on an unpublished article page

* Added specs for 'Click to edit' link on scheduled article preview page

* ContextNotification model

* Articles::Publish worker

* Added specs for articles publish worker

* Schedule publish articles worker

* Added tests to check for scheduled posts in feeds

* Don't allow managing scheduled articles

* Don't send notifications for scheduled articles

* Set published_at in Articles::Updater when publishing

* Published_at value in post options

* Pass timezone and set published_at accordingly

* Limit setting published_at to the future

* Readonly published_at for articles that were already published

* Chagning published_at format in editor v1 (start)

* Changed published_at format in frontmatter, specs

* Added specs for updating published_at from frontmatter

* Fixed accepting past published_at for articles published_from_feed

* Enabled published_at validation: don't allow updating published_at for already published articles

* Validate published_at on create

* Added a spec for updating published_at for exported articles

* Fixed specs related to creating articles with past published_at

* Fixed specs related to past published_at for articles

* Added a hack so that admins would be able to update published_at

* Switch button text schedule/publish when changin publishedAt

* Fixed saving published_at with timezone

* Added a feature flag for scheduling articles

* Default text in markdown editor depends on feature flag

* Enable article editor cache again

* Fixed the default value in the markdown editor

* Fix sitemaps spec

* Removed tooltip

* Fixed articles update specs

* Added missing locales

* Fixed article create specs

* Fixed spec

* Removed commented code

* Returned enabling extensions in the schema

* Returned accidentally deleted constraint

* Make articles query spec more stable

Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>

* Removed commented code

* Removed unused code

* A clearer policy

Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>

* Use StringInquirer for article current state

* Added a note and todo to articles factory past trait

* Remove duplicated PropType

Co-authored-by: Suzanne Aitchison <suzanne@forem.com>

* Refactor query in the Articles::PublishWorker

Co-authored-by: Mac Siri <krairit.siri@gmail.com>

* Refactor articleForm.jsx

Co-authored-by: Mac Siri <krairit.siri@gmail.com>

* Removed specs that are no longer relevant

* Removed useless onKeyUp on a hidden input

* Refactored articleForm

* Hide scheduling from post options when published_at is readonly

* Run sends notifications worker every 5 minutes instead of every minute

Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
2022-07-07 17:32:49 +03:00
Jeremy Friesen
127e3ac248
Adding page view goal for experiments (#17696)
With comments, publishing articles, and reactions we had symmetry on two
goals:

- Create a _subject_ (e.g. Comment, Published Article, Reaction)
- Create four _subjects_ within a week.

For page views we only had "Create four _subjects_ within a week."  This
PR addes the "Create a _subject_".

To do this required adjusting some tests as they were too specific in
nature (looking at an expected count).

In addition, in consultation with Jennie, I've updated the experiment
order to better reflect some hierarchical importance.

**Rollback considerations:**

This also includes a feature flag that we can explicitly disable if we
overload the application with workers handling reaction goals.  Using
`FeatureFlag.accessible?(:field_test_event_single_create_pageview)`
returns `true` unless we explicitly disable this flag.

Related to forem/forem#17673
Related to forem/forem#17669
Closes forem/forem#17691
2022-05-16 15:07:42 -04:00
Jeremy Friesen
8a4e1ebeba
Adding more goals to feed experiments (#17673)
This commit adds three new feed experiment goals:

* user publishes four posts within a week
* user reacts with a "heart", "unicorn", or "reading list" to an article
* user reacts with a "heart", "unicorn", or "reading list" to an article
  four times in a week

In addition it adds it to the existing experiment.  Adding it to the
existing experiment is acceptable because:

1. We can ignore the results
2. The experiments are structured such that these new goals could be met
   with prior data.
3. Both variants are playing by the same rules, so the results relative
   to each other are valid.

**There is a nuanced assumption in how we handle reactions:**

This implementation does not count by unique article reactions. However,
most folks will ❤️ , 🦄 , and 🔖 in one swoop; thus
those 3 reactions are all grouped into happening on one day.

Further, it's a reflection of the person taking an action after reading
a post, not how much action on that particular post.

**Rollback considerations:**

This also includes a feature flag that we can explicitly disable if we
overload the application with workers handling reaction goals.  Using
`FeatureFlag.accessible?(:field_test_event_for_reactions)` returns
`true` unless we explicitly disable this flag.

Closes forem/forem#17669
2022-05-13 13:22:06 -04:00
Jeremy Friesen
27b82996ce
Adding new relevancy lever for privileged reactions (#17598)
* Adding new relevancy lever for privileged reactions

This PR includes four changes:

1. Renaming the variant to remove Jennie's name
2. Starting a new experiment
3. Adding a new relevancy lever
4. Adding a new variant that uses the relevancy lever

1 and 2 is somewhat straight-forward.

For 3, we're looking at having a total of 5 "slots" for privileged user
reactions.  Hence the four boundary values.

For 4, we're configuring the granular levers with the following:

- `[-∞..-5)` should have a 0.05
- `[-5..0)` should have a 0.5
- `[0..5)` should have a 0.9
- `[5..10)` should have a 0.98
- `[10..∞)` should have a 1.0

The 20220509-variant.json is a copy of 20220422-variant.json but
replaces the `privileged_user_reaction` lever with the
`privileged_user_reaction_granular` lever.

Below is the diff to highlight the similarities and differences between
the two variants.

```shell
❯ diff config/feed-variants/20220509-variant.json config/feed-variants/20220422-variant.json
90c90
<     "privileged_user_reaction_granular": {
---
>     "privileged_user_reaction": {
92,96c92,93
<         [-2, 0.05],
<         [-1, 0.5],
<         [0, 0.9],
<         [1, 0.98],
<         [2, 1]
---
>         [-1, 0.2],
>         [1, 1]
98,102c95,97
<       "fallback": 0.9,
<       "very_negative_reaction_threshold": -5,
<       "negative_reaction_threshold": 0,
<       "positive_reaction_threshold": 5,
<       "very_positive_reaction_threshold": 10
---
>       "fallback": 0.95,
>       "negative_reaction_threshold": -10,
>       "positive_reaction_threshold": 10
```

Closes forem/forem#17584

* Update app/models/articles/feeds.rb

Co-authored-by: Josh Puetz <josh@dev.to>

* Adjusting testing logic to reflect latest experiment

* Adjusting negative threshold

Co-authored-by: Josh Puetz <josh@dev.to>
2022-05-09 11:32:57 -04:00
Jeremy Friesen
66119985b6
Adding a field test option for post publishing goals (#17589)
* Adding a field test option for post publishing goals

We have page view and comment view "goals" but have not looked at
publication "goals".  We're using these "goals" to help us refine the
feed towards sustainable engagement and community health.

The conjecture on publication goals is that the content on the feed
itself can nudge folks in their decision to publish posts.

Closes forem/forem#17585

* Fixing broken test
2022-05-06 13:54:50 -04:00
Jeremy Friesen
fe2e53cc50
Moving Experiments into AbExperiment namespace (#17532)
* Moving Experiments into AbExperiment namespace

This commit entails two major concepts:

1. Extracting logic out of a worker.
2. Cleaning the conditions in which we convert experiments.

In addition, I revisited some spec names to tidy them up for
clarification and structure.

**Extracting Logic...**

For workers, my preference is that they be a simple router to another
object.  A non-worker object, with it's initialize method, can both
better manage instance variables.  Further, workers are somewhat "flat"
in hierarchy,

In this case, having the AbExperiment be knowledgeable both in setting
the experiment and handling conversions makes (to me) organizational
sense.  For example, the AbExperiment is constructed as a wrapper to the
FieldTest gem.  But the RecordFieldTestEventWorker had knowledge of
FieldTest.

With this refactor, it does not have that knowledge.  This also provided
an opportunity to replace magic strings with constants.

**Cleaning the conditions...**

This relates directly to and closes forem/forem#17530.  I was looking at
the experiments after 2 days, and realized that there shouldn't be
results for
`user_views_pages_on_at_least_nine_different_days_within_two_weeks`;
after all the experiment started 2 days ago, how can we have results
that look at 2 weeks.

By adding the "max" function calls, we ensure a clear boundary of
"before the experiment began" versus "while the experiment is running".

Closes forem/forem#17530

* Update spec/models/ab_experiment_spec.rb

Co-authored-by: Mac Siri <krairit.siri@gmail.com>

* Apply suggestions from code review

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>

Co-authored-by: Mac Siri <krairit.siri@gmail.com>
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
2022-05-03 21:34:12 -04:00