73e560f7a2
* . * . * . * . * . * specs * specs * . * . * merge * moving cheese around * layout * . * merge * Fixed invalid selector for search. * Fixed issue with URL not updating after a search. * do things better * kinda merge * whoops * whoops * . * . * im a backend developer * tooltips 1.0.1 * tooltips 1.0.1 * fix * specs * . * reduce the spacing * schema drop * reverts * revert * reverts * revert * drop unused nav * drop unused nav * drop unused nav * drop unused nav * typos * paths fixes * speeding up JS * js * search * Reworked initializeTopBarIcons.js * Reworked initializeNavigation.js * Small refactor and updated exported function API docs. * Fixed a bug I created when refactoring the top navigation icons. * sidebar nav links * spec * Fixed issues with logged out experience for top nav icons. * Small refactor of top nav icons. * Small refactor and added some API docs. * Added @testing-library/dom * Added tests for top navigation utilities. * spec * spec Co-authored-by: Nick Taylor <nick@dev.to>
56 lines
2.2 KiB
Markdown
56 lines
2.2 KiB
Markdown
# A/B testing
|
|
|
|
We use the [Field Test](https://github.com/ankane/field_test) gem for conducting
|
|
simple A/B tests.
|
|
|
|
If you want to propose an A/B test of a feature, you may make a pull request
|
|
which defines the hypotheses and what admins should look for to declare a winner.
|
|
As A/B tests are going to have results that may differ from Forem to Forem, the
|
|
process is relatively immature. In the future we may have more studies that can return anonymous ecosystem-wide results.
|
|
|
|
A/B tests are inherently the most useful in _large_ Forems, where qualitative
|
|
feedback may be more useful on small Forems. As such, [DEV](https://dev.to)
|
|
is our largest Forem and therefore can provide us with the most feedback for
|
|
our existing A/B tests. However, we must keep in mind that DEV results may
|
|
not apply well to future large Forems. We should seek to re-run useful experiments within the ecosystem after time has passed.
|
|
|
|
## Creating a new A/B test
|
|
|
|
Follow the guidelines of the field test gem and add the test info to [config/field_test.yml](https://github.com/forem/forem/blob/master/config/field_test.yml).
|
|
|
|
Then where you want to trigger the variant, you'll add some code like this:
|
|
|
|
```ruby
|
|
test_variant = field_test(:follow_implicit_points, participant: user)
|
|
case test_variant
|
|
when "no_implicit_score"
|
|
0
|
|
when "half_weight_after_log"
|
|
Math.log(occurrences + bonus + 1) * 0.5
|
|
when "double_weight_after_log"
|
|
Math.log(occurrences + bonus + 1) * 2.0
|
|
when "double_bonus_before_log"
|
|
Math.log(occurrences + (bonus * 2) + 1)
|
|
when "without_weighting_bonus"
|
|
Math.log(occurrences + 1)
|
|
else # base - Our current "default" implementation
|
|
Math.log(occurrences + bonus + 1) # + 1 in all cases is to avoid log(0) => -infinity
|
|
end
|
|
```
|
|
|
|
Which would find or create the test variant for that user in particular. If
|
|
this code is not called in the controller or view, you'll need to first
|
|
include the gem helpers at the top of the file...
|
|
|
|
```
|
|
include FieldTest::Helpers
|
|
```
|
|
|
|
To record a successful field test outcome, you should call something like this
|
|
|
|
```ruby
|
|
Users::RecordFieldTestEventWorker
|
|
.perform_async(user_id, :follow_implicit_points, "user_creates_reaction")
|
|
```
|
|
|
|
And modify that class as needed to determine whether to record the successful trial.
|