* Refresh an individual user's segmentation
* Refresh segment if creating a published article
* Quick refactor
* Refresh segment if update publishing
* Quick refactor
* Refresh segment if update user settings
* Refresh segments after user update + role change
* Using latest_article_updated_at
* Sidekiq uses JSON for arguments, needs basic types
* Fix test issues with last_updated_at and sidekiq params
* Tests update continues
* Consolidate dependencies
* Method names acknowledging manual exclusive
* Fix test name copypasta
* Rubocop
* Try to clarify Creator#series
* Try to avoid naming predicate
* Remove erroneously included helper
* Revert "(Temporarily) Remove multiple reactions from #index (#19142)"
This reverts commit a45d300639.
* Clean up reaction category count mechanism for #index
* Remove, unnecessary
* Expose reaction categories in JSON response
* Update test with new categories
* Fix flaky sample / minimum floor
* buildArticle (infinite scroll) with multiple reactions
* If we're doing #index front-end, we aren't feature-flag'd
* nbsp
* react (home feed) has multiple_reactions
* Update app/assets/javascripts/utilities/buildArticleHTML.js
Co-authored-by: Rajat Talesra <rajat@forem.com>
* Clean up Reactable#reaction_categories
* Use 'multiple_reactions_icons_container'
* Try adding a ReactionCount test
* Adapt memory fix from 76dd53d
* Try adding categories to fixture
* Attempt to eager-load distinct public categories
* Setting dependent to the default for rubocop
* Try making image assets more public?
* Revert "Fix flaky sample / minimum floor"
---------
Co-authored-by: Rajat Talesra <rajat@forem.com>
Co-authored-by: Mac Siri <mac@forem.com>
* Ensure spam doesn't hit logged-out /latest
* Restore load-bearing constant
* Narrow selector avoids the new 'latest' banner
* Test scenario for signed-in low-score content
* Remove false-negative test
* Actual test for the logged-in scenario
* Signed-in users get poor quality content
* Feeds (RSS) use an entirely different query
* Pass @stories
* Update tests with new minimum score
* Don't change published_at for articles that have already been published (even if unpublished later)
* Removed unused code related to changing published_at
* Added specs for updating published_from_feed articles, fixed specs
* Removed old validation
* Added specs for article validation published_from_feed
* Added and reorganized specs for the Article model (setting published_at)
* Remove the commented code, added a spec
* Added specs for unpublishing
* Removed unused update_published_at from article attributes spec
* Fixed nullifying published_at seconds when republishing from rich editor
Prior to this commit we had a hard-coded 0; there are other hard-coded
score ranges throughout the code-base. These are unexplained magic
numbers. The hope with this change is to provide a mechanism for Forem
owners to adjust a configuration and see a change in behavior.
Related to forem/forem-internal-eng#484
* Article query spec for scheduled articles
* Added scheduled article badge on the user dashboard
* Added published_at field to editor options
* Accept and validate published_at from editor
* Refactor published_at validation
* Allow 1-minute difference in published_at
* Notice on an unpublished article page
* Added specs for 'Click to edit' link on scheduled article preview page
* ContextNotification model
* Articles::Publish worker
* Added specs for articles publish worker
* Schedule publish articles worker
* Added tests to check for scheduled posts in feeds
* Don't allow managing scheduled articles
* Don't send notifications for scheduled articles
* Set published_at in Articles::Updater when publishing
* Published_at value in post options
* Pass timezone and set published_at accordingly
* Limit setting published_at to the future
* Readonly published_at for articles that were already published
* Chagning published_at format in editor v1 (start)
* Changed published_at format in frontmatter, specs
* Added specs for updating published_at from frontmatter
* Fixed accepting past published_at for articles published_from_feed
* Enabled published_at validation: don't allow updating published_at for already published articles
* Validate published_at on create
* Added a spec for updating published_at for exported articles
* Fixed specs related to creating articles with past published_at
* Fixed specs related to past published_at for articles
* Added a hack so that admins would be able to update published_at
* Switch button text schedule/publish when changin publishedAt
* Fixed saving published_at with timezone
* Added a feature flag for scheduling articles
* Default text in markdown editor depends on feature flag
* Enable article editor cache again
* Fixed the default value in the markdown editor
* Fix sitemaps spec
* Removed tooltip
* Fixed articles update specs
* Added missing locales
* Fixed article create specs
* Fixed spec
* Removed commented code
* Returned enabling extensions in the schema
* Returned accidentally deleted constraint
* Make articles query spec more stable
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
* Removed commented code
* Removed unused code
* A clearer policy
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
* Use StringInquirer for article current state
* Added a note and todo to articles factory past trait
* Remove duplicated PropType
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
* Refactor query in the Articles::PublishWorker
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Refactor articleForm.jsx
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Removed specs that are no longer relevant
* Removed useless onKeyUp on a hidden input
* Refactored articleForm
* Hide scheduling from post options when published_at is readonly
* Run sends notifications worker every 5 minutes instead of every minute
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
Prior to this change, we setup our feed variant query to have a cached
randomization seed. For a given user this seed is cached for 15
minutes. The goal of this is to provide mostly consistent sort orders
on the feed articles during that 15 minute window.
With this change, we allow for a feed variant to say ignore the cached
seed and generate a new one for this call.
I have also added explicit settings to each of the feed variants to
reflect our desired intentions for those features.
Relates to
- forem/forem#17833
- forem/forem#17826Closesforem/forem#17940
Prior to this commit, we were always using a new random number for a
query. With this commit we cache the randomized seed; which allows us
to have "a repeatable random" experience within a windo of time.
Closesforem/forem#17826
The commit includes three changes that work to allow us to "seed" the
randomization. When using the same seed, and randomizer should/must
return the same sequence of numbers.
First, I added the "seed" parameter to the variant query. If you want
the same sequence of random numbers from query to query, pass the same
seed. Otherwise, we'll use a Ruby generated random seed.
Second, I added a virtual column to the virtual `article_relevancies`
table. If you provide a seed, and call the query twice, the first row
in each of `article_relevancies` will have the same `randomized_value`,
likewise the second row, etc.
Third, I amended the existing `order_by_lever` that had a `RANDOM()`
postgresql function call. I replaced that with the `randomized_value`
which is computed based on the provided seed.
Fundamentally the idea is to allow for randomness but introduce possible
repeatability; a hard thing considering that some of the inner selection
criteria is not fixed (e.g. number of reactions can change from moment
to moment).
I wrote a [complimentary blog post][1] to further explain what's
happening.
Related to forem/forem#17826
[1]:https://takeonrules.com/2022/06/03/adding-reproducible-randomization-to-sql-queries/
* Adding more context to `/admin/abtests`
Prior to this commit, the `/admin/abtests` was fully rendered by the
field_test gem (see
https://github.com/ankane/field_test/blob/master/app/views/field_test/experiments/index.html.erb
and
https://github.com/ankane/field_test/blob/master/app/views/field_test/experiments/show.html.erb).
With this commit, we're pre-pending our own
partial (e.g. `app/views/field_test/experiments/_experiments.html.erb`)
in the view paths; which means instead of rendering
https://github.com/ankane/field_test/blob/master/app/views/field_test/experiments/_experiments.html.erb
we render our newly created
`app/views/field_test/experiments/_experiments.html.erb`)
Why the ugly antics in the view? As I'm not fully certain if this will
"meet" the full needs of those monitoring the experiments. I'm also
constructing this to most closely match the spreadsheet that has been
assembled for tracking this information.
Why introduce another caching layer for the variant assembly? The
original caching
layer (e.g. `Articles::Feeds::VariantAssembler.pre_assembled_variants`)
is in place to provide the quickest access to the assembled variants.
But due to the nature of the implemented view, we are needing a
programmatic representation of all variants. And the added Rails cache
is there to minimize disk reads.
Closesforem/forem#17820
* Update app/models/articles/feeds/variant_assembler.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* Update app/models/articles/feeds/variant_assembler.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* Add tooltip for cover image button
* Add cover img tooltip to v1 editor
* Use a slightly different message
* Fix tests
* Use proper comment syntax for front matter
* Fix Cypress test
* Use regex to find matching label
Prior to this commit, the SQL fragments included variables that were set
configured at a global level.
With this commit, we're now saying that each lever "knows" what variable
it needs; and providing the means at lever declaration time to "say"
what those variable names are. (e.g. `Articles::Feeds::LEVER_CATALOG`).
Then as part of the variant configuration (in the
`./config/feed-variants/*.json` files) we now include the expected value
of those parameters; which by convention (and coercion) are integers.
This relates to forem/forem#17584 because we want to move from a
privileged user reaction that has two values (`negative` and `positive`)
into four values (`very_negative`, `negative`, `positive`, and
`very_positive`). To do that, we'll create a new lever; but that's for
another pull request.
You had a good ride, but your successor, the
`Articles::Feeds::VariantQuery`, is doing well in production. And has
been for quite awhile.
You helped us get to a better spot in regards to the feed algorithm, but
your time has come to rest as a memory in our git history.
This commit does three things:
1. Breaks the inheritance of the VariantQuery.
2. Revisits the in-line documentation to better reflect current state.
3. Removes branching logic that we don't need.
My goal is to have a VariantQuery that is what we need, conforms to the
method interface of WeightedQueryStrategy but does not rely on it.
After we merge this, we can observe if things break on DEV and fallback
to WeightedQueryStrategy if this doesn't work. Once the
`Articles::Feeds::VariantQuery` has fledged on it's own, we can remove
the feature flag and then finally remove the weighted query strategy.
But that's the future steps.
Related to forem/forem#17393
* Allowing VariantQuery for Feed Generation
Apologies for the breadth of this pull request, I had considered many
small commits, but felt that would've been more effort for the value
provided.
This commit includes the following:
- Documentation updates to the feed variant (though not the final pass)
- Renaming and adding RelevancyLevers that help differentiate
- Adding RelevancyLever#range to provide documentation
- Reducing redundent controller logic by making a
`Articles::Feeds.feed_for` method.
- Adding some configuration validation for RelevancyLevers
- Adding constants for better clarification
- Testing unhappy paths for feed configuration
- Adjusting the module namespace of some objects
- Exposing top-level configurations for variants (along with their
defaults)
- Creating the VariantyQuery that at present inherits from the
`Articles::Feeds::WeightedQueryStrategy`
As implemented, we can deploy this code to production without using the
new VariantQuery. Once we toggle on the
`:feed_uses_variant_query_feature` FeatureFlag, it will switch to using
the VariantQuery. The VariantQuery's two variants and the
internal configuration of `Articles::Feeds::WeightedQueryStrategy`
produce the same query.
The goal of this factor is to allow for a quick on and off toggle of the
feed query; to ensure that what we introduce remains performant.
- Closesforem/forem#17272
- Closesforem/forem#17276
- Closesforem/forem#17216
In addition, I will be recording a code-walkthrough and linking that
recording to the pull request.
* Apply suggestions from code review
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
**tl;dr** This PR is looking to provide a means to programmatically
assemble, from system configurations, what is currently the
[Articles::Feeds::WeightedQueryStrategy::SCORING_METHOD_CONFIGURATIONS][1].
Once we merge this PR, instead of having that constant, we'll initialize
the Articles::Feeds::WeightedQueryStrategy with the variant
configuration that we've assembled.
**Introduction**
This pull request has quite a bit going on, but as of now the production
code does not use any of it.
*Note: None of this code is bleeding into production code paths.*
Put your Ruby hat on and let's go for a ride. And apologizes for not
making this a smaller pull request. As I built this, I made many small
commits, but this became the smallest commit that provided the most
context without integrating into production code.
Before we get started, you may want to familiarize yourself with where
we're going. The `./app/models/articles/feeds/README.md` provides
further guidance; but fair warning reader, I have not reconciled the
README's language with what emerged as I wrote this pull request.
Those of you who are part of Forem Core team, you can read the
[Refinements to Our Present Feed Configuration][2]
to provide some insight into what's happening.
**Why not reconcile?** *Because I want to have all of the language and
thoughts available for you to help consider how best to name and model
this.*
**On to the Review**
The purpose of this pull request is to provide a mechanism for
engineering to provide a series of variant query levers. And to allow
us to easily configure those available query levers into a variant
query.
*Why the mix of Ruby configuration and JSON?*
The Ruby levers are meant to indicate that this is code we don't want to
expose in text based configuration because it might create SQL inject
points. (More on that when we integrate the code of this PR into the
production implementation)
The JSON levers are meant to indicate that "anyone" can pick the
appropriate/available levers and configure how they are set. In other
words, these values do not create SQL injection issues.
**What I Need from You**
I am putting this forward as a draft so you can read this code ahead of
time. I'm then going to schedule a synchronous code review where we
record our collective walk through of the implementation.
Closes:
- forem/forem#17268
- forem/forem#17269
- forem/forem#17270
- forem/forem#17271
- forem/forem#17273
Relates to:
- forem/forem#17308
- forem/forem#17245
[1]:6818ef3ed0/app/services/articles/feeds/weighted_query_strategy.rb (L84-L231)
[2]:https://forem.team/jeremy/refinements-to-our-present-feed-configuration-1p0c
This commit modifies two things:
1. The sort order of the feed.
2. Locking in the next incumbent for the feed.
In consultation with Jamie regarding the SQL, I've switched from a WHERE
IN type clause to creating a JOIN. This allows for the calculated
relevancy score to become a sortable value for the feed.
Second the incumbent feed is a slight modification of the past winner;
based on the results of a long running experiment. This adjustment is
per conversations with product.
The past experiment ran since early February, so it's time to retire it,
regroup and move forward.
Closesforem/forem#17307
* ContextNotification model
* Create and destroy context notifications when notifications are sent/unsent
* Added a transaction when creating notifications and context notifications
* Documentation for context notifications
There are two problems engrained in this:
1. The undocumented expectation of `/new` needed to other sites linking
with prefill information.
2. The prefill parameters being lost during the authentication process.
The first bug was introduced as part forem/forem#16536. The second is
likely introduced as part of a Rails upgrade (namely `request.path` most
likely once included the query parameters but with our current
implementation it does not).
Closesforem/forem#17292
Prior to this commit, we were looking at trusted user's reaction
counts. This is not the desired column to be testing.
See
f1d6291d00/app/models/reaction.rb (L25-L28)
```ruby
counter_culture :reactable,
column_name: proc { |model|
PUBLIC_CATEGORIES.include?(model.category) ? "public_reactions_count" : "reactions_count"
}
```
In the above code, all public reactions (hearts, unicorns, reading
list) go to public_reactions_count. Other things go to the
`reactions_count`; which is the thumbs up and thumbs down.
Fixesforem/forem#17260
* feat: allow reply_to and email_from to be set for an SMTP config
* feat: use these SMTP values in the mailers
* spec: test the application_mailer
* fix: validate with email, and not url
* feat: mimic macs changes from https://github.com/forem/forem/pull/16216 to use in this PR
* setup packs for admin
* refactor: order the keys and add a const for the auth methods
* feat: rename the header to a more user friendlly name
* chore: move the section with Emails
* feat: add a toggle that will show and hide the SMTP form under certain conditions
* feat: add the javaScript to handle the toggles
* feat: add a better description until we convert to a dropdown
* feat: ensure that we have declared sendgrid_enabled
* chore: add anote to the config controller
* chore: remove references of the email addresses to keep brnach scoped
* feat: tweak js
* test: cypress workflow to update smtp settings
* feat : update the smtp tests
* remove comments
* update test
* chore: rename NOTE
* feat: polisha dn test ForemInstance.only_sendgrid_enabled?
* chore: remove specs
* Update app/lib/constants/settings/smtp.rb
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* Update spec/system/admin/config/admin_updates_smtp_settings_spec.rb
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* Update spec/system/admin/config/admin_updates_smtp_settings_spec.rb
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* Update spec/system/admin/config/admin_updates_smtp_settings_spec.rb
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* Update app/javascript/packs/admin/config/smtp.js
Co-authored-by: Nick Taylor <nick@iamdeveloper.com>
* refactor js as per comments
* refactor as per comments
* Update cypress/integration/seededFlows/adminFlows/config/emailServerSettingsSection.spec.js
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
* refactor: update the Cypress tests
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
Co-authored-by: Nick Taylor <nick@iamdeveloper.com>
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
* Removing Articles::Builder making policy decision
This change is a refactoring through triangulation. Given that
`ArticlePolicy#new?` returned true, I'm prepared to assume that calling
`authorize(Article)` in all cases is acceptable.
So to narrow the Builder making a policy decision I renamed the returned
value to reflect what it was actually doing in the logic. And in
renaming, flipped the polarity of the boolean. Why the flip? Because
`needs_authorization == !store_location`.
In consultation with Allison and Jennie, I'm proceeding with a short-cut
to get me unstuck. That unstuck is namely "I need to ensure that the
articles#new action can go through authorization."
Given that I'll be spending time in the authorization layer, I hope
these noted short-cuts and comments will be useful in future spelunking
efforts regarding authorization.
Closes#16529
* Update app/policies/article_policy.rb
Co-authored-by: Michael Kohl <me@citizen428.net>
* Disabling spec
* Update app/policies/article_policy.rb
Co-authored-by: Dwight Scott <dwight@forem.com>
* Update app/policies/article_policy.rb
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Michael Kohl <me@citizen428.net>
Co-authored-by: Dwight Scott <dwight@forem.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Select max from a subquery of two things
max(int, int) not a function, says postgres, once it stopped yelling
about parenthesis balancing.
* prefer greatest to select max from subquery
* Ensuring that we don't divide by zero
Adding one to a "always-ish greater than or equal to 0" value. This
should resolve a flakey test and a seemingly erratic production error.
Resolves https://app.honeybadger.io/projects/66984/faults/83596547
* Favoring min over adding 1
Due to syncrhonization antics, let's not assume time is consistent
across servers.
Timey Whimey Wibbley Wobbley
* Update app/services/articles/feeds/weighted_query_strategy.rb
Co-authored-by: Daniel Uber <djuber@gmail.com>
Co-authored-by: Daniel Uber <djuber@gmail.com>
* Ensuring we don't track views of author or unpublished
Prior to this commit, I was surprised to learn that we:
1) Tracked an author's view of their article.
2) Tracked views of an unpublished article.
This came up from a Forem creator asking if they could reset the view
counter. Or trigger the reset on publication.
I think a general business logic policy of don't track views for the
author and don't track views for unpublished articles is a reasonable
default.
Were we to pursue the clear views on publication, we'd need to consider
something that went from unpublished -> published -> unpublished ->
published. Without a more explicit state machine, triggering a
busineiss logic behavior seems a bit unexpected.
In other words, I wrote an article. There are 20 views when I realize
that I need to unpublished it. I make the changes in the unpublished
state, and re-publish. I'd assume that those 20 views would still be
"recorded" and counted towards my article's view counts.
* Adjusting condition structure
Prior to this commit, the `if` clause was rather far to the right. This
helps make the if clause more pronounced.
In my quest to find `where(attribute: value)` in controllers and even
services, I stumbled upon this pattern.
As of this commit (and prior) an Article's associated user is it's author.
* Initial work
* Implement strategy injection into WeightedQueryStrategy
* Modify field_test config and adding variants
* Change orginal to constant and make some other adjustments
* Fix hardcoded test values
