* Add http_header to FastText for Fastly compatibility
* Update app/services/articles/enrich_image_attributes.rb
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Update app/services/html/parser.rb
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Update spec/models/display_ad_spec.rb
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Fix rubocop
* Also accept canges to main image for this re-calc
* Rename display_ad_spec for compatibility with main
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* feat: update the layout for organizations
* feat: update the layout for an organization
* feat: organization logos are square
* feat: change the text on the org visit button
* feat: initialize the dropdown menu
* feat: trigger the modal
* feat: update the text in the model
* feat: tweak wording
* WIP/feat: hook up a route with an action that calls an organization delete worker.
* feat: redirect after showing the notice
* feat: update the authorization required to be super admin
* feat: add the notice to localization
* spec: integration test
* spec: fix
* spec: delete worker
* chore: update the link
* feat: add safe navigation
* feat: make sure that we show an error if the organization has credits
* fix: error message for modal unspent credits
* feat add the spec for the helper
* fix: remove rspec preface
* fix: small fixes to tests and I18n
* fix: updated the number of orgs
* feat: update the message
* feat: update the message
* chore: remove inclusion of helper
* feat: change the name of the parameter
* chore: newline
* Try to mitigate flaky spec by clearing mock space
* Fix for flaky article published spec
* Rubocop
* Tidy up the specs a bit, update the comment
* Modernize article factory (thanks, rubocop)
* Update update spec to use factory
* Use PublishWorker instead of callback
* nth_published_at -> from_newish_author
* Separate published/unpublished specs
* published_at is correct if unchanged
* Add spec for scheduled post
* update moderation notification view and message for articles
* update models to allow first published articles to notify mod after create
* update mod notifications worker and spec to support articles
* update mod notification send service and spec to support articles
* update helper and request spec to support article mod notifications
* update spec language to match updated notification message
* Update spec/workers/notifications/create_round_robin_moderation_notifications_worker_spec.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* Update spec/workers/notifications/create_round_robin_moderation_notifications_worker_spec.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* Update spec/workers/notifications/create_round_robin_moderation_notifications_worker_spec.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* Update spec/workers/notifications/create_round_robin_moderation_notifications_worker_spec.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* fix comment and refactor order of send_moderation_notification steps
* add fr.yml article mod notification config
* fix spacing around equal sign per rubocop
* fix spacing around equal sign per rubocop
* Update spec/workers/notifications/create_round_robin_moderation_notifications_worker_spec.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
---------
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* Refresh an individual user's segmentation
* Refresh segment if creating a published article
* Quick refactor
* Refresh segment if update publishing
* Quick refactor
* Refresh segment if update user settings
* Refresh segments after user update + role change
* Using latest_article_updated_at
* Sidekiq uses JSON for arguments, needs basic types
* Fix test issues with last_updated_at and sidekiq params
* Tests update continues
* Consolidate dependencies
* Method names acknowledging manual exclusive
* Fix test name copypasta
* Rubocop
* Try to clarify Creator#series
* Try to avoid naming predicate
* Remove erroneously included helper
* Trying some models for audience segmentation
* AudienceSegment basics
* Attach AudienceSegment to DisplayAd
* Possibly use a DUS to populate AudienceSegments
* Add to display ad form UI
* Add to display ad API
* Refresh strategy for audience segments
* Add user_id to async ads query
* Maybe :testing -> :manual, for no-refresh segment
* Test & tweak segment refresh
* Testing audience_segment#refresh logic
* Coverage: testing human_readable
* Scope segment refresh to recently active users
* Tweak logic for when to refresh
* Tweak experience levels to match SettingsHelper
* Test for front-end logic
* Fix test, hope this helps coverage?
* Better test names
* One worker for all, many workers for each, perform_bulk
* Fix audience segment UI, needs to use id, not enum
* cron/schedule should RefreshAll
* Singular id in RefreshWorker
* chore: rubocop
* feat: rename the worker in accordance to our guidelines and to be more verbose
* refactor: move the NotificationsModeration service to be a users query, and move the constants out to new files
* refactor: removed subscribed_to_mod_roundrobin_notifications? which was only being used once in a test
* refactor: remove the aliass from the user method and add them to notification_setting + update references
* Remove jQuery
* Added jquery back to videos
* feat: increase the available moderator count
* Removed jquery code
* Nit fix
* Revert incorrect code
* Updated jquery code to javascript
* Removed jquery from comments
* feat: updates the last_reacted_at value when saving a creation
* feat: update the query for available moderators
* spec: write a test for the available moderators
* fic: update the reaction spec with current time
* fix: add last_reacted_at to the send_spec
* fix: add last_reacted_at to the user
* chore: move the queries around
* fix: change timecop block
* fix: update the last_reacted_at for the user
* chore: try to fix timecop
* Update app/services/notifications/moderation.rb
Co-authored-by: Mac Siri <mac@forem.com>
* feat: update the tests
Co-authored-by: rt4914 <rajattalesra4914@gmail.com>
Co-authored-by: Rajat Talesra <rajat@forem.com>
Co-authored-by: Mac Siri <mac@forem.com>
* Try a ReactionCategory abstract 'model'
* Use ReactionCategory
* Lookup category, pre-reaction
* Lookup category > constant
* Category can sometimes be nil?
* Try something friendlier for code reloading
* Refactor: extract point calculation out of Reaction
* Reaction Category can have color
* Rename reaction points service to follow convention
* method signature changed as well, calculate -> call
* Fix typo, leading comment, and rename data structures for clarity
* Restore DisplayAdEvents.counts_for
* Try a rollup service for DisplayAdEvent
* Schedule daily rollup worker for one month prior
* Refactor aggregator
* More testing for rollup
* Add test for rollup attribute fidelity
* DisplayAd success_rate based on counts_for
* Moved logging and deleting comments to Moderator::UnpublishAllArticlesWorker
* Added logging for admin action (unpublish_all_articles)
* Extracted code from UnpublishAllArticlesWorker to the service
* Make it possible to add a note while unpublishing all via admin action
* Added ability to add notes for unpublishing all from moderator action panel
* Fixed setting slug for the AuditLog on unpublishing + spec
* Remove context_notifications when unpublishing all posts
* Fixed typo
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Fixed typo
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Article query spec for scheduled articles
* Added scheduled article badge on the user dashboard
* Added published_at field to editor options
* Accept and validate published_at from editor
* Refactor published_at validation
* Allow 1-minute difference in published_at
* Notice on an unpublished article page
* Added specs for 'Click to edit' link on scheduled article preview page
* ContextNotification model
* Articles::Publish worker
* Added specs for articles publish worker
* Schedule publish articles worker
* Added tests to check for scheduled posts in feeds
* Don't allow managing scheduled articles
* Don't send notifications for scheduled articles
* Set published_at in Articles::Updater when publishing
* Published_at value in post options
* Pass timezone and set published_at accordingly
* Limit setting published_at to the future
* Readonly published_at for articles that were already published
* Chagning published_at format in editor v1 (start)
* Changed published_at format in frontmatter, specs
* Added specs for updating published_at from frontmatter
* Fixed accepting past published_at for articles published_from_feed
* Enabled published_at validation: don't allow updating published_at for already published articles
* Validate published_at on create
* Added a spec for updating published_at for exported articles
* Fixed specs related to creating articles with past published_at
* Fixed specs related to past published_at for articles
* Added a hack so that admins would be able to update published_at
* Switch button text schedule/publish when changin publishedAt
* Fixed saving published_at with timezone
* Added a feature flag for scheduling articles
* Default text in markdown editor depends on feature flag
* Enable article editor cache again
* Fixed the default value in the markdown editor
* Fix sitemaps spec
* Removed tooltip
* Fixed articles update specs
* Added missing locales
* Fixed article create specs
* Fixed spec
* Removed commented code
* Returned enabling extensions in the schema
* Returned accidentally deleted constraint
* Make articles query spec more stable
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
* Removed commented code
* Removed unused code
* A clearer policy
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
* Use StringInquirer for article current state
* Added a note and todo to articles factory past trait
* Remove duplicated PropType
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
* Refactor query in the Articles::PublishWorker
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Refactor articleForm.jsx
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Removed specs that are no longer relevant
* Removed useless onKeyUp on a hidden input
* Refactored articleForm
* Hide scheduling from post options when published_at is readonly
* Run sends notifications worker every 5 minutes instead of every minute
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Moving Experiments into AbExperiment namespace
This commit entails two major concepts:
1. Extracting logic out of a worker.
2. Cleaning the conditions in which we convert experiments.
In addition, I revisited some spec names to tidy them up for
clarification and structure.
**Extracting Logic...**
For workers, my preference is that they be a simple router to another
object. A non-worker object, with it's initialize method, can both
better manage instance variables. Further, workers are somewhat "flat"
in hierarchy,
In this case, having the AbExperiment be knowledgeable both in setting
the experiment and handling conversions makes (to me) organizational
sense. For example, the AbExperiment is constructed as a wrapper to the
FieldTest gem. But the RecordFieldTestEventWorker had knowledge of
FieldTest.
With this refactor, it does not have that knowledge. This also provided
an opportunity to replace magic strings with constants.
**Cleaning the conditions...**
This relates directly to and closesforem/forem#17530. I was looking at
the experiments after 2 days, and realized that there shouldn't be
results for
`user_views_pages_on_at_least_nine_different_days_within_two_weeks`;
after all the experiment started 2 days ago, how can we have results
that look at 2 weeks.
By adding the "max" function calls, we ensure a clear boundary of
"before the experiment began" versus "while the experiment is running".
Closesforem/forem#17530
* Update spec/models/ab_experiment_spec.rb
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Apply suggestions from code review
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* feat: update all the paths as a first order of business
* feat: update all the folders to no longer appear under users
* feat: update specs
* feat: update specs
* feat: remove User namespace
* feat: remove route namespace
* feat: path change
* feat: update url
We avoid busting a user cache for 15 minutes but the tag cache is 5
hours. So we want to "nudge things along".
What this is trying to solve is the server side rendering of whether we
hide or show a button.
Related to forem/forem#17324
Related to forem/forem#17119
Prior to this commit, when we change attributes for a user we might
resave each article. The purpose of the resave is to update some of the
cached attributes of an article.
This removes some redundant code. The following removed code has
an `article.save` call. Which, the callbacks in the article (see below)
already clear the cache.
```ruby
def resave_articles
articles.find_each do |article|
if article.path
cache_bust = EdgeCache::Bust.new
cache_bust.call(article.path)
cache_bust.call("#{article.path}?i=i")
end
article.save
end
end
```
8e6981aac5/app/models/article.rb (L164)8e6981aac5/app/models/article.rb (L881-L888)
This was discovered in triaging forem/forem#17041
* Limit feeds import fanout to users with feeds
This filtering for feed settings occurs in
Feeds::Import#filter_users_from already, so the enqueued ForUser
jobs were mostly no-op, but the queue latency spiked as hundreds of
thousands of jobs were added to default.
Prescreen users to avoid enqueuing a noop job every hour for every single
user without a feed.
* Update tests to check only jobs for users with feeds are enqueued
* spec cleanup
Only include alice when we check no job is enqueued for her.
Assert no job is enqueued for alice.
Rename bob to user when alice isnt there to contrast.
* Update spec
Use update_columns to bypass setting validation (checking feed is
valid/reachable)
Remove unneeded timecop block when passing a set time
Remove sidekiq: fake and allow Import.call to be received
* Empty commit to trigger CI rebuild
* Fan out Feeds::ImportArticlesWorker
Doing all that work within a single Sidekiq job has begun taking over an
hour on DEV. Regardless of the reasons we did it that way originally, we
should be able to handle this concurrently. If we cannot, we need to
investigate why and handle it properly rather than consigning it to
sequential work.
* Fix specs
The specs make assumptions about how the code under test is implemented.
This commit does not change that, as much as I would like to. Instead,
it just aligns the assumptions with the new implementation.
The previous tests didn't actually represent reality though, since we
can't get a Time or ActiveSupport::TimeWithZone instance inside of the
`perform` method while running it through Sidekiq. Instead, the specs
seem to be relying on the fact that the time instance gets serialized to
ISO-8601/RFC3339 format and that that format is understood by Postgres.
Otherwise, I'm not sure how it would work in production as written.
The new specs reflect reality more closely. The `earlier_than` value
will be converted into an ISO8601/RFC3339 string when passed through
Sidekiq.
* Add parens to perform_bulk call
Turns out, we actually do this pretty consistently. I could've sworn I
saw a bunch of these calls without parens. ¯\_(ツ)_/¯
* Improve variable naming
This is not a list of ids, it's a list of lists of arguments for Sidekiq
jobs, the inner of which contains an id, but that's not the only thing
it contains.
This change makes it easier to resolveforem/forem#16487.
What do I mean by that? To address forem/forem#16487, I need to only
select user's who are authorized to create articles. In
forem/forem#16732 I added a method that will allow chaining of a User
scope. So with this current commit and forem/forem#16732, I'm
triangulating on an approach that will make that change easier.
I did not want to conflate those two, as mixing this PR's change and
what is necessary for the closing PR would create a more complicated
review. Not unduly complicated, but one that will require more tests
and a change in logic. And for someone reviewing the diff, those
concerns could easily be lost.
Yes, I have changed the method signature, but that method signature is
limited to one location:
```shell
❯ rg "Feeds::Import.call"
app/workers/feeds/import_articles_worker.rb
21: ::Feeds::Import.call(users_scope: users_scope, earlier_than: earlier_than)
```
So I look to the method signature a bit as an internal API, hence the change.
Related to forem/forem#16487
* Convert symbol hash keys to strings when calling .perform_async
Fixes a warning from Sidekiq 6.4.0+ about perform_async arguments
which are not equal when passed to perform (`JSON.parse(JSON.dump(arg))`
should equal arg).
This is a safety measure to prevent passing objects (like classes, or
model instances) rather than their representations (like a class name,
or a model's attributes hash).
This warning will be an error in sidekiq 7
* Turn warning into an error in non-production environments
* Use string keys for reaction notification and article fetched
Missed these two on the first pass
* Update example argument hashes for #enqueues_on_correct_queue
Since this calls perform_async under the hood we need to pass json
safe hashes in the test cases as well.
https://github.com/forem/forem/blob/main/spec/workers/shared_examples/enqueues_on_correct_queue.rb
* Convert keys from FollowData#to_h to string before perform_async
I'm not sure enough where else (outside of notification) to_h is being
called, so I'm converting here when building args, rather than in
FollowData#to_h, which might be my next step.
* Let FollowData#to_h return a hash with string keys
Update spec to use string keys as well.
* Make to_h return string keys for ReactionData
Like FollowData, the #to_h method is only used to call
notifications (this is used to enqueue sidekiq jobs).
* Remove a key that was in the hash
Since reaction_data calls to_h, it gets string and not symbol,
keys. Call Hash#except with a key that was actually there.
* Ensuring we don't track views of author or unpublished
Prior to this commit, I was surprised to learn that we:
1) Tracked an author's view of their article.
2) Tracked views of an unpublished article.
This came up from a Forem creator asking if they could reset the view
counter. Or trigger the reset on publication.
I think a general business logic policy of don't track views for the
author and don't track views for unpublished articles is a reasonable
default.
Were we to pursue the clear views on publication, we'd need to consider
something that went from unpublished -> published -> unpublished ->
published. Without a more explicit state machine, triggering a
busineiss logic behavior seems a bit unexpected.
In other words, I wrote an article. There are 20 views when I realize
that I need to unpublished it. I make the changes in the unpublished
state, and re-publish. I'd assume that those 20 views would still be
"recorded" and counted towards my article's view counts.
* Adjusting condition structure
Prior to this commit, the `if` clause was rather far to the right. This
helps make the if clause more pronounced.
* Restoring tracking of custom impressions without GA
In #15967 I (@jeremyf) introduced the "don't do tracking if Google
Analytics ID is blank". However, we still likely want to
trackCustomImpressions().
Further, I did some sleuthing and adding a couple of comments on whether
we should record page views for articles in draft status or any views by
the author.
* Ensuring we don't track views of author or unpublished
Prior to this commit, I was surprised to learn that we:
1) Tracked an author's view of their article.
2) Tracked views of an unpublished article.
This came up from a Forem creator asking if they could reset the view
counter. Or trigger the reset on publication.
I think a general business logic policy of don't track views for the
author and don't track views for unpublished articles is a reasonable
default.
Were we to pursue the clear views on publication, we'd need to consider
something that went from unpublished -> published -> unpublished ->
published. Without a more explicit state machine, triggering a
busineiss logic behavior seems a bit unexpected.
In other words, I wrote an article. There are 20 views when I realize
that I need to unpublished it. I make the changes in the unpublished
state, and re-publish. I'd assume that those 20 views would still be
"recorded" and counted towards my article's view counts.
* Revert "Ensuring we don't track views of author or unpublished"
This reverts commit 321ecbed0ac4552e175d39bf17655a2fc93b265c.
The download app broadcast (in the Generator) returns early if the
user was created less than 7 days ago. It seems counterproductive to
filter out _only_ users created in the last 7 days.
Add one day to the query range, so users created less than 8 days ago,
but more than 7 days ago, receive the final message.
* Name the created after timestamp and remove temporary variables
The logic used to calculate the live at time, the week ago time, then
compare them to find the more recent value.
Move all of this logic to a method (there's a guard to exit if
the setting is nil, so this should be safe).
* Mark method private
This is not part of the public api (for workers, this should be
`#perform` alone). Mark it private
* Proposing a new feed experiment
This commit involves tweaking a few subtle aspects of the challenger:
1) Remove the sort by published date on the relevance score. The
articles will now be listed in the order of perceived relevance.
2) Disable a few levers that did little. There really aren't any
`articles.featured = true` articles in the database.
3) Gently increase the weight of each comment in a linear manner.
4) Give a little more weight to posts that don't have followed tags.
In addition, it involves renaming the conversions to better convey their
implementation. This renaming helps create more descriptive labels for
the test results at `/admin/abtests`.
This commit also adds logic to repurpose the AbExperiment feed_strategy
logic; I envision adjusting the weighted feed strategy levers with some
frequency.
Per discussions, I'm also disabling the "not logged in" feed testing.
We'll use the LargeForemExperimental for this.
* Flipping attribute name to positive
Mentally "not disabled" is harder to parse than "enabled". This change
helps with setting an optimistic attribute.
* Bump for travis
* Bump for travis
* Removing file committed by mistake
* Extracting method
* Load article before creating a page view for an invalid one
This should resolve a validation error in PageView.create! when the
article does not exist (perhaps it was deleted, or the user suspended,
or it was invalid data sent from the client).
This had been happening dozens of times per day for the last 10
months.
* Use an intention revealing symbol instead of calculated id
Since we only require that find_by not find anything, pass a symbol
that's not going to be the id for any article under any conditions.
As an added benefit, this provides a clear indication of the purpose
of the symbol, without needing to mentally confirm that
```sql
SELECT * FROM articles
WHERE id IN (
SELECT (1 + MAX(id)) FROM articles
) LIMIT 1
```
actually never gives any articles back.
* Use the tag_list method, rather than the cached_tag_list attribute
This is a follow on to #15456 and #15637
Saving the article in #15637 was a work-around to force the article to
have a valid cached tag list. However, we can rely on the tag_list
method to find the associated tags.
Update the worker to use tag_list (rather than the decorator's
cached_tag_list.array) and flatten the tag list rather than flat
mapping String#split over the non-nil cached_tag_lists.
* Remove unused join on tags
The query logging showed each article's tags were loaded twice in a
row (select by id, immediately followed by select by id). Removing the
join removed this behavior. I suspect mapping `#tag_list` over
articles is ignoring the loaded tags.
Performance (in my local environment) improved about 30% when this join was removed.
