* Add AlgoliaSearchable::SearchableUser
* Update spec
* Refactor
* Refactor
* Add more test
* Refactor
* Update seeder to clear Algolia index if it exist
* All Algolia directly in update_score
* Fix broken spec
* Remove new queue
* Account for more user statuses
* Remove unncessary test for now
* Move calculating comments score from worker to a service
* Display displayed comments count in feed
* Recalculating displayed comments counts on article show (if needed)
* Rubocop fixes
* Fixed lambda
* Add downstream score effects from spam role
* Move async score calc down low
* Add tests
* Add stub
* Fix typo
* No need for guard
* No need for guard
* Add http_header to FastText for Fastly compatibility
* Update app/services/articles/enrich_image_attributes.rb
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Update app/services/html/parser.rb
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Update spec/models/display_ad_spec.rb
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Fix rubocop
* Also accept canges to main image for this re-calc
* Rename display_ad_spec for compatibility with main
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Renamed display ad variables and descriptions in tests
* Renamed api docs display_ads => billboards
* Updated billboard cypress seeds
* Fixed api billboards specs
* Update db/schema.rb
Co-authored-by: Duke Greene <dukegreene@gmail.com>
---------
Co-authored-by: Ben Halpern <bendhalpern@gmail.com>
Co-authored-by: Duke Greene <dukegreene@gmail.com>
* feat: update the layout for organizations
* feat: update the layout for an organization
* feat: organization logos are square
* feat: change the text on the org visit button
* feat: initialize the dropdown menu
* feat: trigger the modal
* feat: update the text in the model
* feat: tweak wording
* WIP/feat: hook up a route with an action that calls an organization delete worker.
* feat: redirect after showing the notice
* feat: update the authorization required to be super admin
* feat: add the notice to localization
* spec: integration test
* spec: fix
* spec: delete worker
* chore: update the link
* feat: add safe navigation
* feat: make sure that we show an error if the organization has credits
* fix: error message for modal unspent credits
* feat add the spec for the helper
* fix: remove rspec preface
* fix: small fixes to tests and I18n
* fix: updated the number of orgs
* feat: update the message
* feat: update the message
* chore: remove inclusion of helper
* feat: change the name of the parameter
* chore: newline
* Try to mitigate flaky spec by clearing mock space
* Fix for flaky article published spec
* Rubocop
* Tidy up the specs a bit, update the comment
* Modernize article factory (thanks, rubocop)
* Update update spec to use factory
* Use PublishWorker instead of callback
* nth_published_at -> from_newish_author
* Separate published/unpublished specs
* published_at is correct if unchanged
* Add spec for scheduled post
* update moderation notification view and message for articles
* update models to allow first published articles to notify mod after create
* update mod notifications worker and spec to support articles
* update mod notification send service and spec to support articles
* update helper and request spec to support article mod notifications
* update spec language to match updated notification message
* Update spec/workers/notifications/create_round_robin_moderation_notifications_worker_spec.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* Update spec/workers/notifications/create_round_robin_moderation_notifications_worker_spec.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* Update spec/workers/notifications/create_round_robin_moderation_notifications_worker_spec.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* Update spec/workers/notifications/create_round_robin_moderation_notifications_worker_spec.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* fix comment and refactor order of send_moderation_notification steps
* add fr.yml article mod notification config
* fix spacing around equal sign per rubocop
* fix spacing around equal sign per rubocop
* Update spec/workers/notifications/create_round_robin_moderation_notifications_worker_spec.rb
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
---------
Co-authored-by: Ridhwana <Ridhwana.Khan16@gmail.com>
* Refresh an individual user's segmentation
* Refresh segment if creating a published article
* Quick refactor
* Refresh segment if update publishing
* Quick refactor
* Refresh segment if update user settings
* Refresh segments after user update + role change
* Using latest_article_updated_at
* Sidekiq uses JSON for arguments, needs basic types
* Fix test issues with last_updated_at and sidekiq params
* Tests update continues
* Consolidate dependencies
* Method names acknowledging manual exclusive
* Fix test name copypasta
* Rubocop
* Try to clarify Creator#series
* Try to avoid naming predicate
* Remove erroneously included helper
* Trying some models for audience segmentation
* AudienceSegment basics
* Attach AudienceSegment to DisplayAd
* Possibly use a DUS to populate AudienceSegments
* Add to display ad form UI
* Add to display ad API
* Refresh strategy for audience segments
* Add user_id to async ads query
* Maybe :testing -> :manual, for no-refresh segment
* Test & tweak segment refresh
* Testing audience_segment#refresh logic
* Coverage: testing human_readable
* Scope segment refresh to recently active users
* Tweak logic for when to refresh
* Tweak experience levels to match SettingsHelper
* Test for front-end logic
* Fix test, hope this helps coverage?
* Better test names
* One worker for all, many workers for each, perform_bulk
* Fix audience segment UI, needs to use id, not enum
* cron/schedule should RefreshAll
* Singular id in RefreshWorker
* chore: rubocop
* feat: rename the worker in accordance to our guidelines and to be more verbose
* refactor: move the NotificationsModeration service to be a users query, and move the constants out to new files
* refactor: removed subscribed_to_mod_roundrobin_notifications? which was only being used once in a test
* refactor: remove the aliass from the user method and add them to notification_setting + update references
* Remove jQuery
* Added jquery back to videos
* feat: increase the available moderator count
* Removed jquery code
* Nit fix
* Revert incorrect code
* Updated jquery code to javascript
* Removed jquery from comments
* feat: updates the last_reacted_at value when saving a creation
* feat: update the query for available moderators
* spec: write a test for the available moderators
* fic: update the reaction spec with current time
* fix: add last_reacted_at to the send_spec
* fix: add last_reacted_at to the user
* chore: move the queries around
* fix: change timecop block
* fix: update the last_reacted_at for the user
* chore: try to fix timecop
* Update app/services/notifications/moderation.rb
Co-authored-by: Mac Siri <mac@forem.com>
* feat: update the tests
Co-authored-by: rt4914 <rajattalesra4914@gmail.com>
Co-authored-by: Rajat Talesra <rajat@forem.com>
Co-authored-by: Mac Siri <mac@forem.com>
* Restore DisplayAdEvents.counts_for
* Try a rollup service for DisplayAdEvent
* Schedule daily rollup worker for one month prior
* Refactor aggregator
* More testing for rollup
* Add test for rollup attribute fidelity
* DisplayAd success_rate based on counts_for
* Moved logging and deleting comments to Moderator::UnpublishAllArticlesWorker
* Added logging for admin action (unpublish_all_articles)
* Extracted code from UnpublishAllArticlesWorker to the service
* Make it possible to add a note while unpublishing all via admin action
* Added ability to add notes for unpublishing all from moderator action panel
* Fixed setting slug for the AuditLog on unpublishing + spec
* Remove context_notifications when unpublishing all posts
* Fixed typo
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Fixed typo
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Article query spec for scheduled articles
* Added scheduled article badge on the user dashboard
* Added published_at field to editor options
* Accept and validate published_at from editor
* Refactor published_at validation
* Allow 1-minute difference in published_at
* Notice on an unpublished article page
* Added specs for 'Click to edit' link on scheduled article preview page
* ContextNotification model
* Articles::Publish worker
* Added specs for articles publish worker
* Schedule publish articles worker
* Added tests to check for scheduled posts in feeds
* Don't allow managing scheduled articles
* Don't send notifications for scheduled articles
* Set published_at in Articles::Updater when publishing
* Published_at value in post options
* Pass timezone and set published_at accordingly
* Limit setting published_at to the future
* Readonly published_at for articles that were already published
* Chagning published_at format in editor v1 (start)
* Changed published_at format in frontmatter, specs
* Added specs for updating published_at from frontmatter
* Fixed accepting past published_at for articles published_from_feed
* Enabled published_at validation: don't allow updating published_at for already published articles
* Validate published_at on create
* Added a spec for updating published_at for exported articles
* Fixed specs related to creating articles with past published_at
* Fixed specs related to past published_at for articles
* Added a hack so that admins would be able to update published_at
* Switch button text schedule/publish when changin publishedAt
* Fixed saving published_at with timezone
* Added a feature flag for scheduling articles
* Default text in markdown editor depends on feature flag
* Enable article editor cache again
* Fixed the default value in the markdown editor
* Fix sitemaps spec
* Removed tooltip
* Fixed articles update specs
* Added missing locales
* Fixed article create specs
* Fixed spec
* Removed commented code
* Returned enabling extensions in the schema
* Returned accidentally deleted constraint
* Make articles query spec more stable
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
* Removed commented code
* Removed unused code
* A clearer policy
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
* Use StringInquirer for article current state
* Added a note and todo to articles factory past trait
* Remove duplicated PropType
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
* Refactor query in the Articles::PublishWorker
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Refactor articleForm.jsx
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Removed specs that are no longer relevant
* Removed useless onKeyUp on a hidden input
* Refactored articleForm
* Hide scheduling from post options when published_at is readonly
* Run sends notifications worker every 5 minutes instead of every minute
Co-authored-by: Jeremy Friesen <jeremy.n.friesen@gmail.com>
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Moving Experiments into AbExperiment namespace
This commit entails two major concepts:
1. Extracting logic out of a worker.
2. Cleaning the conditions in which we convert experiments.
In addition, I revisited some spec names to tidy them up for
clarification and structure.
**Extracting Logic...**
For workers, my preference is that they be a simple router to another
object. A non-worker object, with it's initialize method, can both
better manage instance variables. Further, workers are somewhat "flat"
in hierarchy,
In this case, having the AbExperiment be knowledgeable both in setting
the experiment and handling conversions makes (to me) organizational
sense. For example, the AbExperiment is constructed as a wrapper to the
FieldTest gem. But the RecordFieldTestEventWorker had knowledge of
FieldTest.
With this refactor, it does not have that knowledge. This also provided
an opportunity to replace magic strings with constants.
**Cleaning the conditions...**
This relates directly to and closesforem/forem#17530. I was looking at
the experiments after 2 days, and realized that there shouldn't be
results for
`user_views_pages_on_at_least_nine_different_days_within_two_weeks`;
after all the experiment started 2 days ago, how can we have results
that look at 2 weeks.
By adding the "max" function calls, we ensure a clear boundary of
"before the experiment began" versus "while the experiment is running".
Closesforem/forem#17530
* Update spec/models/ab_experiment_spec.rb
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
* Apply suggestions from code review
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* feat: update all the paths as a first order of business
* feat: update all the folders to no longer appear under users
* feat: update specs
* feat: update specs
* feat: remove User namespace
* feat: remove route namespace
* feat: path change
* feat: update url
We avoid busting a user cache for 15 minutes but the tag cache is 5
hours. So we want to "nudge things along".
What this is trying to solve is the server side rendering of whether we
hide or show a button.
Related to forem/forem#17324
Related to forem/forem#17119
* transliterate when generating attribute names to avoid emptiness
If you enter a non-ascii (non `\w` matching) string as the profile
field label, the attribute name is the empty string.
This causes problems outlined in #16391
To avoid losing user provided data, transliterate (Sterile is the same
tool we're using for Article#title_to_slug) before matching against
the word regex.
If we persist an empty string, or persist a non-`\w` name, the
coordinating regex in Profile::ATTRIBUTE_NAME_REGEX will leave a nil
match and raise NoMethodError (the method missing is for
match[:attribute_name] when match was nil, not Profile - that's the
next commit.
* Guard against nil matches
If an attribute name doesn't match the regex, the match is nil, and
trying to access (nil)[:attribute_name] raises a NoMethodError.
If there was no match, assume profile does not respond to the
selector, and don't handle it in method missing.
* Ensure generated attribute name is valid before saving
This raises a validation error if the generated name (from the label)
would be empty.
It's not an optimal error message (since the user can't see the
internal attribute name) but it prevents persisting broken/empty data
* actually raise error when validating
validate/valid? only return true or false (and set errors on the
object). In order to reject the creation, we need to raise a
validation error, not only call validate. I think this is because the
execution of before_create hooks happens after validation (which is
why the validation was only checked on update, not create).
* Generate an attribute name completely independent of the field label
This prevents mistakenly labeling a field "Class" or "Association" or
one of the other hundred public methods an AR model like Profile
exposes. Since attribute_name will be passed to `profile.public_send`
we really shouldn't build selectors from user supplied inputs.
* Use the admin supplied label in the sidebar
The profile decorator is used in the Users#show page to populate the
sidebar fields. Don't use the attribute name (which we mangled during
creation, and now generate randomly) as the label, use the label.
* Make the label lookup null safe, and filter attributes more
* Update data update script to not expect predictable labels
This is low impact since it ran in july, but we no longer know what
attribute name a label will create.
* Fix moderator spec
"Test Field" label no longer predictably generates :test_field as an
attribute name. Ask the field what it's name is before asking profile
about it.
* Fix profile preview card request spec
Remove the assumption that profile responds to a method name based on
the label for Work and Education fields.
* Update old DUS and its test
This can probably be archived
* Update profile spec to use fields generated attribute names
We used to "know" how attributes were generated from labels. Now we don't.
* update e2e seeds for profile field change
* Don't expect attribute name to be based on the label
* expect created profile fields respond to their attribute name
* Update system test
The label, not the attribute name, is shown on the profile form (the
field has an id related to the attribute name, but the view shows the
mutable/human-readable label).
* Keep the field title lowercase when sending the json preview card
The userMetadata component expects "work" and "education" to be
attributes of the metadata, but the ui_attributes_for() method was
titlizing these (for display).
Ideally we wouldn't have "special purposed" these two field names, but
they're there.
* update profile field by attribute name, not based on label
* Update profile field removal assumptions
We don't know what the method selectors will be, we have to ask.
* remove old test
* Update translations for Education and Work
Since the ui_attributes_for(area:) now gives the label, not the
attribute, we need to match the label of the profile field.
Note to self: this exposes an issue in localizing the custom profile
fields (probably a bigger problem for large, international communities
like DEV than some others, but trying to match static translation
files against user-modifiable database records seems like a problem
we'll see again).
* Empty commit to retrigger buildkite
* Remove profile field migration update scripts
Cloned the specs from the other "remove unused scripts" script.
* Remove unused scripts
The data update script removes the entry from the table (recording
that these have run) - we also want to remove the files (preventing
them from running again).
* remove unneeded spec for removed file
* when translation for header area field not found, use the title
Only Work and Education already have keys in the yml translation file,
and there's not a great (or easy?) way to make multiple translations
on these fields right now.
Since an admin can create a new field, and assign it to the header
area, we can't assume the code has a configured translation key for
this field.
Fallback to the title (we do this in another context already) if
there's no translation.
* PR feedback: Avoid n+1 query for labels
the original implementation of "label_for_attribute" had an n+1 query
looping over each matched key.
Follow suggested improvement and pull labels and attributes at once
from the db and modify the returned hash.
* Downcase title before looking for translation key
This avoids putting "odd" capitalized keys into the yml translation
file
Revert addition of "Work" and "Education" to the users files.
* use a let binding for duplicated test data
* Update app/models/profile_field.rb
prefer SecureRandom.hex for a dashless uuid (instead of removing the dashes).
Co-authored-by: Jamie Gaskins <jgaskins@hey.com>
Co-authored-by: Jamie Gaskins <jgaskins@hey.com>
* Limit feeds import fanout to users with feeds
This filtering for feed settings occurs in
Feeds::Import#filter_users_from already, so the enqueued ForUser
jobs were mostly no-op, but the queue latency spiked as hundreds of
thousands of jobs were added to default.
Prescreen users to avoid enqueuing a noop job every hour for every single
user without a feed.
* Update tests to check only jobs for users with feeds are enqueued
* spec cleanup
Only include alice when we check no job is enqueued for her.
Assert no job is enqueued for alice.
Rename bob to user when alice isnt there to contrast.
* Update spec
Use update_columns to bypass setting validation (checking feed is
valid/reachable)
Remove unneeded timecop block when passing a set time
Remove sidekiq: fake and allow Import.call to be received
* Empty commit to trigger CI rebuild
* Fan out Feeds::ImportArticlesWorker
Doing all that work within a single Sidekiq job has begun taking over an
hour on DEV. Regardless of the reasons we did it that way originally, we
should be able to handle this concurrently. If we cannot, we need to
investigate why and handle it properly rather than consigning it to
sequential work.
* Fix specs
The specs make assumptions about how the code under test is implemented.
This commit does not change that, as much as I would like to. Instead,
it just aligns the assumptions with the new implementation.
The previous tests didn't actually represent reality though, since we
can't get a Time or ActiveSupport::TimeWithZone instance inside of the
`perform` method while running it through Sidekiq. Instead, the specs
seem to be relying on the fact that the time instance gets serialized to
ISO-8601/RFC3339 format and that that format is understood by Postgres.
Otherwise, I'm not sure how it would work in production as written.
The new specs reflect reality more closely. The `earlier_than` value
will be converted into an ISO8601/RFC3339 string when passed through
Sidekiq.
* Add parens to perform_bulk call
Turns out, we actually do this pretty consistently. I could've sworn I
saw a bunch of these calls without parens. ¯\_(ツ)_/¯
* Improve variable naming
This is not a list of ids, it's a list of lists of arguments for Sidekiq
jobs, the inner of which contains an id, but that's not the only thing
it contains.
This change makes it easier to resolveforem/forem#16487.
What do I mean by that? To address forem/forem#16487, I need to only
select user's who are authorized to create articles. In
forem/forem#16732 I added a method that will allow chaining of a User
scope. So with this current commit and forem/forem#16732, I'm
triangulating on an approach that will make that change easier.
I did not want to conflate those two, as mixing this PR's change and
what is necessary for the closing PR would create a more complicated
review. Not unduly complicated, but one that will require more tests
and a change in logic. And for someone reviewing the diff, those
concerns could easily be lost.
Yes, I have changed the method signature, but that method signature is
limited to one location:
```shell
❯ rg "Feeds::Import.call"
app/workers/feeds/import_articles_worker.rb
21: ::Feeds::Import.call(users_scope: users_scope, earlier_than: earlier_than)
```
So I look to the method signature a bit as an internal API, hence the change.
Related to forem/forem#16487
* Convert symbol hash keys to strings when calling .perform_async
Fixes a warning from Sidekiq 6.4.0+ about perform_async arguments
which are not equal when passed to perform (`JSON.parse(JSON.dump(arg))`
should equal arg).
This is a safety measure to prevent passing objects (like classes, or
model instances) rather than their representations (like a class name,
or a model's attributes hash).
This warning will be an error in sidekiq 7
* Turn warning into an error in non-production environments
* Use string keys for reaction notification and article fetched
Missed these two on the first pass
* Update example argument hashes for #enqueues_on_correct_queue
Since this calls perform_async under the hood we need to pass json
safe hashes in the test cases as well.
https://github.com/forem/forem/blob/main/spec/workers/shared_examples/enqueues_on_correct_queue.rb
* Convert keys from FollowData#to_h to string before perform_async
I'm not sure enough where else (outside of notification) to_h is being
called, so I'm converting here when building args, rather than in
FollowData#to_h, which might be my next step.
* Let FollowData#to_h return a hash with string keys
Update spec to use string keys as well.
* Make to_h return string keys for ReactionData
Like FollowData, the #to_h method is only used to call
notifications (this is used to enqueue sidekiq jobs).
* Remove a key that was in the hash
Since reaction_data calls to_h, it gets string and not symbol,
keys. Call Hash#except with a key that was actually there.
