Commit graph

753 commits

Author SHA1 Message Date
rhymes
9e4de49b79
[Hotfix] [Search 2.0] Optimize performance of listings and articles and fix bugs (#13577)
* Add tsvector index on listings

* Fix sorting order when fetching tag flares

* Add published_at as a sorting condition for Homepage::ArticlesQuery

* Re-added param needed by ES, this got lost somewhere down the line
2021-04-29 17:37:59 +02:00
Michael Kohl
3173bb4bc2
Remove <provider>_created_at columns (#13264)
* Remove <provider>_created_at columns from User

* Update code

* Clean up

* Fix authenticator specs

* Restore old behavior

* Update provider account age logic, again
2021-04-29 10:24:16 -05:00
rhymes
c20c4abe3f
[Search 2.0] Use Article.cached_tagged_with* in queries (#13566)
* Add Symbol to Article.cached_tagged_with* and fix error bug

* Use Article.cached_tagged_with* in search

* Apply PR feedback suggestions
2021-04-29 12:00:49 +02:00
Alex
11138c4f18
[Search 2.0] Change podcast_episode query (#13568)
* Change podcast_episode query

* Add the index of all indexes

* Update ATTRIBUTES
2021-04-29 08:49:47 +02:00
Vaidehi Joshi
f9a6067662
Do not create mentions when embedded within a comment liquid tag (#13552)
Fixes https://github.com/forem/forem/issues/13348.
2021-04-28 08:27:26 -07:00
Alex
0211a7c341
[Search 2.0] Fix podcast values (#13550)
* Fix podcast values

* Also select slug from podcast_episodes
2021-04-28 08:58:45 -04:00
rhymes
8762815709
[Search 2.0] Match both word boundaries when filtering tags (#13554) 2021-04-28 11:24:45 +02:00
Michael Kohl
1c7de11bef
Split Settings::UserExperience (#13495)
* Add Settings::UserExperience

* Add DUS

* Update usage

* Add controller

* Add e2e test

* Fix specs
2021-04-28 13:29:25 +07:00
Jamie Gaskins
d8f7518e5c
Optimize Articles::Suggest query (#13548) 2021-04-27 14:30:06 -04:00
rhymes
f48cf141a6
Use regexp operator to avoid matching partial tags (#13547) 2021-04-27 20:27:11 +02:00
Alex
315dca98c9
[Search 2.0] Podcasts on search page (#13475)
* Working MVP

* Add specs

* Add partial index on podcasts.published

* Add tsvector indexes to podcast_episodes

* Filter for reachable PodcastEpisodes

* Fix indentation/formatting

* Add podcast attribute to serializer
2021-04-27 13:29:22 -04:00
rhymes
2ff317687e
[Search 2.0] Articles (#13540)
* Rename the search scope to reflect its generic usage

* Add and use Search::Postgres::Article
2021-04-27 18:59:01 +02:00
Jamie Gaskins
b167fb7fb1
Optimize GetUserStickies (#13541) 2021-04-27 12:47:49 -04:00
Andy Zhao
dc30eadad8
Remove unused attributes from markdown parser (#13484)
* Remove definitely unused attributes

* Refactor and remove unused attributes

* Add parentheses

* Prevent alt tag from having Liquid tags inside

* Add test

* Remove more unused attributes

* Scrub any valid attributes

* Remove unused tags and add more tests

* Remove <i> tag; <em> is used by parser

* Remove <center> b/c it's deprecated though still valid

* <cite> is a safe tag and used by some

* Eh I changed my mind about center lol

* Add underline <u> back in

* Don't allow width changes

* Use frozen constant for regex

* Use #remove over #gsub
2021-04-27 09:33:01 -04:00
rhymes
8a55c3e888
[Search 2.0] Add query filters for profile, organization and tag index pages (#13510)
* Re-order test cases to increase tests readability

* Add user_id filter for articles query

* Add organization_id and tags filters

* Add new params to SearchController#feed_content

* Add current_user param so that it can be tested for a small pool of users
2021-04-27 08:49:47 +02:00
Michael Kohl
610f6151e6
Split Settings::Community from SiteConfig (#13403)
* Add settings_community_contents model

* Add settings_communities model

* Update usage

* Add controller and update code

* Add e2e test

* Add data update script

* Update schema.rb

* Fix specs

* PR feedback

* Remove experience_* from Settings::Community

* Update spec

* Fix spec
2021-04-26 10:46:35 +02:00
Michael Kohl
f2f5e911cf
Add Settings::Mascot (#13451)
* Add Settings::Mascot

* Add DUS

* Update usage

* Fix e2e test and controller

* Fix specs

* Fix remaining spec
2021-04-26 11:39:19 +07:00
rhymes
996f6e980f
[Search 2.0] Users (#13453)
* Add Search::Postgres::User

* Add sorting params

* Use a better algorithm to filter out suspended users

* Add tsvector index on users.name

* Add specs

* Add UserRole spec

* Update spec/services/search/postgres/user_spec.rb

Co-authored-by: Michael Kohl <citizen428@dev.to>

* Fix search with quotes in users's names

Co-authored-by: Michael Kohl <citizen428@dev.to>
2021-04-24 19:06:06 +02:00
rhymes
3292f27d53
Homepage query should not fail for nil article's cached tag list (#13492) 2021-04-24 18:36:34 +02:00
Alex
f233be805b
Preload users instead of querying separately when searching for comments (#13490)
* Preload users instead of querying separately

* Add conditional to comment.user
2021-04-23 16:30:06 -04:00
Alex
1b6f753f73
Performance improvements to comments search (#13489)
* Explicitly pluck user_id from comments table

* Change comment query

* Remove indexes and add partial indexes

* Update schema
2021-04-23 13:21:30 -04:00
Daniel Uber
c1a93ad3c3
Cleanup articles cache busting (#13467)
* Extract repeated calls to cache_bust.call to iteration over list

This mirrors what the EdgeCache::BustUser service looks like.

It might make sense to have cached_article_paths be a method accepting
a block and yielding paths, and including the collections cache paths
as well.

* Move article path information into a separate method

BustArticle.call now only knows to bust cache for each path associated
with the article, the information about which paths to clear is moved
to a separate `paths_for` method.

I may have gone overboard on this.

* Remove extra blank line
2021-04-23 09:35:40 -05:00
dependabot[bot]
29dcea3750
Bump rubocop-performance from 1.10.2 to 1.11.0 (#13471)
* Bump rubocop-performance from 1.10.2 to 1.11.0

Bumps [rubocop-performance](https://github.com/rubocop/rubocop-performance) from 1.10.2 to 1.11.0.
- [Release notes](https://github.com/rubocop/rubocop-performance/releases)
- [Changelog](https://github.com/rubocop/rubocop-performance/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop/rubocop-performance/compare/v1.10.2...v1.11.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Enable new cops and fix violations

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: rhymes <rhymes@hey.com>
2021-04-22 09:31:25 -05:00
Daniel Uber
3e432c9047
When busting user cache, also clear the api response (#13465)
* When busting user cache, also clear the api response

This should fix #13293 by clearing both the UI copy and the api copy of the
user's page.

* Update spec to include the api path
2021-04-22 09:16:15 -05:00
rhymes
1fd38aa172
[Search 2.0] Allow mention autocomplete to search users names (#13462)
* Add tsvector index on users.name

* [Search 2.0] Allow mention autocomplete to search users names
2021-04-21 18:47:15 +02:00
rhymes
14d98fc590
[Search 2.0] Homepage query (#13339)
* Added basic homepage query object

* Add serializers and initial service object

* Add flare tags, select only required columns

* Test default Homepage::FetchArticles behind a feature flag

* Add all needed params for the homepage feeds

* Add Homepage::ArticlesQuery specs

* Add Homepage::FetchTagFlares specs

* Add Homepage::FetchArticles specs

* Add approved filter to Homepage::ArticlesQuery

* Add specs for SearchController

* Addressed feedback

* Apply PR feedback to cleanup SearchController#feed_content

* Update app/services/homepage/fetch_tag_flares.rb

Co-authored-by: Jamie Gaskins <jamie@forem.com>

* Update spec/queries/homepage/articles_query_spec.rb

Co-authored-by: Jamie Gaskins <jamie@forem.com>

* Update spec/queries/homepage/articles_query_spec.rb

Co-authored-by: Jamie Gaskins <jamie@forem.com>

* Apply PR feedback

* Modify travis

* Revert "Modify travis"

This reverts commit b42548e866804af69542602556db47d37a31034e.

Co-authored-by: Jamie Gaskins <jamie@forem.com>
Co-authored-by: Mac Siri <krairit.siri@gmail.com>
2021-04-21 12:41:24 +02:00
Alex
cf0baf05f9
[Search 2.0] Comments on search page (#13401)
* Working MVP

* Add comment to COMMENTABLE_TYPES

* Add specs to search_spec

* Add service specs and fix ordering

* Fixed queries and specs

* Clarify COMMENTABLE_TYPES comment

* Move FORCED_EAGER_LOAD_QUERY to Comment

* Add highlighting to Comment search

* Add explicit .to_s on class_name params

* Remove Comment#forced_eager_load_serialized_data

* Revert CommentSerializer changes

* Hardcode class_name attribute

* Add PR feedback

* Use one liner syntax in serializer

* Update user comment

* Update path attribute logic

* Fix user_ids

* Remove PodcastEpisode skipped spec

* Add sort order to Search::Postgres::Comment

* Fix search_specs

* Move pagination TODO comment
2021-04-20 12:34:21 -04:00
Michael Kohl
28dd50e718
Create settings model for rate limits (#13381)
Co-authored-by: rhymes <rhymes@hey.com>
2021-04-20 11:29:22 +07:00
Anna Buianova
ba1c13bbb6
Asynchronous organization delete (#13388)
* Asynchronous organization destroy

* Notification after an organization was deleted

* Send data to Datadog and Honeybadger when org deletion failed

* Delete unused code

* Fixed Organization#destroyable? and organization delete specs

* Reorganized organization deletion specs

* Removed redundant specs

* Improved org deleted email text template

* Don't peform the org delete when a user or an org were not found

* Renamed user_id in the org delete worker

* Audit logging when deleten an organization

* Removed specs for tracking pixel and UTM params

* Changed slug for audit log on org delete

* Fixed schema.rb
2021-04-19 18:30:35 +03:00
Josh Puetz
c3174cade8
Push Notification stats instrumentation (#13118) 2021-04-19 09:42:01 -05:00
Michael Kohl
a86c178fe0
Split Settings::Campaign from SiteConfig (#13238) 2021-04-19 10:01:03 +07:00
Vaidehi Joshi
956caf69c7
Rename comment-mentioned-user class added by Html::Parser (#13263)
* Rename comment-mentioned-user class added by Html::Parser

* Add DataUpdateScripts for resaving articles, comments, and messages

* Swap order of messages and articles DUS
2021-04-13 14:33:22 -07:00
rhymes
b6e562f14c
[Search 2.0] Optimize search reading list (#13275)
* Use trigger and tsvector column to speed up reading list search

* Add organization destroy spec and todo note

* Fix failing data update script due to not null constraint

* Remove the leading anchor in the trigger regexp

* Fix reading list specs

* Address feedback
2021-04-12 16:40:40 +02:00
Michael Kohl
5406b0576e
Split Settings::Authentication from SiteConfig (#13095)
* Split Settings::Authentication from SiteConfig

* Move specs

* Sort fields

* Update settings usages

* Update recaptcha usages

* Add data update script

* Update spec

* Rename SiteConfigParams concern

* Fixes, new route, new controller

* Controller and service refactoring

* More controller and service updates

* Spec updates

* More spec fixes

* Move file

* Fix FeedbackMessagesController

* Update admin/configs_spec

* Fix remaining specs in admin/configs_spec

* Fix configs API

* Formatting

* Clean up old service object

* Various fixes

* Update DUS

* Add model argument to admin_config_label

* Fix key name

* Fix specs

* Add distinct request caches for settings classes

* Fix e2e tests

* Fix remaining system spec

* Make DUS idempotent

* Move routes block

* Cleanup

* Switch to ActiveSupport::CurrentAttributes

* Pinned rails-settings-cached

* Update e2e test

* Update lib/data_update_scripts/20210316091354_move_authentication_settings.rb

Co-authored-by: rhymes <rhymes@hey.com>

* Add guard to DUS

* Temporarily re-add two SiteConfig fields

* Fix config show view

Co-authored-by: rhymes <rhymes@hey.com>
2021-04-12 09:41:09 +02:00
Jacob Herrington
c043d3b376
Rename unidiomatic methods (#13328) 2021-04-09 10:59:56 -05:00
Jamie Gaskins
afa3178b79
Fix rescue in EdgeCache::Bust (#13303)
We were trying to report an exception to Honeybadger but we never
actually captured the exception.
2021-04-07 15:31:30 -04:00
Jacob Herrington
f76a8e1e99
[15-minute fix] Don't allow markdown to use CSS classes (#13282)
* Don't allow markdown to use CSS classes

* Fix code highlighting

* Add spec for ignoring CSS classes

* Add comment
2021-04-07 11:40:39 -04:00
Vaidehi Joshi
97678456d6
[15 Min Fix]: Extract a post's followers out of service and onto Article model (#13229)
* Move authors followers out of Notifications::NotifiableAction::Send and into Article model

* Rename authors_followers to followers

* Update followers method in service, oops

* Update app/services/notifications/notifiable_action/send.rb

Co-authored-by: Michael Kohl <citizen428@dev.to>

Co-authored-by: Michael Kohl <citizen428@dev.to>
2021-04-07 06:46:32 -07:00
Alex
3891c7d468
✂️ Remove search from Connect (#13235)
* Remove search from connect

* Remove alternate approach

* Alphabetize CHAT_CHANNEL_PARAMS

* Remove default value for user_ids

* Add spec for result ordering
2021-04-06 13:03:31 -04:00
Josh Puetz
f6dc638190
Upgrade to Ahoy 2.0.x (#13085)
* First set of changes for Ahoy 2.0

* Fix line length

* Change test

* Remove unused test

* Remove ahoy_messages.opened_at

* Simply digest email sending calculation to not rely on last_opened

* Remove outdated comment

* Fix typo in test

* Rework site config email digest to a single value

* Fix merge duplication

* Remove UTM references from ahoy emails

* Fix credits tests

* Remove UTM from expected email params
2021-04-06 10:12:32 -05:00
Jacob Herrington
a5b2d109d5
Rename banned and comment_banned roles (#12270)
* Rename banned and comment_banned roles

* Add data update script to rename roles containing 'ban'

* Add named error for Suspended users

* Update unidiomatic method names

* Rename misc banned to suspended

* Apply suggestions from code review

Co-authored-by: Michael Kohl <me@citizen428.net>

* Add unit tests for suspended methods

This commit also adds TODO comments for removing banned and
comment_banned from the codebase after data update scripts have
successfully run on all of our Forems.

Co-authored-by: Michael Kohl <me@citizen428.net>
2021-04-06 10:12:14 -05:00
rhymes
b6360fb80c
Upgrade Rails to 6.1.3.1 (#11333)
* Upgrade Rails to 6.1

* Switch to unreleased bullet

* Run rails app:update

* Add deprecation notices after reading changelogs

* Fix app:update error

* Move middleware in the correct place

* Temporarily disable ransack which does not support Rails 6.1

* Remove wrongly merged file

* Re-run spring binstub

* Fix double quotes

* Track ransack branch with Rails 6.1 support

* Fix deprecation

* Switch to Ransack 2.4 which supports Rails 6.1 rc1

* Fix missing default params for the duration substitution

* Fix new behavior of relation.pluck with contradictory queries

* Fix uploaders specs by tracking rspec-rails main repo

* Disable bullet temporarily

* Fix remaining fixture_file_upload usages

* Add default seconds for video article duration

* Trigger Travis CI

* Upgrade Rails to 6.1.0.rc2

* Remove file deleted on master

* Add Rails 6.1 gem to Gemfile

* Trigger Travis CI

* Revert "Disable bullet temporarily"

This reverts commit cee0c2ce61fb72cbc16d52c94b12ee681e873031.

* Fix bullet version

* Upgrade to acts-as-taggable 7 and fix conflict

* Update Gemfile and rspec-* gems

* Fix nokogiri in Gemfile.lock

* Switch to rspec-rails main branch

* Remove leftover vendored cached items

* Fix path for Rails 6.1

* Use latest release of erb_lint

* Re-run rails app:update to incorporate new changes

* Disable erb_lint's ErbSafety checks

* Fix Gemfile.lock and re-add platform specific gems of Nokogiri

* Add mini_portile2 as well

* Fix latest merge conflict by removing now unused faraday_middleware

* Upgrade to Rails 6.1.3.1

* Regenerate Gemfile.lock and vendor/cache

* Add x86_64 linux gem

* Mark spec as flaky

* Revert "Mark spec as flaky"

This reverts commit 3caba94b33645f9b59c84ba78ee8df042fc7aee0.

Co-authored-by: Mac Siri <krairit.siri@gmail.com>
2021-04-05 10:39:48 -04:00
Mac Siri
009c14653a
Fix app/services' ruby 2.7 kwargs warnings (#13177) 2021-04-01 18:40:50 -04:00
Jacob Herrington
18ce0c635c
Add endpoint for articles sorted by published date (#13137) 2021-04-01 13:36:36 -05:00
Jamie Gaskins
e953b67ce5
Fix user_id tag in EDAC traces (#13213) 2021-03-31 15:58:08 -04:00
Molly Struve
20c6b39a30
Optimization:Remove Unused Twitter Counts from User (#13196) 2021-03-31 10:57:03 -05:00
Vaidehi Joshi
95055b2a89
Remove pro role + expose analytics to all users via dashboard (#13156)
* Remove pro role on user, expose pro dashboard to all users as analytics

* Remove pro from Elasticsearch mappings

* Update user role docs to use :trusted over :pro

* Remove pro from Role model spec

* Remove more references to pro, as noted by @rhymes
2021-03-30 15:02:18 -07:00
Anna Buianova
40b989ca62
Configure badge credits by admins (#13145)
* Added credits_awarded to /admin/badges forms

* Award credits only if badge has them

* Removed information about credits and listings in the new badge email when not needed

* Specs for number of credits awarded for badges

* Added missing newline

* Fix typo

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>

* Improve new_badge_email.text.erb

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>

* Fixed number of credits in notification

* Added specs for number of credits in notifications

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
2021-03-30 10:58:48 +03:00
Alex
1597318db8
[Search 2.0] Listings (#13133)
* Add PgSearch to Listing model

* Working MVP - pre-optimizations

* Remove includes

* Update search_spec

* Add more specs

* Fix specs

* Refactor filter_by_category

* Update block parameter name

* Add published index to classified_listings table

* Preload user, organization, and listing_category

* Working MVP - pre-optimizations

* Refactor filter_by_category

* Update schema

* Fix schema for real

* Remove constant specs and make them private
2021-03-29 13:22:16 -04:00
Jamie Gaskins
5a0a46887c
Instrument EmailDigestArticleCollector (#13153)
* Instrument EmailDigestArticleCollector

On DEV, since we skip this with sidekiq-cron and instead process them
inline using Heroku Scheduler (which is not instrumented), we miss out
on all instrumentation for this process and have no idea how long it
takes to process each user.

With this instrumentation in place, we can see how long it takes to
process each user and aggregate them to see how long it takes to process
all users.

This commit also adds an Instrumentation mixin that we can use to
instrument blocks easily:

    instrument "MyClass.my_method", tags: { article: article.id } do |span|
      # ...
    end

* Add Instrumentation mixin

Oops, completely forgot to commit this file
2021-03-29 09:33:32 -04:00