Commit graph

77 commits

Author SHA1 Message Date
Jeremy Friesen
6d37f4303c
Favoring single SQL and computation over enum (#15865)
* Favoring single SQL and computation over enum

Prior to this commit, we ran a SQL statement to generate a list of
articles then applied some minor randomization.

With this commit, I'm removing the randomization in favor of expected
values, and collapsing the result set into a single inline SQL and some
post query simple arithmatic.

Let's check my math.  For each article we sum:

* `article.score`
* `article.comments_count` * 14
* a random number between 0 and 5 (`rand(6)`) with expected value of 2.5
* the tag's (taggings_count + 1) / 2

The new SQL query sums the `article.score` and the
`article.comments_count` * 14.  I then reduce the remainder of the
equation.  From "for each article sum `(rand(5) + (taggings_count + 1) /
2)`" we have the following:

`article_count * (2.5 + (taggings_count + 1) / 2)`

Which is equivalent to:

`article_count * ((taggings_count + 1 + (2.5 * 2)) / 2)`

Which is equivalent to:

`article_count * ((taggings_count + 6) / 2)`

This change reduces computation time by favoring expected values.

* Fixing broken behavior and adding comments
2021-12-22 14:59:37 -05:00
Michael Kohl
09828853f6
✂✂✂ Remove Connect (#14734)
* Remove Connect

* Remove more Connect specs

* Remove a lot more Connect code

* 🚮

* It all has to go

* Explicitly add httpclient

* Update application layout

* Remove messages association from User

* Start fixing specs

* reintroduce util function and refactor references

* Remove Connect Cypress test

* Fix more specs

* Remove Connect from listings

* Ignore contact_via_connect column on listings

* Remove contact_via_connect usages

* Ignore mod_chat_channel_id on tags

* Drop Connect tables

* Remove email_connect_messages from user notification settings

* Re-add httpclient 2.8.3

This was mistakenly removed as a merge conflict

* Don't need to exclude removed chat channel file

* Remove unneeded style for chat channels

* Remove unneeded channel list prop type

* Remove chat channels index/connect-link from getPageEntries

* Re-add comment from httpclient in Gemfile

* Remove connect references from mailers

Tag Moderators no longer have a chat channel

No longer will users be notified about new messages (there won't be
any)

No longer will users be notified about channel invites (you can't
invite anyone anymore)

* Don't configure Pusher and remove PUSHER_* from .env_sample

since it's removed from gemfile, the Pusher constant will not resolve, if this is
configured in the environment variables we'll fail to boot.

Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
Co-authored-by: Dan Uber <dan@forem.com>
2021-11-18 08:21:00 -06:00
Suzanne Aitchison
58e3d6efd5
Suggest top tags when tag search is empty (#14817)
* show top tags when search is empty

* add cypress tests

* nudge travis

* tweak to e2e test

* show a 'top tags' heading

* only suggest tags that are supported

* Add supported scope to Tag

Co-authored-by: Michael Kohl <citizen428@forem.com>
2021-09-30 09:41:20 +01:00
Daniel Uber
21e9966ccb
Relax tag validation restriction to permit non-ascii strings (#14710)
* Relax tag validation restriction to permit alphanumeric strings

Non-word symbols like musical notes are still blocked, but most
languages should work.

* Handle redirect after tag edit

The URI.parse(tag.name) was not updated in #13720

Move reused logic into tag_path method in URL module.

* FactoryBot generated tags are non-ascii

This should help expose any cases where we're mistakenly counting on
ascii-only content in tags by causing all generated tags without a
specified name to be of the form `tag你好#{n}` instead of `tag#{n}`.

* Revert "FactoryBot generated tags are non-ascii"

This reverts commit ed1035cd1e91619b2c1599e521b6ddf962c97f80.

It was useful to see how many places the path knowledge is in the
tests (a lot of requests have GET /t/tag.name and need to have GET
URL.tag_path instead).

https://app.travis-ci.com/github/forem/forem/builds/237542600 has the
results.

Reverting because those test-side changes seem like a secondary
concern to the change in this PR (I'll file a follow on issue)
2021-09-14 14:03:54 -05:00
Michael Kohl
c16ec1d53d
Remove redundant freeze calls (#14596) 2021-08-26 10:01:08 -04:00
Khadija Sidhpuri
3d1727b3a8
Delete tag colors cache on updating tags. (#14427)
* add: article cache on update tag

* minor styling fix

* add: published scope to fetch articles

* update: tag_spec:125 to not fetch articles

* modify: tag.rb:bust_cache to delete server cache

* add: tests to verify cache delete
2021-08-11 12:56:04 +02:00
rhymes
e081551dc0
Revert "Clear recent articles cache on updating tags (#14298)" (#14330)
This reverts commit 7159199402.
2021-07-26 11:12:33 +02:00
Khadija Sidhpuri
7159199402
Clear recent articles cache on updating tags (#14298)
* add: article cache on update tag

* minor styling fix

* add: published scope to fetch articles

* update: tag_spec:125 to not fetch articles
2021-07-26 10:24:24 +02:00
Alex
393ba00221
Remove Elasticsearch ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ (#13606)
* Attempt number 1

* Fix rack_attack specs

* Fix users_searches_users spec

* Fix display_users_search_spec

* Fix comment typo

* Remove search:destroy task from cypress

* Remove port 9300 from gitpod

* Stub response in attack_spec
2021-05-03 11:09:45 -04:00
Michael Kohl
1732396abc
Remove ignored Buffer columns (#13139) 2021-03-26 17:37:29 +01:00
rhymes
83852038e4
[Search 2.0] Reading list (#13052)
* Step one in populating the reading list with PG

This first attempt tries to recycle the `Search::ArticleSerializer` which is only
used in input in ES, but we're using it in output in PG.
For this reason it's currently 15.55x times slower

* Serialize only what is requested by the frontend

`Search::ArticleSerializer` which is only used in ES in the indexing step aims
to add as much info as possible for broader purposes, in this case
(with serialization in output) we should aim to save only what's requested from
the frontend.

* Optimize selection of articles columns

* Select only needed columns for users

* Compute total of reading list items

* Attach the basic filtering based on PG on the search controller

* Restructure in methods

* Add tags support

* Use LIKE on articles.cached_tag_list

* Fix tags as nil

* Fix default pagination

* Add optional FTS for reading list

* Reworded the tags comment explaining why

* Add index to reactions.status

* Fix total counter in Preact readingList component

* Fix total count in reading list backend search

* Add GIN index to articles.cached_tag_list

* Add service tests

* Add search request specs

* Added missing early return

* Update spec/requests/search_spec.rb

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>

* Extract MAX_PER_PAGE constant and add comments

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
2021-03-24 15:40:00 +01:00
Michael Kohl
7c31a9cc98
Remove buffer gem and related code: step 1 (#12993)
* Remove buffer gem and related code

* Remove more Buffer-related code

* Remove one more Buffer spec

* Remove DUS

* Remove more Buffer code
2021-03-24 09:58:38 +07:00
rhymes
c0ea4e5878
[Search 2.0] Add Search::Postgres::Tag behind a feature flag (#12937)
* Add Search::Postgres::Tag to retrieve tags for autocompletion

* Add Search::Postgres::Tag in SearchController behind a feature flag

* Update spec/services/search/postgres/tag_spec.rb

Co-authored-by: Michael Kohl <citizen428@dev.to>

* Add index to tags.supported

* Fix hotness_score ordering

Co-authored-by: Michael Kohl <citizen428@dev.to>
2021-03-11 19:12:30 +01:00
Mac Siri
dfdb37971c
Add has_many: articles to Tag model (#12574) 2021-02-04 11:46:10 -05:00
Alex
a5e6f7942c
Rename MarkdownParser to MarkdownProcessor::Parser (#12248)
* Rename MarkdownParser to MarkdownProcessor::Parse

* Wake up, Travis

* Fix typo
2021-01-13 10:39:13 -05:00
Anna Buianova
f4c88341ce
Rubocop fixes (#11862) 2020-12-11 17:36:42 +03:00
Andy Zhao
a8c3ff8c5a
Add trusted role to new tag moderators (#11242)
* Don't add to mailchimp for N/A circumstance

* Use class << self over self. methods

* Refactor and make add/remove tag mod idempotent

* Add missing param wiki_body_markdown

* Add errors_as_sentence b/c Tag not inheriting from AppRecord

* Refactor and Crayonsify tag page

* Use proper for attr to make label checkable

* Crayonsify moderator section

* Make placeholder white

* Remove remote: false for sync form submit

* Fix more merge conflicts

* Minor styling adjustments

* Fix color field value

* Don't add to Mailchimp community mod list if not enabled

* Add tests for new tag mod routes

* Add missing line oops

* Use self.method over class << self for new Rubyists

* Use more efficient query for getting tag mods

* Use parentheses to follow convention

* Use cleaner way of routing and controller

* Update tests to match new configuration
2020-11-05 10:36:26 -05:00
rhymes
10efa25c0a
Add missing presence validators to models (#10495)
* Add missing presence validator for Tag

* Add missing presence validators to Follow

* Add missing presence validators to ProfileField

* Add missing presence validators to Comment

* Add missing presence validators to Profile

* Add missing presence validators to Organization

* Add missing presence validators to Badge

* Add missing presence validators to Webhook::Endpoint

* Add missing presence validators to NotificationSubscription

* Add missing presence validators to PodcastEpisode

* Add missing presence validators to Sponsorship

* Add missing presence validators to PollOption

* Add missing presence validators to Poll

* Add missing presence validators to Article

* Add missing presence validators to EmailAuthorization

* Add missing presence validators to AuditLog

* Add missing presence validators to DataUpdateScript

* Add missing presence validators to User

* Add missing presence validators to SiteConfig

* Fix specs
2020-10-01 16:15:32 +02:00
rhymes
d7976a75d6
[deploy] Add missing FKs to BadgeAchievements BanishedUsers BufferUpdates (#10366)
* Add missing FKs to BackupData BadgeAchievements BanishedUsers BufferUpdates

* Add relations and specs

* Fix specs

* Trigger Travis CI

* Remove traces of backup_data has it has been removed on master

* Fix trailing whitespace
2020-09-29 21:43:14 +02:00
rhymes
0ca4e8b371
[deploy] Add missing FKs to FeedbackMessage, Note, Taggings, Tag, and UserRole (#10376)
* Add missing FKs to FeedbackMessage, Note, Taggings, Tag, and UserRole

* Add missing relations and specs

* Fix spec
2020-09-29 16:49:47 +02:00
rhymes
5a2cae4fbb
Rubocop: enable and fix new Rails cops (#9537)
* Enable new Rails/* cops and use autocorrect on them

* Fixed Rails/PluckInWhere leftovers

* Fix Rails/DefaultScope

* Enable and fix Rails/PluckId

* Fix manual mistake with forcing autocorrection on Rails/PluckId

* Apply PR feedback to remove Rails/PluckId inline disables

* Apply PR feedback to get rid of Rails/PluckInWhere inline
2020-07-29 11:14:19 +02:00
Michael Kohl
a4dadfb728
Standardize ActiveRecord order clauses (#9395)
* Change simple order clauses

* Change nested order clauses
2020-07-20 10:00:51 -04:00
rhymes
696c712883
Rubocop: routine fixes (#9345)
* rubocop -A

* Fix Style/HashLikeCase

* regenerated todo file
2020-07-16 18:18:13 +02:00
rhymes
f1ec04a0c9
Rubocop: move dot in multi-line calls to leading position (#9262) 2020-07-16 15:51:11 +02:00
rhymes
1166370818
Rubocop: activate Layout/ClassStructure (#9304) 2020-07-14 18:28:18 +02:00
rhymes
3ab9b91575
Routine Rubocop fixes (#9302)
* rubocop -A

* rubocop --auto-gen-config

* Fix sorting in Bundler/OrderedGems to consider punctuation
2020-07-14 17:24:03 +02:00
Jacob Herrington
83ba6b64c9
Stop allowing accented characters in tags (#9006)
* Stop allowing accented characters in tags

This commit will add strict frontend and backend validation for tags
disallowing accented characters.

* Update tags validation error message

* Add more character sets to tag validation spec
2020-07-01 11:48:25 +02:00
Mac Siri
1b839d9328
[deploy] Streamline Tag parsing for both editors (#8175) 2020-06-05 08:46:24 -04:00
Molly Struve
b10668a454
[deploy] Create a Bulk Index Elasticsearch Worker (#7265) 2020-04-15 09:08:59 -05:00
Molly Struve
91972dd2c4
ensure that when Tag and ActsAsTaggableOn::Tag are updated their respective related docs are as well (#7218) 2020-04-14 08:38:09 -05:00
Alex
7dfffac3db
[deploy] Copy ActiveRecord methods from fastly-rails (#7145)
* Add comment referencing Fastly gem

* Create purgable concern and include it

* Typo: deprecate --> deprecated

* Include Purgable on the Tag model

* Rename Purgable to Purgeable

* Reorganize fastly and service methods in Purgeable

* Add and fix specs

* Add development env guards
2020-04-13 10:46:15 -04:00
Molly Struve
29b1d0c32c
[deploy] Index Tags on Creation to via ActsAsTaggableOn in Elasticsearch (#7104) 2020-04-07 10:14:24 -05:00
rhymes
857de5ebb9
[deploy] Remove unused tags columns (#6977) 2020-04-03 10:40:45 -04:00
Molly Struve
2c49b34ebf
Hotness Score for Tags Should Use Article positive_reactions_count (#6808) [deploy]
Co-authored-by: Ben Halpern <bendhalpern@gmail.com>
2020-03-31 09:04:20 -05:00
rhymes
466ec8bd17
Ignore unused tags columns (#6947) 2020-03-30 16:57:13 +02:00
rhymes
86129395eb
Cleanup tag validation a bit (#6604)
* Cleanup tag validation a bit

* Use symbols
2020-03-12 15:41:08 +01:00
Anna Buianova
df93e8d41e
Custom social preview template for tags (#6290) [deploy]
* Custom social preview template for tags

* Added index on tags social_preview_template + small refactoring

* Add checks if index exists to tags migration
2020-02-27 17:56:21 -05:00
Alex
da259107bd
Refactor logic to index to Elasticsearch (#6286) [deploy]
* Refactor logic to index to Elasticsearch

* Rename worker argument to object_class

* Change to .constantize for better error messaging
2020-02-25 11:06:42 -08:00
Molly Struve
318ee32cdd
Add worker for removing Elasticsearch docs (#6213) [deploy] 2020-02-21 08:13:36 -05:00
Molly Struve
567eb49ee1
Refactor searchable interface into a module for AR models (#6160) [deploy] 2020-02-18 15:35:34 -05:00
Molly Struve
04dcca0b13
remove algolia from Tag model (#6123) 2020-02-17 12:28:10 -05:00
Molly Struve
3fed9338d2
Index Tags to Elasticsearch after they are created (#5958) [deploy]
* Index tags to Elasticsearch after they are created
* add rules_html to Tag index since we show it in search drop down
* change get method to find_document to make it more explicit
2020-02-11 11:58:58 -05:00
Ben Halpern
f3c86c6377
Add mod channel automation to tags (#5731) [deploy]
* Add mod channel automation to tags

* Change mod_relations to mod_relations_admin

* Run migration properly

* Make queries consistent and fix test
2020-02-06 16:27:12 -05:00
Alex
0c42ea2b43
Move Tags::BustCacheJob to Sidekiq (#5870) [deploy]
* Create Tags::BustCacheWorker

* Create Tags::BustCacheWorker spec

* Update references to new worker
2020-02-04 09:24:37 -08:00
Ben Halpern
f78b89e763
Add suggested tags to SiteConfig (#5426) [deploy]
* Initial base work

* Add suggested tags config

* Downcase suggested tag input
2020-01-10 12:33:37 -05:00
Bolarinwa Balogun
8c6adac7ba Remove special characters in tags for articles created via API (#4239) [deploy] 2019-12-13 11:29:02 -05:00
Molly Struve
7c650dcb1c
Remove RedisRailsCache and replace with Rails.cache (#5059) [deploy] 2019-12-10 13:11:08 -06:00
jeffin sam
fbbb02e241 Move bufferized tags to RedisCache (#4817) 2019-11-14 17:37:20 -05:00
Colby Melvin
4531679c23 fix tag input swallowing numeric keycodes [deploy] (#4283)
* fix tag input swallowing numeric keycodes

* add AR validation for Tags class

* allow whitespace in Tag.name

* fix specs

* whitespace

* reformat
2019-11-05 16:20:34 -05:00
Brad Pauly
b59aba415d ArticlesController#feed refactor (#4498)
* Move query details into model.
Reorganize filtering initial results by params.

* Fixups from feedback.
2019-10-30 12:35:53 -04:00