Commit graph

92 commits

Author SHA1 Message Date
Daniel Uber
c879c5b3d8
Remove slash characters from user name search term (#15867)
* Remove slash characters from user supplied user search input

Prevents an error when the search term includes '\'

    PG::SyntaxError: ERROR:  syntax error in tsquery

https://app.honeybadger.io/projects/66984/faults/79391397

I had originally thought to add this cleanup to Search::Username but
decided to move it as close to the generated (invalid) query as
possible to prevent alternate paths finding their way here.

* Add spec

Since there's no existing tests for the scope - I put the test code on
the caller (Search::Username) rather than the model (User), this seems reasonable.

* When the term is empty (or only slashes) just return null relation

* Handle nil input (search for nothing) correctly

One of the request specs sends a username search with no query, so we
can't call nil.delete or nil.empty?, use blank? of empty?
2021-12-27 12:30:36 -06:00
Jeremy Friesen
9df5f6af49
Ensuring reading list excludes unpublished articles (#15490)
* Ensuring reading list excludes unpublished articles

Prior to this commit, if a given user adds an article to their reading
list then the author unpublishes the article, the article remains in the
reading list.  When the given user would then "click" on the now
unpublished article, they would get a 404 Not Found notice.

With this commit, we now exclude unpublished articles from the search
results for the reading list.

This should also address the issue of attempting to archive the reading
list item (because it won't be visible in the listing).

One of the behaviors that is expected is if, in the above scenario, the
author again publishes the article, that article will again "appear" on
the reading list for the given user.

Closes #14796

* Updating documentation and adding spec

* Addressing pull request feedback

Yes, I should've used a scope!  And also, no need to test present.  Just
let it's "truthy"-ness speak for itself!

* Update spec/services/search/reading_list_spec.rb

Co-authored-by: Jamie Gaskins <jamie@forem.com>

Co-authored-by: Jamie Gaskins <jamie@forem.com>
2021-11-24 13:22:26 -05:00
Michael Kohl
09828853f6
✂✂✂ Remove Connect (#14734)
* Remove Connect

* Remove more Connect specs

* Remove a lot more Connect code

* 🚮

* It all has to go

* Explicitly add httpclient

* Update application layout

* Remove messages association from User

* Start fixing specs

* reintroduce util function and refactor references

* Remove Connect Cypress test

* Fix more specs

* Remove Connect from listings

* Ignore contact_via_connect column on listings

* Remove contact_via_connect usages

* Ignore mod_chat_channel_id on tags

* Drop Connect tables

* Remove email_connect_messages from user notification settings

* Re-add httpclient 2.8.3

This was mistakenly removed as a merge conflict

* Don't need to exclude removed chat channel file

* Remove unneeded style for chat channels

* Remove unneeded channel list prop type

* Remove chat channels index/connect-link from getPageEntries

* Re-add comment from httpclient in Gemfile

* Remove connect references from mailers

Tag Moderators no longer have a chat channel

No longer will users be notified about new messages (there won't be
any)

No longer will users be notified about channel invites (you can't
invite anyone anymore)

* Don't configure Pusher and remove PUSHER_* from .env_sample

since it's removed from gemfile, the Pusher constant will not resolve, if this is
configured in the environment variables we'll fail to boot.

Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
Co-authored-by: Dan Uber <dan@forem.com>
2021-11-18 08:21:00 -06:00
Daniel Uber
d585ae2bf8
Prevent occasional matching on random name by specifying one (#14087)
We could mistakenly be picking up matches on user "name" rather than
user username due to faker giving a name like "Gale" or "Haley" or
"Annalee" which match the search term "ale".

Give an explicitly non-matching pattern for the name string to prevent
this.

Sampling from Faker::Name.name and filtering for case insensitive
matching against the search term shows slightly higher than 1% of
results will match this "ale" triple. Low enough to go unnoticed for a
long time.
2021-06-28 09:48:27 -05:00
rhymes
4bc56b8ac0
Search results: prioritize articles titles (#13918)
* Add HairTrigger spec

* Add weights to columns tsvectors

* Add correct sorting for search results

* Remove Timecop.travel

* Changed order of columns in trigger based on weights
2021-06-09 14:28:19 +02:00
Alex
a0ddb6a848
Cleanup search classes and serializers (#13645) 2021-05-05 09:20:59 -04:00
Daniel Uber
73529e71eb
Specify body and title of listings that shouldn't match a term (#13633)
Noticed the Faker book title included "No Country for Old Men" which
caused a failure expecting search for "old" to be empty (you can set
the title in the let block to replicate this).

Choose title and body that do not include "old" in order to prevent
false failures.
2021-05-03 15:47:12 -05:00
Alex
393ba00221
Remove Elasticsearch ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ ✂️ (#13606)
* Attempt number 1

* Fix rack_attack specs

* Fix users_searches_users spec

* Fix display_users_search_spec

* Fix comment typo

* Remove search:destroy task from cypress

* Remove port 9300 from gitpod

* Stub response in attack_spec
2021-05-03 11:09:45 -04:00
Alex
0211a7c341
[Search 2.0] Fix podcast values (#13550)
* Fix podcast values

* Also select slug from podcast_episodes
2021-04-28 08:58:45 -04:00
rhymes
8762815709
[Search 2.0] Match both word boundaries when filtering tags (#13554) 2021-04-28 11:24:45 +02:00
rhymes
f48cf141a6
Use regexp operator to avoid matching partial tags (#13547) 2021-04-27 20:27:11 +02:00
Alex
315dca98c9
[Search 2.0] Podcasts on search page (#13475)
* Working MVP

* Add specs

* Add partial index on podcasts.published

* Add tsvector indexes to podcast_episodes

* Filter for reachable PodcastEpisodes

* Fix indentation/formatting

* Add podcast attribute to serializer
2021-04-27 13:29:22 -04:00
rhymes
2ff317687e
[Search 2.0] Articles (#13540)
* Rename the search scope to reflect its generic usage

* Add and use Search::Postgres::Article
2021-04-27 18:59:01 +02:00
Michael Kohl
f2f5e911cf
Add Settings::Mascot (#13451)
* Add Settings::Mascot

* Add DUS

* Update usage

* Fix e2e test and controller

* Fix specs

* Fix remaining spec
2021-04-26 11:39:19 +07:00
rhymes
996f6e980f
[Search 2.0] Users (#13453)
* Add Search::Postgres::User

* Add sorting params

* Use a better algorithm to filter out suspended users

* Add tsvector index on users.name

* Add specs

* Add UserRole spec

* Update spec/services/search/postgres/user_spec.rb

Co-authored-by: Michael Kohl <citizen428@dev.to>

* Fix search with quotes in users's names

Co-authored-by: Michael Kohl <citizen428@dev.to>
2021-04-24 19:06:06 +02:00
rhymes
0cf99fd683
Use a manual query to find users by their names and usernames (#13474)
* Use a manual query to find users by their names and usernames

* Remove spec for accented names

* Remove superflous comment
2021-04-22 19:46:43 +02:00
rhymes
1fd38aa172
[Search 2.0] Allow mention autocomplete to search users names (#13462)
* Add tsvector index on users.name

* [Search 2.0] Allow mention autocomplete to search users names
2021-04-21 18:47:15 +02:00
Alex
cf0baf05f9
[Search 2.0] Comments on search page (#13401)
* Working MVP

* Add comment to COMMENTABLE_TYPES

* Add specs to search_spec

* Add service specs and fix ordering

* Fixed queries and specs

* Clarify COMMENTABLE_TYPES comment

* Move FORCED_EAGER_LOAD_QUERY to Comment

* Add highlighting to Comment search

* Add explicit .to_s on class_name params

* Remove Comment#forced_eager_load_serialized_data

* Revert CommentSerializer changes

* Hardcode class_name attribute

* Add PR feedback

* Use one liner syntax in serializer

* Update user comment

* Update path attribute logic

* Fix user_ids

* Remove PodcastEpisode skipped spec

* Add sort order to Search::Postgres::Comment

* Fix search_specs

* Move pagination TODO comment
2021-04-20 12:34:21 -04:00
rhymes
b6e562f14c
[Search 2.0] Optimize search reading list (#13275)
* Use trigger and tsvector column to speed up reading list search

* Add organization destroy spec and todo note

* Fix failing data update script due to not null constraint

* Remove the leading anchor in the trigger regexp

* Fix reading list specs

* Address feedback
2021-04-12 16:40:40 +02:00
Alex
3891c7d468
✂️ Remove search from Connect (#13235)
* Remove search from connect

* Remove alternate approach

* Alphabetize CHAT_CHANNEL_PARAMS

* Remove default value for user_ids

* Add spec for result ordering
2021-04-06 13:03:31 -04:00
Jacob Herrington
a5b2d109d5
Rename banned and comment_banned roles (#12270)
* Rename banned and comment_banned roles

* Add data update script to rename roles containing 'ban'

* Add named error for Suspended users

* Update unidiomatic method names

* Rename misc banned to suspended

* Apply suggestions from code review

Co-authored-by: Michael Kohl <me@citizen428.net>

* Add unit tests for suspended methods

This commit also adds TODO comments for removing banned and
comment_banned from the codebase after data update scripts have
successfully run on all of our Forems.

Co-authored-by: Michael Kohl <me@citizen428.net>
2021-04-06 10:12:14 -05:00
Alex
1597318db8
[Search 2.0] Listings (#13133)
* Add PgSearch to Listing model

* Working MVP - pre-optimizations

* Remove includes

* Update search_spec

* Add more specs

* Fix specs

* Refactor filter_by_category

* Update block parameter name

* Add published index to classified_listings table

* Preload user, organization, and listing_category

* Working MVP - pre-optimizations

* Refactor filter_by_category

* Update schema

* Fix schema for real

* Remove constant specs and make them private
2021-03-29 13:22:16 -04:00
rhymes
83852038e4
[Search 2.0] Reading list (#13052)
* Step one in populating the reading list with PG

This first attempt tries to recycle the `Search::ArticleSerializer` which is only
used in input in ES, but we're using it in output in PG.
For this reason it's currently 15.55x times slower

* Serialize only what is requested by the frontend

`Search::ArticleSerializer` which is only used in ES in the indexing step aims
to add as much info as possible for broader purposes, in this case
(with serialization in output) we should aim to save only what's requested from
the frontend.

* Optimize selection of articles columns

* Select only needed columns for users

* Compute total of reading list items

* Attach the basic filtering based on PG on the search controller

* Restructure in methods

* Add tags support

* Use LIKE on articles.cached_tag_list

* Fix tags as nil

* Fix default pagination

* Add optional FTS for reading list

* Reworded the tags comment explaining why

* Add index to reactions.status

* Fix total counter in Preact readingList component

* Fix total count in reading list backend search

* Add GIN index to articles.cached_tag_list

* Add service tests

* Add search request specs

* Added missing early return

* Update spec/requests/search_spec.rb

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>

* Extract MAX_PER_PAGE constant and add comments

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
2021-03-24 15:40:00 +01:00
Alex
eb741029b7
Use serializer over as_json (#13029) 2021-03-19 11:16:52 -04:00
rhymes
8b9f2d546c
Use faster serializers for Search::Postgres::Tag (#13022)
* Use faster serializers for Search::Postgres::Tag

* Fix spec
2021-03-18 09:30:56 +01:00
Alex
1e4d4db562
[Search 2.0] Add Search::Postgres::Username behind a feature flag (#12975)
* Add PostgreSQL FTS for usernames

* Change profile_image_90 logic

* Make search_users private

* Add tsvector index on usernames

* Limit the number of search results

* Update index name
2021-03-17 13:45:58 -04:00
rhymes
c0ea4e5878
[Search 2.0] Add Search::Postgres::Tag behind a feature flag (#12937)
* Add Search::Postgres::Tag to retrieve tags for autocompletion

* Add Search::Postgres::Tag in SearchController behind a feature flag

* Update spec/services/search/postgres/tag_spec.rb

Co-authored-by: Michael Kohl <citizen428@dev.to>

* Add index to tags.supported

* Fix hotness_score ordering

Co-authored-by: Michael Kohl <citizen428@dev.to>
2021-03-11 19:12:30 +01:00
Molly Struve
37f609e5c5
Return Additional Data Needed for Autocomplete (#12823)
* Return Additional Data Needed for Autocomplete

* return name instead of username twice
2021-02-25 10:55:46 -06:00
Molly Struve
fd8f83dc7d
Feature:Backend Username Search Endpoint for Mention Completion (#11479)
* Feature:Backend Username Search Endpoint for Mention Completion

* ensure user is authenticated to use usernames endpoint

* update spec with authaa
2020-11-18 16:43:59 -06:00
Alex
99c3fb276d
Auto generate mascot accounts (#11380)
* Auto generate mascot accounts

* Refactor to remove extra query

* Update registration specs

* Fix search query spec

* Fix reaction specs

* Create with bang and raise error
2020-11-12 16:33:16 -05:00
Krzysztof Rybka
20a19daa67
Copy username to search_fields (#11045)
* Copy username to search_fields

* Add test for searching by a username

* Add data update script for reindexing users for username search

* Comment out ReindexUsersForProfiles update script

Co-authored-by: rhymes <rhymes@hey.com>
2020-11-02 08:38:15 -06:00
Molly Struve
e9cb7b2fd9
[deploy] Refactor:Remove Reaction Search Code (#10377) 2020-09-23 12:27:50 -05:00
Molly Struve
86cb66e858
[deploy] Feature: Serve Reading List from FeedContent Index (#10294)
* Feature:Serve Reading List from FeedContent Index
* Fix paging, order ID desc, improve comments
2020-09-17 17:20:14 -05:00
Molly Struve
313bfef28c
[deploy] Feature:Allow for Search by ID for FeedContent (#10289) 2020-09-11 14:45:23 -05:00
Molly Struve
9c7c4e7062
[deploy] Optimization:Increase refresh_interval For Possible High Indexing Indexes (#10215) 2020-09-09 17:07:15 -05:00
Molly Struve
d0d3c2a2ab
[deploy] Bug Fix:Sort Reactions by created_at (#10171) 2020-09-03 14:18:02 -05:00
Molly Struve
31acf735cb
Comment out Flaky Reaction Search Sort Spec (#10118) 2020-08-31 19:16:45 -04:00
rhymes
c9e72858b7
Enable pending Style/SingleArgumentDig (#9842) 2020-08-18 10:26:40 -04:00
Molly Struve
d120fc7e54
[deploy] Bug Fix:Rename Listing Search Field in Elasticsearch to Fix Searching (#9609) 2020-08-03 10:18:17 -05:00
Ben Halpern
eb768a24ad
[deploy] Add explicit robots instruction meta tag and adjust cloudinary (#9601)
* Add explicit robots instruction meta tag and adjust cloudinary

* Fix tests

* Fix test to account for random input

* Skip test
2020-08-02 11:25:46 -04:00
Molly Struve
6ac637b9ef
[deploy] Boost Scores For Adjacent Words (#9562)
* Feature:Boost Scores For Adjacent Words

* using equal for spec array to ensure order
2020-07-30 15:24:11 +02:00
rhymes
5b62811c98
[deploy] Rubocop: fix violations of Layout/LineLength (#9197) 2020-07-08 08:36:36 -05:00
Molly Struve
85a10524d4
Change Highlight Tags to mark instead of em (#9098)
* Change Highlight Tags to mark instead of em

* Ensure User ID is never present and stub Sitconfig
2020-07-03 21:50:06 +02:00
Dana Scheider
f6a4c844cd
Implement chronological sorting of search results (#8193)
* Implement search ordering by allowing sort_by and sort_direction
params to be passed through from the search form/page.

* Change ID of select to 'sort' instead of 'order' for consistency

* Fix styling and data attributes of select options

* Enable front end to pass sort_by and sort_direction params for searches to the back end

* Ensure that only query string params that are supposed to be changed are changed in each function

* Respond to review comments (additional commits to come but these were the quick changes)

* Move options for select into template

* Use addEventListener instead of using onchange in response to review comments

* Add tabs for sorting search results

* Finish implementing tabs for ordering search results

* Remove CSS rule applying to element that no longer exists

* Remove more rules created for an element that has been removed

* Remove more code added for the select dropdown that I've removed

* Remove duplicate jobs banner
2020-06-11 13:11:01 +02:00
Molly Struve
5d22e71ad6
nothing to see here....except maybe reverting a push to master (#8308)
🙈
2020-06-05 20:34:21 -05:00
mstruve
533cab50d6 synchronous roles 2020-06-05 20:02:41 -05:00
Michael Kohl
7f75f99560
[deploy] Rename classified listings (#7910)
* Change models and related files

* Update controllers and specs

* More renaming

* Seek and destroy, I mean search and replace

* Round up the stragglers

* Ground control to Major Travis...

* More fixes

* PR feedback

* Various fixes

* Rename view

* Fix list query builder

* Unify request specs

* Fix some API spec errors

* Fix remaining API specs

* Make spec conform to API

* Fix leftover problems

* Fix JS tests

* Fix column name in select

* Fix API specs

* Fix search specs

* Paging Mr. Travis
2020-05-27 13:35:09 +00:00
Mac Siri
9c58ebc61e
[deploy] Apply HTML encoder to FeedContent's query (#7985)
* Use .textContent to apply Search Snippets

* Improve var name

* Apply HTML encoder to FeedContent's query

* Add spec

* Expand FeedContent specs

* Actually let's not use approval
2020-05-25 13:06:35 -04:00
Sarthak Sharma
90d60bf73f
[deploy] 🚀 Feature: Ability to search discoverable channels and send request to join (#7385)
* Feature 🚀 : Ability to delete messages in chat channels

- Sending message ID to frontend
- Deleting Message
- Use pusher to delete message realtime

* Minor Bug 🐞: Show message action only for current user

- User can delete or edit their own messages

* Test cases added

* Bug 🐞: Update message id for receiver

Message id was not sent to receiver by pusher

* Refactoring🛠: Message controller refactoring

* Test Cases📝 : Specs for Delete message added

* Feature 🚀 : Ability to edit messages

* Test Cases📝 : Specs for Edit message added

* added new column discoverable for public channel and changes in elasticsearch

* fix corrections

* changes in serializer file, added channel_discoverable

* added checkbox with discoverable policy

* changes in code for discoverable channels

* accept and reject member invitation by mod

* add joining request to closed groups

* fixed merge error

* changes in joining member to closed groups

* join to closed group

* changes in message action of joining

* changes in chat upopened json

* 🚀Feature : Ability to search Global Channels

* touch updated_at in after commit update

* 🚀Feature : Ability to send request to discoverable channel

* 🚀Feature : Ability to send request to discoverable channel

* created new route for joining closed channel

* 🚀 Success handle on joining request sent

* removed redundant code

* join_channel improvement, removed authorization for join channel

* list of joining requests

* added joining_request status channels on search list

* changes in mailer and query builder optimized

* 🛠 Adding filter for new Query

* added rspec for add membership method in controller

* rspec for sending join channel request

* invite join request channel list rspec

* test case for query builder discoverable

* viewable and discoverable channel list

* refactored logic for search channels

* 🚀 Check if Request already sent

* 🛠 Optimizing code and making channelButton Component

* 🛠 Optimizing codefor SVG problem

* 🛠 Optimized action.js

* changes in search controller query

* hot fix

* removed unwanted code

* 🛠  Fix the Channel Name problem

* 🛠  Optimizing code further for CodeClimate

* added new column discoverable for public channel and changes in elasticsearch

* fix corrections

* changes in serializer file, added channel_discoverable

* added checkbox with discoverable policy

* changes in code for discoverable channels

* accept and reject member invitation by mod

* add joining request to closed groups

* fixed merge error

* changes in joining member to closed groups

* join to closed group

* changes in message action of joining

* changes in chat upopened json

* touch updated_at in after commit update

* 🚀Feature : Ability to search Global Channels

* 🚀Feature : Ability to send request to discoverable channel

* 🚀Feature : Ability to send request to discoverable channel

* created new route for joining closed channel

* 🚀 Success handle on joining request sent

* removed redundant code

* join_channel improvement, removed authorization for join channel

* list of joining requests

* added joining_request status channels on search list

* changes in mailer and query builder optimized

* added rspec for add membership method in controller

* rspec for sending join channel request

* invite join request channel list rspec

* 🛠 Adding filter for new Query

* test case for query builder discoverable

* viewable and discoverable channel list

* refactored logic for search channels

* 🚀 Check if Request already sent

* 🛠 Optimizing code and making channelButton Component

* 🛠 Optimizing codefor SVG problem

* 🛠 Optimized action.js

* changes in search controller query

* hot fix

* 🛠  Fix the Channel Name problem

* 🛠  Fixing merge problem

* 🛠  Optimizing code further for CodeClimate

* updated UI for membership edit

* fixed test casses and fixed UI for chat_channel_edit

* 🛠napshots added

* test cases fixed

* 🛠 Test cases added for new component

* channel settings accesible only by mod

* 🛠 More Test cases added

* 🛠 Svg code optimized

* 🛠 Svg code optimized in videocontent

* optimized code for search query

* fix test case for query builder

* changes in joining closed channel logic

* refactored join channel

* redirect to edit channel after joining request

* changes in test case for redirect

* optimized code

* 🛠 Eslint bugs fixed

* test case fixed

* optimization

* olving channel repetition problem

* optimization of code for query builder

* changes in query builder

* test cases fixed

* 🛠 Handling reduntant data on frontend

* 🛠  Don't show data if no filter query

* 🛠 Optimized code for fixing bugs and code coverage

* 🛠  Optimizing code further for CodeClimate

Co-authored-by: Parasgr-code <paras.gaur@skynox.tech>
2020-05-11 09:29:15 -04:00
Michael Kohl
46f94b135d
[deploy] Classified listings refactor part 3 (#7498)
* Ignore category column

* Move view oriented class methods to helper

* Start refactoring ClassifiedListingsToolkit

* Add attr_accessor instead of category

* Replace old category column with association

* More cleanup

* Trigger Travis

* Don't rename associations, category_slug -> category

* Indiana Kohl and the missing newline

* More refactoring and spec fixes

* Add dashboard for classified listing categories

* Remove TODO comment

Co-authored-by: rhymes <rhymesete@gmail.com>
2020-04-30 18:57:01 -04:00