Commit graph

85 commits

Author SHA1 Message Date
Alex
1597318db8
[Search 2.0] Listings (#13133)
* Add PgSearch to Listing model

* Working MVP - pre-optimizations

* Remove includes

* Update search_spec

* Add more specs

* Fix specs

* Refactor filter_by_category

* Update block parameter name

* Add published index to classified_listings table

* Preload user, organization, and listing_category

* Working MVP - pre-optimizations

* Refactor filter_by_category

* Update schema

* Fix schema for real

* Remove constant specs and make them private
2021-03-29 13:22:16 -04:00
rhymes
83852038e4
[Search 2.0] Reading list (#13052)
* Step one in populating the reading list with PG

This first attempt tries to recycle the `Search::ArticleSerializer` which is only
used in input in ES, but we're using it in output in PG.
For this reason it's currently 15.55x times slower

* Serialize only what is requested by the frontend

`Search::ArticleSerializer` which is only used in ES in the indexing step aims
to add as much info as possible for broader purposes, in this case
(with serialization in output) we should aim to save only what's requested from
the frontend.

* Optimize selection of articles columns

* Select only needed columns for users

* Compute total of reading list items

* Attach the basic filtering based on PG on the search controller

* Restructure in methods

* Add tags support

* Use LIKE on articles.cached_tag_list

* Fix tags as nil

* Fix default pagination

* Add optional FTS for reading list

* Reworded the tags comment explaining why

* Add index to reactions.status

* Fix total counter in Preact readingList component

* Fix total count in reading list backend search

* Add GIN index to articles.cached_tag_list

* Add service tests

* Add search request specs

* Added missing early return

* Update spec/requests/search_spec.rb

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>

* Extract MAX_PER_PAGE constant and add comments

Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
2021-03-24 15:40:00 +01:00
Alex
eb741029b7
Use serializer over as_json (#13029) 2021-03-19 11:16:52 -04:00
rhymes
8b9f2d546c
Use faster serializers for Search::Postgres::Tag (#13022)
* Use faster serializers for Search::Postgres::Tag

* Fix spec
2021-03-18 09:30:56 +01:00
Alex
1e4d4db562
[Search 2.0] Add Search::Postgres::Username behind a feature flag (#12975)
* Add PostgreSQL FTS for usernames

* Change profile_image_90 logic

* Make search_users private

* Add tsvector index on usernames

* Limit the number of search results

* Update index name
2021-03-17 13:45:58 -04:00
rhymes
c0ea4e5878
[Search 2.0] Add Search::Postgres::Tag behind a feature flag (#12937)
* Add Search::Postgres::Tag to retrieve tags for autocompletion

* Add Search::Postgres::Tag in SearchController behind a feature flag

* Update spec/services/search/postgres/tag_spec.rb

Co-authored-by: Michael Kohl <citizen428@dev.to>

* Add index to tags.supported

* Fix hotness_score ordering

Co-authored-by: Michael Kohl <citizen428@dev.to>
2021-03-11 19:12:30 +01:00
Molly Struve
37f609e5c5
Return Additional Data Needed for Autocomplete (#12823)
* Return Additional Data Needed for Autocomplete

* return name instead of username twice
2021-02-25 10:55:46 -06:00
Kirk Haines
ed74f2f245
Abstract DatadogStatsClient to ForemStatsClient (#12304)
* This change abstracts the DatadogStatsClient into a ForemStatsClient.
The purpose of this abstraction is to set the foundation for a subsequent PR that will allow one to use New Relic for recording Forem stats, instead of Datadog, if there is a New Relic configuration found.
This specific change creates an abstraction layer that can be built upon, without changing any actual default behavior. All specs still pass.

* Use delegate instead of explicit methods.

* Delegate instead of explicit methods.

* Fix the error.

* Refactor according to the suggestions in the comments.

* Ooops.  Stats work better when all of the code is committed.

* Removing the alias of count to increment since that was done in error.
2021-01-27 11:25:44 -05:00
Molly Struve
fd8f83dc7d
Feature:Backend Username Search Endpoint for Mention Completion (#11479)
* Feature:Backend Username Search Endpoint for Mention Completion

* ensure user is authenticated to use usernames endpoint

* update spec with authaa
2020-11-18 16:43:59 -06:00
rhymes
fe75dcde1c
Routine rubocop fixes (#10478) 2020-09-29 10:14:26 -04:00
Molly Struve
e9cb7b2fd9
[deploy] Refactor:Remove Reaction Search Code (#10377) 2020-09-23 12:27:50 -05:00
Molly Struve
86cb66e858
[deploy] Feature: Serve Reading List from FeedContent Index (#10294)
* Feature:Serve Reading List from FeedContent Index
* Fix paging, order ID desc, improve comments
2020-09-17 17:20:14 -05:00
Molly Struve
313bfef28c
[deploy] Feature:Allow for Search by ID for FeedContent (#10289) 2020-09-11 14:45:23 -05:00
Molly Struve
9c7c4e7062
[deploy] Optimization:Increase refresh_interval For Possible High Indexing Indexes (#10215) 2020-09-09 17:07:15 -05:00
rhymes
2d881ef571
Routine rubocop fixes (#10237) 2020-09-08 12:06:47 -04:00
Molly Struve
d0d3c2a2ab
[deploy] Bug Fix:Sort Reactions by created_at (#10171) 2020-09-03 14:18:02 -05:00
rhymes
c9e72858b7
Enable pending Style/SingleArgumentDig (#9842) 2020-08-18 10:26:40 -04:00
Molly Struve
db98a8e224
[deploy] Bug Fix:Remove User Roles From Elasticsearch Returned Data (#9831) 2020-08-17 13:17:53 -05:00
rhymes
085c60992b
Rubocop: Enable and fix Style/OptionalBooleanParameter cop (#9711)
* Enable and fix Style/OptionalBooleanParameter cop

* Keep regular parameters for workers

* Keep regular parameters for workers

* Fix spec

* Trigger Travis correctly
2020-08-10 19:35:55 +02:00
rhymes
cc8bfcb5c0
Enable Rubocop 0.89 cops and fix Lint/ violations (#9709)
* Enable new Lint/ cops and run rubocop -a

* Fixing last remaining things with rubocop -a

* Enable and fix Style/ExplicitBlockArgument and Style/GlobalStdStream

* Forgot parenthesis
2020-08-10 16:57:12 +02:00
Molly Struve
d120fc7e54
[deploy] Bug Fix:Rename Listing Search Field in Elasticsearch to Fix Searching (#9609) 2020-08-03 10:18:17 -05:00
Molly Struve
6ac637b9ef
[deploy] Boost Scores For Adjacent Words (#9562)
* Feature:Boost Scores For Adjacent Words

* using equal for spec array to ensure order
2020-07-30 15:24:11 +02:00
luigi
7c22026d30
Use Enumerable#filter_map in more places (#9389)
because it's a little faster without lose readability
2020-07-20 18:54:38 +02:00
rhymes
5b62811c98
[deploy] Rubocop: fix violations of Layout/LineLength (#9197) 2020-07-08 08:36:36 -05:00
Molly Struve
85a10524d4
Change Highlight Tags to mark instead of em (#9098)
* Change Highlight Tags to mark instead of em

* Ensure User ID is never present and stub Sitconfig
2020-07-03 21:50:06 +02:00
Michael Kohl
7f75f99560
[deploy] Rename classified listings (#7910)
* Change models and related files

* Update controllers and specs

* More renaming

* Seek and destroy, I mean search and replace

* Round up the stragglers

* Ground control to Major Travis...

* More fixes

* PR feedback

* Various fixes

* Rename view

* Fix list query builder

* Unify request specs

* Fix some API spec errors

* Fix remaining API specs

* Make spec conform to API

* Fix leftover problems

* Fix JS tests

* Fix column name in select

* Fix API specs

* Fix search specs

* Paging Mr. Travis
2020-05-27 13:35:09 +00:00
Andy Zhao
c42fd3461e
[deploy] Rename positive_reactions_count to public_reactions_count in logic (#7926)
* Rename positive_reactions_count to public_reactions_count

* Add positive reactions count back in so we can remove it

* Use public_category method for reactions

* Add positive_reactions_count in case any old caches rely on it

* Add positive_rxn_count to account for API endpoints

* Remove unused method

* One more spot...

* Add method back in because of caches

* Update specs to match new functionality

* Fix typo

* Remove unused methods
2020-05-26 12:36:28 -04:00
Mac Siri
9c58ebc61e
[deploy] Apply HTML encoder to FeedContent's query (#7985)
* Use .textContent to apply Search Snippets

* Improve var name

* Apply HTML encoder to FeedContent's query

* Add spec

* Expand FeedContent specs

* Actually let's not use approval
2020-05-25 13:06:35 -04:00
Andy Zhao
e686d63d78
[deploy] Add public reactions count fields to Elasticsearch (#7956)
* Add public_reactions_count field to elasticsearch

* Add new data update scripts and update old ones

* Use BulkIndexWorker to batch indexing

* Use proper search class for handling feed content

* Remove redundant script oops

* Rename data update script and remove old specs

* Combine specs into one file
2020-05-20 13:06:48 -04:00
Molly Struve
dbb5c752ab
[deploy] Sample Noisy Honeycomb Spans (#7891) 2020-05-18 10:36:01 -05:00
rhymes
3130261384
Refactor GitHub::Client to extract Github::OauthClient for users (#7848)
* Introduce Github::OAuthClient and refactor shortcut Github::Client

* Better stubbing and fix specs

* Fix Badge rewarder

* Add dummy values for Travis

* Fix spec?

* Simplify

* Fix spec take 2?

* Is this the right one?

* Hopefully this is the last time, for realz

* Fix spec for the nth time?

* Which deity do I have to offer a sacrifice to?

* This is it

* Fix Honeycomb name
2020-05-18 14:23:55 +02:00
Sarthak Sharma
90d60bf73f
[deploy] 🚀 Feature: Ability to search discoverable channels and send request to join (#7385)
* Feature 🚀 : Ability to delete messages in chat channels

- Sending message ID to frontend
- Deleting Message
- Use pusher to delete message realtime

* Minor Bug 🐞: Show message action only for current user

- User can delete or edit their own messages

* Test cases added

* Bug 🐞: Update message id for receiver

Message id was not sent to receiver by pusher

* Refactoring🛠: Message controller refactoring

* Test Cases📝 : Specs for Delete message added

* Feature 🚀 : Ability to edit messages

* Test Cases📝 : Specs for Edit message added

* added new column discoverable for public channel and changes in elasticsearch

* fix corrections

* changes in serializer file, added channel_discoverable

* added checkbox with discoverable policy

* changes in code for discoverable channels

* accept and reject member invitation by mod

* add joining request to closed groups

* fixed merge error

* changes in joining member to closed groups

* join to closed group

* changes in message action of joining

* changes in chat upopened json

* 🚀Feature : Ability to search Global Channels

* touch updated_at in after commit update

* 🚀Feature : Ability to send request to discoverable channel

* 🚀Feature : Ability to send request to discoverable channel

* created new route for joining closed channel

* 🚀 Success handle on joining request sent

* removed redundant code

* join_channel improvement, removed authorization for join channel

* list of joining requests

* added joining_request status channels on search list

* changes in mailer and query builder optimized

* 🛠 Adding filter for new Query

* added rspec for add membership method in controller

* rspec for sending join channel request

* invite join request channel list rspec

* test case for query builder discoverable

* viewable and discoverable channel list

* refactored logic for search channels

* 🚀 Check if Request already sent

* 🛠 Optimizing code and making channelButton Component

* 🛠 Optimizing codefor SVG problem

* 🛠 Optimized action.js

* changes in search controller query

* hot fix

* removed unwanted code

* 🛠  Fix the Channel Name problem

* 🛠  Optimizing code further for CodeClimate

* added new column discoverable for public channel and changes in elasticsearch

* fix corrections

* changes in serializer file, added channel_discoverable

* added checkbox with discoverable policy

* changes in code for discoverable channels

* accept and reject member invitation by mod

* add joining request to closed groups

* fixed merge error

* changes in joining member to closed groups

* join to closed group

* changes in message action of joining

* changes in chat upopened json

* touch updated_at in after commit update

* 🚀Feature : Ability to search Global Channels

* 🚀Feature : Ability to send request to discoverable channel

* 🚀Feature : Ability to send request to discoverable channel

* created new route for joining closed channel

* 🚀 Success handle on joining request sent

* removed redundant code

* join_channel improvement, removed authorization for join channel

* list of joining requests

* added joining_request status channels on search list

* changes in mailer and query builder optimized

* added rspec for add membership method in controller

* rspec for sending join channel request

* invite join request channel list rspec

* 🛠 Adding filter for new Query

* test case for query builder discoverable

* viewable and discoverable channel list

* refactored logic for search channels

* 🚀 Check if Request already sent

* 🛠 Optimizing code and making channelButton Component

* 🛠 Optimizing codefor SVG problem

* 🛠 Optimized action.js

* changes in search controller query

* hot fix

* 🛠  Fix the Channel Name problem

* 🛠  Fixing merge problem

* 🛠  Optimizing code further for CodeClimate

* updated UI for membership edit

* fixed test casses and fixed UI for chat_channel_edit

* 🛠napshots added

* test cases fixed

* 🛠 Test cases added for new component

* channel settings accesible only by mod

* 🛠 More Test cases added

* 🛠 Svg code optimized

* 🛠 Svg code optimized in videocontent

* optimized code for search query

* fix test case for query builder

* changes in joining closed channel logic

* refactored join channel

* redirect to edit channel after joining request

* changes in test case for redirect

* optimized code

* 🛠 Eslint bugs fixed

* test case fixed

* optimization

* olving channel repetition problem

* optimization of code for query builder

* changes in query builder

* test cases fixed

* 🛠 Handling reduntant data on frontend

* 🛠  Don't show data if no filter query

* 🛠 Optimized code for fixing bugs and code coverage

* 🛠  Optimizing code further for CodeClimate

Co-authored-by: Parasgr-code <paras.gaur@skynox.tech>
2020-05-11 09:29:15 -04:00
Molly Struve
ea1d38d3b8
[deploy] Bulk Fetch Follow Button Data for Users (#7676) 2020-05-08 13:58:58 -05:00
Sai Bhargav
2d20d67594
[deploy] Fix listing tags bug (#7533)
* Fix listing tags bug

* Fix failing specs

* Fix failing specs

* Fix failig spec

* Fix failing specs

* Refactor code

* Fix failing specs

* Support AND/OR queries based on params

* Add tag_boolean to search params
2020-04-28 16:04:09 +02:00
Molly Struve
4ab202ab0a
[deploy] Execute Reading List searches in Elasticsearch (#7440)
* Execute Reading List searches in Elasticsearch

* add additional readinglist front tests
2020-04-24 10:30:17 -04:00
Molly Struve
73120a67c9
[deploy] Bug Fix: Use strings with colons for all Datadog tags (#7442) 2020-04-22 12:24:57 -05:00
Molly Struve
556bb565bf
[deploy] allow option to search by all tags in Elasticsearch for reactions (#7425) 2020-04-21 18:30:04 -04:00
Molly Struve
4f374acbc5
[deploy] filter Elasticsearch by org id properly (#7389) 2020-04-20 10:01:30 -05:00
Ben Halpern
3d4944ebae
[deploy] Add video duration to search results (#7377) 2020-04-18 15:16:38 -04:00
Ben Halpern
a0504f5f77
[deploy] Include organization in feed_content results (#7374) 2020-04-18 13:07:59 -04:00
Molly Struve
2c99424205
[deploy] Execute Feed Searches through Elasticsearch (Round 2) (#7231) 2020-04-17 08:38:07 -05:00
Molly Struve
2d40174a92
implement backend search functionality for readinglist Reactions (#7308) 2020-04-17 08:37:22 -05:00
Molly Struve
7be90d1b4b
[deploy] overwrite name field for honeycomb when sending Elasticsearch requests (#7338) 2020-04-17 12:30:29 +02:00
Molly Struve
223703f0d3
[deploy] Update URLSearchParams to handle arrays correctly for Rails (#7300) 2020-04-15 14:58:18 -05:00
Molly Struve
b10668a454
[deploy] Create a Bulk Index Elasticsearch Worker (#7265) 2020-04-15 09:08:59 -05:00
Molly Struve
07050baa28
[deploy] Refactor search_documents Method Into Search Base.rb Class (#7260) 2020-04-14 08:39:50 -05:00
rhymes
63893e4ac1
Upgrade twilio-ruby to 5.33.0 and elasticsearch to 7.6.0 (#7208)
* Upgrade gems

* Switch ES client to Patron
2020-04-10 15:30:51 +02:00
Molly Struve
26acabce51
[deploy] add custom sorting and scoring for feed content (#7199) 2020-04-09 18:30:27 -04:00
Molly Struve
578d09a54e
[deploy] return cloudinary url to ensure videos are viewable in feeds (#7157) 2020-04-08 15:47:47 -04:00
Guilherme Vinicius Moreira
9618ccc927
[deploy] Add new user attributes to elasticsearch (#7123)
* chore(user): add new user attributes to elasticsearch

* Fix schema.rb
2020-04-07 13:16:15 -04:00