* Move static profile fields to profiles columns
In order to remove some ambiguity around the availability of certain
profile fields, we can designate certain fields as "static." These
fields are intended to exist on every Forem regardless of configuration;
they contain basic personal info that most Forem's will likely use.
Because these fields already exist on some Forems it is necessary to
migrate the data from existing profile's data column to their respective
columns.
This change should behave as expected irrespective of the existence of
the static fields having associated ProfileFields, however, the UI that
is rendered in a user's settings still depends on the presence of the
ProfileFields. We can address that in a future change when we are
prepared to delete those ProfileFields entirely. We should make sure the
migration occurs without issue before moving to that step, in my
opinion.
* Apply suggestions from code review
Co-authored-by: Michael Kohl <me@citizen428.net>
* Add clarifying comments
Co-authored-by: Michael Kohl <me@citizen428.net>
Co-authored-by: Michael Kohl <me@citizen428.net>
* schema file undelete description
* feat: v1 of the script
* Flesh out remaining enums under their categories
* complete UsersSettings data update script
* complete DUS for relevant attributes in users and profiles tables
* complete DUS for users_notification_settings
* alphabetize user_settings sql file
* safeguard against null values for "null: false" settings
* Set up actual UsersSettings DUS and specs files
* fix broken DUS script
* complete specs for UsersSetting DUS
* Address QA of specs
* complete specs for users_notification_settings DUS
* fix the typos (thanks Julianna!)
* add missing attribute "email_membership_newsletter"
Co-authored-by: Arit Amana <msarit@gmail.com>
* Rename SiteConfig
* More renaming
* Update spec
* Update mandatory settings mapping
* More renaming
* e2e test fixes
* You have a rename, and you have a rename
* Spec fix
* More changes
* Temporarily disable specs
* After-merge update
* Undo rename for migration
* undo rename of DUS
* Fix DUS
* Fix merge problem
* Remove redundant DUS
* Fix specs
* Remove unused code
* Change wrong class name
* More cleanup
* Re-add missing values to constant
* Fix constant
* Fix spec
* Remove obsolete fields
* Add accidentally removed field
* Update spec
* Move methods from Settings::General to ForemInstance
* Remove unneeded model
* Change mentions of 'site config'
* Add tsvector index on listings
* Fix sorting order when fetching tag flares
* Add published_at as a sorting condition for Homepage::ArticlesQuery
* Re-added param needed by ES, this got lost somewhere down the line
* Use trigger and tsvector column to speed up reading list search
* Add organization destroy spec and todo note
* Fix failing data update script due to not null constraint
* Remove the leading anchor in the trigger regexp
* Fix reading list specs
* Address feedback
* First set of changes for Ahoy 2.0
* Fix line length
* Change test
* Remove unused test
* Remove ahoy_messages.opened_at
* Simply digest email sending calculation to not rely on last_opened
* Remove outdated comment
* Fix typo in test
* Rework site config email digest to a single value
* Fix merge duplication
* Remove UTM references from ahoy emails
* Fix credits tests
* Remove UTM from expected email params
* feat: add the two new models users_setting and users_notification_setting
* feat: add the settings and notification_settings table to the schema
* feat: add the user and notification models
* feat: add the user_id foreign key to the model
* chore: sneaky indent
* feat: add some fields from the profile attributes
* Revert "feat: add some fields from the profile attributes"
This reverts commit 376828746ded063a243505d317140fa5339227cf.
* chore: add some profile field attributes
* chore: remove language_settings
* chore: update indent
* chore: remove language_settings
* feat: changes to the tables
* chore: remove validation in favor of the foreign keys
* chore: add default for editor version
* Address PR review suggestions
* setting_spec.rb needs to be fixed; need help
* Working on PR review comments
* Continue with addressing PR review comments
* Remove normalize_config_values method; pass correct values from forms
* Address Travis failures
* revert some unnecessary changes in spec file
Co-authored-by: Arit Amana <msarit@gmail.com>
* Add PgSearch to Listing model
* Working MVP - pre-optimizations
* Remove includes
* Update search_spec
* Add more specs
* Fix specs
* Refactor filter_by_category
* Update block parameter name
* Add published index to classified_listings table
* Preload user, organization, and listing_category
* Working MVP - pre-optimizations
* Refactor filter_by_category
* Update schema
* Fix schema for real
* Remove constant specs and make them private
We have a btree index on this table but it is not invoked with a `LIKE`
operation that uses `%` - only when using `LIKE` with an exact match.
This query is DEV's second-most intense query by total time spent with
a p50 latency of ~130ms. This index brings it down to 5ms at p90.
* Step one in populating the reading list with PG
This first attempt tries to recycle the `Search::ArticleSerializer` which is only
used in input in ES, but we're using it in output in PG.
For this reason it's currently 15.55x times slower
* Serialize only what is requested by the frontend
`Search::ArticleSerializer` which is only used in ES in the indexing step aims
to add as much info as possible for broader purposes, in this case
(with serialization in output) we should aim to save only what's requested from
the frontend.
* Optimize selection of articles columns
* Select only needed columns for users
* Compute total of reading list items
* Attach the basic filtering based on PG on the search controller
* Restructure in methods
* Add tags support
* Use LIKE on articles.cached_tag_list
* Fix tags as nil
* Fix default pagination
* Add optional FTS for reading list
* Reworded the tags comment explaining why
* Add index to reactions.status
* Fix total counter in Preact readingList component
* Fix total count in reading list backend search
* Add GIN index to articles.cached_tag_list
* Add service tests
* Add search request specs
* Added missing early return
* Update spec/requests/search_spec.rb
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* Extract MAX_PER_PAGE constant and add comments
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* Added credits_awarded to badges
* Data update script for updating credits_awarded
* Data update script for updating credits_awarded
* Added null: false to badges.credits_awarded
* Add PostgreSQL FTS for usernames
* Change profile_image_90 logic
* Make search_users private
* Add tsvector index on usernames
* Limit the number of search results
* Update index name
* Add Search::Postgres::Tag to retrieve tags for autocompletion
* Add Search::Postgres::Tag in SearchController behind a feature flag
* Update spec/services/search/postgres/tag_spec.rb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Add index to tags.supported
* Fix hotness_score ordering
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Add new table and model
* Store banned user username hash on delete
* Prevent previously banned user from signing up again
* Update method name
* Refactor code and add more specs
* Test improvements
* Don't override existing global flash
* Fix typo
* Update spec description
* Update schema.rb
* More schema.rb fixes
* Simplify spec
* Update migration
* Clean up migration
* Rename method
* Add DataDog counter
* Revisit error handling
* Remove spurious empty line
* Make model name more explicit
* Added custom seed task
* Added a test for the initial login for the intial admin of a forem instance.
* Renamed admin login seed data file.
* Pulled functions out of main Cypress plugin file and import them now.
* Added some comments.
* Added a comment about Cypress tasks requiring a return value to be considered successful.
* Moved admin_login_setup seed file to spec folder.
* Made change requests in regards to rake task.
* Fixed reference to SiteConfig
* Created the seeder class to be shared for all seed files.
* Added a missing param to function comment.
* Fixed seeder require in e2e test seed file.
* Fixed seeder require in seeds.rb. Why is Rails.root required?
* Added an environment guard for not being production in the e2e seed task.
* Made seeder require relative.
* Trying something for the Elastic Search issue in CI.
* Revert "Trying something for the Elastic Search issue in CI."
This reverts commit 7cb2a963c8ac1f9242c612a1b9fe8ff814605df6.
* Search indices are now removed via bundle exec rake search:destroy
* Now rake search:destroy is used in the Cypress task for resetting data as well.
* Moved Seeder class to app/lib
* Added cypress-rails
* Got two flows working with db rollbacks.
* Trimmed down the e2e dataset for now.
* Added a custom Cypress command to encompass test setup.
* removed unused e2e rake task
* added some user login/password e2e tests.
* Added a note about Cypress not cleaning cookies, so we do for the moment.
* Removed code no longer required.
* Removed comment in test that is no longer pertinent to the test suite.
* Removed data cleaning plugins as cypress rails gem handles it now.
* Added @citizen428's recommendation got raising an error if trying to seed production.
* Temporarily have paralleziation of e2e tests disabled to figure out cypres-rails/knapsack pro integration.
* Doh! Forgot a semi-colon in the Travis config.
* Added a comment about not integrating cypress-rails/knapsack pro right now.
* Removed knapsack pro npm package for now as we aren't using it.
* Reworded comment about cypress-rails/kanpsackpro.
* Removed seeding for tags to follow for onboarding in e2e seed data.
* Made pree2e script e2e:setup so it only runs explicitly when needed.
* Renamed bin/e2e to bin/e2e-ci.
* Created a new script for local e2e testing.
* Removed RAILS_ENV as it's in the e2e-ci script already.
* Fixed an auto corrected command.
* Fixed wording in e2e script prompt.
* Removed bundle exec rake data_updates:run that I had added. Doesn't appear to be necessary.
* Added missing new lines.
* Renamed e2e seed file and only use that one now. THere are no others.
* Some script cleanup.
* Added a check for the E2E environment so as to not pollute system tests.
* Now e2e test server cleanup occurs when the server shuts down.
* Put puts as this is really just for testing.
* Put environment variable in the proper place for runnning bundle exec.
* Added a check to only run for E2E tests.
* Some bash script formatting.
* Removed copy paste irrelevant comments.
* Updated e2e server shutdown message.
* Now a data-testid attribute is used for finding the user account errors panel.
* Made test selectors ignore casing of text.
* Removed prod paranoia check in e2e seed file.
* Refactored spec to use new cy.loginUser(user) command.
* Update app/lib/seeder.rb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Fixed flakiness that @aitchiss was experiencing.
* Rubocop fix.
* Due to a Cypress issue, fixing cookie clearing with another tweak.
* Bumped the wait time to 500ms if cookies don't clear.
* Added a 'bundle check' to the bin/e2e script as suggested by @katiedavis.
* Removed some white space
Co-authored-by: Michael Kohl <citizen428@dev.to>
DEV has 30+ million email messages, to display some of them to the admin, they
all are scanned to find those that belong to the requested feedback messages.
This results in SQL queries that can take up to 15 seconds.
By adding an index, we cut the SQL response time to milliseconds.
