* feat: add a hidden tags route + controller + view
* refactor: make a re-useable template for hidden_tags and following_tags
* chore: add some jsdocs to document/help follow the code
* chore: add some jsdocs to document/help follow the code
* feat: add soem comment changes
* refactor: use more readable string concatenation
* refactor: use more readable string concatenation
* fix: whoops error in template literal
* feat: potential solution to being able to query the paginated hidden tags whilst using the existing following_tags functionality
* feat: typo
* chore: add comment
* update the count for the dashboard sidebar
* feat: add the correct count on the mobile action menu
* spec: update the request spec to ensure that we have the correct tags being displayed
* refactor: query for hidden tags
* chore: update initscrolling language in comments
* refactor: prioritize readability
* refactor: update where clause
* chore: add more clarifying comments
* feat: make sure that we only rename if action exists in initScrolling
* feat: update the styling for the update weights section
* feat: include followable on tag page
* feat: update points to explicit_points in the followings_controller
* feat: abstract out the uncommon UI elements and move them to the top level template
* feat: remove unused html
* chore: remove the antispam tag
* feat: update the tags template and corresponding localization keys
* chore: update the comment
* feat: update the following_tags spec
* spec: update accordingly
* refactor: use URLSearchParams
* feat: update the follow and antifollow/hidden tags
* feat: use a range
* fix: use explicit points
* Empty commit
* feat: update the pagination styles
* update localization
* Renamed display ad variables and constants
* Renamed display_ads associations
* Fixed variable name in display ads
* Renamed display_ads variables in views
* Fixed admin billboard view spec
* Fixed variable name in FilteredAdsQuery
* Prepare: relocate user suggestions
* Prepare: relocate users suggestion service -> query
* Organization query for orgs with above-average scores
* Rubocop
* Limit to last 3 weeks
* Tweak recent scope, limit to 5 orgs
* Onboarding routes are also always JSON
* Divide by zero makes NaN means
* Add Orgs suggester into suggestions
* Rubocop
* select distinct orgs
* Fix for weird edge-case with bad local data
* Include type_identifier in JSON payload
* Update follows API to allow org_ids as input
* Update onboarding front-end to distinguish users/orgs
* Fix: i18n issues
* Fix: type_identifier in json output
* Fix: distinct is weird
* Fix: JS linter
* Continue tweaking front-end
* Audit import order
* Cleanup @todo note
* Try renaming controller action
* Move Article average calculation to postgres and fix math
* Refactor decorated type_identifier
* Refactor SuggestProminent, return more orgs, fix spec math
* Use FeatureFlag for organization suggestions
* This might fix the jest
* Onboarding suggested tags should include supported
* Migrate tags to add suggested boolean column
* Use new suggested attribute
* Sync Settings.suggested_tags with new form setting
* Better sort order
* Test logic update
* Add a little comment
* feat: Add a pack file that pulls in the MultiSelect Component
* feat: move the tags to its own component
* save tags
* refactor: create a getCSRFToken function in the packs files so that it can be used in the admin
* feat: import the new module in request.js
* feat: remove unnecessary id
* feat: first pass of csrf token test
* chore: update the test
* fix: csrf token
* feat: hide the enw functionality behinda feature flag
* fix: loading form twice
* refactor: import for csrftoken
* chore: update the description of the function
* feat: use acts_on_taggable to craete a relationship between display_ad and tag
* feat: add a tag field to the display_ad form
* feat: add the selected tags from the multiselect autocomplete component to the input text field that references the tag_list
* feat: add the tag_list to the controller so that we can save it to the db with the display ad parameters
* feat: pull out the tag validation from the article and the display_ads into a concern
* feat: write soem tests for validating the tag on the display_ads model
* feat: add the tag_list as a hidden field
* feat: set the selected tags on edit
* feat: remove initializeSponsorshipVisibility and related code
* feat: remove sponsorships from sidebar
* feat: remove sponsorships from the admin - route, controller, view, spec
* feat: remove the admin menu item
* feat: remove the i8n for admin sponsors controller
* feat: sponsorship decorator was not being used anywhere
* feat: sponsorship slack messenger was not being used anywhere
* feat: remove the sponsorship_headline that gets configures on the admin
* feat: remove the /sponsors page
* feat: remove renedring of single_sponsor partial and associated partials
* feat: remove the navigation link rake task for sponsors
* feat: remove sponsorship from tags
* feat: remove i8n constants used
* remove sponsor references in text to the privacy page
* feat: remove the sponsorship detail from the organization page
* feat: remove the sponsors css that was used for app/views/pages/sponsors.html.erb
* feat: remove the sponsorship i8n that was used in the slack messengers
* feat: swap out the decorators to use Article as an example
* feat: remove spec to show sponsors on home page
* feat: update the specs to use Article Decorator instead of the Sponsorship Decorator
* fix: use direct and not all
* fix: remove tests for tag sponsorship
* fix: remove organization sponsorship test
* feat: remove more i8n
* remove unused css from app/views/organizations/_sidebar_additional.html.erb
* feat: remove sponsorship relationship from tag and organization
* remove seed
* feat: remove all Sponsor related specs
* feat: remove model, associated i8n and specs
* feat: remove sponsor from navbar
* feat: add a guard clause if we're unable to constantinize the purchase_type
* feat; remove sponsorship related tests
* feat: remove sponsorship relationship
* feat: remov especs related to sponsorship
* feat: remove tag sponsorship validation
* feat: remove the sponsorship factory
In looking at forem/forem#16787, it felt like we would benefit from a
similar approach as we adopted for nullifying blank strings.
This refactor helps pave the way for a "normalize_text_for" method (or
some such thing; naming things is hard).
This change also involved introducing a `describe` block into the
associated spec.
* Adding indirection for Tag#name to ease future work
In pairing with Arit, we were able to capture a few small wins in
regards to favoring an accessible tag name. In other words, where ever
you see a tag's name, we want to render it as camel-case. This will
greatly help those using a screen reader and frankly visually scanning
the tag (idontknowaboutyoubutthisiskindahardtoread).
However, as we explored the
code base, we found several places that will require further
consideration. These revolve around the API and caching of tags for
articles and listings as well as the JSON documents we use for
populating drop-down lists.
So, instead of unleashing a large pull request, we're opting to provide
a small non-breaking refactor that demonstrates where we're going and
keeps production working as expected while allowing future
development/testing to benefit from these captured gains.
Our next steps are to revisit the related issue and do a proper task
breakdown, becuase there are too many considerations to call this a
"small win".
I also encourage reviewers to read the comments. This is an emerging
mental pattern that I believe helps us conceptually move our codebase
forward while also guarding against the mega-PR with oodles of commits
and file changes that span numerous contexts.
Related to forem/forem-internal-eng#269
* Expanding specs to address feedback
* Refactoring cached_followed_tags
This refactor consolidates two queries and a loop into a single query.
One thing that is unclear is if we're ever really clearing the cache of
the user by id?
What follows is un-related to the refactor but appears to be related to #14937.
> First, I see we set a cache here:
>
> ebdaaaf15b/app/decorators/user_decorator.rb (L23-L34)
>
> But when I look at what busts the user cache, it doesn’t look like we’re busting it for the above cached location:
>
> ebdaaaf15b/app/services/edge_cache/bust_user.rb (L3-L22)
>
> What if anything am I missing in regards to cache busting?
* Adding some deprecated methods
* Updating documentation
* Bump for travis
* Bump for travis
* Updating comments to fix confusion
NOTE: I might be introducing more confusion, but at least
I'm addressing the prior confusion.
* Bump for travis
* Updating so we don't cache ActiveRecord objects
* Selecting only applicable attributes for caching
* Apply suggestions from code review
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Adjusting code based on feedback
* Bump for travis
* Bump for travis
* Update app/decorators/user_decorator.rb
Co-authored-by: Michael Kohl <citizen428@forem.com>
Co-authored-by: Michael Kohl <citizen428@forem.com>
On the DEV.to database, I ran the following SQL:
```sql
SELECT name, short_summary FROM tags WHERE short_summary LIKE '%<%';
```
There were three tags with a `<` in them:
- changelog
- cnc2018
- javascript
This PR will tidy up our short summary as part of saving a tag. It will
also tidy up the existing tags.
Prior to this commit, we were somewhat naively rendering Hash style data
attributes in our ERB templates. By rendering each hash attribute
separately, we were rendering characters that could break the
javascript (e.g. double hack or backslash `"` or `\`).
By moving to this view_object rendering, we leverage Rails's `to_json`
behavior to ensure properly escaped values. As part of this exercise, I
generalized the method to allow for other places to benefit from this
behavior.
This generalization also helps ensure that we have a more conformant
rendering (e.g. we should always have an :id, :className, and :name
value in our data-info hash).
_Note: I've updated the user's names for Cypress tests as they are more
likely to catch the particular issue than anything else. I assume that
I'm going to break some cypress tests and will need some help fixing
them._
Closes#15916, #14704
Supersedes #15983
How to test locally:
Assuming you have seeded database (e.g. `rails db:seed`), checkout the
"main" branch. Then in `rails console` find a user that's written articles:
```ruby
user = Article.last.user
user.update(name: "\\: #{user.name}")
user.articles.each(&:save)
```
Now, again on the "main" branch, start your application (e.g.,
`bin/startup`).
Then get a logged in and a logged out browser session going. Open your
web inspector and open console. Then go to the local instances homepage
(e.g., http://localhost:3000) and look for JS errors.
On the main branch, you should see an exception around
`JSON.parse(button.data.info)` (assuming that the `user`'s article is
rendered on the homepage).
Then go to the user's page (e.g. https://localhost:3000/:user-slug) and
look for JS parse errors.
On this PR's branch (e.g.,
`jeremyf/take-two-at-resolving-gh-15916`)
you shouldn't see those console errors.
More importantly, the Follow buttons should work.
* Tidying up and documenting Tag model
Prior to this commit we had a custom `where(alias_for: [nil, ""])`
call. That call highlighted that we lacked a term for a Tag that was
not an alias. As part of this commit, I named that a "concrete" tag.
Further, I added scopes to assist in helping "name" those concepts.
This commit also adds a data migration and utilization of
StringAttributeCleaner to hopefully get away from `alias_for == ""`
situations.
As of writing this commit <2022-01-04 Tue 17:22 UTC>, in DEV.to we had 5
tags with `alias_for == ""`:
- actionshackathon21
- regex
- atlashackathon
- hotwire
- foremfest
In https://dev.to/admin/blazer I ran the following:
```sql
SELECT name FROM tags WHERE alias_for = ''
```
* Renaming concrete to direct
* test: whitespace unicode characters cannot be used as titles or tags
* feat: add localized error message
* refactor: use localized error message
* fix: whitespace unicode characters cannot be used as titles or tags
* chore: fix locale after merge
* refactor: fix indentation
* Fix spelling of prohibited in method names
Additionally, rubocop removed a redundant user_id validation since
Article belongs to user.
* Fix spelling
Missed one method call on the same line (fixed the first of two
instances needing changes)
* add failing test cases
The reorganization to remove let was due to a limit on nesting rspec
contexts (it inside context inside describe inside describe when
trying to use different titles in let blocks in contexts)
The first test was clarified (the "U+202D" string that looks like the
code for a unicode point is valid, but the character \u202d is
expected to be invalid
The remaining two tests are based on the feedback I'd given in the PR,
failing because we don't assert title is present after removing
unicode whitespace, and we don't actually set the title (gsub is
non-destructive, returning a value).
* Set title to sanitized title
contains_prohibitied_unicode_characters? was using == (which is not a
good match for strings to regexes), use =~ instead
Change invalid example characters in titles from \u202d (bidi override) to
\u200a (hair space)
Assert that the empty title is blank and can't be blank (even though
it contains no prohibitd characters after replacement)
* change the definition of prohibited characters
From the description, it looks like "unicode space characters" was the
desired rejection set. It was unclear what the existing regex was
matching on (I couldn't get the expected examples to match correctly).
Given the intent, I select "unicode space property, except the ascii
space character", which may _also_ be incorrect but passed the tests.
* Remove now-invalid expectation for presence of user id
Since this was redundant (belongs_to user), we no longer will validate
presence of user id, and we should not test for it.
* Only reject bidirectional text controls
The original issue pointed to the BIDI controls as problematic.
While they called out other "whitespace" characters, as we accept a
wider range of input languages, the need to handle non-printing
punctuation (for example a group separating space for some asian
languages).
It's possible a wider list of characters should be added - if that's
the case I suggest this regex and the sanitization be moved to a
standalone class, and each of the recommendations in
https://www.w3.org/TR/unicode-xml/#Charlist be checked for
applicability.
* Check for blank title after sanitizing disallowed characters
Since validate_title modifies the title, and doesn't set error
messages on the model itself, check for non-empty title _after_
potentially removing any disallowed characters.
* Remove invalid characters before validation
Remove the validate_title and contains_prohibited_characters methods,
and center all logic on the remove prohibitied unicode characters
method.
Remove unneeded and unused arguments (input string is always title,
replacement is always removal/empty string).
* handle case when validating null title
If the title's nil, we don't want to call match? since it will fail.
Title will sometimes be nil when validating.
Co-authored-by: Ben Halpern <bendhalpern@gmail.com>
Co-authored-by: Dan Uber <dan@forem.com>
* Favoring single SQL and computation over enum
Prior to this commit, we ran a SQL statement to generate a list of
articles then applied some minor randomization.
With this commit, I'm removing the randomization in favor of expected
values, and collapsing the result set into a single inline SQL and some
post query simple arithmatic.
Let's check my math. For each article we sum:
* `article.score`
* `article.comments_count` * 14
* a random number between 0 and 5 (`rand(6)`) with expected value of 2.5
* the tag's (taggings_count + 1) / 2
The new SQL query sums the `article.score` and the
`article.comments_count` * 14. I then reduce the remainder of the
equation. From "for each article sum `(rand(5) + (taggings_count + 1) /
2)`" we have the following:
`article_count * (2.5 + (taggings_count + 1) / 2)`
Which is equivalent to:
`article_count * ((taggings_count + 1 + (2.5 * 2)) / 2)`
Which is equivalent to:
`article_count * ((taggings_count + 6) / 2)`
This change reduces computation time by favoring expected values.
* Fixing broken behavior and adding comments
* Remove Connect
* Remove more Connect specs
* Remove a lot more Connect code
* 🚮
* It all has to go
* Explicitly add httpclient
* Update application layout
* Remove messages association from User
* Start fixing specs
* reintroduce util function and refactor references
* Remove Connect Cypress test
* Fix more specs
* Remove Connect from listings
* Ignore contact_via_connect column on listings
* Remove contact_via_connect usages
* Ignore mod_chat_channel_id on tags
* Drop Connect tables
* Remove email_connect_messages from user notification settings
* Re-add httpclient 2.8.3
This was mistakenly removed as a merge conflict
* Don't need to exclude removed chat channel file
* Remove unneeded style for chat channels
* Remove unneeded channel list prop type
* Remove chat channels index/connect-link from getPageEntries
* Re-add comment from httpclient in Gemfile
* Remove connect references from mailers
Tag Moderators no longer have a chat channel
No longer will users be notified about new messages (there won't be
any)
No longer will users be notified about channel invites (you can't
invite anyone anymore)
* Don't configure Pusher and remove PUSHER_* from .env_sample
since it's removed from gemfile, the Pusher constant will not resolve, if this is
configured in the environment variables we'll fail to boot.
Co-authored-by: Suzanne Aitchison <suzanne@forem.com>
Co-authored-by: Dan Uber <dan@forem.com>
* show top tags when search is empty
* add cypress tests
* nudge travis
* tweak to e2e test
* show a 'top tags' heading
* only suggest tags that are supported
* Add supported scope to Tag
Co-authored-by: Michael Kohl <citizen428@forem.com>
* Relax tag validation restriction to permit alphanumeric strings
Non-word symbols like musical notes are still blocked, but most
languages should work.
* Handle redirect after tag edit
The URI.parse(tag.name) was not updated in #13720
Move reused logic into tag_path method in URL module.
* FactoryBot generated tags are non-ascii
This should help expose any cases where we're mistakenly counting on
ascii-only content in tags by causing all generated tags without a
specified name to be of the form `tag你好#{n}` instead of `tag#{n}`.
* Revert "FactoryBot generated tags are non-ascii"
This reverts commit ed1035cd1e91619b2c1599e521b6ddf962c97f80.
It was useful to see how many places the path knowledge is in the
tests (a lot of requests have GET /t/tag.name and need to have GET
URL.tag_path instead).
https://app.travis-ci.com/github/forem/forem/builds/237542600 has the
results.
Reverting because those test-side changes seem like a secondary
concern to the change in this PR (I'll file a follow on issue)
* add: article cache on update tag
* minor styling fix
* add: published scope to fetch articles
* update: tag_spec:125 to not fetch articles
* modify: tag.rb:bust_cache to delete server cache
* add: tests to verify cache delete
* Step one in populating the reading list with PG
This first attempt tries to recycle the `Search::ArticleSerializer` which is only
used in input in ES, but we're using it in output in PG.
For this reason it's currently 15.55x times slower
* Serialize only what is requested by the frontend
`Search::ArticleSerializer` which is only used in ES in the indexing step aims
to add as much info as possible for broader purposes, in this case
(with serialization in output) we should aim to save only what's requested from
the frontend.
* Optimize selection of articles columns
* Select only needed columns for users
* Compute total of reading list items
* Attach the basic filtering based on PG on the search controller
* Restructure in methods
* Add tags support
* Use LIKE on articles.cached_tag_list
* Fix tags as nil
* Fix default pagination
* Add optional FTS for reading list
* Reworded the tags comment explaining why
* Add index to reactions.status
* Fix total counter in Preact readingList component
* Fix total count in reading list backend search
* Add GIN index to articles.cached_tag_list
* Add service tests
* Add search request specs
* Added missing early return
* Update spec/requests/search_spec.rb
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* Extract MAX_PER_PAGE constant and add comments
Co-authored-by: Julianna Tetreault <32834804+juliannatetreault@users.noreply.github.com>
* Add Search::Postgres::Tag to retrieve tags for autocompletion
* Add Search::Postgres::Tag in SearchController behind a feature flag
* Update spec/services/search/postgres/tag_spec.rb
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Add index to tags.supported
* Fix hotness_score ordering
Co-authored-by: Michael Kohl <citizen428@dev.to>
* Don't add to mailchimp for N/A circumstance
* Use class << self over self. methods
* Refactor and make add/remove tag mod idempotent
* Add missing param wiki_body_markdown
* Add errors_as_sentence b/c Tag not inheriting from AppRecord
* Refactor and Crayonsify tag page
* Use proper for attr to make label checkable
* Crayonsify moderator section
* Make placeholder white
* Remove remote: false for sync form submit
* Fix more merge conflicts
* Minor styling adjustments
* Fix color field value
* Don't add to Mailchimp community mod list if not enabled
* Add tests for new tag mod routes
* Add missing line oops
* Use self.method over class << self for new Rubyists
* Use more efficient query for getting tag mods
* Use parentheses to follow convention
* Use cleaner way of routing and controller
* Update tests to match new configuration
* Add missing FKs to BackupData BadgeAchievements BanishedUsers BufferUpdates
* Add relations and specs
* Fix specs
* Trigger Travis CI
* Remove traces of backup_data has it has been removed on master
* Fix trailing whitespace
* Enable new Rails/* cops and use autocorrect on them
* Fixed Rails/PluckInWhere leftovers
* Fix Rails/DefaultScope
* Enable and fix Rails/PluckId
* Fix manual mistake with forcing autocorrection on Rails/PluckId
* Apply PR feedback to remove Rails/PluckId inline disables
* Apply PR feedback to get rid of Rails/PluckInWhere inline
* Stop allowing accented characters in tags
This commit will add strict frontend and backend validation for tags
disallowing accented characters.
* Update tags validation error message
* Add more character sets to tag validation spec
* Add comment referencing Fastly gem
* Create purgable concern and include it
* Typo: deprecate --> deprecated
* Include Purgable on the Tag model
* Rename Purgable to Purgeable
* Reorganize fastly and service methods in Purgeable
* Add and fix specs
* Add development env guards
