Commit graph

81 commits

Author SHA1 Message Date
Molly Struve
e752ff98af
Index ChatChannelMemberships to Elasticsearch (#6162) [deploy] 2020-02-19 14:29:16 -05:00
Alex
8d7b95ea15
Index ClassifiedListings to Elasticsearch (#6132) [deploy]
* Add ClassifiedListingEsIndexWorker & spec

* Update ClassifiedListing model

- Add es index methods (inline and async)
- Index ClassifiedListing to es after_commit
- Add serialized_search_hash
- Update model specs for new methods
- Add serializers referenced in new model method

* Update specs

* Create DataUpdateScript to index ClassifiedListings

* Fix formatting

* Fix codeclimate :/

* Use match_array instead of keys.sort

* Change find_by! to find

* Update spec

* Update mappings for ClassifiedListing

* Update specs

* Implement refactor with Searchable module
2020-02-19 09:55:37 -05:00
rhymes
f3c2adfa45
Routine rubocop fixes (#6138) 2020-02-18 11:21:42 -05:00
Ridhwana
6356089ee0
Re-index existing articles with DataUpdateScript (#6109) [deploy] 2020-02-17 10:14:50 -05:00
Molly Struve
ef9611d745
Create DataUpdateScript to Index all Tags (#6078) [deploy] 2020-02-14 14:17:52 -05:00
Molly Struve
65eba740ab
Document DataUpdateScripts, Add to Deploy Process, Add Development Check (#6068) [deploy] 2020-02-14 11:45:27 -05:00
Molly Struve
0bfbd9bb1f
Set Up Framework For Running Data Update Scripts (#6025) [deploy] 2020-02-13 10:48:23 -05:00
Molly Struve
acfdd11b50
update cluster settings to prevent auto index creation, update destroy to remove ALL indexes (#6042) 2020-02-12 15:21:38 -05:00
Juanito Fatas
33b8014677
Fix indentation in search.rake (#5971)
lib/tasks/search.rake:9:3: C: Layout/IndentationWidth: Use 2 (not 3) spaces for indentation. (https://rubystyle.guide#spaces-indentation)
     Search::Cluster.update_mappings
  ^^^
2020-02-10 09:09:23 -05:00
Molly Struve
4691ae1e34
Log in Datadog When Sidekiq Worker Fails for Final Time (#5942) [deploy] 2020-02-10 08:56:41 -05:00
Maykon Menezes
01dc81afa9
Move Podcasts::GetEpisodesJob to Sidekiq (#5443) [deploy]
* Refactored Podcast::GetEpisodesJob to sidekiq

* fixing specs

* removing env keys

* Change arguments to Hash

* Update specs and add spec for queue

* Update references for Hash argument

* Add comment to explain parameters of #perform

Co-authored-by: Alex <alexandersmith223@gmail.com>
2020-02-06 09:42:55 -08:00
Molly Struve
00fefe5865
Add Code To Setup the Elasticsearch Tag Index (#5919) [deploy] 2020-02-06 11:46:11 -05:00
rhymes
3260270103
Have counters:update_users use a transaction per user and customizable batch size (#5912) [deploy] 2020-02-05 12:26:03 -05:00
rhymes
a1835ae5b9
User counters using PostgreSQL JSON (#5373) [deploy]
Co-authored-by: Ben Halpern <bendhalpern@gmail.com>
2020-02-03 10:35:08 -05:00
Ben Halpern
a4b86e3ff1
Add initial usage metrics for datatog (#5544) [deploy]
* Add initial usage metrics for datatog

* Make measurable name more clear

* Change name of daily usage worker

* Fix test names
2020-01-16 12:59:17 -05:00
Molly Struve
1a3d59c97f
Send Sidekiq data to Honeycomb (#5517) [deploy] 2020-01-16 12:30:43 -05:00
Molly Struve
5962d52a61
record background queue stats in a worker (#5533) [deploy] 2020-01-15 14:44:57 -05:00
Molly Struve
2e5a36e538
Log DB Table sizes in a Sidekiq Worker (#5518) 2020-01-15 12:33:14 -05:00
rhymes
9b2886979a ActiveRecord count optimizations (#5478) [deploy]
* Do not load messages in memory, just to count them
* Add Model.estimated_count
* Use Model.estimated_count in task record_db_table_counts
* Use size and estimated_count
* Use table_name so that .estimated_count works with all models
* Add comment to explain the .load.size + .each pattern
2020-01-14 12:03:38 -05:00
Michael Kohl
dba7e6ca60 Move Twitch webhook registration from ActiveJob to Sidekiq (#5414) 2020-01-14 11:55:35 -05:00
rhymes
c267526ed2 Remove temporary site config script and unused ENV var (#5455) [deploy] 2020-01-14 09:49:31 -06:00
rhymes
102e1a6b9c Routine rubocop fixes (#5386) 2020-01-08 09:05:12 -05:00
Molly Struve
6b23e327aa
Track Queue sizes and latency for Sidekiq (#5352) [deploy] 2020-01-07 11:57:13 -05:00
Molly Struve
bc8c177966
Record db table sizes in datadog where we can use them to generate relevant alerts (#5377) 2020-01-06 15:52:47 -05:00
Molly Struve
660852b711 record counts for important db tables to help track growth (#5253) [deploy] 2019-12-27 11:29:47 -05:00
Molly Struve
f86f93bc8a
Create rake Task to Destroy Old Notifications (#5180) [deploy] 2019-12-26 13:17:34 -06:00
Molly Struve
fb17d0e079
fix title_with_timeframe to handle invalid timeframe and update rake task to not use return (#5241) [deploy] 2019-12-26 08:43:46 -06:00
Ben Halpern
cd9805bc22 Fix tag badge task logic (#5189) [deploy] 2019-12-20 14:39:28 -05:00
Ben Halpern
c6d94d4049
Add rake task to reward tag badges weekly (#5183) [deploy] 2019-12-19 16:34:22 -05:00
Molly Struve
ce1bc8d2a4
Revert "Setup dual Redis store for rolling over fragment cacheing to Redis (#4991)" (#5014) [deploy]
This reverts commit e871a01c01.
2019-12-05 14:03:41 -06:00
Molly Struve
e871a01c01
Setup dual Redis store for rolling over fragment cacheing to Redis (#4991) [deploy] 2019-12-05 13:19:01 -06:00
rhymes
c9a07faf6a Add script to display and migrate environment variables to SiteConfig (#4865) [deploy] 2019-11-22 10:18:22 -05:00
Michael Kohl
b57ced5466 CacheBuster refactoring (#4766)
* Turn CacheBuster into a module

This class used no internal state, so repeatedly creating short-lived objects seems wasteful.

* Consistently use string interpolation and parenthesis

* Destructure arrays into meaningful names, formatting

* Fix request spec for internal classified listings controller

Interestingly this works when asserting directly on the module, but not on a double.
Asserting directly in the module seems sufficient for this test so the indirection
was removed.

* Turn CacheBuster into a module

This class used no internal state, so repeatedly creating short-lived objects seems wasteful.

* Fix specs after rebasing
2019-11-13 10:51:23 -05:00
Molly Struve
00e3ee811b Use ActiveSupport::Cache::RedisCacheStore As RedisRailsCache (#4676) [deploy] 2019-11-01 12:19:12 -04:00
Molly Struve
cbc85100e3 Create RedisRailsCache Object to Mimic Rails.cache And Apply (#4667)
* Create RedisRailsCache object to mimic Rails.cache

* Do not use prefix _ for a variable that is used

* dont shadow outer local variable
2019-10-31 12:12:21 -04:00
rhymes
c8abbe9d60 Pro: memberships (#3461)
* Add ProMembership model

* Implement ProMembershipsController#create

* Implement basic ProMembershipsController#show

* Add ProMembership to ledger

* Populate user history after pro subscription is created

* Add fields for expiration notifications

* Add ProMemberships::ExpirationNotifier to notify users of expiring memberships

* Add tasks for recurring jobs to notify users of expiration

* Add auto_recharge column to ProMembership

* Add ProMemberships::Biller (incomplete)

* Fix specs

* Add ProMembership to Administrate

* Fix spec

* Add has_enough_credits? to User and Organization

* Add Payments::Customer class

* Finish ProMembership::Biller functionality

* Fix ProMemberships::Creator check for credits

* Disable destroy actions for ProMembershipsController

* Correctly authenticate ProMembershipsController actions

* Make sure only pro user's history can be indexed

* Add ProMembershipsController#update action for auto recharge

* Use regular AR to save new credits and add touch to the purchaser

* Clarify Pro membership create policy

* Display information about an existing pro membership

* Add UI to show page

* Add system test for Pro membership creation

* Implement edit membership

* Make sure users with pro memberships can access history and dashboard pro

* Fix padding issue

* Show a different text for a user that has credits but not enough for Pro

* Move Pro Membership functionality inside settings

* Update Pro Membership link in email notifications

* Bust all relevant caches

* Add the Pro checkmark around the website

* Use Users::ResaveArticlesJob instead of delay

* Add/remove user from pro-members chat channel

* Use the appropriate Pro checkmark

* Remove unfinished pro elements

* Remove checkmark JS
2019-09-24 10:38:54 -07:00
Jess Lee
65fccf00a9 add rake task for fixing credit counter cache (#3962) 2019-09-06 13:08:39 -04:00
Ben Halpern
66f72c091a Fix rake task with repeated name (#3845) [ci skip] 2019-08-26 17:33:00 -04:00
Mario See
47c2885810 Listings set custom expiry date (#3770)
* add expire_on column to classified_listings

* add expire_on to valid listing params

* add expire_on to model

* add expire date to new listing form

* add field in edit form

* add expiry dates to dashboard

* update expiry date in listing toolkit

* WIP unpublish job and test

test still has to run properly

* add job to class name

* change job to rake task

* revert snap change

* delete migration file

will add new one and revert schema changes to rename column

* reset schema

* add new migration to schema 'expires_at'

* change expire_on to expires_at
2019-08-23 17:33:50 -04:00
Anna Buianova
fabaeb5461 Introduced published field to podcasts (#3593)
* Published field for podcasts

* Add podcasts migration

* More specs for podcasts availability

* Show published in podcasts internal

* Update schema.rb with db:migrate
2019-08-01 09:29:13 -04:00
Anna Buianova
95a9e13922 Remove logic relying on old podcast status (#3505) 2019-07-19 09:30:51 -04:00
Anna Buianova
a8227b3048 A temp rake task to update podcast statuses (#3496) 2019-07-17 11:10:02 -04:00
Anna Buianova
94ab4e23fe Update podcast episode statuses #2952 (#3450) 2019-07-16 13:36:25 -04:00
rhymes
611f4b8675 Sponsorship: remove old columns (step 3) (#3446)
* Remove old sponsorship columns

* Rename redundant param names
2019-07-11 15:35:24 -04:00
rhymes
ad79116366 Sponsorship: add model (step 1) (#3438)
* Add Sponsorship model

* Add Sponsorship to admin

* Add correct relationships between models

* Add temporary script to migrate sponsorship data
2019-07-10 15:58:37 -04:00
rhymes
97b288330e Remove temp page's views referrers rake task (#3426) [ci skip] 2019-07-09 15:05:08 -04:00
rhymes
7da7a16d8d Pro: add referrers to dashboard and single article stats (#3295)
* Reorganize PageViewsController

* Add domain and path to PageView model

* Add before_create callback to populate domain and path

* Add list of referrers to AnalyticsService

* Add referrers to the UI

* Remove useless referrers card and tweak table line height

* Add referrer stats to article stats page

* Add not null and empty default to domain and path

* Refactor JS analytics client

* create_list is a step back here

* Revert "Add not null and empty default to domain and path"

This reverts commit bc02440076047a887c65d300bccd4661ecc8ffd0.

* Add index on domain concurrently

* Make the script more robust
2019-06-25 13:58:09 -04:00
Anna Buianova
9751e84549 Moved Podcast#pull_all_episodes to a separate ActiveJob #2952 #2497 (#3057)
* Use one PodcastFeed instance while fetching podcasts

* Moved Podcast#pull_all_episodes to a separate ActiveJob
2019-06-06 09:49:10 -04:00
Corey Alexander
ec388804df Embed Twitch Live Streaming (#2591)
* Get a job created that can create a webhook subscription for a twitch user login

* Remove ngork url

* Refactor add store the access token in the cache

* Get a controller stood up to recieve the webhooks. Now they just need to be processed

* Get User columns added and got webhook controller bones working

* Update the webhook job to use the User

* Add a way for the User to input their Twitch User Name. Plus a linter fix

* Delay webhook registration when profile is updated

* Don't add _ in username

* Use String columns

* Quick fix and add some more requests specs

* Specs for the webhook job

* Get a show page Twitch Live Streams. Just a straight embed of the Twitch Everything Embed UI. Works surprisingly well responsively, and works on all screen sizes

* Fix Gemfile.lock from merge issues

* Add support for expired tokens and add spec

* Add secrets to webhook registration and clean up spec to remove token logic

* Verify webhook secret and spec it

* Add rake task to enqueue webhook registration for all Users. This can be used from Heroku Scheduler

* Update the lease seconds to be for 5 days

* Actually lets do 7 so we can refresh twice a week and try to make sure that we can always miss one

* Hijack the existing Twitch logo instead of making a duplicate one

* Remove comment and replace with log line

* Remove some white space

* Log to Airbrake when webhook errors occur

* Move to passing in an id instead of User object

* Extract logic from Job to Service object

* Capitilize in the view

* Move out of models and into services

* Remove letover stub

* Remove one usage of Faraday

* Use HTTParty for all the HTTP here
2019-04-29 18:11:53 -04:00
Ben Halpern
bfedee0561
Fix some listing/credit bugs (#2584)
* Fix some listing/credit bugs

* Tack on codepen team link fix

* Fix test

* Actually fix test
2019-04-26 19:01:35 -04:00