maverick/docbrown
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
8085 commits 1 branch 0 tags 1.6 GiB
Commit graph

10 commits

Author SHA1 Message Date
rhymes
e7f9735354
Remove RssReader (#12169) 
* Add script to remove feature flag

* Remove :feeds_import feature flag and RssReader and related classes

* Change Feeds::ImportArticlesWorker signature to support Sidekiq Cron serialization

* Replace RssReader::Assembler with Feeds::AssembleArticleMarkdown

* Removing Assembler
 2021-01-08 11:45:15 -05:00
rhymes
4aaeb3e955
Feeds::Import: only import feeds that haven't been recently updated (#12126) 
* Add earlier_than to Feeds::Import

* Ignore feeds that have been imported in the last 4 hours

* Make earlier_than mandatory in Feeds::ImportArticlesWorker
 2021-01-06 16:58:42 +01:00
rhymes
00247bed99
Log error message inline when Feeds::Import fails (#12012) 2020-12-22 20:02:07 +01:00
Michael Kohl
1ce8a762b7
Move HtmlCleaner from app/labor into service (#11609) 
* Move HtmlCleaner from app/labor into service

* Move to Feeds:: namespace
 2020-11-27 08:21:57 +07:00
rhymes
1741e52e83
Add missing features to new feed importer (#11501) 
* Add explanatory comment to how RssReaderFetchUserWorker works

* Add Feeds::ImportWorker worker

* Enable Feeds::Import for single user feed fetching behind a feature flag

* Add Feeds::ValidateUrl service

* Add feature flag to user's validate feed URL feature

* Remove todo notices

* Add feature flag to fetch_all_rss task

* Add feature flag to RssReaderWorker

* Bring back short-circuit for Articles::RssReaderWorker
 2020-11-24 18:11:13 +01:00
rhymes
cbc461a850
Refactor RssReader and Feeds::Import common parts: take 1 (#11412) 
* Extract Feeds::CheckMediumReplyItem service

* Extract Feeds::CheckItemPreviouslyImported

* Clarified comment on Feeds::CheckItemMediumReply (thanks @maestromac)

* Refactor a bit to be more Rubeyeyey
 2020-11-18 14:17:53 +01:00
rhymes
910f79e04e
Remove wrong datadog metric in Feeds::Import (#11405) 2020-11-13 16:11:02 -05:00
rhymes
1db388510e
Add custom instrumentation to Feeds::Import (#11297) 
* Add custom instrumentation to Feeds::Import

* Remove superfluous logging instructions
 2020-11-09 17:12:01 +01:00
rhymes
47624ddfcd
Add users param to Feeds::Import (#11234) 
* Add users param to Feeds::Import

* Fix init and specs

* Replace explicit nil check

Co-authored-by: Michael Kohl <me@citizen428.net>

Co-authored-by: Michael Kohl <me@citizen428.net>
 2020-11-03 07:02:17 -06:00
rhymes
51a4a35448
Add Feeds::Import service class (#10998) 
* Add the parallel gem

* Add prototype version of Feeds::Import with parallel URL fetching and parsing

* Tune Feeds::Import and add rake task for local tests

* Add rough measurer tool

* Add specs

* Apply suggestions by @citizen428

* Replace silence with silent

* No need for eager loading

* Add temporary Articles::DevFeedsImportWorker

* Remove temporary rake task

* Add basic error handling, copied from RssReader

* Fix error handling

* Remove temporary measuring rake task

* Remove logging added for development purposes

* Add info and error logging on error level
 2020-10-30 15:01:44 +01:00