Commit graph

2 commits

Author SHA1 Message Date
King Omar
665cd264cc Multi-crawler fetch: try Googlebot + social crawlers, keep richest copy
Hard paywalls (AFR etc.) block Googlebot by IP but serve full articles to
facebookexternalhit/Twitterbot for link previews. fetchBest() now fetches
several crawler identities concurrently and keeps the copy with the most text,
so the Reader works on AFR (0 -> 2256 words). Also rank archives first for
known hard-paywall domains.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 15:21:36 +10:00
King Omar
c3103e6ecb PaywallFree: read paywalled articles free — Radical reader + proxy + method hub
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 15:02:18 +10:00