paywall/src
King Omar 88038a7c0e Wayback fallback: when live site blocks all crawlers, read from a Wayback snapshot
When the multi-crawler fetch can't get article text (FT, WSJ, etc. return
403/JS-only shells to every crawler), fall back to the Internet Archive:
resolve the closest snapshot via the availability API, fetch the raw id_
version, and extract THAT into the Radical Reader with a 'via Wayback Machine,
archived DATE' source note. archive.today can't be used server-side (429 +
captcha on datacenter IPs) so it stays as the user-side button.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 15:54:47 +10:00
..
index.js Wayback fallback: when live site blocks all crawlers, read from a Wayback snapshot 2026-07-25 15:54:47 +10:00
methods.js Multi-crawler fetch: try Googlebot + social crawlers, keep richest copy 2026-07-25 15:21:36 +10:00
proxy.js Wayback fallback: when live site blocks all crawlers, read from a Wayback snapshot 2026-07-25 15:54:47 +10:00
views.js Wayback fallback: when live site blocks all crawlers, read from a Wayback snapshot 2026-07-25 15:54:47 +10:00