search-engine/crawler
King Omar 3aa08e4a4f Add link-graph authority: record edges + PageRank recompute
Crawler now records the deduped cross-domain link graph (edges table).
recompute.js runs real iterative PageRank over that graph, blends it with
accumulated in-degree + the Majestic seed prior, and patches every doc's
`score` (Meili's authority tie-breaker) via partial updates — preserving
titles/bodies/embeddings and triggering no re-embedding. systemd timer runs
it every 3h so ranking sharpens as the 1.5M-page crawl densifies the graph.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 18:35:23 +10:00
..
src Add link-graph authority: record edges + PageRank recompute 2026-07-22 18:35:23 +10:00
package.json Overhaul crawler: depth crawl, link graph, robots, sitemaps, authority 2026-07-22 02:26:22 +10:00