3aa08e4a4f
Crawler now records the deduped cross-domain link graph (edges table). recompute.js runs real iterative PageRank over that graph, blends it with accumulated in-degree + the Majestic seed prior, and patches every doc's `score` (Meili's authority tie-breaker) via partial updates — preserving titles/bodies/embeddings and triggering no re-embedding. systemd timer runs it every 3h so ranking sharpens as the 1.5M-page crawl densifies the graph. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
21 lines
684 B
Desktop File
21 lines
684 B
Desktop File
[Unit]
|
|
Description=RADICAL_SEARCH authority recompute (PageRank -> Meili scores)
|
|
After=network.target meilisearch.service
|
|
Requires=meilisearch.service
|
|
|
|
[Service]
|
|
Type=oneshot
|
|
User=root
|
|
WorkingDirectory=/opt/search-engine/crawler
|
|
ExecStart=/usr/bin/node --experimental-sqlite src/recompute.js
|
|
# Keep the recompute from contending with the live crawl for CPU.
|
|
CPUQuota=120%
|
|
MemoryMax=1500M
|
|
|
|
Environment=FRONTIER_DB=/opt/search-engine/crawler/frontier.db
|
|
Environment=MEILI_URL=http://127.0.0.1:7700
|
|
# MEILI_KEY is injected via an override drop-in on the box (keeps the key out of git).
|
|
Environment=DAMPING=0.85
|
|
Environment=ITERS=40
|
|
Environment=PR_WEIGHT=0.6
|
|
Environment=INDEG_WEIGHT=0.4
|