b0496a5ac1
The crawl stalled when two crawler processes briefly overlapped on the same WAL SQLite frontier (node:sqlite is synchronous, so a blocked write lock froze the event loop). Fixes: - run the crawler as a single-instance systemd service (search-crawler.service) with CPUQuota=150% + MemoryMax=1200M so it can't starve a shared VM - add timeouts to all Meilisearch calls + resilient flush (an untimed hang was the amplifier that wedged every worker) - crawl-diskguard.sh + cron: auto-stop the crawl and ntfy-alert if disk < 800MB Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
10 lines
477 B
Bash
10 lines
477 B
Bash
#!/bin/sh
|
|
# Autonomous safety: stop the crawler if disk gets low, so it can never fill
|
|
# the shared VM and break other services. Runs from cron every 5 min.
|
|
THRESHOLD_KB=800000 # 800 MB
|
|
FREE=$(df --output=avail / | tail -1 | tr -d ' ')
|
|
if [ "$FREE" -lt "$THRESHOLD_KB" ]; then
|
|
systemctl stop search-crawler
|
|
curl -s -d "RADICAL_SEARCH crawler STOPPED on adrian-bingo: disk free ${FREE}KB < ${THRESHOLD_KB}KB" \
|
|
ntfy.sh/radicalparty-vm-alerts-x7k2q9 >/dev/null 2>&1
|
|
fi
|