drones/pipeline

CritterScope drone ingest pipeline

Turns drone footage + telemetry into georeferenced detections and pushes them to a CritterScope instance.

video / RTMP frames   +   telemetry (DJI .SRT)
        │                        │
   extractFrames             parseDjiSrt
        └──────────┬─────────────┘
              detector (stub | ONNX YOLO)     → bbox per frame
                     │
               pixelToGround                   → lat/lon per detection
                     │
             POST /api/ingest                  → D1
                     │
            view at /?survey=<id>

Quick start (no drone needed)

Runs a synthetic lawnmower flight through the stub detector and prints what it would send — proves the whole chain:

cd pipeline
node src/run.js --synthetic --dry-run

Push it into a running local CritterScope (with local D1, see root README):

node src/run.js --synthetic --name "test-survey" --base-url http://localhost:8787
# → View it: http://localhost:8787/?survey=sv-test-survey-...

Real footage

node src/run.js \
  --source flight.MP4 --srt flight.SRT \
  --detector onnx --model yolo-thermal.onnx \
  --width 1920 --height 1080 \
  --drone "DJI Mavic 2 Enterprise Advanced" \
  --base-url https://critterscope.theradicalparty.com

Live RTMP (DJI Fly app livestream)

Point the DJI Fly app's RTMP stream at a local server, then:

node src/run.js --source rtmp://localhost/live/stream --srt live.SRT --fps 1

The three stages you can swap

• Telemetry (src/telemetry.js) — DJI SRT parser + synthetic flight. Add other drones by mapping their telemetry into {lat, lon, altAGL, heading, gimbalPitch, hfovDeg}.
• Detector (src/detector.js) — stub (works now) and onnx (wire your trained YOLO: JPEG decode → letterbox → run → NMS → map class ids to species). Everything downstream is model-agnostic.
• Georeferencer (src/georef.js) — pinhole camera + flat-ground projection. Verified: centre pixel at nadir → drone position; right-edge offset matches altitude·tan(hfov/2); near-horizon rays correctly return no ground hit. Next refinement: intersect the ray with a DEM instead of flat ground.

Requirements

• Node 18+ (uses global fetch)
• ffmpeg on PATH (only for --source)
• onnxruntime-node (optional, only for --detector onnx)

What's real vs. stubbed

• ✅ telemetry parse, frame extraction, georeferencing, survey assembly, ingest, map render
• 🔩 the ONNX detector's tensor plumbing is left for when you have a trained thermal wildlife model — the stub detector exercises the identical downstream path today.