Architecture

Phones are cameras. The Mac is the venue brain.

RefereAI keeps the live loop local: phones stream frames to the Mac, YOLO26 MLX runs on Apple Silicon, and the Mac serves overlays, review artifacts, and private viewing pages.

Capture
Parent phoneSafari camera PWA
Corner phoneofficial angle
Guest phoneoptional feed
LAN / Tailscale WebSocket frames →
Mac server
RefereAI live server aiohttp · /app · /debug/videos/hil · private viewer
frame ingest recording sessions
YOLO26 MLXlocal Apple Silicon inference
Trackerplayers, ball, racket, court hints
Observerssport, teams, score probes, commentary
Artifactsoverlays, summaries, frame logs
→ Served by Mac viewer pages
Outputs
Live overlayviewer toggles
HIL reviewcorrect / bad overlay / demo-worthy
Private linkfamily room, token-gated
Data flow

One loop, three surfaces.

Live capture

The phone opens the Mac-hosted app and sends compressed camera frames over WebSocket.

Local inference

The Mac runs YOLO26 MLX, tracking, sport classification, score probes, and overlay rendering locally.

Review workbench

Recorded clips produce overlay videos, JSON summaries, frame logs, and HIL review cases.

Private viewing

The Mac can serve a private viewing room with overlay and commentary controls.

Why this shape

Local first changes the product.

No cloud inference pathThe model runs on the Mac, which keeps raw capture close to the venue.
Phones stay simpleThey act as cameras and viewers. The heavy work stays on Apple Silicon.
Review is part of the loopBad classifications and bad overlays become feedback, not hidden failure.
Public site stays cleanVercel hosts only public demo assets. Local sessions and debug pages stay on the Mac.
Tech stack

The pieces in the loop.

YOLO26n → MLX

Default engine is models/yolo26n.npz, converted for the yolo26mlx runtime.

MLX 0.30.x

pyproject.toml pins mlx>=0.30.3,<0.31 for Apple-Silicon-native inference.

Python + aiohttp

The Mac server serves the PWA, WebSocket frame ingest, debug workbench, HIL review, and private viewer pages.

OpenCV renderer

Offline and live overlays are rendered locally with boxes, HUD panels, frame logs, and summary JSON.

Tracker + observers

Local tracking, sport inference, team/court hints, score probes, commentary, and replay metadata.

Engine catalog

Catalog includes yolo26n by default and placeholders for yolo26s/m/l/x when local weights are installed.

Optional vision add-ons

Feature flags reserve paths for SAM-style masks, OCR probes, and local narration models.

Static public site

Vercel serves only public pages and demo assets; local capture/review stays on the Mac.

Try it

Run the Mac server, then scan from a phone.

git clone https://github.com/jravinder/refereai-mlx
cd refereai-mlx
bash scripts/bootstrap.sh
.venv/bin/refereai serve --port 8765
Install guide Watch demos GitHub
Source reads

Implementation references.

READMEREADME.md
Product overview, architecture, privacy boundary, and local run path.
InterfacesINTERFACES.md
Contracts for tracking, classifier verdicts, sport plugins, and WebSocket payloads.
SecuritySECURITY.md
What stays private, what is safe to publish, and how to handle private media.