WIUT Hackathon 2026 · Computer Vision track

Traffic events from one fixed camera, as time segments

NeuroLens watches an intersection CCTV clip and reports every violation it sees as [start_sec, end_sec, label], plus a causal per-frame risk that an accident starts within 5 seconds. It runs offline with one small pretrained detector; everything above detection is explicit, inspectable rules.

Try the live demo See results on the samples Repository
–
Score A on our dev labels
7 / 14
classes predicted
19 MB
weights (YOLO11s, COCO)
0.40×
real time for Part A tracking (Apple M4 GPU)

The problem

Part A · event detection

Given a several-minute .mp4 from one fixed camera, return every traffic event as a time segment with one of 14 classes. Segments are matched to hidden labels by temporal IoU at 0.3, 0.5 and 0.7, and scored as macro F1 over classes. Boundaries matter as much as detection: a correct event with sloppy edges fails at tIoU 0.7.

Part B · accident anticipation

Frames arrive one at a time; at each one, output P(accident starts within 5 s) using only the past. Scored by chance-normalised AP, alarm F1 at θ = 0.5 and mean time-to-accident.

We got four unlabelled sample clips and no camera.md. So we labelled the samples ourselves, described the scene ourselves (scene/camera.md), and built everything so it can be checked against those labels with the official evaluate.py.

The pipeline

learned  pretrained network weights rule-based  hand-written geometry / temporal logic data  inputs and outputs
NeuroLens pipeline: Part A decodes every third frame, detects and tracks road users, aligns the annotated scene, reads the traffic signal and applies seven event rules. Part B tracks every fifth frame and turns pairwise time-to-collision into a risk score. PART A · detect_events(video_path) .mp4 clip3840×2160, 29.97 fps Decodeffmpeg: every 3rd framescaled to 1920 px YOLO11s detectCOCO, imgsz 1280, FP16person, 2-wheelers, cars… ByteTracktrack ids across frames→ foot points, speeds 7 event rulestrajectories × scene× signal state Segment post-processmerge gaps, drop blips,pad / trim edges per class 25 keyframesmedian background Scene alignmentSIFT + RANSAC to reference scene.jsonlanes, crossings, stop line Signal croplamp region, every frame Signal readercolour contrast → R/A/G [[s, e, label], …]harness → predictions.json PART B · RiskEstimator.step(frame, t) — causal frame tone at a time YOLO11s + ByteTrackevery 5th frame,imgsz 640 Pairwise TTCclosest approach of movingtracks, miss < 0.04 fh risk = e^(−TTC/2)max over pairs, TTC ≤ 5 sheld between samples score ∈ [0, 1]per frame

Models and data, and why

Detector · YOLO11s learned

Ultralytics YOLO11s, COCO-pretrained, no fine-tuning (weights in the repo, 19 MB, AGPL-3.0). We keep person, bicycle, car, motorcycle, bus and truck. Why: no labelled boxes exist for this camera, COCO already covers every road-user class we need, and the small model at imgsz 1280 still sees pedestrians that are 40–100 px tall while staying far inside the 3× budget.

Tracker · ByteTrack learned inputs

Ultralytics' ByteTrack (src/bytetrack.yaml) on every 3rd frame (~10 Hz). Why: every event is defined by what one road user does over time. ByteTrack keeps low-confidence boxes in the association step, which holds ids through partial occlusion in queues.

Everything else rule-based

Scene alignment, signal reading, all seven event classes, segment post-processing and the Part B risk are hand-written. Why: the camera is fixed and hard-coding its geometry is allowed; the rules are exact, explainable, deterministic, and can be tuned directly against tIoU on our labels. With four unlabelled clips there is nothing to train an event model on.

DataUsed forLicence
MS COCO (through the YOLO11s checkpoint)Pretrained detector onlyCOCO terms / Ultralytics AGPL-3.0
Our labels of the 4 sample clips (dev_labels.json)Rule tuning and evaluationTeam NeuroLens
Our scene description (scene/scene.json, reference.jpg)Geometry for the rulesTeam NeuroLens

No other datasets were used for training. No hosted models or paid APIs at inference.

Event rules

All thresholds are in normalised frame units (fh = frame heights, x stretched by 16/9 so distances are isotropic). Constants are quoted from src/events/*.py.

Rebuild it

  1. Decode every 3rd frame, scaled to 1920 px wide, through an ffmpeg pipe (src/video.py).
  2. Detect and track with YOLO11s (imgsz 1280, conf 0.2, FP16 on CUDA) + ByteTrack; store one row per (frame, track): time, id, class, box (src/tracking.py).
  3. Align the scene: take the median of 25 evenly spaced frames as background, match SIFT features against scene/reference.jpg, fit a similarity transform with RANSAC (≥ 40 inliers, else identity), warp every polygon of scene.json (src/align.py, src/scene.py).
  4. Read the signal: per analysed frame, crop the opposite-approach signal head; score each lamp section by colour contrast (red = R − max(G, B), amber = min(R, G) − B, green = G − R) against a noise-scaled threshold; runs shorter than 5 samples take the previous state (src/signal.py).
  5. Features per track row: foot point = bottom-centre of the box; speed = central difference over ±2 samples; people overlapping a two-wheeler (≥ 0.3) or inside a vehicle (≥ 0.6) are riders or passengers, not pedestrians (src/events/common.py).
  6. Rules (table above) produce per-frame masks, turned into segments and post-processed: merge gaps, minimum length, padding, and trimming of sparse edges (src/segments.py).
  7. Part B runs its own causal tracker on every 5th frame (imgsz 640); for every pair of moving tracks it predicts the closest approach under constant velocity. If they come within 0.04 fh at TTC ≤ 5 s, risk = exp(−TTC / 2); the frame score is the max over pairs (src/risk.py).

Seeds are fixed (SEED = 0). Everything runs with python run_submission.py --videos /data/test --out predictions.json.