brian_hacker

 Screenshot_20260719_144205_Chrome.jpg   45,57 Ko   0 téléchargement(s)

@Shockley you may review my PR

**VLM Police Patrol - Stage 3 (Launch) COMPLETE!**

Hey everyone! Just finished Stage 3 of the Robotics Dream Keeper Challenge. My autonomous patrol robot on the **RDK X5** is fully built, demoed, and packaged.

**What it does**
An autonomous patrol robot that detects people on-board with **YOLO11n (BPU)**, triages each one against an operator's reference photo using an on-device **Vision-Language Model (InternVL2.5-1B)**, and drops confirmed matches onto a live **SLAM map** - all driven from a Flask web dashboard with holonomic mecanum teleop.
> Surface-feature triage only (clothing colour/type + hairstyle) - not face recognition.

**Highlights**
Two BPU-accelerated models on-device - YOLO11n detector + InternVL2.5-1B ViT
Two concurrent AI workloads: continuous detection gates the slower VLM match
Off-board `slam_gmapping` mapping + on-board **AMCL** localization
RP2040 micro-ROS mecanum base - PID, `cmd_vel` watchdog + hardware **RF E-stop**
EKF sensor fusion (wheel odom + MPU-9250 IMU)
Full stack + VLM: ~70 °C / ~90 % CPU, and teleop *still* smooth (motion runs on the Pico, not the X5)
One-shot `master.sh` brings up all 9 nodes

**Links**
Demo video:
Repo: https://github.com/z...M-Police-Patrol
Showcase PR: [Showcase] Brian - VLM Police Patrol (Stage 3)

Happy to answer any questions about the build!

sure will do:yes:

**VLM Police Patrol - Stage 3 in progress** (Robotics Dream Keeper Challenge)

Repo: https://github.com/z...M-Police-Patrol
Stage 3 brief: https://github.com/D...tage3-launch.md

Full stack integrated and running end-to-end on the RDK X5 - a patrol robot that spots people on-board, triages them against a reference photo via VLM, and drops confirmed matches onto a live map from a browser dashboard.

** Technical writing**
Architecture, ROS 2 graph, calibration, and known issues all written up in `docs/technical.md`. README maps every piece straight to the challenge's own competency checklist (bring-up, model deployment, ROS 2 tuning, thermal/profiling).

** Benchmarks**
- YOLO11n person detector on the **BPU** (640×640 NV12, int8) - continuous stream
- InternVL2.5-1B VLM - ViT encode on **BPU**, Qwen2.5-0.5B decode on CPU (llama.cpp)
- Two workloads running concurrently: live detection never stalls while a VLM triage is in flight
- Full stack + VLM active: ~70°C / ~90% CPU - teleop stays smooth since motion control lives on the Pico, not the X5

Full numbers: https://github.com/z...s/benchmarks.md

** End-to-end bring-up**
One script (`sh/master.sh`) sequences all 9 nodes 2s apart so dependencies are ready before consumers start: base/micro-ROS/EKF -> camera -> YOLO -> VLM -> matcher -> AMCL -> localizer -> dashboard -> OLED. cmd_vel watchdog auto-brakes the bot if the connection ever drops.

** Left to do - visual data**
- Photo of the robot
- URDF render
- Dashboard screenshot (with overlays)
- OLED status display photo
- Demo video (full loop)