24 réponses à ce sujet

[brian_hacker]

Hi everyone! I'm Brian.

My technical background: Robotics engineer from Malaysia with experience in ROS/ROS 2, autonomous mobile robots (AMR), SLAM, navigation, sensor integration, simulation, and industrial automation. I enjoy developing intelligent robotic systems and solving real-world robotics challenges.

My robot/AI dream: To develop intelligent autonomous robots that can perceive, understand, and navigate complex environments safely and efficiently, leveraging AI and advanced robotics technologies to solve real-world problems.

Excited to join the challenge and learn with you all!

#RoboticsDreamKeeper #RDKX5

[evelyn041059]

looking forward to your creation:clapping:

[brian_hacker]

Stage 1 - Ignite complete!

Board bring-up: flashed RDK X5 via RDK Studio, connected over network, and confirmed SSH access
Sensor explorer: brought up the LeTMC-520 RGBD camera (Orbbec Astra Pro clone) using `ros2_astra_camera` and verified a live feed
First AI task: ran YOLO object detection on-device using `rdk_model_zoo`, with real-time bounding boxes on detected persons

This is feeding into my project, **VLM Police Patrol** - a ROS 2 patrol robot combining YOLO detection with VLM-based suspect appearance matching (`suspect_matcher`) and a live web dashboard (`dashboard_flask`) on a mecanum drive base.

Repo:
1. https://github.com/z...PolicePatrol.md
2. https://github.com/z...M-Police-Patrol
Onward to Stage 2

[brian_hacker]

 yolo.png   663,19 Ko   0 téléchargement(s)

[brian_hacker]

@:robot:Lisa for the submission, as long as i completed each stage within July 15, still considered complete the competition
 Screenshot_20260706_133421_Chrome.jpg   63,13 Ko   0 téléchargement(s)

[drobo_community]

Yes, exactly!

[drobo_community]

@Shockley Please review

[shockley6668]

@brian_hacker Great work! One small suggestion: try updating RDK Studio to the latest version, it may help improve the experience.

[brian_hacker]

sure no problem:laugh_squint:

[brian_hacker]

Stage 2 - Build submitted!

Just pushed my proposal for **VLM Police Patrol** - a ROS 2 patrol robot that combines on-device VLM suspect matching with live SLAM mapping on the RDK X5.

What's in the Stage 2 doc:
**Concept** - indoor patrol robot that detects a person of interest by appearance (clothing + hairstyle) and marks their location on a live map
**Architecture** - gated two-tier inference: YOLO runs real-time on the BPU, and an InternVL VLM is invoked on throttled crops (~30s per match cycle on-device)
**SLAM** - gmapping builds the map, AMCL localizes during patrol, MPU-9250 IMU fused with wheel odometry, and a `suspect_locator` node drops a marker in the map frame on a positive match
**Hardware** - mecanum base, 4× MC520 encoder motors on a TB6612 4-ch driver, RP2040 clone running micro-ROS for closed-loop control with ESP8285, LeTMC-520 RGBD camera, 0.96" OLED status display

Full proposal (architecture diagrams, BOM, engineering plan):
https://github.com/z...PolicePatrol.md

Code repo:
https://github.com/z...M-Police-Patrol

Stage 3 build + demo coming before the deadline. Onward!

[brian_hacker]

System Design
 mermaid-flowchart-2026-07-08T13-48-59.png   983,4 Ko   0 téléchargement(s)

[brian_hacker]

ROS2 node
 mermaid-flowchart-2026-07-08T13-49-53.png   781,61 Ko   0 téléchargement(s)

[shockley6668]

Hi Brian, thanks - the Stage 2 doc looks solid.

But, I don't see a PR to the official D-Robotics repo yet; it looks like it's only on your fork branch.

So, please open a PR from `showcase/zimbot97-stage2` to `D-Robotics/Robotics-Dream-Keeper-Challenge:develop`.

Also change `Stage completed: <2>` to `Stage completed: 2`.

Send me the PR link once it's open and I'll review it there.

[brian_hacker]

https://github.com/D...hallenge/pull/7

[shockley6668]

Hi Brian, thanks - I reviewed the PR on GitHub.

Stage 2 content looks good overall. Just one issue before merge: the PR also deletes `projects/Example-Project-SampleVision-project.md`.

Please restore that file and keep the PR limited to adding your `projects/Brian-Project-VLMPolicePatrol.md`. After that it should be ready for approval.

[brian_hacker]

you may check now if i done correctly

[shockley6668]

Thanks Brian. I checked again, and it looks good now. I've approved it on GitHub.

[brian_hacker]

@:robot:Lisa the deadline for stage 3 is 15 July is it? and what timezone does it follow? i am 95% done on stage 3 and curious on the deadline timeframe

[shockley6668]

Hi @brian_hacker , the deadline for stage 3 is delay to 22 July.

[brian_hacker]

**VLM Police Patrol - Stage 3 in progress** (Robotics Dream Keeper Challenge)

Repo: https://github.com/z...M-Police-Patrol
Stage 3 brief: https://github.com/D...tage3-launch.md

Full stack integrated and running end-to-end on the RDK X5 - a patrol robot that spots people on-board, triages them against a reference photo via VLM, and drops confirmed matches onto a live map from a browser dashboard.

** Technical writing**
Architecture, ROS 2 graph, calibration, and known issues all written up in `docs/technical.md`. README maps every piece straight to the challenge's own competency checklist (bring-up, model deployment, ROS 2 tuning, thermal/profiling).

** Benchmarks**
- YOLO11n person detector on the **BPU** (640×640 NV12, int8) - continuous stream
- InternVL2.5-1B VLM - ViT encode on **BPU**, Qwen2.5-0.5B decode on CPU (llama.cpp)
- Two workloads running concurrently: live detection never stalls while a VLM triage is in flight
- Full stack + VLM active: ~70°C / ~90% CPU - teleop stays smooth since motion control lives on the Pico, not the X5

Full numbers: https://github.com/z...s/benchmarks.md

** End-to-end bring-up**
One script (`sh/master.sh`) sequences all 9 nodes 2s apart so dependencies are ready before consumers start: base/micro-ROS/EKF -> camera -> YOLO -> VLM -> matcher -> AMCL -> localizer -> dashboard -> OLED. cmd_vel watchdog auto-brakes the bot if the connection ever drops.

** Left to do - visual data**
- Photo of the robot
- URDF render
- Dashboard screenshot (with overlays)
- OLED status display photo
- Demo video (full loop)