20 réponses à ce sujet

[naren_bhau_77570]

My technical background: 3rd year B.Tech ECE
(IoT) student at IIIT Nagpur. Co-Head of
Robotics Wing of my institute. Experience with
ROS 2, Raspberry Pi, ESP32, Arduino, Python, and embedded systems. Previously built a quadcopter drone, weather station with environmental sensors,
and ROS 2 target tracking system. Interned at
India Space Lab working on PCB design and
drone mechanics.

My robot/AI dream: Building ClassMind - a
Smart Classroom Intelligence System using
RDK X5. The system uses burst-capture face
detection with YOLO on BPU for automated
student attendance across 50 students
simultaneously, combined with occupancy-based
energy management via ESP32 relay control for
lights and fans. Full ROS 2 backbone with
real deployment in my actual college lab at
IIIT Nagpur.

GitHub: https://github.com/N...classmind-rdkx5

Excited to build and learn with this
community!

#RoboticsDreamKeeper #RDKX5

[naren_bhau_77570]

Week 1 Progress Update - Laptop Phase Complete!
Big update from Team ClassMind!
While waiting for my RDK X5 board to arrive, I have been using this time to build and test the entire ClassMind pipeline on my laptop. Here's what's working RIGHT NOW:

What I Built This Week
OpenCV Complete
1. Image processing fundamentals
2. Live webcam face detection
3. Multi-face detection on group photos
4. Red object tracker with centroid calculation
5. Contour detection and shape analysis

YOLO v11 Mastery
1. Person detection at 86% confidence - 10.5ms per frame.
2. Object tracking and segmentation
3. Custom dataset training via Roboflow
4. Trained number plate detection model
5. Trained cat/dog classification model

DeepFace Recognition - ClassMind Core
1. Live webcam recognizing me by name
"narendra" displayed with green bounding box
2. Face database system working
This is exactly what ClassMind attendance needs

The moment RDK X5 arrives - all this laptop code gets ported to BPU. Face detection will go from CPU to 10 TOPS hardware acceleration.
Stay tuned!

[naren_bhau_77570]

 Screenshot_2026-06-05_141644.png   642,98 Ko   0 téléchargement(s)
 Screenshot_2026-06-06_000951.png   705,83 Ko   0 téléchargement(s)
 Screenshot_2026-06-09_203433.png   1,95 Mo   0 téléchargement(s)
 Screenshot_2026-06-05_102947.PNG   741,04 Ko   0 téléchargement(s)

[naren_bhau_77570]

RDK X5 Unboxing !!
The moment I've been waiting for!
My D-Robotics RDK X5 developer kit has arrived. Huge thanks to Lisa and Yujia from the D-Robotics team for making this possible for a student from India!
What's next--->
Getting SD card today -> Flashing OS -> First boot tonight!
ClassMind - Smart Classroom Intelligence System is about to go LIVE on RDK X5 BPU !!!
 IMG_20260616_134331618_HDR2.jpg   481,65 Ko   0 téléchargement(s)

[drobo_community]

@naren Glad to hear from you! Please make sure to deliver your first stage tasks based on the RDK X5 ecosystem.

[naren_bhau_77570]

Already on it, stage one evidence by tonight.

[naren_bhau_77570]

Stage 1 Complete!
Successfully completed all Stage 1 challenges on RDK X5:
Board flashed, booted, SSH working

Phone camera (DroidCam) -> YOLO detection on BPU - 91% confidence person detection, 13ms inference!

Showcase submitted + PR opened!
GitHub: https://github.com/N...classmind-rdkx5
Onward to Stage 2 - Build Challenge!
#RoboticsDreamKeeper #RDKX5 #Stage1Complete

[naren_bhau_77570]

VID-20260619-WA0037.mp4   3,87 Mo

[naren_bhau_77570]

Update !!!!
connected my ESP32 to RDK X5 over wifi today. when YOLO detects a person on the BPU, it sends a signal to ESP32 and the built in RGB led blinks red. when no person detected the light turns off automatically.
attached a short video showing it working live, used my phone camera (droidcam) as input since i don't have a proper camera module yet.
this is basically the foundation for ClassMind's occupancy detection - same logic will control the actual classroom lights/fan through a relay later.
next working on the attendance side, trying to get face recognition working alongside this.

[shockley6668]

Congratulations! This is a great example of combining AI perception with real-world device control. Looking forward to seeing the attendance and face recognition in the next Stage.

[drobo_community]

Congratulations, you have unlocked your RDK Explorer title!

[naren_bhau_77570]

Thank you so much

[naren_bhau_77570]

**Today's Update **

Today I worked on the face recognition part of my **ClassMind** project on the **RDK X5**.
I successfully got **DeepFace** running on the board and tested face recognition using both **FaceNet** and **SFace** models with a small face database.

**Observations:**

* **FaceNet:** Better recognition accuracy, but it took around **19-20 seconds** to recognize a single face, which is too slow for a real-time classroom attendance system.
* **SFace:** Recognition was much faster (**around 2-3 seconds**), but the accuracy wasn't as good on my current dataset and sometimes produced incorrect matches.

Although DeepFace helped me validate that the complete face recognition pipeline works on the RDK X5, I don't think it's the right choice for my final system because of the inference time and limited control over the recognition pipeline.
My next step is to move to **InsightFace**. From what I've researched, it provides better face detection/alignment, ArcFace embeddings, lower overhead, and will also make it easier to integrate **FAISS** later for faster face search.
Looking forward to benchmarking InsightFace on the RDK X5 and seeing how much performance I can gain. Any suggestions or experiences with InsightFace on RDK X5 are welcome.

[naren_bhau_77570]

 image.png   47,56 Ko   0 téléchargement(s)

[naren_bhau_77570]

Completed **Stage 2** of the Robotics Dream Keeper Challenge!

Over the past few days I've been working on **ClassMind**, an AI-powered smart classroom system built on the **RDK X5**.

### Current Progress

* Flask-based web application
* Live camera streaming
* YOLO11n running on the **RDK X5 BPU** for person detection
* InsightFace (buffalo_s) for face recognition
* Automatic attendance generation with CSV export
* Classroom occupancy monitoring
* ESP32 integration for classroom automation (Light/Fan control via HTTP)
* Modular project architecture for future ROS 2 integration

During development I also worked through several practical challenges such as optimizing memory usage, integrating BPU and CPU workloads, handling camera streaming, and organizing the project into reusable modules.

### Current Architecture

```
Camera -> Camera Manager
:box_drawings_light_vertical_and_right::box_drawings_light_horizontal::box_drawings_light_horizontal: Occupancy -> YOLO (BPU) -> ESP32 -> Relay
:box_drawings_light_up_and_right::box_drawings_light_horizontal::box_drawings_light_horizontal: Attendance -> YOLO (BPU) -> InsightFace -> CSV
```

Next, I'll be focusing on:

* ROS 2 integration
* MIPI camera support
* Database integration
* Improved web dashboard

GitHub Repository:
https://github.com/N...classmind-rdkx5

Looking forward to feedback from the community.

[drobo_community]

Congratulations! @Shockley please check and leave your comments.

[shockley6668]

Great proposal! It's very detailed and well thought out. Looking forward to seeing the project come to life. Best of luck with Stage 3!

[naren_bhau_77570]

Thanks

[naren_bhau_77570]

Moving forward to complete the stage 3!!
 rn_image_picker_lib_temp_4bc5c0df-c880-48f8-840c-968824b58d04.jpg   3,95 Mo   0 téléchargement(s)

[naren_bhau_77570]

ClassMind Stage 3 - Complete System Demo

45 days on RDK X5. One command starts everything:

./launch_classmind.sh

YOLO11n BPU - 13-15ms real-time detection
InsightFace recognition - 1.5s/person
ROS 2 sensor_bridge -> /classmind/gas topic
ROS 2 Decision Node - MQ baseline calibration + alert
ESP32 relay - occupancy light/fan control
ESP32 RGB LED - environmental alert
Buzzer - attendance window signal
Flask teacher portal - live feed, results, CSV

Demo video:
GitHub: https://github.com/N...classmind-rdkx5
Release: v1.0-demo

#RoboticsDreamKeeper #RDKX5 #Stage3Complete