1110 réponses à ce sujet

[guosheng_xu]

https://discord.com/...ype=0&scope=bot

[_coronavirus]

What is this?

[_coronavirus]

This is the log after running the dnn_node_example.launcher.py with the saving argumnet hardcoded to yes. Also, are you really a bot or is htat just your username?  logrdk5.txt   10,45 Ko   0 téléchargement(s)

[_coronavirus]

My config file is just a copy of the original yolo11 but with my class list and my own 640x640 yolo model.

[_coronavirus]

I added debug level  debug_level_DNN_node_example.txt   26,83 Ko   0 téléchargement(s)

[_coronavirus]

Maybe this has something to do with the mismatch between Yolov11 output anzd the yolov8 parser used? Is this a common issue?

[chao_ma]

Hi! That makes perfect sense. For a product-level implementation with tight control over latency and resources, moving to a standalone C/C++ pipeline on RDK X5 is the correct decision.
Here are the precise API references and workflow for the X5 platform:
1. Memory Management (UVC :rightwards_arrow: HW Memory) Since standard UVC drivers do not export DMABUF, you must manually copy the frame.
API: Use [HB_SYS_Alloc](https://developer.d-...oc#hb_sys_alloc) to allocate a physically contiguous memory block.
This API provides both the Virtual Address (for your memcpy) and the Physical Address (for the hardware decoder).
Workflow: UVC Buffer -> memcpy -> HB_SYS_Alloc Buffer (Virtual Addr) -> HB_VDEC_SendStream (Physical Addr).
2. Hardware Decoding
API: [HB_VDEC_SendStream](https://developer.d-...vdec_sendstream)
Input: Prepare your VIDEO_STREAM_S structure using the physical address from step 1 and pass it to this API.

[chao_ma]

3. VPS (Video Processing Subsystem) is Essential We strongly recommend the Decode -> VPS -> BPU pipeline.
Why:
Resizing: Your 1080p camera input needs to be resized (e.g., to 640x640) for YOLO.
Alignment: The BPU requires strict memory alignment. VPS handles the format conversion (NV12) and alignment perfectly.
Reference Sample: Please check the VPS Sample included in the system image at: /app/cdev_demo/ (You can see the usage of command line arguments like -iheight, -oheight in the source code, which maps to the hardware scaling engine).
4. Official References For the exact C code implementation, please refer to the Multimedia SP Dev API documentation and samples here: [RDK X5 C-Dev Demos & API Reference](https://developer.d-...h-6cd5:-yolov5s)
This path (UVC -> HB_SYS_Alloc -> HB_VDEC -> VPS -> BPU) is the standard high-performance path for the X5 chip.
Hope this helps you build an efficient pipeline!

[chao_ma]

To answer your technical question: You hit the nail on the head with your last guess. The issue is indeed the mismatch between the YOLOv11 model output and the YOLOv8 parser.

Currently, TROS does not natively support YOLOv11 post-processing yet. The yolov8 parser in the config expects a specific tensor structure that likely differs from your v11 model's output, which is why you see no bounding boxes (and why the JSON config ignores your parameters).

Here is the recommended path to fix this:

Model Conversion Check: First, could you share which guide or repo you referenced to convert your .pt to .bin? To ensure the basic model graph is correct, we highly recommend following our official Model Zoo conversion guide for Ultralytics models: [Link to RDK Model Zoo / Ultralytics] (This ensures the model compiles correctly for the BPU)

Post-Processing (The Hard Part): Since there is no built-in yolov11 parser in the pre-compiled libraries, you cannot just use a JSON config. You need to modify the C++ source code.

Action: You should pull the dnn_node ROS package source code (or the C++ examples).

Modify: Locate the post-processing logic (where it handles YOLOv8 output tensors) and manually adapt the code to match YOLOv11's specific output head (shape, strides, anchors).

Recompile: Rebuild the package to apply your custom parser.

Regarding the logs: The camera_type is None warning suggests you might need to export the camera type before running (e.g., export CAM_TYPE=usb if you are using a USB webcam), otherwise it defaults to MIPI and might show a black screen.

Hope this points you in the right direction!

[tototek]

Thanks for replying. I was going to try asking on the Dietpi forum if they could support the X5, but just like on the d-robotics forum, my account is temporarily suspended and I don't know how to get it back.

[_coronavirus]

I got the updated Github repo on the RDK X5, but is Colcon suppossed not to work?

[_coronavirus]

Can you please help me figure out how I can build the hobot_dnn repo from ws/src? All of it is located in there.

[chao_ma]

u can read the Cmakelists and set PLATFORM_X5

[olivier]

Hi Marcelo,
Thanks a lot for the detailed and precise explanation - this is extremely helpful.
I
Key points we have validated:
UVC MJPEG input
Since standard UVC drivers do not export DMABUF, we will manually copy the bitstream.
We will allocate physically contiguous memory using HB_SYS_Alloc, use the virtual address for memcpy, and pass the physical address to HB_VDEC_SendStream.

Hardware decoding
We will use HB_VDEC_SendStream with VIDEO_STREAM_S, using the physical address obtained from HB_SYS_Alloc.

VPS usage
We will always use the Decode :rightwards_arrow: VPS :rightwards_arrow: BPU pipeline.

We still have a few clarification questions to make sure we fully follow best practices on X5:

1) BPU scheduling in multi-camera scenarios
For multiple parallel video pipelines feeding the BPU, is the intended approach to use a single shared BPU context with a user-level scheduler (round-robin or priority-based)?
Or are there advantages to maintaining per-pipeline BPU contexts?
We are currently leaning toward a centralized user-level scheduler with frame-drop policies to avoid backlog.

2) Recommended buffer pool sizing
Is there a recommended number of pre-allocated HB_SYS_Alloc buffers per camera (for decode input and VPS output) to avoid stalls while keeping memory usage bounded?
Any guidance on typical pool sizes used in reference designs?

3)VPS multi-output usage
Is it considered best practice to use VPS to generate multiple outputs (for example one resized stream for BPU inference and another for display or streaming), or should VPS be kept strictly limited to the inference path?
Aside from these points, everything is now very clear, and we are confident moving forward with the SP Multimedia + BPU APIs in C.

Thanks again for the support and for the quality of the documentation and samples - they make building a product-grade pipeline on RDK X5 much easier.

Best regards,
Olivier

[_coronavirus]

Do you support Yolov10? I just switched over, but I get the same error with exit code -11...

[_coronavirus]

{
"model_file": "/home/sunrise/dnn/models/best_yolov10s_bayese_640x640_nv12.bin",
"task_num": 4,
"dnn_Parser": "yolov10",
"model_output_count": 6,
"reg_max": 16,
"class_num": 1,
"cls_names_list": "/home/sunrise/dnn/drone.list",
"strides": [8, 16, 32],
"score_threshold": 0.25,
"top_k": 300,
"output_order": [0,1,2,3,4,5]
}

[_coronavirus]

The config used

[_coronavirus]

This is my new error.  yolov10.txt   13,6 Ko   0 téléchargement(s)

[_coronavirus]

I think your output parser is wrong even for yolov10 and 8 I used the guide here: https://github.com/D...l_zoo/tree/main

[_coronavirus]

Can you perhaps expand on this?