How to address resource constraints

[sansi-zhang]

Hi @maltanar ，I'm sorry to bother you again.

I encountered an error while building a hardware streaming structure for my custom model on ZCU104, and the error message was Synthesis failed, no bitfile found.

Upon inspecting the log files, I identified three DRC UTLZ-1 errors. The primary issue is related to the over-utilization of RAMB18 and RAMB36/FIFO resources in the top-level design. The error message stated:

ERROR: [DRC UTLZ-1] Resource utilization: RAMB18 and RAMB36/FIFO over-utilized in Top Level Design (This design requires more RAMB18 and RAMB36/FIFO cells than are available in the target device. This design requires 1068 of such cell types but only 624 compatible sites are available in the target device. Please analyze your synthesis results and constraints to ensure the design is mapped to Xilinx primitives as expected. If so, please consider targeting a larger device.)

I reviewed my network, and the ONNX file for my network is only around 5MB. However, I need to set the input size as (1, 9, 432, 432), which is relatively large.

Therefore, I need to make adjustments to my hardware design to address this issue. Here are the methods I have tried:

1. I attempted to introduce "block," "distributed," and "block" modes in decoupled mode, but it did not provide a resolution.
2. I tried reducing FIFO usage in "const" mode, but it did not solve the problem.
3. Combining the above two methods did not yield a solution.
4. I intended to use the "external" mode, but I encountered an error: 'NoneType' object has no attribute 'shape'.

In conclusion, I'm seeking guidance on how to resolve this issue, or how to correctly use the "external" mode.

Your assistance is greatly appreciated.
Thank you.

[fpjentzsch]

Hi,

which folding configuration (SIMD/PE) are you using throughout your model? Did you apply the automatic SetFolding with a performance target that was too high?

What does the resource breakdown between layers look like? Is only the input layer problematic? I would recommend to scale down your input or reduce the overall depth and/or width of your model to get a first working prototype. Afterwards you can still make the model larger or faster (by increasing SIMD/PE) until you hit the resource limits of your device.

Please continue any discussion on the external mode here: #904

[sansi-zhang]

Hi,

which folding configuration (SIMD/PE) are you using throughout your model? Did you apply the automatic SetFolding with a performance target that was too high?

What does the resource breakdown between layers look like? Is only the input layer problematic? I would recommend to scale down your input or reduce the overall depth and/or width of your model to get a first working prototype. Afterwards you can still make the model larger or faster (by increasing SIMD/PE) until you hit the resource limits of your device.

Please continue any discussion on the external mode here: #904

Thank you for your reply.

I used my own configuration of simd/pe, and now I can avoid this problem by adjusting my network model.

In #904 and I will write some of the problems I met, but in this discussion I still have some questions:

1. if I can through the use of external mode to alleviate this one phenomenon?

2. In theory, setting const mode should result in a decrease in fifo (no longer needed when writing weights), but I did not find a significant decrease or even an increase when I tested it.

3. In addition to the buffer fifo and weight buffer fifo between layers in the design, whether the ram consumption is also mvu? In which part? swu?

[fpjentzsch]

1. If the MVAU/VVAU weight storage is indeed the cause for the high BRAM utilization, external mode should help here, but you might run into performance bottlenecks when weights are streamed from memory to many layers in parallel.
2. You mean the FIFO between weight streamer and MVAU/VVAU? Yes, that will be eliminated since the weights are hard-coded into the compute unit. However, for large weight matrices or low SIMD/PE values, the synthesis tools will likely select BRAM storage for these weights anyways because a LUT-based implementation is not feasible. In this case you might see very similar resource usage to decoupled mode.
3. Synthesis reports should give you resource consumption figures for each individual layer in the hierarchy. I would guess that weight storage of the MVAUs contributes the most to BRAM consumption, but the sliding window unit can also consume significant BRAM, depending on the kernel size and feature map dimensions. FIFOs have a ram_style attribute which should give you some control over the memory resource used.

[sansi-zhang]

Thank you very much for your answer.

I already know something about it.
Also, as you guess, I met a lot of problems in trying to use external mode, as described in "#904 (comment)" for me.