What model do existing pacing offloads supported by NICs use?

[maolson-msft]

Pacing can be done two ways: either each packet is posted to the NIC with a timestamp (which feels like what we ideally want to have), or you can plub a rate limit to the nic, post packets without any metadata (or maybe with a flow ID to match them up with a per-flow plumbed rate limit), and the NIC decides when to send each packet.

I did some prelim research on this topic a while back and was surprised to read in some docs (IIRC from Mellanox) that the second model above is used.

We probably want the first model, but if NICs are already doing the second model that will be an ugly ask.

[maolson-msft]

(I'm really hoping the above experience was really just a nightmare I was having. This Issue is to basically find out whether that's the case.)

[nibanks]

The first one is the model we're going with.

[maolson-msft]

That doesn't answer my question, though. I'm asking, what model do the NICs currently use?

[maolson-msft]

By the way, I found what I was reading before, here's a link: https://docs.nvidia.com/networking/display/VMAv875/Advanced+Features#AdvancedFeatures-PacketPacing

They describe an API that smells like a global pacing rate for the whole interface, which isn't what we want; then they mention something called the "rivermax library" for more "advanced" pacing, but it's a dead link.

[BorisPis]

Pacing can be done two ways: either each packet is posted to the NIC with a timestamp (which feels like what we ideally want to have), or you can plub a rate limit to the nic, post packets without any metadata (or maybe with a flow ID to match them up with a per-flow plumbed rate limit), and the NIC decides when to send each packet.

I did some prelim research on this topic a while back and was surprised to read in some docs (IIRC from Mellanox) that the second model above is used.

Both models should be possible, we have work in DPDK with the second model, here are some references:
• RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME
• https://doc.dpdk.org/api/rte__mbuf__dyn_8h.html#aaef84c5eadedb4c2fad3d49eabd7f0df
• Implemented in the testpmd sample application, see here: https://github.com/DPDK/dpdk/blob/main/app/test-pmd/txonly.c#L259