ConnectionShutdown cancels sending even after stream graceful shutdown complete?

[jamboree]

The sender side does this:

• Read file to buffer
• StreamSend(QUIC_SEND_FLAG_NONE)
• Wait for QUIC_STREAM_EVENT_SEND_COMPLETE
• Repeat above steps until EOF
• StreamShutdown(QUIC_STREAM_SHUTDOWN_FLAG_GRACEFUL)
• Wait for QUIC_STREAM_EVENT_SHUTDOWN_COMPLETE
• ConnectionShutdown(QUIC_CONNECTION_SHUTDOWN_FLAG_NONE)

But the receiver side may not receive all data, not even QUIC_STREAM_EVENT_PEER_SEND_SHUTDOWN.
Instead, QUIC_CONNECTION_EVENT_SHUTDOWN_INITIATED_BY_PEER will be observed, followed by QUIC_STREAM_EVENT_SHUTDOWN_COMPLETE and QUIC_CONNECTION_EVENT_SHUTDOWN_COMPLETE.

It seems that ConnectionShutdown somehow forcibly cancels sending and QUIC_STREAM_EVENT_SHUTDOWN_COMPLETE does not guarantee what previously send to be finished?

[nibanks]

Just because one side received stream shutdown complete doesn't mean the peer has processed all the received data. It only means the peer transport stack has received and queued it. You should either let the connection shut down based on idle timeout (this is what HTTP does) or you need to have some explicit signal at the application layer.

[jamboree]

It only means the peer transport stack has received and queued it.

If the peer has received and queued it, why didn't it get event notifications?
Will shutdown initiated by peer somehow clear the receive queue before we get any chance to process the queued data?

[nibanks]

It's simply a race condition. The original sender can get the shutdown complete before the app on the other side gets the notification.

[jamboree]

So, does that mean shutdown initiated by peer will clear the receive queue?

[nibanks]

A connection shutdown throws away any streams, which throws away any queued data.

[jamboree]

Thanks for the clarification. Since I may not consume all RECEIVE data on QUIC_STREAM_EVENT_RECEIVE, and will call StreamReceiveSetEnabled for further receive, I found that it actually got notified with the received data, just that I didn't consume all of it, then it got CONNECTION_EVENT_SHUTDOWN_INITIATED_BY_PEER, and the next call to StreamReceiveSetEnabled didn't have any effect anymore.