Pulsar is good performance on publish rate but it is slow for delivery

[rocklim]

Current metrics

Local Delivery rate is 1k msg/s

Local publish rate has 6k msg/s

It does able to push high sometimes I restarted my consumer/pulsar to 14000 msg/s delivery rate, but it will eventually falls down to 1k msg/s or lower.

This is my current setup

AWS EKS v1.28
6 i3en.2xlarge

config:
managedLedgerDefaultEnsembleSize: "6"
managedLedgerDefaultWriteQuorum: "4"
managedLedgerDefaultAckQuorum: "3"
managedLedgerMaxEntriesPerLedger: "5000000"
brokerDeleteInactiveTopicsEnabled: "false"
dispatcherMaxReadBatchSize: "50000"
dispatcherMaxReadSizeBytes: "52428800"

Bookie on local storage PV with nvme SSD

My Application

Producer -> Pulsar -> Flink -> Clickhouse

Is there any guidance i can improve my dispatch rate?

[lhotari]

config:
managedLedgerDefaultEnsembleSize: "6"
managedLedgerDefaultWriteQuorum: "4"
managedLedgerDefaultAckQuorum: "3"
managedLedgerMaxEntriesPerLedger: "5000000"

dispatcherMaxReadBatchSize: "50000"
dispatcherMaxReadSizeBytes: "52428800"

What is the rational behind these configs? I'd assume that the default values are more optimal for your use case.