Rpc calls made by web3.js over websockets do not get a response.

[RobbeVerhelst]

I found this issue when trying to benchmark different Hyperledger Besu nodes using Hyperledger Caliper. Our setup is x amount of non-validator nodes behind a dshackle loadbalancer.

When running Caliper we saw that we didn't get any response from dshackle over websockets. When we ran Caliper directly on a besu node everything worked fine. After some further investigation we saw Caliper uses web3.js to make rpc calls. So to test this out we made a small web3 project and found that when web3 tries to send rpc calls over websockets to dshackle it does not get a reply. We do however see the rpc call coming into dshackle (in a grafana dashboard), but the web3 project does not get a reply. When we try to do the rpc calls over https we do get a reply from dshackle, unfortunately Caliper is not able to use anything else but websockets.

Somewhere in the websocket part of dshackle it is not able to send the response to the web3 project.

[splix]

What method do you call over websockets? It's possible that those methods are just not enabled by default

[RobbeVerhelst]

Unfortunately that's not possible, I call eth_blockNumber and am able to get the response from that using wscat and even ethers.js using ws.
It's specifically web3.js that's not getting a response for some reason

[splix]

Do you mean that the a request for eth_blockNumber made with wscat works, but the exactly same request with web3.js doesn't work?

[splix]

I mean, I'm sure there is some difference in the calls that makes one request working and another one not

[RobbeVerhelst]

Yes, I'm sending the same request using web3.js as I am with wscat, but also with ethers.js I get a response. Specifically web3.js is not getting the answer from dshackle, but dshackle does get the request and it is "successful" when checking the grafana dashboard. But on the web3 side I get nothing. So web3.js does something different then ethers.js or wscat. But when I change the url in my web3 project to the url of the besu node, then it does work. So it's not that the web3 project doesn't work at all.

[splix]

Can you trace the requests with Wireshark and provide me with the details?

[PDVJAM]

Any updates on this? We actually have the same issue with web3 and websockets.

[splix]

@PDVJAM can you provide with traffic traces between the dshackle and the upstreams? We need something to look at to find out what is the problem

[PDVJAM]

@PDVJAM can you provide with traffic traces between the dshackle and the upstreams? We need something to look at to find out what is the problem

Our dshackle deployed to the k8s, so I don't think I can do some wireshark for you.
@RobbeVerhelst were you able to solve the issue?

[splix]

The latest version of the code in master supports logging of all requests, including requests to the upstreams. So if you are able try with a version from master that could provide enough details

[PDVJAM]

The latest version of the code in master supports logging of all requests, including requests to the upstreams. So if you are able try with a version from master that could provide enough details

Is there a docker image already or must be built manually?

[brixial]

i guess this also has the same context?
#224