[Python] How find process which have problem with TCP, TCPOFOQueue?

In my project I use NATS JetStream, I use publish to stream rps request, this means that the client will wait for a response from the server, but often there is a timeout error, what the problem is, it is not clear.

I started collecting metrics with node_exporter and viewing them in grafana. in the TCP Errors panel you can see a lot of errors with the following description: TCPOFOQueue - TCP layer receives an out of order packet and has enough memory to queue it

Grafana graph

Moreover, these errors coincide in time with the errors that occur in my application.

Can you please tell me where to dig to find out and fix this problem?

I send usual request via python client for nats and receive exception nats: timeout

await connection.request(
message=b"some_message",
subject="order.cancel",
timeout=1
)

Continue reading...