How it works

From a server's perspective

Every time a message is sent to a client connection, a message sequence number is incremented on the server side and the message is wrapped into a sequenced message, which contains the incremental sequence number and message data. Flow control checks are not made for all of the messages, they are only made for messages where the sequence number divides by the server’s check interval value (modulo equal to zero).

For the flow control check, the delta between the sent sequence number (sequence number from the message) and the received sequence number (received by the client connection) is calculated. If this delta is larger than a predefined value on the server (not fixed, but ~50,000 messages) and is continuously growing, then the server will issue a too slow warning message. This control message doesn't close the connection, it just warns clients that this connection might be closed soon if client performance does not improve soon. If the delta begins to reduce in size, we decrement a counter for clients and eventually their “too slow” state will be reset back to zero (ok). If the delta continues to surpass server maximums and continuously grows, eventually after numerous flow control checks, the connection will be closed with response code 450 and reasoning “Too Slow”. There are additional checks on the server to consider message spikes (to allow connections to catch up) in the flow control checks.

From a client's perspective

The flow control message check interval is initialized by a value sent in the connect response. Every time a new message arrives, the client does a quick calculation to see if it’s time to send a flow control message to the server (done every X messages, usually in the 10s of thousands).

This is applicable for all of the clients.

What can cause connection to be closed due to being "Too slow"

Connections are closed with a "Too slow" reason when clients can't keep up with the amount of messages either because it takes too long to process every message on the client side (rendering graphs, doing calculations, etc or the amount of messages is simply too much even if processing is optimized). The two main ways to improve performance are to decrease the message processing time, processing them quickly and not blocking or to split symbol subscriptions across additional connections, giving more resources to each connection.