What's the best way to transmit data over a high-latency networks

[evanzhang87]

I want to transmit data between two places that are far apart, eg: EU -> ASIA, and I want to send msg from vector to vector,

input -> [EU]vector-source -> (encode)[EU]vector-sink ---> (decode)[ASIA]vector-source -> [ASIA]vector-sink 

If vector->vector too slow, queue will block, and it will affect input rate, how to mitigate this situation?
There are a lot of source and sink pairs for me to use, eg: vector->vector, http->http-server... Which is recommended to use?

There are my configs:

sinks:
  vector:
    type: vector
    inputs:
      - input
    address: xxx
    compression: true
    acknowledgements: false
    request:
      retry_attempts: 1

sources:
  vector:
    type: vector
    address: 0.0.0.0:6000

[jszwedko]

For Vector to Vector communication, the vector source/sink will be the most efficient. You could try setting a static request concurrency to see if that helps in this high-latency situation: request.concurrency = 10 (or 20, 30, 50 etc.) in the vector sink.

[evanzhang87]

It works, thanks.