[P2P] Native TCP support

[YangZhou1997]

Description

Please include a summary of the changes and the related issue.

Fixes # (issue)

Type of Change

• Bug fix
• New feature
• Documentation update

How Has This Been Tested?

Include any tests here.

• Unit tests
• Integration tests
• Manual testing

Checklist

• My code follows the style guidelines, e.g. format.sh.
• I have run build_and_install.sh to verify compilation.
• I have removed redundant variables and comments.
• I have updated the documentation.
• I have added tests.

[YangZhou1997]

To build:

# Ignore the libuccl_p2p.so install error (caused by permission issues)
make -f Makefile.rocm -j USE_TCP=1 install

To test:

UCCL_P2P_LOG_LEVEL=INFO UCCL_P2P_TCP_THREADS=20 UCCL_P2P_TCP_IFNAME=enp49s0f0np0,enp49s0f1np1 \
torchrun --nnodes=2 --nproc_per_node=1 --node-rank=0 --master_addr=10.162.224.129 \
benchmarks/benchmark_uccl_readwrite.py --mode=write --sizes=1048576

UCCL_P2P_LOG_LEVEL=INFO UCCL_P2P_TCP_THREADS=20 UCCL_P2P_TCP_IFNAME=enp49s0f0np0,enp49s0f1np1 \
torchrun --nnodes=2 --nproc_per_node=1 --node-rank=1 --master_addr=10.162.224.129 \
benchmarks/benchmark_uccl_readwrite.py --mode=write --sizes=1048576

[praveingk]

Nice 👍🏽 This is great.

[praveingk]

@YangZhou1997 TCP/EFA support will need integrations into uccl_engine.cc as well, similar to TCP-X.

[YangZhou1997]

So far, we keep the engine.h interface, so hopefully uccl_engine.cc will be compatible

[YangZhou1997]

NCCL performance is actually extremely high:

NCCL_IB_DISABLE=1 NCCL_SOCKET_IFNAME=enp49s0f0np0,enp49s0f1np1 \
NCCL_NCHANNELS_PER_NET_PEER=4 NCCL_DEBUG=INFO \
torchrun --nnodes=2 --nproc_per_node=1 --node-rank=0 \
--master_addr=10.162.224.129 benchmarks/benchmark_nccl.py

[Client]     256 B :    0.05 Gbps |    0.01 GB/s
[Client]    1.0 KB :    0.23 Gbps |    0.03 GB/s
[Client]    4.0 KB :    0.98 Gbps |    0.12 GB/s
[Client]   16.0 KB :    4.04 Gbps |    0.51 GB/s
[Client]   64.0 KB :   13.63 Gbps |    1.70 GB/s
[Client]  256.0 KB :   31.55 Gbps |    3.94 GB/s
[Client]    1.0 MB :   51.77 Gbps |    6.47 GB/s
[Client]   10.0 MB :   49.13 Gbps |    6.14 GB/s
[Client]   16.0 MB :   40.15 Gbps |    5.02 GB/s
[Client]  100.0 MB :   40.76 Gbps |    5.10 GB/s

[YangZhou1997]

I have been playing with TCP support over the break, and now my take is that it is gonna be hard to beat NCCL. There are two reasons:

1. NCCL seems to support multi-NIC on Ethernet: Do NCCL support multi-NIC on ethernet? NVIDIA/nccl#601 and my test results above
2. NCCL implements semi-persistent GPU kernels to pipeline GPU<->CPU data copy and CPU<->network data transfer, which is already optimal.

Because of this, I think we should support TCP in a similar way to TCPX by just layering on top of NCCL. Maybe that would need helps from @DanielDanyang @derekwin

Also cc @praveingk @zhongjiechen @MaoZiming

[praveingk]

@YangZhou1997 Broadly I Agree.
Few questions :

1. Do we plan to support multi-nic in future. It can be useful, and could be done at a higher level in engine.cc (by maintaining multiple endpoints and choosing one by Round-robin, etc)
2. https://blog.vllm.ai/2026/01/08/kv-offloading-connector.html has similar observations w.r.t. GPU-CPU transfers. We also compared this offloading connector to UCX's GPU<->CPU transfer and found that UCX is 4-5x slower (yet to dig deeper on the reason).
   @YangZhou1997 Are we using cudaMemcpyAsync, since the offloading connector of vLLM observes it performs better than custom kernel for larger blocks.

[DanielDanyang]

@YangZhou1997 I'd like to try implementing TCP based on NCCL, but this time it should be directly under the engine, rather than uccl_engine like TCPX before, correct?
Can I start directly from the current PR and rewrite the contents of the tcp/ folder?

[YangZhou1997]

Hi @DanielDanyang , great! Yes, let's try to organically integrate into engine.h/cc. You can create a new branch in uccl and open a PR from there. I would like to keep this pr for the record.

[YangZhou1997]

Hi @praveingk

1. I guess you are talking about two RDMA NICs per GPU. We have supported in EFA, and could easily support for non-EFA. We currently just lack that type of testbed to develop. For TCP, NCCL supports it natively by specifying multiple interfaces in NCCL_SOCKET_IFNAME.

2. Very useful blog! I tried to use cudaMemcpyAsync for TCP transfer, but there is a challenging dilemma: 1) do a big cudaMemcpyAsync for the whole data once, then transfer, finally cudaMemcpyAsync again on the receiver, but this will bring high copy latency on the critical path, 2) do chunk-by-chunk small cudaMemcpyAsync and overlap it with TCP transfer, but cudaMemcpyAsync performs poorly under small chunks as there is only one copy engine per GPU. NCCL ends up with the second approach, but uses custom kernels to guarantee high performance even for small chunks. (iirc, NIXL/UCX should use the first approach, which is much smaller than NCCL's)

My understanding is that for TCP, NCCL's solution is already optimal. And it is also pretty challenging to implement efficiently, thus I think we should just use NCCL for TCP transfer.

[praveingk]

Thanks @YangZhou1997 for the clarification.
Regarding 1) I have a testbed for testing the multi-NIC for RDMA, I can try to port the changes that have been done for EFA to non-EFA RDMA.
Regarding 2) Yes, that makes a lot of sense. The blog is focussed only on GPU-CPU transfers, and not pipelining network transfers. Better to re-use NCCL's implementation here.