tcptunnelchecker shows some failures for tobaru

[vi]

I have developed a special tool that checks corner cases of TCP forwarding regarding TCP resets/hangups while being backpressured.
It shows discrepancy between connecting back through tobaru and, for example, through socat.

$ cat config.json
{
  // Listen on all interfaces, on port 8080.
  "bindAddress": "127.0.0.1:1234",
  "target": {
    // Forward to 192.168.8.1 port 80.
    "address": "127.0.0.1:1235",
    // Allow connections from all addresses. 'all' is a special alias for 0.0.0.0/0.
    "allowlist": "all"
  }
}

$ tobaru ./config.json
Listening (TCP): 127.0.0.1:1234
[2022-03-16T08:31:00Z ERROR tobaru::tcp] 127.0.0.1:33892 finished with error: Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }
[2022-03-16T08:31:01Z ERROR tobaru::tcp] 127.0.0.1:33896 finished with error: Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }
[2022-03-16T08:31:01Z ERROR tobaru::tcp] 127.0.0.1:33908 finished with error: Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }
[2022-03-16T08:31:02Z ERROR tobaru::tcp] 127.0.0.1:33952 finished with error: Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }
[2022-03-16T08:31:03Z ERROR tobaru::tcp] 127.0.0.1:33956 finished with error: Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }
[2022-03-16T08:31:03Z ERROR tobaru::tcp] 127.0.0.1:33960 finished with error: Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }

$ ./tcptunnelchecker 127.0.0.1:1235 127.0.0.1:1234
[ OK ] Trivial test 1
[ OK ] Trivial test 2
[ OK ] Clogged close test passed: outDrainCheck_inClose
[ OK ] Clogged close test passed: outDrainCheck_inDrainClose
[ OK ] Clogged close test passed: outClogCheck_inClose
[ OK ] Clogged close test passed: outClogDrainCheck_inClose
[ OK ] Clogged close test passed: outShutDrainCheck_inClose
[ OK ] Clogged close test passed: outShutDrainCheck_inDrainClose
[ OK ] Clogged close test passed: outClogCheck_inClogClose
[ OK ] Clogged close test passed: outDrainCheck_inShutClose
[ OK ] Clogged close test passed: outDrainCheck_inShutDrainClose
Broken pipe (os error 32)
Broken pipe (os error 32)
[ OK ] Clogged close test passed: outShutDrainCheck_inShutClose
[ OK ] Clogged close test passed: outShutDrainCheck_inShutDrainClose
[ OK ] Clogged close test passed: outClose_inDrainCheck
[ OK ] Clogged close test passed: outDrainClose_inDrainCheck
[ OK ] Clogged close test passed: outShutClose_inDrainCheck
[ OK ] Clogged close test passed: outShutDrainClose_inDrainCheck
[ OK ] Clogged close test passed: outClose_inClogCheck
[ OK ] Clogged close test passed: outClose_inClogDrainCheck
[ OK ] Clogged close test passed: outClogClose_inClogCheck
Broken pipe (os error 32)
Broken pipe (os error 32)
[ OK ] Clogged close test passed: outClose_inShutDrainCheck
[ OK ] Clogged close test passed: outDrainClose_inShutDrainCheck
[ OK ] Clogged close test passed: outShutClose_inShutDrainCheck
[ OK ] Clogged close test passed: outShutDrainClose_inShutDrainCheck

In comparison, the tool does not show errors for socat-based forwarder:

$ socat tcp-listen:1234,fork,reuseaddr tcp-connect:127.0.0.1:1235
2022/03/16 11:31:53 socat[21951] E write(5, 0x55b7cf2c7000, 8192): Connection reset by peer
2022/03/16 11:31:53 socat[21954] E write(5, 0x55b7cf2c7000, 8192): Connection reset by peer
2022/03/16 11:31:54 socat[21963] E write(5, 0x55b7cf2c7000, 8192): Connection reset by peer
2022/03/16 11:31:54 socat[21973] E write(5, 0x55b7cf2c7000, 8192): Broken pipe
2022/03/16 11:31:54 socat[21976] E write(5, 0x55b7cf2c7000, 8192): Broken pipe
2022/03/16 11:31:55 socat[22000] E write(6, 0x55b7cf2c7000, 8192): Broken pipe
2022/03/16 11:31:56 socat[22002] E write(6, 0x55b7cf2c7000, 8192): Broken pipe
2022/03/16 11:31:56 socat[22006] E write(6, 0x55b7cf2c7000, 8192): Connection reset by peer
2022/03/16 11:31:56 socat[22010] E write(6, 0x55b7cf2c7000, 8192): Broken pipe
2022/03/16 11:31:57 socat[22012] E write(6, 0x55b7cf2c7000, 8192): Broken pipe

$ tcptunnelchecker 127.0.0.1:1235 127.0.0.1:1234
[ OK ] Trivial test 1
[ OK ] Trivial test 2
[ OK ] Clogged close test passed: outDrainCheck_inClose
[ OK ] Clogged close test passed: outDrainCheck_inDrainClose
[ OK ] Clogged close test passed: outClogCheck_inClose
[ OK ] Clogged close test passed: outClogDrainCheck_inClose
[ OK ] Clogged close test passed: outShutDrainCheck_inClose
[ OK ] Clogged close test passed: outShutDrainCheck_inDrainClose
[ OK ] Clogged close test passed: outClogCheck_inClogClose
[ OK ] Clogged close test passed: outDrainCheck_inShutClose
[ OK ] Clogged close test passed: outDrainCheck_inShutDrainClose
[ OK ] Clogged close test passed: outClogCheck_inShutClose
[ OK ] Clogged close test passed: outClogDrainCheck_inShutClose
[ OK ] Clogged close test passed: outShutDrainCheck_inShutClose
[ OK ] Clogged close test passed: outShutDrainCheck_inShutDrainClose
[ OK ] Clogged close test passed: outClose_inDrainCheck
[ OK ] Clogged close test passed: outDrainClose_inDrainCheck
[ OK ] Clogged close test passed: outShutClose_inDrainCheck
[ OK ] Clogged close test passed: outShutDrainClose_inDrainCheck
[ OK ] Clogged close test passed: outClose_inClogCheck
[ OK ] Clogged close test passed: outClose_inClogDrainCheck
[ OK ] Clogged close test passed: outClogClose_inClogCheck
[ OK ] Clogged close test passed: outShutClose_inClogCheck
[ OK ] Clogged close test passed: outShutClose_inClogDrainCheck
[ OK ] Clogged close test passed: outClose_inShutDrainCheck
[ OK ] Clogged close test passed: outDrainClose_inShutDrainCheck
[ OK ] Clogged close test passed: outShutClose_inShutDrainCheck
[ OK ] Clogged close test passed: outShutDrainClose_inShutDrainCheck

Loopback test tcptunnelchecker 127.0.0.1:1234 127.0.0.1:1234 also shows results similar to socat ones.

[cfal]

very cool, thanks for creating the issue. will take a look.

[vi]

Note that fixing it portably may be problematic due to missing support of waiting for hangups without also waiting for read- or write-readiness. See tokio-rs/tokio#3467 or tokio-rs/mio#1476.