net: tcp: Fix possible deadlock in tcp_in()

After introducing SO_SNDBUF socket option, a possible deadlock situation
slipped into the TCP implementation. The scenario for the deadlock:

  * application thread tries to send some data, it enters
    net_context_send() which locks the context mutex,
  * internal context_sendto() blocks on a TX packet allocation, if the
    TX pool is empty rescheduling takes place,
  * now, if at the same time some incoming packet has arrived (ACK for
    example), TCP stack enters tcp_in() function from a different
    thread. The function locks the TCP connection mutex, and tries to
    obtain the SNDBUF option value. net_context_get_option() tries to
    lock the context mutex, but it is already held by the transmitting
    thread, so the receiver thread blocks
  * when TX packet is available again, the transmitting thread unblocks
    and tries to pass the packet down to TCP stack. net_tcp_queue_data()
    is called which attempts to lock the TCP connection mutex, but it is
    already held by the receiving thread. Both threads are in a deadlock
    now with no chance to recover.

Fix this, by obtaining the SNDBUF option value in tcp_in() before
locking the TCP connection mutex.

Signed-off-by: Robert Lubos <[email protected]>

Author: rlubos

subsys/net/ip/tcp.c

@@ -1744,12 +1744,19 @@ static void tcp_in(struct tcp *conn, struct net_pkt *pkt)
 	struct k_fifo *recv_data_fifo;
 	size_t len;
 	int ret;
+	int sndbuf_opt = 0;
 
 	if (th) {
 		/* Currently we ignore ECN and CWR flags */
 		fl = th_flags(th) & ~(ECN | CWR);
 	}
 
+	if (IS_ENABLED(CONFIG_NET_CONTEXT_SNDBUF) &&
+	    conn->state != TCP_SYN_SENT) {
+		(void)net_context_get_option(conn->context, NET_OPT_SNDBUF,
+					     &sndbuf_opt, NULL);
+	}
+
 	k_mutex_lock(&conn->lock, K_FOREVER);
 
 	NET_DBG("%s", log_strdup(tcp_conn_state(conn, pkt)));
@@ -1783,9 +1790,6 @@ static void tcp_in(struct tcp *conn, struct net_pkt *pkt)
 
 	if (th) {
 		size_t max_win;
-		int sndbuf;
-		size_t sndbuf_len;
-
 
 		conn->send_win = ntohs(th_win(th));
 
@@ -1802,15 +1806,8 @@ static void tcp_in(struct tcp *conn, struct net_pkt *pkt)
 				   CONFIG_NET_BUF_DATA_SIZE) / 3;
 		}
 
-		if (IS_ENABLED(CONFIG_NET_CONTEXT_SNDBUF) &&
-			conn->state != TCP_SYN_SENT &&
-			net_context_get_option(conn->context,
-					       NET_OPT_SNDBUF,
-					       &sndbuf,
-					       &sndbuf_len) == 0) {
-			if (sndbuf > 0) {
-				max_win = sndbuf;
-			}
+		if (sndbuf_opt > 0) {
+			max_win = sndbuf_opt;
 		}
 
 		max_win = MAX(max_win, NET_IPV6_MTU);