large package was cut into several parts?

[bethebest0622]

I was using commit tag 028d4dc2fb59dd90a6997eea04bcaed899adb0fe for a long time.

Recently, I want to follow up the newest version, So I git pull on branch main.

Then something wrong in my code.

When I received data from websocket server (binance), if the package is large, it will call event loop functon (int shm_event_cb(struct lws *wsi, enum lws_callback_reasons reason, void *user, void *in, size_t len)) many times, each part maxium size is 4092.

The data is:

{"action":"snapshot","arg":{"instType":"usdt-futures","topic":"books","symbol":"ASTERUSDT"},"data":[{"a":[["1.1608","958"],["1.1609","666"],["1.1610","4466"],["1.1611","9121"],["1.1612","3454"],["1.1613","4447"],["1.1614","7124"],["1.1615","19424"],["1.1616","7327"],["1.1617","14990"],["1.1618","17562"],["1.1619","8810"],["1.1620","6235"],["1.1621","15662"],["1.1622","9334"],["1.1623","9258"],["1.1624","15474"],["1.1625","4446"],["1.1626","21295"],["1.1627","20473"],["1.1628","4522"],["1.1629","9979"],["1.1630","42868"],["1.1631","8601"],["1.1632","5761"],["1.1633","48504"],["1.1634","23609"],["1.1635","43192"],["1.1636","15759"],["1.1637","40638"],["1.1638","19980"],["1.1639","4527"],["1.1640","18877"],["1.1641","23040"],["1.1642","4458"],["1.1643","3098"],["1.1644","39312"],["1.1645","4619"],["1.1646","1443"],["1.1647","36235"],["1.1648","3951"],["1.1649","27127"],["1.1650","12925"],["1.1651","6933"],["1.1652","111"],["1.1653","7961"],["1.1654","569"],["1.1655","17935"],["1.1656","41817"],["1.1657","100"],["1.1658","19986"],["1.1659","915"],["1.1660","1202"],["1.1661","13317"],["1.1662","19879"],["1.1663","21195"],["1.1664","310"],["1.1665","65"],["1.1666","968"],["1.1667","13613"],["1.1668","3558"],["1.1669","5085"],["1.1670","76709"],["1.1671","7990"],["1.1672","9789"],["1.1673","36670"],["1.1674","26966"],["1.1675","31919"],["1.1676","1128"],["1.1677","153"],["1.1678","1791"],["1.1679","272"],["1.1680","5442"],["1.1681","435"],["1.1682","51"],["1.1683","237"],["1.1684","16281"],["1.1685","8203"],["1.1686","870"],["1.1687","34310"],["1.1688","1171"],["1.1689","257"],["1.1690","987"],["1.1691","4432"],["1.1692","148"],["1.1693","835"],["1.1694","34299"],["1.1695","24233"],["1.1696","31287"],["1.1697","25861"],["1.1698","131"],["1.1699","921"],["1.1700","225"],["1.1701","957"],["1.1702","20015"],["1.1703","26006"],["1.1704","382"],["1.1705","161337"],["1.1706","2"],["1.1707","43351"],["1.1708","113"],["1.1709","59820"],["1.1710","6"],["1.1711","184"],["1.1712","30552"],["1.1713","3591"],["1.1714","103"],["1.1715","73"],["1.1716","23"],["1.1717","13638"],["1.1718","9"],["1.1720","31"],["1.1721","21"],["1.1722","49522"],["1.1723","1330"],["1.1724","9"],["1.1725","20"],["1.1726","384"],["1.1727","10"],["1.1728","2"],["1.1729","27"],["1.1730","28165"],["1.1731","2"],["1.1733","28138"],["1.1735","159"],["1.1736","58"],["1.1738","29"],["1.1739","57723"],["1.1740","19"],["1.1741","49"],["1.1742","9"],["1.1744","62"],["1.1745","35"],["1.1746","30151"],["1.1748","24"],["1.1749","16"],["1.1750","12"],["1.1751","1"],["1.1752","10"],["1.1753","10"],["1.1754","354"],["1.1755","12"],["1.1756","286"],["1.1757","196"],["1.1758","48025"],["1.1759","9"],["1.1760","50344"],["1.1761","15"],["1.1762","68"],["1.1763","27791"],["1.1764","98"],["1.1765","52943"],["1.1766","176"],["1.1767","49"],["1.1768","3"],["1.1769","34"],["1.1770","39137"],["1.1772","57687"],["1.1773","29854"],["1.1774","1"],["1.1775","31"],["1.1777","30370"],["1.1778","126"],["1.1779","30"],["1.1780","60"],["1.1781","48342"],["1.1782","16"],["1.1783","13"],["1.1785","35"],["1.1786","54283"],["1.1788","20"],["1.1789","38141"],["1.1790","879"],["1.1791","3494"],["1.1792","8"],["1.1793","47"],["1.1794","26151"],["1.1795","5"],["1.1796","158"],["1.1797","48"],["1.1800","59768"],["1.1802","143"],["1.1803","64"],["1.1804","15"],["1.1805","1"],["1.1806","13"],["1.1807","23"],["1.1809","72"],["1.1810","26"],["1.1811","91123"],["1.1814","87"],["1.1815","5"],["1.1816","2"],["1.1817","30"],["1.1818","8"],["1.1819","9270"],["1.1820","9"],["1.1822","5"],["1.1823","16"],["1.1824","25"]],"b":[["1.1606","186"],["1.1605","1186"],["1.1604","13572"],["1.1603","4747"],["1.1602","11035"],["1.1601","13507"],["1.1600","3543"],["1.1599","6792"],["1.1598","14426"],["1.1597","7251"],["1.1596","15203"],["1.1595","18123"],["1.1594","18811"],["1.1593","10685"],["1.1592","16123"],["1.1591","13013"],["1.1590","17518"],["1.1589","2608"],["1.1588","18311"],["1.1587","38901"],["1.1586","43754"],["1.1585","12868"],["1.1584","26802"],["1.1583","724"],["1.1582","2660"],["1.1581","19785"],["1.1580","25151"],["1.1579","3371"],["1.1578","13652"],["1.1577","25295"],["1.1576","7724"],["1.1575","25205"],["1.1574","444"],["1.1573","25795"],["1.1572","16577"],["1.1571","49725"],["1.1570","18248"],["1.1569","61156"],["1.1568","11523"],["1.1567","15650"],["1.1566","32701"],["1.1565","2938"],["1.1564","4022"],["1.1563","1519"],["1.1562","744"],["1.1561","1279"],["1.1560","302"],["1.1559","529"],["1.1558","31872"],["1.1557","14168"],["1.1556","6004"],["1.1555","31179"],["1.1554","4978"],["1.1553","225"],["1.1552","946"],["1.1551","4"],["1.1550","13853"],["1.1549","381"],["1.1548","943"],["1.1547","628"],["1.1546","8458"],["1.1545","5204"],["1.1544","69273"],["1.1543","845"],["1.1542","163316"],["1.1541","9965"],["1.1540","3746"],["1.1539","330"],["1.1538","605"],["1.1537","131"],["1.1536","3131"],["1.1535","7"],["1.1534","9143"],["1.1533","46636"],["1.1532","10"],["1.1531","225"],["1.1530","365"],["1.1529","1"],["1.1528","338"],["1.1527","32446"],["1.1526","12476"],["1.1525","775"],["1.1524","41116"],["1.1523","573"],["1.1522","11864"],["1.1521","1"],["1.1520","41"],["1.1519","84"],["1.1518","47481"],["1.1517","298"],["1.1516","30"],["1.1515","313"],["1.1514","1807"],["1.1513","28"],["1.1512","101"],["1.1511","2275"],["1.1510","71"],["1.1509","10"],["1.1508","20687"],["1.1507","26883"],["1.1506","48"],["1.1505","542"],["1.1504","64961"],["1.1503","643"],["1.1502","9"],["1.1501","82"],["1.1500","45115"],["1.1499","12"],["1.1498","59"],["1.1497","29"],["1.1496","23"],["1.1495","23"],["1.1494","178"],["1.1493","13"],["1.1492","30"],["1.1491","20"],["1.1490","437"],["1.1489","86"],["1.1488","23207"],["1.1487","24898"],["1.1486","32061"],["1.1485","212"],["1.1484","29"],["1.1483","3491"],["1.1482","710"],["1.1481","2"],["1.1480","628"],["1.1479","60572"],["1.1478","4"],["1.1477","35"],["1.1476","37"],["1.1475","4"],["1.1474","33"],["1.1473","20"],["1.1472","54306"],["1.1469","24"],["1.1468","15"],["1.1467","101"],["1.1466","105116"],["1.1465","58"],["1.1464","237"],["1.1463","100"],["1.1461","5"],["1.1460","42"],["1.1459","44686"],["1.1458","2"],["1.1457","5"],["1.1456","8"],["1.1455","12"],["1.1454","235"],["1.1453","12"],["1.1452","69"],["1.1451","32"],["1.1450","2933"],["1.1449","22"],["1.1448","12"],["1.1446","2"],["1.1445","17"],["1.1444","301"],["1.1443","94049"],["1.1442","3"],["1.1441","28"],["1.1440","4"],["1.1439","212"],["1.1438","8"],["1.1437","45525"],["1.1436","15"],["1.1435","35"],["1.1434","7"],["1.1433","166"],["1.1432","67"],["1.1431","96"],["1.1430","7"],["1.1429","15"],["1.1428","38"],["1.1426","54666"],["1.1425","18"],["1.1424","21"],["1.1423","47614"],["1.1422","13"],["1.1421","895"],["1.1420","3530"],["1.1418","40"],["1.1417","1"],["1.1416","4"],["1.1415","116"],["1.1414","87"],["1.1413","64"],["1.1412","37"],["1.1411","16"],["1.1410","56232"],["1.1409","23"],["1.1408","70"],["1.1407","3"],["1.1406","22"],["1.1405","92523"],["1.1404","31"],["1.1403","55798"],["1.1402","3"],["1.1401","61879"]],"checksum":0,"pseq":0,"seq":1377211448039727113,"ts":"1764078717241"}],"ts":1764078717368}

The json data size is 9136, the old version will just call event loop function once, the last argument(len) is 9136

the new version will call twice, first one size if 4092, the second size is 3044

I can ensure all the os setting is same, the only change is i git pull branch main.

I have set the protocols as this:

struct lws_protocols protocols[] = {{"ws", &event_cb, 4096, 65535, 1, NULL}, LWS_PROTOCOL_LIST_TERM};

and the context info as this:

info.pt_serv_buf_size = 65535;

the rx_buff is 65535, I think this is enough.

could you let me know, why the new version cut the large packages into several parts? Will this be more effective than old version?

Is there anything I can do to make the large packages not be cut?

Thanks

[lws-team]

It has always been that lws will pass up whatever it has received to the user code as it receives it. There's no guarantee that will consist of a complete message.

You seem to be telling lws that your protocol should only receive up to 4096 bytes at a time (rx_buffer_size in the protocol struct). You can increase this to the same as pt_serv_buf_size to get rid of this cutting at 4KB. But there's still no quarantee your message will come in one callback, especially if the message is > 16KB (due to tls buffer limits).

The best way is to understand you can receive fragments and either copy the pieces sequentially until you have the whole message in memory, or process it in chunks. You can see where you are in a message by using

int
lws_is_final_fragment(struct lws *wsi);

 int
lws_is_first_fragment(struct lws *wsi);

[bethebest0622]

It has always been that lws will pass up whatever it has received to the user code as it receives it. There's no guarantee that will consist of a complete message.

You seem to be telling lws that your protocol should only receive up to 4096 bytes at a time (rx_buffer_size in the protocol struct). You can increase this to the same as pt_serv_buf_size to get rid of this cutting at 4KB. But there's still no quarantee your message will come in one callback, especially if the message is > 16KB (due to tls buffer limits).

The best way is to understand you can receive fragments and either copy the pieces sequentially until you have the whole message in memory, or process it in chunks. You can see where you are in a message by using

int
lws_is_final_fragment(struct lws *wsi);

 int
lws_is_first_fragment(struct lws *wsi);

I can understand there is no gurantee the message will be completed.

But I think the old version has the same problem.

But the fact is:

the old version was always completed.

the new version always be splited if size > 8188

Is there anything may cause the difference in recent commits? or it maybe caused by other external library's change?

[bethebest0622]

and chatgpt's answer:

✅ Why long messages >8192 bytes are now split after updating LWS

When you upgraded to the latest libwebsockets, you inherited a major internal change:

✔ The default per-connection output buffer (LWS_MAX_SOCKET_IO_BUF) was reduced to 8192 bytes.

This value determines how much payload LWS can place into one single WebSocket frame.

In recent LWS versions (5.x):

LWS_MAX_SOCKET_IO_BUF = 8192   // new default

Because of this:

• Any message larger than 8192 bytes

• Cannot fit into a single WebSocket frame inside LWS

• So LWS automatically fragments the message into multiple frames

This is completely normal and expected in the new LWS.

---

❗ Important: WebSocket fragmentation is allowed and LWS does it automatically

The WebSocket protocol explicitly allows fragmentation:

• Large messages may be split across multiple frames

• LWS decides fragmentation based on its internal buffer size

• You cannot force LWS to "never fragment" unless you change its compile-time limits

This behavior is not a bug—it is a design decision.

---

✔ Why the old version did NOT fragment?

Earlier versions of LWS used a larger buffer (e.g. 32 KB), so your messages fit into a single frame.

After upgrading, the smaller default buffer (8192 bytes) causes messages to be split.

---

🔧 Can you stop fragmentation completely?

❌ No — you cannot completely disable fragmentation in libwebsockets

(Unless you bypass LWS and write your own WS framing logic.)

But you can make fragmentation rare or even “invisible” by increasing the buffer limit.

---

✔ Solution: Increase the LWS send buffer size (recommended)

Rebuild libwebsockets with a larger LWS_MAX_SOCKET_IO_BUF.

Example: set it to 64 KB

CMake:

cmake .. -DLWS_MAX_SOCKET_IO_BUF=65536

or add to your build:

#define LWS_MAX_SOCKET_IO_BUF 65536

This allows one WebSocket message frame up to 64 KB.

---

✔ Larger options (choose one)

Maximum frame size | Compile flag -- | -- 64 KB | -DLWS_MAX_SOCKET_IO_BUF=65536 128 KB | -DLWS_MAX_SOCKET_IO_BUF=131072 256 KB | -DLWS_MAX_SOCKET_IO_BUF=262144

After rebuilding, messages up to the new limit will no longer be fragmented.

---

✔ Alternative: Reassemble at application level (practical recommendation)

Even if you increase the buffer, fragmentation may still occur:

• Some TLS implementations fragment internally

• Kernel socket buffers may split writes

• Some proxies split frames

So the most robust solution is:

• Read data across multiple callbacks

• Reassemble fragments until LWS indicates the final frame (LWS_WRITE_NO_FIN vs FIN)

This is how most production WebSocket code works.

---

📌 Final Summary

Why fragmentation happens now:
Because new LWS uses 8192-byte output buffer, so long messages are automatically split.

Can you prevent fragmentation:
Not completely, but you can reduce it dramatically.

How to reduce it:
Recompile LWS with a larger LWS_MAX_SOCKET_IO_BUF, e.g.:

-DLWS_MAX_SOCKET_IO_BUF=65536

Best practice:
Handle fragmentation in your application logic.

The complie flag LWS_MAX_SOCKET_IO_BUF not exisited, but I think there may be another flag/setting can do the similiar thing?

[lws-team]

Lol... so there's both your insistence black is white, and you paste a totally hallucinated fabrication by an LLM (which you admit to not have any relationship to reality), and little old me who merely conceived and wrote it, raising my hand and explaining how it works just bounces off.

Did you change the 4096 limit for your protocol, or since whatever the author says is worth less than chatgpt's pink slime outpouring, did that seem not worth attending to?

[laoshanxi]

this is epoll, not select, I recieving data what ever large or small like this: https://github.com/laoshanxi/app-mesh/blob/main/src/common/websockets/WebSocketService.cpp#L437

[lws-team]

... if you use tls, data up to the tls buffer size (by default 16KB) limit is pre-sent and the crypto done when the encrypted data is all present on the receiving side. It's decrypted and 'received' all at once.

If you're on mbedtls these buffers can be set to smaller sizes creating frag limits.

Without tls, you get what was received at the time, nothing stops packets being delayed or lws event loop going around before all packets of the message were sent or reveived, no matter the rx buf size.

For arbitrary size messages including tls and endless / huge messages, you cannot expect to receive them atomically.

[bethebest0622]

Apologize for the mess answer of llm

I think I am using TLS, as the address is wss://xxxx

But according to your describption, It seems the old version used TLS(so the event LWS_CALLBACK_CLIENT_RECEIVE will only called when last message arrive), the new version, for some reason, it failed to use tls

I have set the protocols as this:

struct lws_protocols protocols[] = {{"ws", &event_cb, 4096, 65535, 1, NULL}, LWS_PROTOCOL_LIST_TERM}; // rx_buffer is 65535

and the context info as this:

info.pt_serv_buf_size = 65535;

The setting is same, only difference is libwebsocket's version.