feat(lua): Lua binding subsystem (hloop/event/http/ws/redis/mqtt) + redis TcpClient refactor - #857
Open
ithewei wants to merge 33 commits into
Open
feat(lua): Lua binding subsystem (hloop/event/http/ws/redis/mqtt) + redis TcpClient refactor#857ithewei wants to merge 33 commits into
ithewei wants to merge 33 commits into
Conversation
…/hv.core/hv.dns bindings + hvlua runtime
Run HTTP Lua handlers in a per-request coroutine on the server IO thread's per-loop lua_State, so scripts use synchronous-style async APIs (hloop.sleep, hv.dns.resolve) that yield to the loop and complete the response via the async HttpResponseWriter. Adds hvlua_start_task scheduler entry, per-(loop,file) script env cache with hot reload, and http_lua_async_test proving concurrent non-blocking execution.
…th server IO thread hvlua now wraps its hloop in an hv::EventLoop and publishes it as the thread's currentThreadEventLoop, matching the environment lua bindings see inside an HTTP server IO thread. This is the correct foundation for future hv.* client bindings that resume coroutines via EventLoop::queueInLoop.
Per the two-layer design, the lower layer (coroutine scheduler, hloop.* timers/ sleep/run/stop, hv.dns) is now plain C with no C++ dependency: std::map task tracking replaced by a registry slot keyed on the coroutine pointer, new/delete replaced by HV_ALLOC/HV_FREE. Only hv.json (lua_hv_core.cpp) stays C++ for nlohmann::json. hv_lua.h is now dual-mode (C via BEGIN/END_EXTERN_C, C++ via hv:: using-aliases). Verified: pure-C99 strict compile with no C++ symbols, Makefile+CMake builds, all lua unittests and hvlua examples pass.
…_json hv.log/hv.now move to lua_hv_base.c (pure C, bounded stack buffer instead of std::string); hv.json stays in lua_hv_json.cpp (nlohmann). hvlua_state now calls hvlua_open_base + hvlua_open_json. lua/ now has a single C++ file (json), the rest is pure C. Verified: Makefile+CMake compile base.c as C / json.cpp as C++, all lua unittests pass, hv.log + hv.json roundtrip work in hvlua.
- Merge async DNS into the event module (event/ layer): hv.dns.resolve ->
hv.resolveDns, alongside hv.setTimeout/setInterval/clearTimer/sleep/run/stop.
- Collapse the hloop.* global into the single hv.* table; every script-facing
function now lives under hv.* (loop primitives operate on the current thread's
loop).
- Rename files to match the hvlua_* symbol/executable prefix and mirror libhv's
C-layer dirs: hv_lua.{c,h}->hvlua.{c,h}, lua_hloop.c->hvlua_event.c (event/),
lua_hv_base.c->hvlua_base.c (base/), lua_hv_json.cpp->hvlua_json.cpp (cpputil/);
lua_hv_dns.c removed (merged into event). Registration order base->event->json.
- base module: drop hv.now (Lua has os.time), add hv.version (first), and add
log levels hv.logd/logi/logw/loge with hv.log as an alias of logi.
- Update examples, scripts, docs, and unittests to the hv.* API.
Verified: pure-C strict compile of hvlua*.c with no C++ symbols; Makefile+CMake
build (base/event as C, json as C++); all lua unittests and hvlua examples pass;
hv.version/hv.log levels confirmed at runtime.
Add hv.connect / hv.tcpServer / hv.udpClient / hv.udpServer and conn/sock
methods (read/readline/readuntil/readbytes/setUnpack/write/close/peeraddr/fd,
sendto/recvfrom) to hvlua_event.c. All built directly on event/ hio (no evpp,
no extra thread): each op suspends the running coroutine and resumes on the
same loop thread when the hio callback fires, so scripts write synchronous-style
non-blocking IO. TCP server runs on_conn(conn) in a fresh coroutine per accepted
connection; conn:setUnpack maps to hio_set_unpack for framed-packet reads.
lua_io_test covers TCP echo, length_field unpack, and UDP echo on a single loop
(no external processes). examples/lua/{tcp,udp}_{client,server}.lua + tcp_unpack.
hvlua runtime now line-buffers stdout so long-running server scripts log promptly.
Verified: Makefile + CMake builds (hvlua_event.c compiled as C), all lua
unittests pass, connect-failure path returns nil,err safely.
Add hv.http.{get,post,put,delete,request} to the Lua runtime. Requests are
issued through a per-lua_State AsyncHttpClient bound to the current loop
(currentThreadEventLoopPtr), so completion callbacks fire on the same loop
thread and resume the suspended coroutine directly — no cross-thread hop,
single-loop model. Enabled via -DHVLUA_WITH_HTTP (WITH_LUA && WITH_HTTP).
Supporting changes:
- EventLoop: inherit enable_shared_from_this and add currentThreadEventLoopPtr
so a raw EventLoop* (from TLS) can recover the owning EventLoopPtr to hand to
AsyncHttpClient. Falls back to NULL for non-shared (stack) loops.
- EventLoop::run(): null an owned (AUTO_FREE) loop_ after hloop_run returns,
since hloop_run already freed it; prevents a dangling loop_ when the loop was
stopped via the raw C hloop_stop() (e.g. Lua hv.stop()) instead of stop().
- AsyncHttpClient: add is_loop_owner so an externally-provided loop is not
stopped/freed on destruction (mirrors evpp/TcpClient). Clear per-connection
onclose before member teardown to avoid a keep-alive UAF where ~channels ->
~Channel -> close -> onclose touches the already-destroyed conn_pools map.
- hvlua / example runtime now use make_shared<EventLoop>() directly instead of
hloop_new(0) + wrapper + manual TLS/free.
Tests: unittest/lua_http_test.cpp (in-process server + hv.http.get) wired into
Makefile, CMake and scripts/unittest.sh; examples/lua/http_client.lua demo.
Consolidate the duplicated is_loop_owner flag out of each subclass (TcpClient/TcpServer/UdpClient/UdpServer) into EventLoopThread, exposed via isLoopOwner(). The base ctor derives it from loop==NULL, and stop() guards on ownership so it is a no-op for an external, not-yet-running loop the caller owns. AsyncHttpClient, AsyncRedisClient and RedisSubscriber now query isLoopOwner() instead of carrying their own copy.
AsyncRedisClient and RedisSubscriber now inherit TcpClientTmpl<SocketChannel> (like WebSocketClient) instead of composing a TcpClientEventLoopTmpl member. The base class provides connect/reconnect/DNS/loop-ownership, so the duplicated tcp_client member, the mirrored host/port/timeout/reconnect fields, and every isLoopOwner() branch are gone; each Impl keeps only the RESP protocol layer (handshake, request pipelining, reply dispatch) driven through the inherited channel/send/startConnect/setReconnect API. This unifies external-loop semantics with TcpClient: starting on an external, not-yet-running loop now spins the client's own worker thread instead of the old redis-specific ERR_CONNECT rejection. redis_async_client_test is updated to assert the unified contract. All redis unit tests (protocol/async_client/client/batch/subscriber) pass.
Add three script-facing client modules under the hv.* table, all following
the single-loop coroutine-synchronous model of hv.http (client bound to the
current loop; completion callbacks fire on the same thread and resume the
coroutine directly, no cross-thread hop):
- hv.redis: hv.redis.new{host,port,auth,db,timeout} + r:command(...) and
get/set/del/incr/... sugar; RESP reply -> Lua value, error reply -> nil,err.
- hv.ws: hv.ws.connect(url[,headers]) + ws:send/recv/close; message-driven, so
inbound frames are buffered and ws:recv() suspends until one arrives.
- hv.mqtt: hv.mqtt.connect{...} + m:publish/subscribe/unsubscribe/recv/
disconnect; binds the raw mqtt_client_t C API and installs its own dispatch
callback (MqttClient::run() would block the shared loop), recv() buffered.
Modules are conditionally compiled on HVLUA_WITH_HTTP/REDIS/MQTT (set by the
build when WITH_HTTP/REDIS/MQTT are on) and registered by hvlua_state().
Tests: lua_redis_test (FakeRedisServer, full command + error mapping),
lua_ws_test (in-process WebSocket echo server, send/recv round-trip),
lua_mqtt_test (registration + connect-failure path). Example scripts under
examples/lua/. All pass.
Two code-review findings on the lua-binding work: 1. CMake never defined HVLUA_WITH_HTTP/REDIS/MQTT (Makefile.in did). Under a CMake build (CI Windows/macOS/Android/iOS) the http/redis/ws/mqtt binding bodies were #ifdef'd out, so hv.http/redis/ws/mqtt silently became nil. Add the definitions in the WITH_LUA branch, gated on WITH_HTTP/REDIS/MQTT, mirroring Makefile.in. 2. hv.redis's __gc deleted the client without first neutralizing in-flight command callbacks. Destroying the client runs ~AsyncRedisClient -> stop -> failPending synchronously on the loop thread (i.e. from inside the __gc metamethod), firing the pending lua callback which called lua_resume -- re-entering the VM from within GC. Add a 'destroyed' flag on the box, set it before delete, and have the command callback release the suspend token (hvlua_cancel, __gc-safe) instead of resuming when destroyed. This matches the teardown-safety pattern already used by hv.ws / hv.mqtt / hvlua_event. Verified: make libhv + redis_async_client/batch/subscriber + lua_redis tests all pass.
…iases Expose the event-layer client/server entries under names mirroring the C++ classes (hv.tcpClient / hv.tcpServer / hv.udpClient / hv.udpServer) for a consistent client/server x tcp/udp matrix, and keep the idiomatic verbs hv.connect (= tcpClient) and hv.listen (= tcpServer) as aliases. Examples use the connect/listen aliases. redis keeps hv.redis.new (its 'new' is a pure constructor with lazy/queued connect + reconnect, unlike ws/mqtt connect() which suspends until the handshake completes, so the naming difference is intentional). lua_io_test unchanged: it already covers tcpServer (primary) + connect (alias) + udpServer/udpClient. All lua/redis tests pass.
The test bound a stack EventLoop to TLS. It works today (the scripts finish synchronously and never reach a client binding), but is inconsistent with the other lua tests (lua_http/ws/redis/mqtt all use make_shared) and would silently trap a future script that calls hv.http/redis/ws/mqtt: currentThreadEventLoopPtr -> shared_from_this() on a stack object throws bad_weak_ptr and returns nullptr, so the binding would report 'no shared loop' instead of actually running. Switch to make_shared<EventLoop> and store loop.get() in TLS (TLS holds a raw EventLoop*, matching EventLoop::run). ~EventLoop frees the never-run AUTO_FREE hloop via stop(), so no leak. lua_binding_test / lua_io_test are unaffected (pure hloop_new, no EventLoop object).
Rename the two HTTP Lua handler tests so each name reflects what it verifies: - http_lua_handler_test.cpp (synchronous HttpScriptHandler unit test: .lua dispatch, .py -> 501, script-dir routing, path-traversal guard, ctx API) -> http_script_handler_test.cpp (its subject is HttpScriptHandler, the language-agnostic dispatcher) - http_lua_async_test.cpp (real HttpServer + concurrent requests proving .lua handlers run as coroutines concurrently on one IO thread) -> http_lua_handler_test.cpp Update the file bodies (headers, tmp dirs, PASSED banners) and all build wiring: Makefile, unittest/CMakeLists.txt, scripts/unittest.sh. Both tests build and pass under Make and CMake.
The configure script's module help text didn't mention the lua build option added for the Lua binding. Add the --with-lua line so ./configure --help documents it alongside --with-http/mqtt/redis.
- docs/cn/lua.md: user-facing reference for the hv.* Lua API — build/run, the coroutine-synchronous model, return-value convention, and every module (hv.log/json, timers/sleep, resolveDns, tcp/udp, http, ws, redis, mqtt). - docs/cn/README.md: link it under a new 'lua接口' section. - docs/PLAN.md: move 'lua binding' from Plan to Done.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a coroutine-based Lua binding subsystem for libhv (per-loop lua_State, yielding/resuming to keep the event loop non-blocking), refactors Redis clients to build on the unified TcpClientTmpl<SocketChannel> model, and updates build/test scaffolding plus documentation/examples to support the new Lua runtime (hvlua) and bindings.
Changes:
- Add Lua core runtime/scheduler and
hv.*bindings for timers/sleep/DNS + TCP/UDP + JSON, plus optional HTTP/WS/Redis/MQTT bindings behindHVLUA_WITH_*. - Rework
HttpLuaHandlerto run per-request coroutines on the server IO thread using the per-looplua_State, completing responses asynchronously when scripts yield. - Refactor
AsyncRedisClientandRedisSubscriberto inheritTcpClientTmpl<SocketChannel>and adjust tests/build scripts accordingly.
Reviewed changes
Copilot reviewed 60 out of 60 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| unittest/redis_async_client_test.cpp | Updates external-loop expectations for Redis client under unified TcpClient model. |
| unittest/lua_ws_test.cpp | Adds end-to-end Lua ws binding test against an in-process echo server. |
| unittest/lua_redis_test.cpp | Adds Lua redis binding test using FakeRedisServer. |
| unittest/lua_mqtt_test.cpp | Adds Lua mqtt binding smoke test validating failure path without a broker. |
| unittest/lua_io_test.cpp | Adds deterministic Lua event-layer IO tests (tcp/udp + unpack). |
| unittest/lua_http_test.cpp | Adds Lua http binding test against an in-process HttpServer. |
| unittest/lua_binding_test.cpp | Adds core Lua binding tests (json/timers/sleep/coroutine interleave). |
| unittest/http_script_handler_test.cpp | Adds tests for script handler routing + safety behavior. |
| unittest/http_lua_handler_test.cpp | Replaces prior unit-style checks with coroutine concurrency integration test. |
| unittest/CMakeLists.txt | Adds Lua-related unittest executables and ties them into the CMake unittest target. |
| scripts/unittest.sh | Runs new Lua unittests when present. |
| redis/RedisSubscriber.h | Switches subscriber to inherit from TcpClientTmpl<SocketChannel>. |
| redis/RedisSubscriber.cpp | Refactors subscriber implementation to use inherited TcpClient transport. |
| redis/AsyncRedisClient.h | Switches async client to inherit from TcpClientTmpl<SocketChannel>. |
| redis/AsyncRedisClient.cpp | Refactors async client implementation to use inherited TcpClient transport. |
| Makefile.in | Adds HVLUA_WITH_* gating macros for module-aware Lua bindings. |
| Makefile | Adds lua sources/headers and builds hvlua; adjusts unittest/exemplar build rules. |
| lua/hvlua.h | Defines public Lua binding API (state/task/suspend/resume) usable from C/C++. |
| lua/hvlua.c | Implements per-loop Lua state lifecycle + coroutine scheduler/task runner. |
| lua/hvlua_ws.cpp | Implements coroutine-synchronous hv.ws with inbox + recv() yielding. |
| lua/hvlua_redis.cpp | Implements coroutine-synchronous hv.redis with reply-to-Lua mapping + sugar. |
| lua/hvlua_mqtt.cpp | Implements coroutine-synchronous hv.mqtt using C mqtt client API + inbox. |
| lua/hvlua_json.cpp | Adds hv.json.encode/decode backed by nlohmann::json. |
| lua/hvlua_http.cpp | Adds coroutine-synchronous hv.http.* using per-state AsyncHttpClient. |
| lua/hvlua_event.c | Adds hv.setTimeout/setInterval/clearTimer/sleep/resolveDns + tcp/udp bindings. |
| lua/hvlua_base.c | Adds hv.version and hv.log* base utilities. |
| http/server/HttpLuaHandler.h | Documents the new per-loop coroutine execution model and simplifies state. |
| http/server/HttpLuaHandler.cpp | Reworks script loading/caching per loop + coroutine execution + async response flush. |
| http/client/AsyncHttpClient.h | Fixes destructor ordering/UAF by clearing per-channel close callbacks. |
| examples/scripts/async.lua | Adds async-yielding script handler demo (sleep + resolveDns). |
| examples/lua/ws_client.lua | Adds Lua ws client demo script. |
| examples/lua/udp_server.lua | Adds Lua UDP echo server demo script. |
| examples/lua/udp_client.lua | Adds Lua UDP client demo script. |
| examples/lua/timer.lua | Adds Lua timer demo script. |
| examples/lua/tcp_unpack.lua | Adds Lua TCP unpack/framing demo script. |
| examples/lua/tcp_server.lua | Adds Lua TCP echo server demo script. |
| examples/lua/tcp_client.lua | Adds Lua TCP client echo demo script. |
| examples/lua/sleep.lua | Adds Lua coroutine sleep/concurrency demo script. |
| examples/lua/redis_client.lua | Adds Lua redis client demo script. |
| examples/lua/mqtt_client.lua | Adds Lua mqtt client demo script. |
| examples/lua/http_client.lua | Adds Lua http client demo script. |
| examples/lua/dns.lua | Adds Lua async DNS demo script. |
| examples/hvlua.cpp | Adds standalone hvlua runtime on top of EventLoop + per-loop Lua state. |
| examples/http_server_test.cpp | Adds a comment/example callout for async script handler. |
| examples/CMakeLists.txt | Builds and installs hvlua example when WITH_LUA is enabled. |
| evpp/UdpServer.h | Removes duplicated loop-ownership flag; relies on EventLoopThread ownership rule. |
| evpp/UdpClient.h | Removes duplicated loop-ownership flag; relies on EventLoopThread ownership rule. |
| evpp/TcpServer.h | Removes duplicated loop-ownership flag; relies on EventLoopThread ownership rule. |
| evpp/TcpClient.h | Removes duplicated loop-ownership flag; relies on EventLoopThread ownership rule. |
| evpp/EventLoopThread.h | Centralizes loop-ownership via isLoopOwner() and adjusts stop() guard. |
| evpp/EventLoop.h | Adds enable_shared_from_this + TLS shared_ptr accessor; clears owned loop_ after run. |
| event/hloop.h | Adds per-loop opaque lua_state storage API. |
| event/hloop.c | Destroys per-loop lua_state during loop cleanup and implements getters/setters. |
| event/hevent.h | Stores lua_state + destructor in hloop_t. |
| docs/PLAN.md | Updates plan to mark Lua binding as done. |
| docs/cn/README.md | Links new Lua binding documentation. |
| docs/cn/lua.md | Adds comprehensive Chinese documentation for hv.* Lua API and build/run instructions. |
| docs/cn/HttpLuaHandler.md | Updates HttpLuaHandler docs to reflect coroutine model + expanded hv.* API. |
| configure | Adds --with-lua configure flag. |
| CMakeLists.txt | Adds HVLUA_WITH_* definitions and includes lua sources/headers when enabled. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+173
to
+176
| HV_ALLOC_SIZEOF(s); | ||
| s->co = hvlua_suspend(L); | ||
| s->timer = htimer_add(loop, on_sleep_timer, ms, 1); | ||
| hevent_set_userdata(s->timer, s); |
Comment on lines
+183
to
+207
| HvLuaCoroutine* co = hvlua_suspend(L); | ||
| box->client->command(cmd, [co, box](const RedisResult& result) { | ||
| // If the client is being destroyed (this callback fires synchronously | ||
| // from ~AsyncRedisClient -> stop -> failPending, which runs inside the | ||
| // Lua __gc metamethod), we must NOT lua_resume — that would re-enter the | ||
| // VM from within GC. Just release the suspend token (hvlua_cancel is | ||
| // __gc-safe: luaL_unref + free, no lua_resume). | ||
| if (box->destroyed) { hvlua_cancel(co); return; } | ||
| lua_State* cur = hvlua_coroutine_state(co); | ||
| if (cur == NULL) { hvlua_cancel(co); return; } // coroutine gone | ||
| if (result.code != 0) { | ||
| lua_pushnil(cur); | ||
| lua_pushfstring(cur, "hv.redis: request failed (%d)", result.code); | ||
| hvlua_resume(co, 2); | ||
| return; | ||
| } | ||
| if (result.reply.isError()) { | ||
| lua_pushnil(cur); | ||
| lua_pushlstring(cur, result.reply.str.data(), result.reply.str.size()); | ||
| hvlua_resume(co, 2); | ||
| return; | ||
| } | ||
| reply_push(cur, result.reply); | ||
| hvlua_resume(co, 1); | ||
| }); |
Comment on lines
93
to
+97
| if(WITH_LUA) | ||
| add_executable(lua_binding_test lua_binding_test.cpp) | ||
| target_include_directories(lua_binding_test PRIVATE .. ../base ../ssl ../event ../cpputil ../evpp ../lua) | ||
| target_link_libraries(lua_binding_test ${HV_LIBRARIES}) | ||
| add_executable(lua_io_test lua_io_test.cpp) |
All four review comments verified against source and fixed: 1. HttpLuaHandler push_script_env: the lua_pcall failure path popped 3 stack slots (err, env, chunk) but the stack held 4 (scripts, chunk, env, err), so the SCRIPTS_REG table leaked on the stack for every script with a runtime load error, accumulating until stack corruption. Pop 4. 2. hv.sleep: htimer_add() could return NULL (NULL loop / out of resources) and was dereferenced by hevent_set_userdata, and the coroutine would already be suspended with nothing to wake it. Create the timer before suspending and fail fast with (nil, err) if it is NULL. 3. hv.redis command: AsyncRedisClient::command may invoke the callback synchronously (e.g. loop not running -> enqueue rejected). The binding suspended the coroutine before calling command(), so a synchronous callback called lua_resume on the still-running coroutine (illegal) and the coroutine then yielded forever. Track sync vs async completion: on synchronous completion, return the results directly without yielding. 4. CMake only built lua_http_test (under WITH_HTTP); lua_ws_test / lua_mqtt_test / lua_redis_test were Make-only, so the new ws/mqtt/redis bindings had no CMake test coverage. Add them under WITH_HTTP / WITH_MQTT / WITH_REDIS. Verified: full lua/redis suite passes under both Make and CMake.
The httpd example now reads a [script] section from httpd.conf and maps each
URL prefix to a script directory via HttpService::Script, mirroring how the
[proxy] section maps reverse-proxy routes. A request to /script/foo then runs
examples/scripts/foo.lua's get/post/... or handle(ctx).
- httpd.cpp: iterate ini.GetKeys("script"), ltrim the '=>' value, call
g_http_service.Script(path, dir). Guarded by #ifdef WITH_LUA (Script is only
compiled with WITH_LUA); without it, a non-empty [script] section logs a
warning instead of silently doing nothing.
- etc/httpd.conf: point /script/ at examples/scripts/ (which has handle(ctx)
scripts) instead of examples/lua/ (standalone hvlua runtime scripts), and
document the mapping.
Verified end-to-end: built with WITH_LUA, GET/POST /script/hello run the lua
get/post handlers, /script/async exercises coroutine-synchronous hv.resolveDns.
Verified each against source before fixing. C1 (data loss / hang): l_conn_read* issued hio_read_until_* BEFORE arming the read callback + suspending. hio_read_until_length/delim invoke the read callback INLINE when buffered data already satisfies the request (hevent.c:765/789), at which point c->co is unset -> on_conn_read dropped the data and the coroutine suspended forever. Now arm callback + enter a 'reading' window first, issue the read, and if it completed synchronously return the data directly without suspending (mirrors the redis RedisCmdState pattern). Regression-tested with pipelined readbytes. C2 (UAF + double-free): a fired once-timer's htimer_t is freed by the loop after its callback, but on_lua_timer freed LuaTimer without clearing the timer's back-pointer, and the script still held the raw handle. clearTimer(stale) then read hevent_userdata of freed memory -> UAF + double-free. Add a live-timer registry; clearTimer validates the handle against it (stale -> safe no-op), and lua_timer_free unregisters + clears hevent_userdata. C3 (UAF): mqtt_client_free freed cli without closing cli->io or deleting cli->timer or clearing their hevent_userdata(cli); a later on_close / connect_timeout_cb / reconnect_timer_cb dereferenced the freed cli. Tear down io (detach close cb + close) and timer (detach + del) inside mqtt_client_free. H2 (UAF): ~WebSocketClient didn't clear channel->onclose, so ~Channel -> close() fired the onclose lambda capturing the destroying TcpClient. Clear it first (mirrors AsyncHttpClient::~AsyncHttpClient). H4 (leak): on_dns_resolved returned without hvlua_cancel when the coroutine was gone, leaking the suspend token. Cancel it. M1 (CI gap): scripts/unittest.sh ran lua_http_test but not lua_ws_test / lua_mqtt_test / lua_redis_test. Add them. Verified: full lua/redis suite (12 tests) passes under Make; clean rebuild.
mqtt_client_run/stop unconditionally called hloop_run/hloop_stop on cli->loop, even when the loop was supplied by the caller. Driving or stopping a loop the client doesn't own is wrong: for an external loop the caller already runs it (e.g. an IO thread or a runtime loop), so mqtt_client_run would block/double- drive it and mqtt_client_stop would stop someone else's loop. Track ownership with an is_loop_owner flag (set when mqtt_client_new creates its own loop, i.e. loop==NULL), aligned with the evpp EventLoopThread model. mqtt_client_run/stop are now no-ops for a caller-supplied loop; the self-created loop path (examples/mqtt: MqttClient with no loop) is unchanged and still auto-frees via HLOOP_FLAG_AUTO_FREE. mqtt_client_free needs no change. This is what the hv.mqtt Lua binding relies on: it binds the client to the shared runtime loop and never calls mqtt_client_run, so run/stop must leave that loop alone. Verified: libhv builds; examples/mqtt links; full lua/redis suite passes.
Pin down the (previously implicit) contract in EventLoopThread: when an external loop is supplied to the constructor, the caller must have it already running before start(). This is what all real usages do (HttpServer/EventLoopThreadPool loops; the Lua bindings obtain the loop via currentThreadEventLoopPtr, which is only non-NULL on an already-running loop thread). If a not-yet-running external loop is passed and start() is called, start() intentionally spins its own worker thread to drive it and stop() joins that thread — a deliberate, safe fallback (exercised by redis_async_client_test), not a bug. Documenting it so the behavior isn't repeatedly misread as a lifetime defect. Comment-only; no logic change.
…fter finish Second-review Point 1 (fragility, not a bug): hvlua_start_task called hvlua_task_step(co) then task_get(co) to decide sync-vs-async completion. But if the task finished synchronously, hvlua_task_step already released the coroutine's thread ref (luaL_unref), so re-reading co via task_get relied on 'GC won't run between these two lines' — safe in single-threaded Lua with no intervening allocation, but fragile. Make hvlua_task_step return 1 (finished) / 0 (yielded) and have hvlua_start_task use that return value instead of touching co again. The other caller (hvlua_resume) ignores the return, unchanged. Comment documents that callers must not touch co after a 'finished' return. Verified: full lua/redis suite passes (lua_binding_test covers sync + async task completion).
Expose the underlying WebSocketClient / mqtt_client reconnect capability through
the bindings, with a symmetric config-table API:
hv.ws.connect(url, { headers=, ping_interval=, reconnect={min_delay, max_delay,
delay_policy, max_retry} })
hv.mqtt.connect({ ..., reconnect={min_delay, max_delay, delay_policy, max_retry} })
Semantics (both):
- reconnect sub-table -> reconn_setting_t, applied via setReconnect (ws, before
open) / mqtt_client_set_reconnect (mqtt, before connect).
- On (re)connect, connected is set and closed is reset, so recv()/send() resume
working after a reconnect.
- While disconnected with reconnect enabled, recv()/send() return
(nil,reconnecting) — a transient signal distinct from the terminal
(nil,closed) — so a script can keep waiting or bail, and send() never
silently drops during a gap.
- Explicit ws:close() / m:disconnect() are terminal: they disable reconnect so
recv() reports closed.
hv.ws.connect's 2nd arg is now an opts table (headers moved under opts.headers);
existing hv.ws.connect(url) callers are unaffected. lua_ws_test extended to
exercise the opts/reconnect path; full-drop reconnect behavior is covered by the
C++ websocket_client_test example. Docs + example scripts updated.
Verified: full lua/redis suite (12 tests) passes; example scripts run.
All reproduced locally before fixing; regression tests added. A1 json cyclic-ref segfault: hv.json.encode / ctx:json recursion had no depth limit — a self-referential table (t.self=t) recursed until stack overflow. Add a depth cap AND lua_checkstack per level (the Lua value stack, not just C stack, was overflowing -> crash in Lua's table internals). A2 non-UTF-8 abort (remote-triggerable): json.dump throws type_error.316 on invalid UTF-8; it was uncaught -> process abort. Reachable via an HTTP handler doing ctx:json(user_input). Wrap dump in try/catch: hv.json.encode returns (nil,err); http dump_json returns empty. (dump_json in http_content.cpp is master-level but is the remote sink, so guarded here.) A3 timer UAF: add_lua_timer stored the CALLING coroutine's lua_State in lt->L; if that coroutine was GC'd before the timer fired (e.g. a timer registered inside another timer's callback), on_lua_timer's lua_newthread(lt->L) was a use-after-free. Store the per-loop main state (hloop_lua_state), matching on_server_accept / on_udp_server_read. B lua_yieldk skips C++ destructors: lua_yieldk longjmps (liblua is C) and never unwinds C++ frames, so any non-trivial local alive at the yield leaks every call. Confined the C++ objects to a scope ending before the yield at all four sites: hvlua_http (shared_ptr<HttpRequest>), hvlua_ws (http_headers), hvlua_redis (shared_ptr<RedisCmdState> + RedisCommand), hvlua_mqtt (EventLoopPtr + std::strings). Documented the rule in hvlua.h. UDP recvfrom inline-callback: l_udp_recvfrom suspended before hio_read, but hio_read can invoke the read callback inline (buffered datagram), resuming a not-yet-yielded coroutine. Reuse the TCP begin/end reading-window (reading_L) so a synchronous datagram is returned directly. Refactor: the duplicated lua<->json conversion (hvlua_json.cpp + HttpLuaHandler) is unified into lua/hvlua_json.h + hvlua_json.cpp (hv::hvlua_lua_to_json / hvlua_json_to_lua), so the A1/A2 guards live in ONE place; HttpLuaHandler reuses it. Tests: lua_binding_test gains cyclic-json / non-utf8-json / timer-in-callback+GC regressions (each crashed pre-fix). Full lua/redis suite (12) passes.
…ed helper)
hv.ws and hv.mqtt each parsed the reconnect config sub-table
({min_delay,max_delay,delay_policy,max_retry} -> reconn_setting_t) with
near-identical code. Move it to a shared pure-C helper hvlua_parse_reconnect in
lua/hvlua_util.{h,c} that both reuse. Pure C (C-includable) so future C or
C++ bindings can share it too; C++-only helpers stay in hvlua_json.h.
No behavior change; ws/mqtt/binding/io tests pass.
…lass The __gc + __index=self + luaL_setfuncs metatable setup was byte-identical across hv.ws / hv.mqtt and near-identical in hv.redis (plus verb sugar). Consolidate into hvlua_new_class() in the shared hvlua_util helper; redis reuses it and only appends its verb closures when the mt is newly created.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a coroutine-synchronous Lua binding subsystem for libhv, and refactors the
redis clients onto TcpClient.
Scripts express async IO in a synchronous style (
local data = conn:read(),local resp = hv.http.get(url),local v = r:get(k)) while the event loopnever blocks — bindings
yieldinternally and are resumed on the same loopthread when the libhv async callback fires (OpenResty-style model).
Lua bindings (
WITH_LUA)hloop.*+hv.*): per-looplua_State,coroutine scheduler (suspend/resume),
hv.setTimeout/setInterval/clearTimer/sleep,hv.resolveDns,hv.json,hv.log*,hv.run/stop; standalonehvluaruntime +
examples/lua/*.lua.hv.tcpClient / tcpServer / udpClient / udpServer(aliaseshv.connect/hv.listen), coroutine-synchronousconn:read/readbytes/ readuntil/readline/write/close+setUnpack, andsendto/recvfromfor UDP.hv.http(needsWITH_HTTP),hv.ws(WITH_HTTP),hv.redis(WITH_REDIS),hv.mqtt(WITH_MQTT) — all coroutine-synchronous.HttpLuaHandlerreworked to a per-looplua_State+ coroutine execution onthe IO thread (async writer completes the deferred response).
redis refactor
AsyncRedisClient/RedisSubscribernow inheritTcpClientTmpl<SocketChannel>(likeWebSocketClient) instead of composing aTcpClientEventLoopTmplmember — dropping the duplicated tcp_client member,the mirrored host/port/timeout/reconnect fields, and every
isLoopOwner()branch. Each
Implkeeps only the RESP protocol layer.evpp
is_loop_ownerintoEventLoopThread; addEventLoop::shared_from_this+currentThreadEventLoopPtrso clients can runon the current loop thread instead of spawning their own.
Tests
lua_binding/lua_io/http_script_handler/http_lua_handler/lua_http/lua_ws/lua_mqtt/lua_redis+redis_*all pass under bothMake and CMake.
Docs
docs/cn/lua.mddocuments the fullhv.*API;docs/PLAN.mdmarks the luabinding done.