Custom handler for access to raw sockets #185
Replies: 5 comments 3 replies
-
|
Hello, |
Beta Was this translation helpful? Give feedback.
-
|
I initially considered that, but it would be great to be able to use the rest of the features of the webserver without having to reimplement them. I mean, I could port my C websocket implementation over, but you've already got one. Oh well. I don't normally like to release competing offerings if something out there is almost what i need, but the gap here seems to large for me without forking. |
Beta Was this translation helpful? Give feedback.
-
|
In that case use ESPAsyncWS but with its API. It already does the parsing for you. Just develop a layer to interface your project with this one. |
Beta Was this translation helpful? Give feedback.
-
|
Note: there won't and will NEVER be a way to do what you want. Request header parsing is happening BEFORE the handler selection and request body parsing is triggered depending on the CT, which trigger the right onBody / onUpload handler. Parsing is the most complicated stuff. The rest is related to ESPASyncWS, which you do not want. So use directly AsyncTCP which provides the socket layer. You will be able to parse and write your own way headers and body. |
Beta Was this translation helpful? Give feedback.
-
|
Maybe I'm not understanding correctly: there's nothing strictly speaking preventing you from subclassing It's worth noting, though, that the raw handle does not support blocking operations like a socket; it's a fairly thin wrapper of the native form of the underlying LwIP TCP stack, which is fundamentally callback-oriented. (The blocking socket implementation exposed in ESP-IDF is actually a wrapper of it; they've abstracted away the callback nature, with the attendant memory and thread-synchronization costs.) As you've observed, the Response class has to take responsibility for tracking relevant state for any generated content greater than the TCP stack can handle at once (ie. the TCP window size, typically about two packets worth), and managing producing new content essentially on a packet-by-packet basis as the remote TCP stack acks each one (hence the callback's name |
Beta Was this translation helpful? Give feedback.
-
|
I see. It doesn't surprise me to hear that the non-blocking socket API is callback oriented, and explains the behavior of ESPAsyncWebServer's handlers. Thanks. I'll show you what I'm working with just so you can see it. I should note that in this generated code the headers and status line are included, but it can simply omit the status line if need be depending on what it's being used with. httpd_content.h Basically I just spit content out to a socket. It's already stored as literals, precompressed, all that mess. If there was a way I could get this server to use this it would be great. The most difficult bit might be the dynamic responses, since they are http chunked and stored encoded as chunks as literals in the code. Maybe take a look and see if you have any ideas? |
Beta Was this translation helpful? Give feedback.
-
|
If the content is strictly guaranteed to be only literals (ie. memory with global lifetime), it's straightforward: the response object could collect the pointer/length pairs and stream them out later, one packet at a time. Chunked encoding wouldn't matter as it'd just be dealing with abstract blocks of bytes. Note that using a raw (incomplete sketch, totally untested:) struct block { const char* data; size_t size; };
struct StaticHTTPResponse_ref;
class StaticHTTPResponse : public AsyncWebServerResponse {
std::vector<block> reply;
size_t total_size;
size_t reply_idx;
size_t block_data_sent;
size_t bytes_acked;
friend struct StaticHTTPResponse_ref;
virtual bool _started() const { return reply.size(); } // unused
virtual bool _finished() const { return _started() && (bytes_acked == total_size); } // not used
virtual bool _failed() const { return false; }; // TODO?
virtual bool _sourceValid() const { return true; }; // unused
virtual void _respond(AsyncWebServerRequest *request);
virtual size_t _ack(AsyncWebServerRequest *request, size_t len, uint32_t time);
public:
StaticHTTPResponse() : total_size(0), reply_idx(0), block_data_sent(0), bytes_acked(0) {};
inline void send_block(const char* data, size_t len) {
reply.push_back({data, len});
total_size += len;
}
};
struct StaticHTTPResponse_ref {
StaticHTTPResponse& response;
};
void httpd_send_block(const char* data, size_t len, void* arg) {
if (!data || !*data || !len) {
return;
}
StaticHTTPResponse_ref* resp_arg = reinterpret_cast<StaticHTTPResponse_ref*>(arg);
resp_arg->response.send_block(data, len);
}
void StaticHTTPResponse::_respond(AsyncWebServerRequest *request) {
// Note we can get away with freeing memory allocated with new on ESP32s
// Allocating it this way guarantees proper alignment
StaticHTTPResponse_ref* ref = new StaticHTTPResponse_ref { *this };
// Do the thing that calls httpd_send_block here
serve_request(request, ref);
// Start sending data
_ack(request, 0, 0);
}
size_t StaticHTTPResponse::_ack(AsyncWebServerRequest *request, size_t len, uint32_t time) {
auto client = request->client();
// Handle ack'd bytes
bytes_acked += len;
if (bytes_acked == total_size) {
// All done!
client->close();
return;
}
// Fill the output buffer with more data
// Note the standard code (AsyncAbstractResponse::_ack) takes on additional
// complexity to try to optimize TCP socket buffer memory usage patterns; this is
// instead the simplest possible implementation
while (reply_idx < reply.size()) {
auto& block = reply[reply_idx];
auto block_to_send = block.size - block_data_sent;
auto newly_sent = client->write(&block.data[block_data_sent], block_to_send);
if (newly_sent != block_to_send) {
// buffer is full, break
// TODO: check for error case (newly_sent == 0)
block_data_sent += newly_sent;
break;
}
// Otherwise we sent the whole block, so keep going
++reply_idx;
block_data_sent = 0;
}
} |
Beta Was this translation helpful? Give feedback.
-
|
Oh interesting, that hadn't occurred to me. Thanks, I'll poke around with this and see how far I get. =) |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I currently can't use this offering with my present tooling. It creates and embeds anything Vite can generate (React/Angular/etc) as C code ready to send over a socket as HTTP chunked responses. On the ESP-IDF this works fine, because you can just get the underlying socket handle and go to town. On win32 winsock, same deal. POSIX same deal. I have no problems serving this generated content from my tooling on any of these platforms. I could on Arduino using your offering except:
What would be nice is if you could register a handler with a path. That handler is a raw handler, and all it does is process the request headers and body. From there, you generate the response. Now, it's possible for my tooling to forgo generating the status line, but it must generate Content-Type/Content-Length/Content-Encoding headers for the static content, so at the very least, the custom handler would have to allow you to append a block of headers AS TEXT as the first part of its response. example:
If I could do this, it would open up me being able to
A) Easily embed React or other modern web frameworks with transpilation into a project
B) Allow me to serve the same content on a local web for easier editying
C) Allow my SSR ASP-like engine to function with your server
Is this completely out of pocket? I haven't dived into the code too much. If I just had a method like response_send_raw(const void* data,size_t len)l or similar from a callback that calls me right after it processes the request body, I'd be golden.
Beta Was this translation helpful? Give feedback.
All reactions