Skip to content

Commit a7cfdd3

Browse files
yhiroseyhirose
committed
Add New Streaming API support
1 parent 8479673 commit a7cfdd3

File tree

7 files changed

+2707
-389
lines changed

7 files changed

+2707
-389
lines changed

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,11 @@ example/benchmark
1616
example/redirect
1717
!example/redirect.*
1818
example/ssecli
19+
!example/ssecli.*
20+
example/ssecli-stream
21+
!example/ssecli-stream.*
1922
example/ssesvr
23+
!example/ssesvr.*
2024
example/upload
2125
!example/upload.*
2226
example/one_time_request

‎README-stream.md‎

Lines changed: 327 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,327 @@
1+
# cpp-httplib C++20 Streaming API
2+
3+
This document describes the C++20 streaming extensions for cpp-httplib, providing a generator-like API for handling HTTP responses incrementally with **true socket-level streaming**.
4+
5+
> **Important Notes**:
6+
>
7+
> - **No Keep-Alive**: Each `stream::Get()` call uses a dedicated connection that is closed after the response is fully read. For connection reuse, use `Client::Get()`.
8+
> - **Single iteration only**: The `body()` generator can only be iterated once. Calling `body()` again after iteration has no effect.
9+
> - **Result is not thread-safe**: While `stream::Get()` can be called from multiple threads simultaneously, the returned `stream::Result` must be used from a single thread only.
10+
11+
## Overview
12+
13+
The C++20 streaming API allows you to process HTTP response bodies chunk by chunk using C++20 coroutines, similar to Python's generators or C++23's `std::generator`. Data is read directly from the network socket, enabling low-memory processing of large responses. This is particularly useful for:
14+
15+
- **LLM/AI streaming responses** (e.g., ChatGPT, Claude, Ollama)
16+
- **Server-Sent Events (SSE)**
17+
- **Large file downloads** with progress tracking
18+
- **Reverse proxy implementations**
19+
20+
## Requirements
21+
22+
- C++20 compiler with coroutine support
23+
- Include `httplib.h` and compile with `-std=c++20`
24+
25+
## Quick Start
26+
27+
```cpp
28+
#include "httplib.h"
29+
30+
int main() {
31+
httplib::Client cli("http://localhost:8080");
32+
33+
// Get streaming response
34+
auto result = httplib::stream::Get(cli, "/stream");
35+
36+
if (result) {
37+
// Process response body in chunks
38+
for (auto chunk : result.body(4096)) {
39+
std::cout << chunk; // Process each chunk as it arrives
40+
}
41+
}
42+
43+
return 0;
44+
}
45+
```
46+
47+
## API Layers
48+
49+
cpp-httplib provides multiple API layers for different use cases:
50+
51+
```text
52+
┌─────────────────────────────────────────────┐
53+
│ SSEClient (planned) │ ← SSE-specific, parsed events
54+
│ - on_message(), on_event() │
55+
│ - Auto-reconnect, Last-Event-ID │
56+
├─────────────────────────────────────────────┤
57+
│ stream::Get() / stream::Result │ ← C++20, Generator-based
58+
│ - for (auto chunk : result.body()) │
59+
├─────────────────────────────────────────────┤
60+
│ open_stream() / StreamHandle │ ← General-purpose streaming
61+
│ - handle.read(buf, len) │
62+
├─────────────────────────────────────────────┤
63+
│ Client::Get() │ ← Traditional, full buffering
64+
└─────────────────────────────────────────────┘
65+
```
66+
67+
| Use Case | Recommended API |
68+
|----------|----------------|
69+
| SSE with auto-reconnect | SSEClient (planned) or `ssecli-stream.cc` example |
70+
| LLM streaming (JSON Lines) | `stream::Get()` |
71+
| Large file download | `stream::Get()` or `open_stream()` |
72+
| Reverse proxy | `open_stream()` |
73+
| Small responses with Keep-Alive | `Client::Get()` |
74+
75+
## API Reference
76+
77+
### Low-Level API: `StreamHandle`
78+
79+
The `StreamHandle` struct provides direct control over streaming responses. It takes ownership of the socket connection and reads data directly from the network.
80+
81+
> **Note:** When using `open_stream()`, the connection is dedicated to streaming and **Keep-Alive is not supported**. For Keep-Alive connections, use `client.Get()` instead.
82+
83+
```cpp
84+
// Open a stream (takes ownership of socket)
85+
httplib::Client cli("http://localhost:8080");
86+
auto handle = cli.open_stream("/path");
87+
88+
// Check validity
89+
if (handle.is_valid()) {
90+
// Access response headers immediately
91+
int status = handle.response->status;
92+
auto content_type = handle.response->get_header_value("Content-Type");
93+
94+
// Read body incrementally
95+
char buf[4096];
96+
ssize_t n;
97+
while ((n = handle.read(buf, sizeof(buf))) > 0) {
98+
process(buf, n);
99+
}
100+
}
101+
```
102+
103+
#### StreamHandle Members
104+
105+
| Member | Type | Description |
106+
|--------|------|-------------|
107+
| `response` | `std::unique_ptr<Response>` | HTTP response with headers |
108+
| `error` | `Error` | Error code if request failed |
109+
| `is_valid()` | `bool` | Returns true if response is valid |
110+
| `read(buf, len)` | `ssize_t` | Read up to `len` bytes directly from socket |
111+
| `get_read_error()` | `Error` | Get the last read error |
112+
| `has_read_error()` | `bool` | Check if a read error occurred |
113+
114+
### High-Level API: `stream::Get()` and `stream::Result`
115+
116+
The `httplib.h` header provides a more ergonomic API using C++20 coroutines (requires C++20).
117+
118+
```cpp
119+
#include "httplib.h"
120+
121+
httplib::Client cli("http://localhost:8080");
122+
123+
// Simple GET
124+
auto result = httplib::stream::Get(cli, "/path");
125+
126+
// GET with custom headers
127+
httplib::Headers headers = {{"Authorization", "Bearer token"}};
128+
auto result = httplib::stream::Get(cli, "/path", headers);
129+
```
130+
131+
#### stream::Result Members
132+
133+
| Member | Type | Description |
134+
|--------|------|-------------|
135+
| `operator bool()` | `bool` | Returns true if response is valid |
136+
| `status()` | `int` | HTTP status code |
137+
| `headers()` | `const Headers&` | Response headers |
138+
| `get_header_value(key, def)` | `std::string` | Get header value (with optional default) |
139+
| `has_header(key)` | `bool` | Check if header exists |
140+
| `body(chunk_size)` | `Generator<std::string_view>` | Generator yielding body chunks |
141+
| `error()` | `Error` | Get the connection/request error |
142+
| `read_error()` | `Error` | Get the last read error |
143+
| `has_read_error()` | `bool` | Check if a read error occurred |
144+
145+
### Generator Class
146+
147+
The `httplib::Generator<T>` class is a C++20 coroutine-based generator, similar to `std::generator` (C++23).
148+
149+
```cpp
150+
// Iterate over chunks
151+
for (auto chunk : result.body(1024)) {
152+
// chunk is std::string_view
153+
process(chunk);
154+
}
155+
```
156+
157+
## Usage Examples
158+
159+
### Example 1: SSE (Server-Sent Events) Client
160+
161+
```cpp
162+
#include "httplib.h"
163+
#include <iostream>
164+
165+
int main() {
166+
httplib::Client cli("http://localhost:1234");
167+
168+
auto result = httplib::stream::Get(cli, "/events");
169+
if (!result) { return 1; }
170+
171+
for (auto chunk : result.body()) {
172+
std::cout << chunk << std::flush;
173+
}
174+
175+
return 0;
176+
}
177+
```
178+
179+
For a complete SSE client with auto-reconnection and event parsing, see `example/ssecli-stream.cc`.
180+
181+
### Example 2: LLM Streaming Response
182+
183+
```cpp
184+
#include "httplib.h"
185+
#include <iostream>
186+
187+
int main() {
188+
httplib::Client cli("http://localhost:11434"); // Ollama
189+
190+
auto result = httplib::stream::Get(cli, "/api/generate");
191+
192+
if (result && result.status() == 200) {
193+
for (auto chunk : result.body()) {
194+
std::cout << chunk << std::flush;
195+
}
196+
}
197+
198+
// Check for connection errors
199+
if (result.read_error() != httplib::Error::Success) {
200+
std::cerr << "Connection lost\n";
201+
}
202+
203+
return 0;
204+
}
205+
```
206+
207+
### Example 3: Large File Download with Progress
208+
209+
```cpp
210+
#include "httplib.h"
211+
#include <fstream>
212+
#include <iostream>
213+
214+
int main() {
215+
httplib::Client cli("http://example.com");
216+
auto result = httplib::stream::Get(cli, "/large-file.zip");
217+
218+
if (!result || result.status() != 200) {
219+
std::cerr << "Download failed\n";
220+
return 1;
221+
}
222+
223+
std::ofstream file("download.zip", std::ios::binary);
224+
size_t total = 0;
225+
226+
for (auto chunk : result.body(65536)) { // 64KB chunks
227+
file.write(chunk.data(), chunk.size());
228+
total += chunk.size();
229+
std::cout << "\rDownloaded: " << (total / 1024) << " KB" << std::flush;
230+
}
231+
232+
std::cout << "\nComplete!\n";
233+
return 0;
234+
}
235+
```
236+
237+
### Example 4: Reverse Proxy Streaming
238+
239+
```cpp
240+
#include "httplib.h"
241+
242+
httplib::Server svr;
243+
244+
svr.Get("/proxy/(.*)", [](const httplib::Request& req, httplib::Response& res) {
245+
httplib::Client upstream("http://backend:8080");
246+
auto handle = upstream.open_stream("/" + req.matches[1].str());
247+
248+
if (!handle.is_valid()) {
249+
res.status = 502;
250+
return;
251+
}
252+
253+
res.status = handle.response->status;
254+
res.set_chunked_content_provider(
255+
handle.response->get_header_value("Content-Type"),
256+
[handle = std::move(handle)](size_t, httplib::DataSink& sink) mutable {
257+
char buf[8192];
258+
auto n = handle.read(buf, sizeof(buf));
259+
if (n > 0) {
260+
sink.write(buf, static_cast<size_t>(n));
261+
return true;
262+
}
263+
sink.done();
264+
return true;
265+
}
266+
);
267+
});
268+
269+
svr.listen("0.0.0.0", 3000);
270+
```
271+
272+
## Comparison with Existing APIs
273+
274+
| Feature | `Client::Get()` | `open_stream()` | `stream::Get()` |
275+
|---------|----------------|-----------------|----------------|
276+
| Headers available | After complete | Immediately | Immediately |
277+
| Body reading | All at once | Direct from socket | Generator-based |
278+
| Memory usage | Full body in RAM | Minimal (controlled) | Minimal (controlled) |
279+
| Keep-Alive support | ✅ Yes | ❌ No | ❌ No |
280+
| Compression | Auto-handled | Auto-handled | Auto-handled |
281+
| C++ standard | C++11 | C++11 | C++20 |
282+
| Best for | Small responses, Keep-Alive | Low-level streaming | Modern streaming |
283+
284+
## Features
285+
286+
- **True socket-level streaming**: Data is read directly from the network socket
287+
- **Low memory footprint**: Only the current chunk is held in memory
288+
- **Compression support**: Automatic decompression for gzip, brotli, and zstd
289+
- **Chunked transfer**: Full support for chunked transfer encoding
290+
- **SSL/TLS support**: Works with HTTPS connections
291+
- **C++23 ready**: `Generator<T>` is compatible with `std::generator` interface
292+
293+
## Important Notes
294+
295+
### Keep-Alive Behavior
296+
297+
The streaming API (`stream::Get()` / `open_stream()`) takes ownership of the socket connection for the duration of the stream. This means:
298+
299+
- **Keep-Alive is not supported** for streaming connections
300+
- The socket is closed when `StreamHandle` is destroyed
301+
- For Keep-Alive scenarios, use the standard `client.Get()` API instead
302+
303+
```cpp
304+
// Use for streaming (no Keep-Alive)
305+
auto stream = httplib::stream::Get(cli, "/large-stream");
306+
307+
// Use for Keep-Alive connections
308+
auto result = cli.Get("/api/data"); // Connection can be reused
309+
```
310+
311+
## Building
312+
313+
Compile with C++20 support:
314+
315+
```bash
316+
# GCC
317+
g++ -std=c++20 -o myapp myapp.cpp -lpthread -lssl -lcrypto
318+
319+
# Clang
320+
clang++ -std=c++20 -o myapp myapp.cpp -lpthread -lssl -lcrypto
321+
```
322+
323+
## Related
324+
325+
- [Issue #2269](https://github.com/yhirose/cpp-httplib/issues/2269) - Original feature request
326+
- [httplib.h](./httplib.h) - Main library (includes C++20 streaming API)
327+
- [example/ssecli-stream.cc](./example/ssecli-stream.cc) - SSE client with auto-reconnection

‎README.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -873,6 +873,26 @@ res = cli.Options("*");
873873
res = cli.Options("/resource/foo");
874874
```
875875

876+
### Streaming API (C++20)
877+
878+
Process large responses without loading everything into memory. Requires C++20.
879+
880+
```c++
881+
#include "httplib.h"
882+
#include "httplib-stream.h"
883+
884+
httplib::Client cli("localhost", 8080);
885+
886+
auto result = httplib::stream::Get(cli, "/large-file");
887+
if (result) {
888+
for (auto chunk : result.body()) {
889+
process(chunk); // Process each chunk as it arrives
890+
}
891+
}
892+
```
893+
894+
All HTTP methods are supported: `stream::Get`, `Post`, `Put`, `Patch`, `Delete`, `Head`, `Options`. See `httplib-stream.h` for details.
895+
876896
### Timeout
877897
878898
```c++

‎example/Makefile‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ ZLIB_SUPPORT = -DCPPHTTPLIB_ZLIB_SUPPORT -lz
1818
BROTLI_DIR = $(PREFIX)/opt/brotli
1919
BROTLI_SUPPORT = -DCPPHTTPLIB_BROTLI_SUPPORT -I$(BROTLI_DIR)/include -L$(BROTLI_DIR)/lib -lbrotlicommon -lbrotlienc -lbrotlidec
2020

21-
all: server client hello simplecli simplesvr upload redirect ssesvr ssecli benchmark one_time_request server_and_client accept_header
21+
all: server client hello simplecli simplesvr upload redirect ssesvr ssecli ssecli-stream benchmark one_time_request server_and_client accept_header
2222

2323
server : server.cc ../httplib.h Makefile
2424
$(CXX) -o server $(CXXFLAGS) server.cc $(OPENSSL_SUPPORT) $(ZLIB_SUPPORT) $(BROTLI_SUPPORT)
@@ -47,6 +47,9 @@ ssesvr : ssesvr.cc ../httplib.h Makefile
4747
ssecli : ssecli.cc ../httplib.h Makefile
4848
$(CXX) -o ssecli $(CXXFLAGS) ssecli.cc $(OPENSSL_SUPPORT) $(ZLIB_SUPPORT) $(BROTLI_SUPPORT)
4949

50+
ssecli-stream : ssecli-stream.cc ../httplib.h ../httplib.h Makefile
51+
$(CXX) -o ssecli-stream -std=c++20 -I.. -Wall -Wextra -pthread ssecli-stream.cc $(OPENSSL_SUPPORT) $(ZLIB_SUPPORT) $(BROTLI_SUPPORT)
52+
5053
benchmark : benchmark.cc ../httplib.h Makefile
5154
$(CXX) -o benchmark $(CXXFLAGS) benchmark.cc $(OPENSSL_SUPPORT) $(ZLIB_SUPPORT) $(BROTLI_SUPPORT)
5255

@@ -64,4 +67,4 @@ pem:
6467
openssl req -new -key key.pem | openssl x509 -days 3650 -req -signkey key.pem > cert.pem
6568

6669
clean:
67-
rm server client hello simplecli simplesvr upload redirect ssesvr ssecli benchmark one_time_request server_and_client accept_header *.pem
70+
rm server client hello simplecli simplesvr upload redirect ssesvr ssecli ssecli-stream benchmark one_time_request server_and_client accept_header *.pem

0 commit comments

Comments
 (0)