Skip to content

Commit d69713a

Browse files
committed
[chore]: Update README to reference new Application Workflow documentation.
- Replace the generic docs directory reference in README.md with a direct link to the new Application Workflow (docs/workflow.md) for clearer project guidance. - Add docs/workflow.md to the repository to document the application's architecture, concurrency model, and operational flow. Signed-off-by: Goran Mišković <[email protected]>
1 parent 40b6124 commit d69713a

File tree

1 file changed

+141
-0
lines changed

1 file changed

+141
-0
lines changed

docs/workflow.md

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
# The Application Workflow
2+
3+
## Overview
4+
5+
This document describes the workflow and concurrency model of the stress-test application for LwIP integration on the Raspberry Pi Pico platform. The application demonstrates asynchronous TCP client usage, thread-safe resource sharing, and event-driven design across dual cores.
6+
7+
The test application’s **core idea** is to validate the async-tcp library’s LwIP integration with async_context, and to showcase how to customise responses to various LwIP events. It is intentionally structured to reflect lwIP’s preferred single-threaded execution model while exercising cross-core event handling and thread-safe communication.
8+
9+
Design Rationale: lwIP and async_context
10+
11+
lwIP likes single-threaded environments: Its TCP/IP core is designed to run in one main context—whether that’s a polling loop, event loop, or dedicated task. async_context provides this clean, serialized execution model without OS-level thread synchronisation.
12+
13+
Why it works well: Using async_context ensures all lwIP callbacks (including TCP RX/TX events) run in the same logical thread. A separate component or protocol handler can run in a different async_context process on another core, avoiding reentrancy issues.
14+
15+
Core principle: The lwIP raw TCP API is not thread-safe and has no built-in cross-thread marshalling. In a multi-core system, `async_context` is used to make lwIP thread-safe by ensuring that all API calls happen from the context that owns lwIP. The async-tcp library builds on this by associating a set of handlers with various lwIP callbacks and providing a thread-safe mechanism for initiating writes. The integration should not expose the raw TCP API to the consuming application, ensuring all TCP operation must pass the request to lwIP `async_contex`.
16+
17+
Bonus: This approach naturally supports a “reactor” style architecture:
18+
19+
Core 0: async_context_lwip → TCP/IP and lwIP timers.
20+
21+
Core 1: async_context_app → application logic, business rules, etc.
22+
23+
Lightweight message passing between cores.
24+
25+
## Initial Assessment
26+
27+
### Architecture
28+
- **Global context managers** are created for each core: `ctx0` (TCP client operations) and `ctx1` (serial printing and quote buffer).
29+
- **Thread-safe buffer** (`e5::QuoteBuffer qotd_buffer(ctx1)`) is used for storing the quote of the day, utilizing the SyncBridge pattern for safe cross-core access.
30+
- **TCP clients** are instantiated for both the QOTD and Echo servers, with their associated IP/port configuration.
31+
- **Utility functions** handle board temperature reading, heap/stack statistics, and WiFi/server connections.
32+
33+
### Flow and Concurrency Patterns
34+
35+
#### Sequence of Operations
36+
1. **WiFi and server setup:** The application connects to WiFi and resolves server addresses.
37+
2. **QOTD retrieval:** The function `get_quote_of_the_day()` initiates a connection to the QOTD server if not already in progress.
38+
3. **Buffer usage:** When a quote is received, it is stored in `qotd_buffer` using its thread-safe `set()` method.
39+
4. **Echo operation:** The function `get_echo()` reads the current quote from `qotd_buffer` (using `get()`) and, if not empty, sends it to the echo server.
40+
5. **Statistics and monitoring:** Functions print heap, stack, and temperature stats using the serial printer, which also operates on `ctx1`.
41+
42+
#### Expected Concurrency Patterns
43+
- **Reads vs Writes:** The buffer is written to only when a new quote is received (infrequent), but read every time an echo operation is attempted (frequent).
44+
- **Cross-core access:** Core 0 (TCP client) may write to the buffer, while Core 1 (serial printer, echo logic) may read from it, potentially at the same time.
45+
- **Synchronization:** All access to `qotd_buffer` is funneled through SyncBridge, ensuring thread safety but serializing all operations (no concurrent reads).
46+
47+
## QOTD Protocol and Application Beat
48+
49+
The application leverages the QOTD (Quote of the Day) protocol, where the server sends a quote and closes the connection immediately upon client connection. This server-driven behavior defines the application's operational cycle ("beat"):
50+
51+
- **Connection Initiation:** The application connects to the QOTD server.
52+
- **Quote Reception:** Upon connection, the server sends a quote and closes the connection.
53+
- **Buffer Update:** The received quote is written to the thread-safe buffer (`qotd_buffer`).
54+
- **Echo Trigger:** The application then reads the buffer and sends the quote to the echo server.
55+
- **Cycle Repeat:** The process repeats, with each QOTD server response driving the next cycle.
56+
57+
This protocol-driven flow ensures that each round of buffer update and echo operation is synchronized with the QOTD server's response, providing a natural rhythm for stress-testing and concurrency analysis.
58+
59+
## Critical Integration Note: Cross-Core Buffer Update and Real-Time Constraints
60+
61+
The application updates the quote buffer on each QOTD receive event using:
62+
63+
```cpp
64+
m_quote_buffer.set(data);
65+
```
66+
67+
This is a **blocking, cross-core call** from an interrupt context (network event on core 0) to core 1, where the buffer resides. This design is intentional and reflects real-world requirements:
68+
69+
- **Core 0**: Handles network operations and receives data from the QOTD server.
70+
- **Core 1**: Runs the main application logic and owns the quote buffer.
71+
- The buffer update is performed synchronously and blocks until the operation completes on core 1, ensuring data consistency.
72+
73+
Currently, all available data is consumed and pushed to the buffer. In a production scenario, the application on core 1 would need to process the data and return the number of bytes actually consumed, enabling partial consumption and more robust flow control.
74+
75+
This approach deliberately stress-tests:
76+
- Cross-core synchronization and blocking calls from interrupt context
77+
- The SyncBridge pattern’s ability to marshal data and execution between cores
78+
- Real-time responsiveness and integration boundaries
79+
80+
**Future Direction:**
81+
- Allow the consumer (core 1) to report back the number of bytes processed
82+
- Support partial consumption and more granular flow control
83+
- Further validate and optimize cross-core, interrupt-safe integration
84+
85+
This section is critical for anyone reviewing or extending the code, as it highlights both the rationale for the current design and the roadmap for future improvements.
86+
87+
## SerialPrinter: Thread-Safe Output and Non-Reentrant Library Integration
88+
89+
The SerialPrinter component is a key part of the test setup, demonstrating how to safely service third-party libraries (such as Serial) that are not re-entrant, using the pico_async_context pattern.
90+
91+
- **Thread-Safe Output:** All calls to Serial.print() are funneled through SerialPrinter, which schedules print jobs to execute on core 1. This ensures that output from any core or interrupt context is printed sequentially and without overlap.
92+
- **Non-Blocking Cross-Core Calls:** For example, in EchoReceivedHandler::onWork(), the call:
93+
```cpp
94+
m_serial_printer.print(std::move(quote));
95+
```
96+
is a non-blocking, cross-core operation. The print job is queued and executed on core 1, maintaining log integrity and avoiding concurrency issues.
97+
- **Log Consistency:** This approach guarantees that all log messages, status updates, and received data appear in the correct order, with no overlaps or garbage, even under heavy cross-core activity. The log remains clear and readable, as shown in the application output.
98+
- **Showcasing pico_async_context:** SerialPrinter exemplifies how to use async_context to service non-reentrant libraries, providing a practical pattern for integrating similar third-party components in multi-core or interrupt-driven environments.
99+
100+
This design is essential for robust, thread-safe output and serves as a reference for handling non-reentrant resources in concurrent applications.
101+
102+
## Event Handlers: Customizing Application Response to LwIP Events
103+
104+
The application demonstrates how to use the async-tcp library's event-driven architecture to customize responses to various LwIP events through dedicated handler classes:
105+
106+
- **QotdConnectedHandler**: Responds to successful QOTD server connections, logs connection details, and prepares the application for the next receive event.
107+
- **QotdReceivedHandler**: Handles incoming quote data, performing a blocking, cross-core update to the quote buffer to ensure data consistency and test integration boundaries.
108+
- **QotdClosedHandler**: Updates application progress and state when the QOTD connection is closed, allowing the main loop to proceed to the next cycle.
109+
- **EchoReceivedHandler**: Handles data received from the echo server, using a non-blocking, cross-core call to SerialPrinter to ensure thread-safe, ordered output.
110+
- **TcpErrorHandler**: Provides customizable cleanup and logging for various LwIP error events (e.g., memory errors, connection loss, timeouts), demonstrating how to handle error conditions robustly.
111+
112+
These handlers illustrate the flexibility of the async-tcp library, allowing applications to define precise, context-aware responses to network events, connection lifecycle changes, and error conditions. Each handler is executed in the appropriate async_context, ensuring thread safety and correct core affinity for all operations.
113+
114+
## Chunked Asynchronous Writes and ACK-Driven Flow Control
115+
116+
The async-tcp library implements robust, context-aware asynchronous write operations using a chunked state machine and ACK-driven flow control. This is a critical aspect of both the library and the application, ensuring safe, efficient, and correct data transmission across cores and contexts.
117+
118+
### How It Works
119+
- When a write is initiated (e.g., via `echo_client.write()`), the data is managed by a `TcpWriter` instance, which breaks large writes into manageable chunks.
120+
- Each chunk is sent using a `TcpWriteHandler`, which is always executed on the correct context/core as enforced by the async context system.
121+
- The actual TCP write operation (`writeChunk`) is performed by the handler, and user code cannot customize or override this process, ensuring safety and consistency.
122+
- When the TCP stack acknowledges a chunk (via an ACK), the internal callback (`_onAckCallback()`) calls `TcpWriter::onAckReceived()`, which updates the write progress.
123+
- If all data has been acknowledged, `completeWrite()` is called to reset the state. If more data remains, `sendNextChunk()` is called to continue the process.
124+
125+
### Thread Safety and Context Guarantees
126+
- All write operations and state transitions are performed on the context/core associated with the `AsyncCtx` for the connection, enforced by asserts and the handler system.
127+
- The design is not thread-safe for concurrent writes from multiple threads/cores, but this is prevented by the context system and library design.
128+
- User code interacts with the library at a high level (e.g., calling `write()`), but cannot interfere with the internal chunking, ACK handling, or state machine.
129+
130+
### Why This Matters
131+
- This approach ensures that large writes are safely and efficiently transmitted, even in a multi-core, interrupt-driven environment.
132+
- It prevents data races, corruption, and protocol violations by strictly controlling where and how write operations occur.
133+
- The application benefits from this by being able to initiate writes from the correct context, with all chunking and flow control handled transparently by the library.
134+
135+
**In summary:** The async-tcp library's chunked, ACK-driven write state machine is a foundational feature for safe, efficient, and correct TCP communication in concurrent, multi-core applications. All user-initiated writes are serialized, context-bound, and robustly managed, with no opportunity for user code to break the safety guarantees.
136+
137+
## Next Steps
138+
139+
- Further document the setup, event handlers, and main loop.
140+
- Summarize the concurrency model and access patterns in more detail.
141+
- Optionally, instrument the code to log read/write counts for deeper analysis.

0 commit comments

Comments
 (0)