@@ -91,6 +91,50 @@ Content Data < | | records | |
9191Fluent Bit API provides backward compatibility with the previous metadata and content
9292format found on series v1.8.
9393
94+ Starting with the Fluent Bit release that introduces direct route persistence, the
95+ fourth metadata byte now carries feature flags. A zero value preserves the legacy
96+ layout, while a non-zero value indicates that additional structures follow the tag.
97+ When the `` FLB_CHUNK_FLAG_DIRECT_ROUTES `` bit is set the tag is terminated with a
98+ single `` \0 `` byte and a routing payload is appended. Fluent Bit v3.0 and later
99+ also set the `` FLB_CHUNK_FLAG_DIRECT_ROUTE_LABELS `` bit to store each destination's
100+ alias (or generated name) alongside its numeric identifier so routes can survive
101+ configuration changes that renumber outputs. If any stored identifier exceeds
102+ 65535 the `` FLB_CHUNK_FLAG_DIRECT_ROUTE_WIDE_IDS `` bit is enabled and each ID is
103+ encoded using four bytes so large configurations remain routable after a restart:
104+
105+ ```
106+ Metadata
107+ -- +---------+-------+
108+ / | 0xF1 | 0x77 | <- Magic Bytes
109+ / +---------+-------+
110+ < | Type | Flags | <- Chunk type and flag bits
111+ \ +---------+-------+
112+ \ | Tag | <- Tag string (no size prefix)
113+ +---------+-------+
114+ | 0x00 | | <- Tag terminator (present when flags != 0)
115+ +---------+-------+
116+ | Routing Length | <- uint16_t big endian
117+ +---------+-------+
118+ | Route Count | <- uint16_t big endian
119+ +---------+-------+
120+ | Output IDs ... | <- Each stored as uint16_t (big endian)
121+ | | or uint32_t when FLB_CHUNK_FLAG_DIRECT_ROUTE_WIDE_IDS
122+ +---------+-------+
123+ | Label Lens ... | <- Present when FLB_CHUNK_FLAG_DIRECT_ROUTE_LABELS
124+ +---------+-------+
125+ | Label Bytes ... | <- Concatenated label strings (no terminators)
126+ -- +-----------------+
127+ ```
128+
129+ The routing payload captures the direct route mapping so that filesystem chunks
130+ loaded by the storage backlog re-use the same outputs after a restart. Chunks
131+ without direct routes keep the legacy layout (flags byte set to zero) and remain
132+ fully backwards compatible across Fluent Bit versions. When labels are stored the
133+ reader first reconstructs routes by matching aliases or numbered names and only
134+ falls back to numeric identifiers if the textual metadata cannot be matched. This
135+ ensures that chunks continue to flow to the intended destinations even when the
136+ output configuration is re-ordered.
137+
94138### Fluent Bit <= v1.8
95139
96140Up to Fluent Bit <= 1.8.x, the metadata and content data is simple, where metadata
0 commit comments