Skip to content

Commit 2260f84

Browse files
authored
🐛 fix encode dot in keys (#37)
* 🦺 enforce decodeDotInKeys and allowDots option consistency; clarify dot decoding in documentation * 🐛 fix dot notation encoding in key splitter; handle top-level dots and bracket depth correctly * ✅ add tests for allowDots and decodeDotInKeys consistency in DecodeOptions * ✅ add comprehensive tests for encoded dot behavior in keys to ensure C# parity and option consistency * ♻️ refactor DecodeOptions to support legacy decoders and unify decode logic; add decodeKey/decodeValue helpers * ✅ add tests for encoded dot handling in keys and custom decoder behavior in DecodeOptions * 🐛 fix list limit enforcement and unify key/value decoding in parser * 🔥 remove unused import of DecodeKind from qs.dart * 💡 update decode.dart comments to clarify key decoding and dot/bracket handling logic * 💡 clarify DecodeOptions docs for allowDots and decodeDotInKeys interaction; improve charsetSentinel and decoder behavior descriptions * ♻️ simplify custom decoder handling in DecodeOptions; remove dynamic invocation and legacy overloads * ✅ update tests to use new decoder signature with DecodeKind; remove legacy dynamic invocation cases * 🚚 move _dotToBracketTopLevel to QS extension as static helper; update decode.dart accordingly * 🐛 preserve leading dot in key decoding except for degenerate ".[" case * ✅ add tests for leading and double dot handling with allowDots=true * 🔥 remove legacy dynamic decoder fallback tests and helper class * 🐛 fix list limit check to account for current list length when splitting comma-separated values * 🐛 fix parameter splitting to correctly enforce limit and wrap excess bracket groups as single segment * ✅ fix custom percent-decoding logic to handle non-encoded characters and improve byte extraction * 💡 update cleanRoot comment * 💡 clarify negative listLimit behavior and list growth checks in decode logic comments * 💡 clarify listLimit negative value behavior and throwOnLimitExceeded interaction in decode options comments * ✅ improve decode tests for nested list handling, list limit error matching, and long input parsing; fix percent-decoding to handle '+' as space * 💡 update comments * 💡 clarify handling of percent-encoded dots in keys and list growth with negative listLimit in decode logic comments
1 parent 489b4ee commit 2260f84

File tree

7 files changed

+883
-206
lines changed

7 files changed

+883
-206
lines changed

lib/src/extensions/decode.dart

Lines changed: 149 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
// ignore_for_file: deprecated_member_use_from_same_package
12
part of '../qs.dart';
23

34
/// Decoder: query-string → nested Dart maps/lists (Node `qs` parity)
@@ -16,17 +17,9 @@ part of '../qs.dart';
1617
/// Implementation notes:
1718
/// - We decode key parts lazily and then "reduce" right-to-left to build the
1819
/// final structure in `_parseObject`.
19-
/// - We never mutate caller-provided containers; fresh maps/lists are created.
20-
/// - No behavioral changes are introduced here; comments only.
21-
22-
/// Split operation result used by the decoder helpers.
23-
/// - `parts`: collected key segments.
24-
/// - `exceeded`: indicates whether a configured limit was exceeded during split.
25-
typedef SplitResult = ({List<String> parts, bool exceeded});
26-
27-
/// Normalizes simple dot notation to bracket notation (e.g. `a.b``a[b]`).
28-
/// Only matches \nondotted, non-bracketed tokens so `a.b.c` becomes `a[b][c]`.
29-
final RegExp _dotToBracket = RegExp(r'\.([^.\[]+)');
20+
/// - We never mutate caller-provided containers; fresh maps/lists are allocated for merges.
21+
/// - The implementation aims to match `qs` semantics; comments explain how each phase maps
22+
/// to the reference behavior.
3023
3124
/// Internal decoding surface grouped under the `QS` extension.
3225
///
@@ -41,6 +34,13 @@ extension _$Decode on QS {
4134
///
4235
/// The `currentListLength` is used to guard incremental growth when we are
4336
/// already building a list for a given key path.
37+
///
38+
/// **Negative `listLimit` semantics:** a negative value disables numeric-index parsing
39+
/// elsewhere (e.g. `[2]` segments become string keys). For comma‑splits specifically:
40+
/// when `throwOnLimitExceeded` is `true` and `listLimit < 0`, any non‑empty split throws
41+
/// immediately; when `false`, growth is effectively capped at zero (the split produces
42+
/// an empty list). Empty‑bracket pushes (`a[]=`) are handled during structure building
43+
/// in `_parseObject`.
4444
static dynamic _parseListValue(
4545
dynamic val,
4646
DecodeOptions options,
@@ -50,18 +50,22 @@ extension _$Decode on QS {
5050
if (val is String && val.isNotEmpty && options.comma && val.contains(',')) {
5151
final List<String> splitVal = val.split(',');
5252
if (options.throwOnLimitExceeded &&
53-
currentListLength + splitVal.length > options.listLimit) {
53+
(currentListLength + splitVal.length) > options.listLimit) {
5454
throw RangeError(
5555
'List limit exceeded. '
5656
'Only ${options.listLimit} element${options.listLimit == 1 ? '' : 's'} allowed in a list.',
5757
);
5858
}
59-
return splitVal;
59+
final int remaining = options.listLimit - currentListLength;
60+
if (remaining <= 0) return const <String>[];
61+
return splitVal.length <= remaining
62+
? splitVal
63+
: splitVal.sublist(0, remaining);
6064
}
6165

6266
// Guard incremental growth of an existing list as we parse additional items.
6367
if (options.throwOnLimitExceeded &&
64-
currentListLength + 1 > options.listLimit) {
68+
currentListLength >= options.listLimit) {
6569
throw RangeError(
6670
'List limit exceeded. '
6771
'Only ${options.listLimit} element${options.listLimit == 1 ? '' : 's'} allowed in a list.',
@@ -73,10 +77,13 @@ extension _$Decode on QS {
7377

7478
/// Tokenizes the raw query-string into a flat key→value map before any
7579
/// structural reconstruction. Handles:
76-
/// - query prefix removal (`?`), percent-decoding via `options.decoder`
80+
/// - query prefix removal (`?`), and kind‑aware decoding via `DecodeOptions.decodeKey` /
81+
/// `DecodeOptions.decodeValue` (by default these percent‑decode)
7782
/// - charset sentinel detection (`utf8=`) per `qs`
7883
/// - duplicate key policy (combine/first/last)
7984
/// - parameter and list limits with optional throwing behavior
85+
/// - Comma‑split growth honors `throwOnLimitExceeded` (see `_parseListValue`);
86+
/// empty‑bracket pushes (`[]=`) are created during structure building in `_parseObject`.
8087
static Map<String, dynamic> _parseQueryStringValues(
8188
String str, [
8289
DecodeOptions options = const DecodeOptions(),
@@ -99,15 +106,12 @@ extension _$Decode on QS {
99106
final List<String> allParts = cleanStr.split(options.delimiter);
100107
late final List<String> parts;
101108
if (limit != null && limit > 0) {
102-
final int takeCount = options.throwOnLimitExceeded ? limit + 1 : limit;
103-
final int count =
104-
allParts.length < takeCount ? allParts.length : takeCount;
105-
parts = allParts.sublist(0, count);
106109
if (options.throwOnLimitExceeded && allParts.length > limit) {
107110
throw RangeError(
108111
'Parameter limit exceeded. Only $limit parameter${limit == 1 ? '' : 's'} allowed.',
109112
);
110113
}
114+
parts = allParts.take(limit).toList();
111115
} else {
112116
parts = allParts;
113117
}
@@ -145,15 +149,14 @@ extension _$Decode on QS {
145149

146150
late final String key;
147151
dynamic val;
148-
// Decode key/value using key-aware decoder, no %2E protection shim.
152+
// Decode key/value via DecodeOptions.decodeKey/decodeValue (kind-aware).
149153
if (pos == -1) {
150-
// Decode bare key (no '=') using key-aware decoder
151-
key = options.decoder(part, charset: charset, kind: DecodeKind.key);
154+
// Decode bare key (no '=') using key-aware decoding
155+
key = options.decodeKey(part, charset: charset) ?? '';
152156
val = options.strictNullHandling ? null : '';
153157
} else {
154-
// Decode key slice using key-aware decoder; values decode as value kind
155-
key = options.decoder(part.slice(0, pos),
156-
charset: charset, kind: DecodeKind.key);
158+
// Decode the key slice as a key; values decode as values
159+
key = options.decodeKey(part.slice(0, pos), charset: charset) ?? '';
157160
// Decode the substring *after* '=', applying list parsing and the configured decoder.
158161
val = Utils.apply<dynamic>(
159162
_parseListValue(
@@ -163,8 +166,7 @@ extension _$Decode on QS {
163166
? (obj[key] as List).length
164167
: 0,
165168
),
166-
(dynamic v) =>
167-
options.decoder(v, charset: charset, kind: DecodeKind.value),
169+
(dynamic v) => options.decodeValue(v as String?, charset: charset),
168170
);
169171
}
170172

@@ -206,6 +208,16 @@ extension _$Decode on QS {
206208
/// - When `allowEmptyLists` is true, an empty string (or `null` under
207209
/// `strictNullHandling`) under a `[]` segment yields an empty list.
208210
/// - `listLimit` applies to explicit numeric indices as an upper bound.
211+
/// - A negative `listLimit` disables numeric‑index parsing (bracketed numbers become map keys).
212+
/// Empty‑bracket pushes (`[]`) still create lists here; this method does not enforce
213+
/// `throwOnLimitExceeded` for that path. Comma‑split growth (if any) has already been
214+
/// handled by `_parseListValue`.
215+
/// - Keys have been decoded per `DecodeOptions.decodeKey`; top‑level splitting applies to
216+
/// literal `.` only (including those produced by percent‑decoding). Percent‑encoded dots may
217+
/// still appear inside bracket segments here; we normalize `%2E`/`%2e` to `.` below when
218+
/// `decodeDotInKeys` is enabled.
219+
/// Whether top‑level dots split was decided earlier by `_splitKeyIntoSegments` (based on
220+
/// `allowDots`). Numeric list indices are only honored for *bracketed* numerics like `[3]`.
209221
static dynamic _parseObject(
210222
List<String> chain,
211223
dynamic val,
@@ -255,8 +267,12 @@ extension _$Decode on QS {
255267
: Utils.combine([], leaf);
256268
} else {
257269
obj = <String, dynamic>{};
258-
// Normalize bracketed segments ("[k]") and optionally decode `%2E` → '.'
259-
// when `decodeDotInKeys` is enabled.
270+
// Normalize bracketed segments ("[k]"). Note: depending on how key decoding is configured,
271+
// percent‑encoded dots *may still be present here* (e.g. `%2E` / `%2e`). We intentionally
272+
// handle the `%2E`→`.` mapping in this phase (see `decodedRoot` below) so that encoded
273+
// dots inside bracket segments can be treated as literal `.` without introducing extra
274+
// dot‑splits. Top‑level dot splitting (which only applies to literal `.`) already
275+
// happened in `_splitKeyIntoSegments`.
260276
final String cleanRoot = root.startsWith('[') && root.endsWith(']')
261277
? root.slice(1, root.length - 1)
262278
: root;
@@ -313,20 +329,22 @@ extension _$Decode on QS {
313329
}
314330

315331
/// Splits a key like `a[b][0][c]` into `['a', '[b]', '[0]', '[c]']` with:
316-
/// - dot-notation normalization (`a.b``a[b]`) when `allowDots` is true
332+
/// - dotnotation normalization (`a.b``a[b]`) when `allowDots` is true (runs before splitting)
317333
/// - depth limiting (depth=0 returns the whole key as a single segment)
318-
/// - bracket group balancing, preserving unterminated tails as a single
319-
/// remainder segment unless `strictDepth` is enabled (then it throws)
334+
/// - balanced bracket grouping; an unterminated `[` causes the *entire key* to be treated as a
335+
/// single literal segment (matching `qs`)
336+
/// - when there are additional groups/text beyond `maxDepth`:
337+
/// • if `strictDepth` is true, we throw;
338+
/// • otherwise the remainder is wrapped as one final bracket segment (e.g., `"[rest]"`)
320339
static List<String> _splitKeyIntoSegments({
321340
required String originalKey,
322341
required bool allowDots,
323342
required int maxDepth,
324343
required bool strictDepth,
325344
}) {
326345
// Optionally normalize `a.b` to `a[b]` before splitting.
327-
final String key = allowDots
328-
? originalKey.replaceAllMapped(_dotToBracket, (m) => '[${m[1]}]')
329-
: originalKey;
346+
final String key =
347+
allowDots ? _dotToBracketTopLevel(originalKey) : originalKey;
330348

331349
// Depth==0 → do not split at all (reference `qs` behavior).
332350
if (maxDepth <= 0) {
@@ -335,67 +353,142 @@ extension _$Decode on QS {
335353

336354
final List<String> segments = [];
337355

338-
// Extract the parent token (before the first '['), if any.
356+
// Parent token before the first '[' (may be empty when key starts with '[')
339357
final int first = key.indexOf('[');
340358
final String parent = first >= 0 ? key.substring(0, first) : key;
341359
if (parent.isNotEmpty) segments.add(parent);
342360

343361
final int n = key.length;
344362
int open = first;
345-
int depth = 0;
363+
int collected = 0;
364+
int lastClose = -1;
346365

347-
while (open >= 0 && depth < maxDepth) {
348-
// Balance nested brackets inside this group: "[ ... possibly [] ... ]"
366+
while (open >= 0 && collected < maxDepth) {
349367
int level = 1;
350368
int i = open + 1;
351369
int close = -1;
352370

371+
// Balance nested '[' and ']' within this group.
353372
while (i < n) {
354-
final int ch = key.codeUnitAt(i);
355-
if (ch == 0x5B) {
356-
// '['
373+
final int cu = key.codeUnitAt(i);
374+
if (cu == 0x5B) {
357375
level++;
358-
} else if (ch == 0x5D) {
359-
// ']'
376+
} else if (cu == 0x5D) {
360377
level--;
361378
if (level == 0) {
362379
close = i;
363380
break;
364381
}
365382
}
366-
// Advance inside the current bracket group until it balances.
367383
i++;
368384
}
369385

370386
if (close < 0) {
371-
// Unterminated group, stop collecting groups
372-
break;
387+
// Unterminated group: treat the entire key as a single literal segment (qs semantics).
388+
return <String>[key];
373389
}
374390

375-
segments.add(key.substring(open, close + 1)); // includes enclosing [ ]
376-
depth++;
391+
segments
392+
.add(key.substring(open, close + 1)); // balanced group, includes [ ]
393+
lastClose = close;
394+
collected++;
377395

378-
// find next group, starting after this one
396+
// Find the next '[' after this balanced group.
379397
open = key.indexOf('[', close + 1);
380398
}
381399

382-
// If additional groups remain beyond the allowed depth, either throw or
383-
// stash the remainder as a single segment, per `strictDepth`.
384-
if (open >= 0) {
385-
// We still have remainder starting with '['
400+
// Trailing text after the last balanced group → one final bracket segment (unless it's just '.').
401+
if (lastClose >= 0 && lastClose + 1 < n) {
402+
final String remainder = key.substring(lastClose + 1);
403+
if (remainder != '.') {
404+
if (strictDepth && open >= 0) {
405+
throw RangeError(
406+
'Input depth exceeded $maxDepth and strictDepth is true');
407+
}
408+
segments.add('[$remainder]');
409+
}
410+
} else if (open >= 0) {
411+
// There are more groups beyond the collected depth.
386412
if (strictDepth) {
387413
throw RangeError(
388414
'Input depth exceeded $maxDepth and strictDepth is true');
389415
}
390-
// Stash the remainder as a single segment (qs behavior)
416+
// Wrap the remaining bracket groups as a single literal segment.
417+
// Example: key="a[b][c][d]", depth=2 → segment="[[c][d]]" which becomes "[c][d]" later.
391418
segments.add('[${key.substring(open)}]');
392419
}
393420

394421
return segments;
395422
}
396423

424+
/// Convert top‑level dots to bracket segments (depth‑aware).
425+
/// - Only dots at depth == 0 split.
426+
/// - Dots inside `[...]` are preserved.
427+
/// - Degenerate cases are preserved and do not create empty segments:
428+
/// * leading '.' (e.g., ".a") keeps the dot literal,
429+
/// * double dots ("a..b") keep the first dot literal,
430+
/// * trailing dot ("a.") keeps the trailing dot (which is ignored by the splitter).
431+
/// - Only literal `.` are considered for splitting here. In this library, keys are normally
432+
/// percent‑decoded before this step; thus a top‑level `%2E` typically becomes a literal `.`
433+
/// and will split when `allowDots` is true.
434+
static String _dotToBracketTopLevel(String s) {
435+
if (s.isEmpty || !s.contains('.')) return s;
436+
final StringBuffer sb = StringBuffer();
437+
int depth = 0;
438+
int i = 0;
439+
while (i < s.length) {
440+
final ch = s[i];
441+
if (ch == '[') {
442+
depth++;
443+
sb.write(ch);
444+
i++;
445+
} else if (ch == ']') {
446+
if (depth > 0) depth--;
447+
sb.write(ch);
448+
i++;
449+
} else if (ch == '.') {
450+
if (depth == 0) {
451+
final bool hasNext = i + 1 < s.length;
452+
final String next = hasNext ? s[i + 1] : '\u0000';
453+
454+
// preserve a *leading* '.' as a literal, unless it's the ".[" degenerate.
455+
if (i == 0 && (!hasNext || next != '[')) {
456+
sb.write('.');
457+
i++;
458+
} else if (hasNext && next == '[') {
459+
// Degenerate ".[" → skip the dot so "a.[b]" behaves like "a[b]".
460+
i++; // consume the '.'
461+
} else if (!hasNext || next == '.') {
462+
// Preserve literal dot for trailing/duplicate dots.
463+
sb.write('.');
464+
i++;
465+
} else {
466+
// Normal split: convert a.b → a[b] at top level.
467+
final int start = ++i;
468+
int j = start;
469+
while (j < s.length && s[j] != '.' && s[j] != '[') {
470+
j++;
471+
}
472+
sb.write('[');
473+
sb.write(s.substring(start, j));
474+
sb.write(']');
475+
i = j;
476+
}
477+
} else {
478+
// Inside brackets, keep '.' as content.
479+
sb.write('.');
480+
i++;
481+
}
482+
} else {
483+
sb.write(ch);
484+
i++;
485+
}
486+
}
487+
return sb.toString();
488+
}
489+
397490
/// Normalizes the raw query-string prior to tokenization:
398-
/// - Optionally drops a single leading `?` (when `ignoreQueryPrefix` is set).
491+
/// - Optionally drops exactly one leading `?` (when `ignoreQueryPrefix` is true).
399492
/// - Rewrites percent-encoded bracket characters (%5B/%5b → '[', %5D/%5d → ']')
400493
/// in a single pass for faster downstream bracket parsing.
401494
static String _cleanQueryString(

0 commit comments

Comments
 (0)