Skip to content

Commit 4caaa2c

Browse files
239573049Copilot
andauthored
Add TOON format encoding and decoding functionality (#2)
* Add TOON format encoding and decoding functionality - Implement StringUtils for string manipulation, including escaping and unescaping. - Create ValidationShared for key validation and safety checks. - Introduce ToonDecodeOptions and ToonEncodeOptions for customizable encoding/decoding settings. - Develop ToonDecoder for parsing TOON format strings into JsonNode objects. - Implement ToonEncoder for converting data structures into TOON format strings. - Add ToonFormatException for error handling during parsing and encoding. - Create unit tests for ToonDecoder and ToonEncoder to ensure functionality and data integrity. - Remove outdated UnitTest1.cs file. * Update src/ToonFormat/Constants.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/ToonFormat/Internal/Encode/Normalize.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/ToonFormat/Internal/Encode/Encoders.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Optimize parsing performance and fix potential issues Optimize the performance of the `ParseDelimitedValues` and `ToParsedLines` methods: - Reduce memory allocation by using `StringBuilder` and `ReadOnlySpan<char>`. - Allocate capacity in advance to avoid dynamic resizing. - Replace string operations with more efficient logic. Fix the code formatting and logic of `IsArrayHeaderAfterHyphen` and `ParseStringLiteral`: - Adjust the format of the `return` statement. - Eliminate redundant code. New `MapRowValuesToPrimitives` method: - Supports mapping string arrays to JSON primitive values. Other minor optimizations: - Fixed the logic for handling blank lines. - Replaced some string operations with more efficient implementations. - Fixed the issue of character encoding in comments, improving code readability. * Update src/ToonFormat/Internal/Encode/Encoders.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/ToonFormat/Internal/Encode/Normalize.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Unified logic for negative zero normalization of floating-point numbers, and a new FloatUtils utility class has been added. Introduce the FloatUtils utility class and provide the NormalizeSignedZero method, to uniformly handle the logic of negated zero normalization for floating-point numbers, replacing the original repetitive implementations. Update the relevant code in Parser.cs and Normalize.cs to enhance readability. Add the NearlyEqual method for approximate comparison of floating-point numbers. Delete the references to the Internal\Shared\ folder. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 8bf7d83 commit 4caaa2c

26 files changed

+3672
-65
lines changed

src/ToonFormat/Class1.cs

Lines changed: 0 additions & 35 deletions
This file was deleted.

src/ToonFormat/Constants.cs

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
using System;
2+
3+
namespace ToonFormat
4+
{
5+
6+
public static class Constants
7+
{
8+
public const char LIST_ITEM_MARKER = '-';
9+
10+
public const string LIST_ITEM_PREFIX = "- ";
11+
12+
// #region Structural characters
13+
public const char COMMA = ',';
14+
public const char COLON = ':';
15+
public const char SPACE = ' ';
16+
public const char PIPE = '|';
17+
public const char HASH = '#';
18+
// #endregion
19+
20+
// #region Brackets and braces
21+
public const char OPEN_BRACKET = '[';
22+
public const char CLOSE_BRACKET = ']';
23+
public const char OPEN_BRACE = '{';
24+
public const char CLOSE_BRACE = '}';
25+
// #endregion
26+
27+
// #region Literals
28+
public const string NULL_LITERAL = "null";
29+
public const string TRUE_LITERAL = "true";
30+
public const string FALSE_LITERAL = "false";
31+
// #endregion
32+
33+
// #region Escape/control characters
34+
public const char BACKSLASH = '\\';
35+
public const char DOUBLE_QUOTE = '"';
36+
public const char NEWLINE = '\n';
37+
public const char CARRIAGE_RETURN = '\r';
38+
public const char TAB = '\t';
39+
40+
// #region Delimiter defaults and mapping
41+
public const ToonDelimiter DEFAULT_DELIMITER_ENUM = ToonDelimiter.COMMA;
42+
43+
/// <summary>Default delimiter character (comma).</summary>
44+
public const char DEFAULT_DELIMITER_CHAR = COMMA;
45+
46+
/// <summary>Maps delimiter enum values to their specific characters.</summary>
47+
public static char ToDelimiterChar(ToonDelimiter delimiter) => delimiter switch
48+
{
49+
ToonDelimiter.COMMA => COMMA,
50+
ToonDelimiter.TAB => TAB,
51+
ToonDelimiter.PIPE => PIPE,
52+
_ => COMMA
53+
};
54+
55+
/// <summary>Maps delimiter characters to enum; unknown characters fall back to comma.</summary>
56+
public static ToonDelimiter FromDelimiterChar(char delimiter) => delimiter switch
57+
{
58+
COMMA => ToonDelimiter.COMMA,
59+
TAB => ToonDelimiter.TAB,
60+
PIPE => ToonDelimiter.PIPE,
61+
_ => ToonDelimiter.COMMA
62+
};
63+
64+
/// <summary>Returns whether the character is a supported delimiter.</summary>
65+
public static bool IsDelimiterChar(char c) => c == COMMA || c == TAB || c == PIPE;
66+
67+
/// <summary>Returns whether the character is a whitespace character (space or tab).</summary>
68+
public static bool IsWhitespace(char c) => c == SPACE || c == TAB;
69+
70+
/// <summary>Returns whether the character is a structural character.</summary>
71+
public static bool IsStructural(char c)
72+
=> c == COLON || c == OPEN_BRACKET || c == CLOSE_BRACKET || c == OPEN_BRACE || c == CLOSE_BRACE;
73+
// #endregion
74+
}
75+
76+
/// <summary>
77+
/// TOON's unified options configuration, styled to align with System.Text.Json. Used to control indentation,
78+
/// delimiters, strict mode, length markers, and underlying JSON behavior.
79+
/// </summary>
80+
public enum ToonDelimiter
81+
{
82+
/// <summary>Comma ,</summary>
83+
COMMA,
84+
85+
/// <summary>Tab \t</summary>
86+
TAB,
87+
88+
/// <summary>Pipe |</summary>
89+
PIPE
90+
}
91+
92+
}
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
using System;
2+
using System.Text.Json;
3+
using System.Text.Json.Serialization;
4+
5+
namespace ToonFormat.Internal.Converters
6+
{
7+
/// <summary>
8+
/// Normalizes double NaN/Infinity to null when writing JSON, keeping original numeric precision otherwise.
9+
/// Reading still uses default handling, no special conversion.
10+
/// Purpose: Consistent with TS spec (NaN/±Infinity -> null), and provides stable JsonElement for subsequent TOON encoding phase.
11+
/// </summary>
12+
internal sealed class DoubleNamedFloatToNullConverter : JsonConverter<double>
13+
{
14+
public override double Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
15+
=> reader.GetDouble();
16+
17+
public override void Write(Utf8JsonWriter writer, double value, JsonSerializerOptions options)
18+
{
19+
if (double.IsNaN(value) || double.IsInfinity(value))
20+
{
21+
writer.WriteNullValue();
22+
return;
23+
}
24+
writer.WriteNumberValue(value);
25+
}
26+
}
27+
}
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
using System.Text.Json;
2+
using System.Text.Json.Serialization;
3+
4+
namespace ToonFormat.Internal.Converters
5+
{
6+
/// <summary>
7+
/// Normalizes float NaN/Infinity to null when writing JSON; reading keeps default behavior.
8+
/// </summary>
9+
internal sealed class SingleNamedFloatToNullConverter : JsonConverter<float>
10+
{
11+
public override float Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
12+
=> reader.GetSingle();
13+
14+
public override void Write(Utf8JsonWriter writer, float value, JsonSerializerOptions options)
15+
{
16+
if (float.IsNaN(value) || float.IsInfinity(value))
17+
{
18+
writer.WriteNullValue();
19+
return;
20+
}
21+
writer.WriteNumberValue(value);
22+
}
23+
}
24+
}

0 commit comments

Comments
 (0)